Metrics As is standard in semantic parsing, we measure accuracy, the fraction of correctly generated examples.
because generating an exact match for complex code structures is nontrivial, we follow Ling et al.