In fact, the accuracy of the NLU-SA labels in this data set, with respect to the gold SAs, is 53%.
while Rules can achieve very good performance with gold SAs, the high frequency of NLU errors causes a significant degradation in policy performance.