We find counts of two types of common mistakes: deletions, where the model fails to realize a value from the input MR, and repetitions, where the model repeats the same value more than once.
9 we compute SER per MR as SER = D+R N , where D and R are the number of deletions and repetitions, and the N is the number of tuples in the MR, and average across the test outputs.