For content and fluency, we compute the average score across all 5 raters for each item, and average those scores to get a final rating for each model, such that higher content and fluency scores are better.
we note that while these measures are common for machine translation, they are not well-suited to this task, since they are based on ngram overlap which is not a constraint within the model; we include them for comparative purposes.