Nevertheless, these reordering errors are not at all detected by automatic error analysis-it detects reordering errors only if exactly the same words in the reference and in the hypothesis are present.
there are more discrepancies between the human and automatic error analysis for all three outputs: called off is detected as missing, disappeared as a lexical error, and in the hierarchical system output have been are considered as extra words.