The samples with different annotations are then reviewed by both annotators to produce the final result.
precision is defined as the percentage of correct NEs of a given class from the automatically extracted ones.