The state of the art performance, using distant supervision (Li et al., 2012), achieves an end-to-end Avg-Q score of 0.678 (on training data), where we achieve 0.679 (on evaluation data).
our scores are not directly comparable since we reduce the number of classes (and the amount of evidence) in our evaluation.