We first find that FNC-1's proposed evaluation metric favors the majority class, which can be easily classified, and thus overestimates the true discriminative power of the methods.
we propose a new F1-based metric yielding a changed system ranking.