We found that absolute grammaticality judgments were hard to achieve on Mechanical Turk; Turkers' ratings of grammaticality were very noisy and they did not consistently rate true article sentences above obviously noised variants.
we implement a deletion-based BiLSTM model for sentence compression (wang et al., 2017) and run the model on top of our extraction output.