We can observe that even without pre-training our models outperform existing models, and pretraining gives further improvements.
pre-trained Grid-CNN gives an improvement of 2.5% over the Grid model, and including entity features pushes the improvement further to 3.7%.