Empirically, we observed that, if with only s + 0 in the AGT network, namely removing s + i for i > 0, the test-set accuracy dropped from 50.5% to 48.5%.
transforming a linear combination of the bag-of-words features was insufficient for obtaining sufficient accuracy for the classification task.