So far, not much has been shown about how neural networks are able to compensate for the removal of the structures used in past models.
shuffling does cause performance to degrade relative to the base parser even when the unshuffled win-  dow is moderately large, indicating that the LSTM is propagating information that depends on the order of words in far-away positions.