We propose a deep network, which not only achieves competitive accuracy for text classification, but also exhibits compositional behavior.
while creating hierarchical representations of a piece of text, such as a sentence, the lower layers of the network distribute their layer-specific attention weights to individual words.