Finally, as a final (and perhaps thoughtprovoking) finding, the highest attention weights associated to the emoji were given to the words game, boys and football, in that order.
standard attention mechanisms only tell us which text fragments are considered impor- Figure 1: A classic attention network (top), and our attentive label-wise network (bottom), with a specific attention module for each label.