Attentive architectures in NLP, in fact, have recently received substantial interest, mostly for sequenceto-sequence models (which are useful for machine translation, summarization or language modeling), and a myriad of modifications have been proposed, including additive (Bahdanau et al., 2015), multiplicative (Luong et al., 2015) or self (Lin et al., 2017) attention mechanisms.
standard attention mechanisms only tell us which text fragments are considered impor- Figure 1: A classic attention network (top), and our attentive label-wise network (bottom), with a specific attention module for each label.