The key techniques of our approach are twofold: memoryshared embeddings and a variant of the key-valuequery (KVQ) self-attention mechanism (Vaswani et al., 2017).
our approach reconstructs well-trained word embeddings using a limited number of embedding vectors that are shared by all the subwords with an effective weighting calculated by the self-attention mechanism.