Assuming U as the embedded x Seq (word sequence form of x), in source memory, we compute the match between the k-th topic and the embedding of the l-th word in x Seq by where [x; y] denotes the merge of x and y, and we use concatenation operation here (Dou, 2017;Chen et al., 2017).
such errors can be reduced by enhancing our NTM to phrase discovery topic models (Lindsey et al., 2012;He, 2016), which is worthy exploring in future work.