Topic embedding with masked attention (TEMA): Since a topic is a distribution over tokens in the vocabulary, we use the mixture of token embeddings to represent the corresponding topic embedding.
topics with large proportions for a document can be considered as extra input tokens of the decoder.