The vocabulary then not only consists of the union over all words w i in any caption of the training set, but also of the union of all attributes a i in any video across the dataset.
an embedding matrix E is used to represent words and we denote E w by an embedding vector of a given w. we obtain input word embedding vectors as: In this section, we introduce the core part of our proposed model, the multi-faceted attention mechanism.