We train sentence encoders with dynamic metaembeddings using two well-known and often-used embedding types: FastText (Mikolov et al., 2018;Bojanowski et al., 2016) and GloVe (Pennington et al., 2014).
we make use of the 300-dimensional embeddings trained on a similar WebCrawl corpus, and compare three scenarios: when used individually, when naively concatenated or in the dynamic meta-embedding setting (unweighted, context-independent DME and contextualized CDME).