To get a dense low-dimenional representation of texts, we test seven representations covering sentence embedding, variants of un-contextualized word embedding and variants of contextualized word embedding.
: (i) Two Google universal sentence encoders: (Cer et al., 2018), an encoder (enc-2) based on deep averaging net (Iyyer et al., 2015) and an encoder (enc-3) based on transformer (Vaswani et al., 2017).