For instance, two enti- ties such as Tesla and Edison could have close representations in the word embedding space, within a scientific-related subset of tokens: the use of a placeholder would thus prevent the use of such information.
we concatenate the contextfree vectors (see 3.2) for a specific token with the corresponding ELMO (Peters et al., 2018) representation at the encoding stage.