Keeping in mind we want an efficient model, we encode words in supporting documents and in the query using only a pre-trained model for contextualized word representations rather than training our own encoder.
we use ELMo 2 (Peters et al., 2018), a pre-trained bi-directional lan-guage model that relies on character-based input representation.