It also includes a cross-attention layer for fusion.
the output of the lexicon encoder is appended with the CoVe vector and passed to the contextual encoding layer, which is a 2-layer Bidirectional LSTM that produces hidden states of the passage H p ∈ R T ×2m and the question H q ∈ R T ×2m , where m is the hidden size of the BiLSTM.