The output of the last layer h L,k is then employed in conjunction with a softmax layer to predict the next token at k + 1.
(2018) posit that the best way to take advantage of ELMo's representational power is to compute a linear combination of BiLSTM layers: where s j is a softmax-normalized task-specific parameter and γ is a task-specific scalar.