More generally, we compute a task specific weighting of all biLM layers: (1) In (1), s task are softmax-normalized weights and the scalar parameter task allows the task model to scale the entire ELMo vector.
we find that including ELMo at the output of the biRNN in task-specific architectures improves overall results for some tasks.