(2015) trained a character LSTM language model on source code and showed that individual neurons in the hidden state track the beginning and end of code blocks.
as  showed the biLM based representations outperformed CoVe in all considered tasks, we focus exclusively on biLMs.