In this section, we examine the intrinsic properties of contextual vectors learned with biLMs, focusing on those that are independent of the architecture details.
we seek to understand how familiar types of linguistic information such as syntactic or coreferent relationships are represented throughout the depth of the network.