We observe a consistent trend across both of our metrics, with the tasks encoded in a natural progression: POS tags processed earliest, followed by constituents, dependencies, semantic roles, and coreference.
it appears that basic syntactic information appears earlier in the network, while high-level semantic information appears at higher layers.