We first confirm that our probe can't simply "learn to parse" on top of any informative representation, unlike parserbased probes (Peters et al., 2018b).
eLMO0 and DeCAY0 fail to substantially outperform a right-branching-tree oracle that encodes the linear sequence of words.