In particular, we observe a decent accuracy in identifying discourse relations between adjacent EDUs, perhaps due to the "next sentence prediction" task in pre-training, as pointed out in (Shi and Demberg, 2019).
our probes fall behind the left-chain baseline, which benefits from its strong structural prior 4 (principal clause mostly in front of its subordinate clause).