"The happy child" vs. "The child" For the Act3 dataset, we observe that middle layers of most models (BERT, Multitask) retain the adjective information well.
surprisingly simple multitask model (lstm 1 forward layer accuracy = 0.89) retains adjective information better than BERT model (layer 7 accuracy = 0.84).