We experimented on two datasets from SemEval 2014 (Pontiki et al., 2014), which consist of reviews and comments from two categories: LAPTOP and RESTAURANT.
giving the superior performance of BERT on downstream tasks, it is natural to ask if BERT is learning an empirically useful structure of language.