It has been shown that simply fine-tuning large, pre-trained language models such as GPT (Radford et al., 2018) and BERT (Devlin et al., 2019) can be a very strong baseline method.
there still exists a large gap between performance of said baselines and human performance.