Considering the unique challenge of COSMOS, we explore two related multiple-choice datasets for knowledge transfer: RACE (Lai et al., 2017), a large-scale reading comprehension dataset, and SWAG (Zellers et al., 2018), a large-scale commonsense inference dataset.
we first fine-tune BERT on RACE or SWAG or both, and directly test on COS-MOS to show the impact of knowledge transfer.