BERT LARGE This is the large BERT model released by the authors, which we used as a baseline.
since MT-DNN uses BERT LARGE to initialize its shared layers, the gain is mainly attributed to the use of MTL in refining the shared layers.