While the second baseline performs well on held-out data from its own sample (90.3% on GIMPEL-DEV), it performs poorly across our out-of-sample test and development sets.
it seems to overfit the sample of tweets described in Gimpel et al.