In fact, in standard English, inside sentences, only proper nouns start with upper-cased letter thus fine-tuning the pre-trained model fails to slough this pattern which is not always respected in Tweets.
to do so, we follow (tamaazousti et al., 2017) and analyse the units of Φ (biLStM layer) before and after finetuning.