We experiment both with using the embeddings as is, and fine-tuning the top layers of the attention encoder (we only try fine-tuning the top, toptwo and top-three layers due to computational constraints).
this usually requires a large amount of fine-grained data annotated at the token level, which in turn can be expensive and cumbersome to obtain.