For example, Peters et al. (2019) retrieve entities' embeddings according to the similarity between Transformer's hidden states and pre-trained graph embeddings, then treat the retrieved embeddings as extra inputs to the next layer.
Bosselut et al. (2019) directly finetune a pre-trained LM on partially-masked triples from a KG, aiming at commonsense KBC tasks.