This is because two domain-specific embeddings are trained separately on source and target datasets, and therefore the two embedding spaces are heterogeneous.
model parameters trained in each model are heterogeneous as well, which hinders the transfer process.