Despite being similar, input and output embeddings solve different tasks.
fully tying input and output embedding sub-networks in subwordaware neural language models is worse than letting them be slightly different.