Each head of the encoder-decoder is fed with one type of input sequence, and they are combined at the end of decoder layer.
the tag prediction becomes more stable than using a simple encoderdecoder without the multi-head.