We observe that adding a component-specific vector to the recurrent cells in the bottom layer of the decoder is the most effective, and results in BLEU scores superior or onpar with the other variants across the four language pairs.
we use this model variant in all experiments for the rest of the paper.