Although the first one has proven to be beneficial during pre-training to some NLP tasks, we want to check how much its influence is to our final translation performance.
as mentioned in Section 3.4, it is interesting to explore the different usage of pre-trained decoders in the MT task.