We can see that attention and back-transfer loss play important roles in the model.
when both of them are removed (last row in Table 4), although the classification accuracy improves, the perplexity and the content preservation drop significantly.