Inspired by these findings, we propose to exploit deep representations for NMT models.
we investigate two types of strategies to better fuse information across layers, ranging from layer aggregation to multi-layer attention.