While some work (Luong et al., 2015b;Luong and Manning, 2016;Wu et al., 2016) has achieved state-of-the-art results using deep networks, others (Jean et al., 2015;Chung et al., 2016;Sennrich et al., 2016b) have produced similar results with far shallower ones.
it is unclear how important depth is, and whether shallow networks are capable of producing results competitive with those of deep networks.