More recently, the system based on multi-layer self-attention (call it Transformer) has shown strong results on several large-scale tasks (Vaswani et al., 2017).
approaches of this kind benefit greatly from a wide network with more hidden states (a.k.a.