Note that we use a slightly updated residual structure as implemented by tensor2tensor 1 than proposed originally.
layer normalization is applied to the input of the residual block instead of applying it between blocks.