The opponent probability P o (y i |y i−1 1 , X) is computed by stacking several layers on top of attention o , and a softmin layer in the end as shown in the right part of Figure (1).
where W is the matrix of the linear projection sublayer.