2 Zero in the sense of non-character like BOS or EOS The model above is exactly an 0 th -order neuralized hidden Markov model (HMM).
p(y i | a i , y <i , x) can be regarded as an emission distribution and p(a i | y <i , x) can be regarded as a transition distribution, which does not condition on the previous alignment.