In this work, we stick to the standard neural approach to modeling the conditional probability.
a trainable neural network is used to encode the context x <t into a fixed size hidden state, which is multiplied with the word embeddings to obtain the logits.