We try to incorporate part-of-speech features in recurrent neural network language model, and use them to predict the next word.
we proposed a parallel structure which contains two recurrent neural networks, one for word sequence modeling and another for part-of-speech sequence modeling.