The hidden layers and word embeddings are opaque and difficult from which to draw conclusions.
a generative model that represents words explicitly as probability distributions allows for easier post-analysis.