Compared with traditional single prototype model, these models have demonstrated significant improvements in many semantic natural language processing (NLP) tasks.
they suffer from a crucial restriction in terms of scalability when facing exploding training text corpus, mainly due to the deep layers and huge amounts of parameters in the neural networks in these models.