The reason is that the number of its parameters is so large that the trained network has overfitted on training data.
it is necessary to capture the potential long-distance dependencies without increasing the size of the context window.