In Table 3, we observe that when we replace the DENSE encoder layers with GRU or biGRU layers, the improvement from the multilingual training is still present.
the multilingual models with shared atten-   tion are superior to alternatives, regardless of the employed encoders.