The proposed architecture introduces 4 additional parameter matrices that are optimised during training: and ← − W q .
the computational complexity and resource requirements of this model during sequence labeling are equal to the baseline from Section 2, since the language modeling components are ignored during testing and these additional weight matrices are not used.