Bidirectional LSTM is used in the feature extractors for all experiments.
f s is a two-layer BiLSTM of hidden size 128 (64 for each direction), and f p is a two-layer BiLSTM of hidden size 128 stacked with a MoE module (see figure 2).