The Hybrid RNN 2 Attention BiRNN : https://github.com/HadoopIt/rnn-nlu and Hybrid RNN : https://github.com/mesnilgr/is13  (Mesnil et al., 2015) applied the RNN to slot filling for the first time, the Encoder-labeler Deep LSTM (Kurata et al., 2016) applied the encoder-decoder concept to slot filling, and the Attention BiRNN (Liu and Lane, 2016) applied attention and multi-task learning to slot filling and intent detection and achieved the state-of-the-art performance among the previous approaches for both.
we did not utilize any kind of pre-training or external data for any of the experiments.