While the value of the outer memory cell in the LSTM is estimated as c outer t = f t c t−1 + i t g t , memory cells of the NLSTMs use the concatenation (f t c t−1 , i t g t ) as input to an inner LSTM (or NLSTM) memory cell, and set c outer t = h inner t .
in compared to the LSTM and stacked LSTM, the inner memories of NLSTMs operate on longer time-scales and effectively capture the context information from the input texts.