For the second input, odd-numbered layers receive h Between the LSTM layers, we use the following connection (Zhou and Xu, 2015), Here, we firstly concatenate x t , and then calculate the inner product of the concatenated vector and the parameter matrix W (ℓ) with the rectified linear units (ReLU).
we obtain the input representation x (ℓ+1) t for the next (ℓ + 1-th) LSTM layer.