For an input we encode this input as $s$ and thus assume that s 0 = $ and s n+1 = $.
our RNNs use ReLUs (Rectified Linear Units), so for every v ∈ Q N we let σ v (the ReLU activation) be the vector the ReLUs act like identities on nonnegative inputs, but clip negative inputs to 0.