The first part can be formulated as Here we use subscripts to denote layers and superscripts to denote timesteps.
h t l represents the hidden state of the L-th layer at timestep t. x w t 2 R H 0 is the input embedding of word w t where H 0 is the input embedding size.