(1) is conducted in a sequential manner based on two adjacent data bags B j−1 and via parameters Ψ of the feature extractor updated bŷ B j−1 .
the state transition probability P(s |s, a) is determined by stochastic optimization and other randomness in training, e.g., dropout (Srivastava et al., 2014).