In hopes of improving the inductive bias of the learner, we partitioned the hidden state vector into three sub-vectors: The mark scoring function f θ (s t−1 , ω t ) was as before, but we restricted the form of g θ , the state update function.
s a t encodes all past marks and depends on the full hidden state so far: we make s x t encode only the sequence of past input marks, ignoring all others.