We assume that there is a one-to-one mapping between decision sequences d 1:j−1 and states s j : that is, we essentially assume that a state encodes the entire history of decisions.
each state can be reached by a unique decision sequence from s † .