While in state s ∈ S and performing action a ∈ A, the learner receives a reward r(s, a), and advances to state s = δ(s, a).
to learn an optimal control strategy that maxi-mizes the cumulative reward over time, an evaluation function Q(s, a) is defined as follows: Q(s, a) is the immediate reward, r(s, a), plus the discounted maximum future reward starting from the new state δ(s, a).