For example, a model can have an optimal policy with a very high ECR value, but have very wide confidence bounds reaching even into negative rewards.
the V-values indicate how much reward one would expect from starting in that state to get to a final state.