In either case, one can build a user simulation model that is the average of different user behaviors or learn a policy from a corpus that contains a variety of interaction patterns, and thus safely assume that single-agent RL techniques will work.
in the latter case if the behavior of the user changes significantly over time then the assumption that the environment is stationary will no longer hold.