Variance reduction by additive control variates has implicitly been used in doubly robust techniques (Dudik et al., 2011;Jiang and Li, 2016).
the connection to Monte Carlo techniques has not been made explicit until Thomas and Brunskill (2016), nor has the control variate technique of optimizing the variance reduction by adjusting a linear interpolation scalar (Ross, 2013) been applied in off-policy learning.