Actor-critic (Sutton and Barto, 1998) methods are often adopted, which involves training an additional value network to predict the expected reward.
2 We use an Ima-geNet pre-trained 101-layered ResNet 3 (He et al., 2016) to extract visual features.