Effectiveness with fewer pre-training steps We followed the standard practice that the models are pre-trained by maximum likelihood before starting reinforcement learning.
such pre-training may have a negative effect in reinforcement learning.