For the parameters φ in the compression model, we firstly define the learning signal, Then, we update the parameters φ by: However, this gradient estimator has a big variance because the learning signal l(s, c (m) ) relies on the samples from q φ (c|s).
following the RE-INFORCE algorithm, we introduce two baselines b and b(s), the centred learning signal and inputdependent baseline respectively, to help reduce the variance.