We also experiment with a variant (+AE) in which the encoded (source) representation must predict its own words as target in addition to those of adjacent sentences.
in FastSent+AE, (1) becomes (2) 8 We searched po, px ∈ {0.1, 0.2, 0.3} and observed best results with po = px = 0.1.