A.1 BoW models outperform more complex models on our sentence content probe In addition to the paragraph embedding models presented in the main paper, we also experiment  with the following embedding methods: We consider an LSTM (Hochreiter and Schmidhuber, 1997) encoder-decoder model paired with a reconstruction objective.
we implement a single-layer bidirectional LSTM encoder and a two-layer unidirectional LSTM decoder.