Although encouraging the invertible constraint leads to slightly poorer performance of f −1 de on unsupervised tasks, it generally leads to better sentence representations when the ensemble of the encoder f en and the inverse of the decoder f −1 de is considered.
for unsupervised tasks, the ensemble is an average of the two vector representations produced from the two encoding functions during the testing time, and for supervised tasks, the concatenation of the two representations is regarded as the representation of a given sentence.