The best bilingual compositional representations are better than non-compositional Glove embeddings (Pennington et al., 2014), but worse than compositional Paragram embeddings (Wieting et al., 2016).
paragram initialization requires large amounts of text and human word similarity judgments for tuning, while our models were initialized randomly.