Following monolingual coherence evaluations (Lau et al., 2014), we present topic pairs to bilingual CrowdFlower users.
as expected, CNPMI outperforms INPMI regardless of reference corpus overall, because INPMI only considers monolingual coherence.