The other potential problem for correlation analysis of human vs. automatic framework is that high corpus-level correlation might not translate to high sentence-level correlation.
high sentence-level correlation is often an important property that machine translation researchers look for.