The length penalty in the BLEU metric, in particular, is only an indirect and weak indicator of adequacy.
sMT work has been driven to optimize systems such that they often produce translations that contain significant role confusion errors despite reading fluently.