We had expected human annotators to pick up on these fluency errors, giving the the advantage to the IR systems.
it appears that MT-CHAT's ability to tailor its response to the status on a fine-grained scale overcame the disadvantage of occasionally introducing fluency errors.