The usual solution is to prune or otherwise approximate the messages at each step.
cotterell and Eisner (2015) gave a principled way to approximate the messages using variablelength n-gram models, using an adaptive variant of Expectation Propagation (Minka, 2001).