The latest work is Bidirectional Beam Search (BiBS) (Sun et al., 2017) which proposes an approximate inference algorithm in BiRNN for image caption infilling.
this method is based on some unrealistic assumptions, such as that given a token, its future sequence of words is independent of its past sequence of words.