Previous works have shown that visual attention alone can perform fairly well for localizing objects  and aiding caption generation (Xu et al., 2015).
most recent image captioning models are based on a CNN-RNN framework (Vinyals et al., 2015;Fang et al., 2015;Karpathy and Fei-Fei, 2015;Xu et al., 2015;You et al., 2016;Krause et al., 2017).