To compare the models comprehensively, we adapt the connectionist temporal classification (CTC) (Graves et al., 2006) method to the prior approaches (e.g., EF-LSTM, MCTN, RAVEN) that cannot be applied directly to the unaligned setting.
these models train to optimize the CTC alignment  objective and the human multimodal objective simultaneously.