(2018) used hierarchical attention strategies to learn multimodal representations, Wang et al.
we hypothesize such performance drop occurs because the asynchronous (and much longer) data streams introduce more difficulty in recognizing important features and computing the appropriate attention.