Human language is often multimodal, which comprehends a mixture of natural language, facial gestures, and acoustic behaviors.
two major challenges in modeling such multimodal human language time-series data exist: 1) inherent data non-alignment due to variable sampling rates for the sequences from each modality; and 2) long-range dependencies between elements across modalities.