We adopt the idea of early-fusion (Srivastava and Salakhutdinov, 2012), which means the bimodal and trimodal interactions are captured by the concatenated features from different modalities.
we define our factorized model as: where ⊕ denotes vector concatenation for bimodal and trimodal features.