One of the main drawbacks of HOCA is the generation of high-order tensor, the size of the high-order tensor will increases exponentially with the number of modalities as n i=1 t i , resulting in a lot of computation.
space (G) Training Time (s) HOCA-U 4.1 16418 HOCA-UBT 9.7 37040 L-HOCA-UBT 5.6 24615 The theoretical complexity of different attention mechanisms is illustrated in section 4.3.