Actually, HFFN can adopt any fusion strategy in local fusion stage other than only outer product, showing high flexibility and applicability.
in our model, when conducting outer product, all three modalities are treated equally, which is probably not the optimal choice.