In multi-channel attention, feature vectors are first fed into a fully connected neural network to get their hidden representations h i .
the same "entity mask" strategy is used which replaces subject (or object) entity with special NER -SUBJ (or NER -OBJ) tokens to avoid overfittting on specific entities and provide entity type information.