Compared to the dot-product attentions in GAT, KGAT's Edge attention focuses on fewer tokens and has a smaller entropy.
the BERt sentence retrieval outperforms ESIM sentence retrieval significantly, thus also helps improve KGAt's reasoning accuracy.