Attention over tokens on the other hand can push the attention weights towards either the subject or object in the KB tuple, based on the query's request.
using both in conjunction helps BOSSNET perform significantly better than the baselines and illustrates the importance of the BOSS memory in comparison to a flat memory layout.