Our architecture combines perceptual input processing, short-term memory storage, an attention mechanism, and a reasoning module.
then, the similarity between the projected memory and the question is computed using the Scaled Dot-Product attention: Next, the memories are combined using the attention weights α i , obtaining an output vector h = j α j m j .