Naturally, we would like to ask whether it is possible to decode, or at least partially infer, a word's property from the neuron behaviours of its context words.
if the model process a verbal phrase 'spill over' or 'in the garden', will the property of the word 'spill', 'in' be combined with the following word and decodable from the model neuron activation behaviours towards the following word, or will the property of the word 'over', 'the garden' be primed by the previous word and decodable from the model neuron behaviours towards the previous word?