The word layer is connected to a convolutional output layer y t by weights summarized in the sparse matrix C. The output layer represents all possible next minimal units, where each MTU entry is only connected to neurons in the word layer representing its source and target words.
the word and MtU layers are then computed as follows: there are a number of computational issues with this model: First, we cannot efficiently factor the word layer w t into classes such as for the atomic MtU RNN model because we require all its activations to compute the MtU output layer y t .