CLM is inherently unidirectional, as the model must not be able to "peek" at the word it has to predict.
representations from Transformers), a baseline featuring the same architecture as QBErT but missing the BiTransformer layer: the outputs of the past and future stacks are simply combined through elementwise sum after the position shift.