BERT is also a transformer encoder that has access to the entire input but this choice requires a special training regime.
they multi-task between predicting a subset of masked input tokens, similar to a denoising autoencoder, and a next sentence prediction task.