Further, due to the limited position index during pre-training, most Transformer-based models have a maximum capacity of input tokens.
they often truncate the length of a document to satisfy the length limitation of the encoder, which may lose some important semantics, especially for long documents.