BERT uses a cross-encoder: Two sentences are passed to the transformer network and the target value is predicted.
this setup is unsuitable for various pair regression tasks due to too many possible combinations.