Pretrained sentence encoders have become particularly popular due to the success of training models for downstream tasks on top of their learned representations, greatly improving the results compared to training from scratch, especially in low-data regimes (see Table 1).
we also probe the usefulness of ConveRT encodings for transfer learning in the intent classification task: the model must classify the user's utterance into one of several predefined classes, that is, intents (e.g., within e-banking intents can be card lost or replace card).