The second task predicts whether a pair of sentences are consecutive in a document to learn the relation between the sentences.
as illustrated in   pre-trained BERT model 1: Obtain a set of phrase alignments A as pairs of spans for each s, t ∈ P 2: WordPiece tokenization of P 3: Accommodate phrase spans in A to BERT's token indexing: Encode b t by the BERT model 7: Compute loss: L(Θ) 8: For phrasal paraphrase task: For sentential paraphrase task: Compute gradient: ∇(Θ) 12: Update the model parameters 13: until convergence token of every input is always the special token of [CLS].