Additionally, the use of CNN layers allows us to explicitly control the window size for phrase modeling, which has been shown to be critical for relevance matching (Dai et al., 2018;Rao et al., 2019).
the contextual encoder enables us to obtain long-distance contextual representations for each token.