The convolution result of each kernel, termed conv i , is further activated with the RELU non-linear activation function (Xu et al., 2015), which is formulated as: Pooling Layer: Because utterance and response are naturally variable-sized, we put a max-overtime pooling layer on the top of temporal convolutional layer (Kim, 2014), which extracts the max value for each kernel, and gets a fixsized representation of length k for utterance and response.
representations constructed by CNN with max-pooling reflect the core meanings of utterance and response.