Here, pre-activation refers to activation being done before weighting instead of after as is typically done.
in the convolu-tion layer of DPCNN, Wσ(x) + b is computed at every location of each document where a column vector x represents a small region (overlapping with each other) of input at each location, σ(•) is a component-wise nonlinear activation, and weights W and biases b (unique to each layer) are the parameters to be trained.