A number of models (Simonyan and Zisserman, 2015;He et al., 2015He et al., , 2016Conneau et al., 2016) increase the number of feature maps whenever downsampling is performed, causing the total computational complexity to be a function of the depth.
we fix the number of feature maps, as we found that increasing the number of feature maps only does harm -increasing computation time substantially without accuracy improvement, as shown later in the experiments.