This means that if hyperparameter search is done carefully, the choice of activation function is less important for these sentence classification tasks.
the (binary) tasks MR and SUBJ appear robust against the choice of activation function, with the difference between the best and worst function being always less than 1pp, in all settings.