Upon inspection of the gathered labels, the high difficulty comes from the fact that there were many Turkers who labeled the data as neutral and also many who labeled it as contradiction.
an example that is easy for humans but difficult for the DNN models (Table  2, row 2) requires more abstract thinking than the earlier example.