We wanted to evaluate performance out-of-the-box because    bootstrapping a supertagger for a new language is one of the main use scenarios we envision: in such a scenario, there is no development data for changing settings and parameters.
we determined a train/test split beforehand and ran the methods exactly as we had for CCGbank.