A very common technique for combination in such a case is by voting (Brill and Wu, 1998;van Halteren et al., 1998;Sang et al., 2000).
it can also be useful to consider performance on only the subset of data for which a system is confident enough to answer, measured by the secondary measure precision.