In order to decide the sequence of sub-systems to add, we first evaluate the performance of pair-wise combinations between each sub-system and its baseline system on the development set.
for each sub-system, we combine its n-best list with the n-best list of its baseline system and perform system combination for MT03 data set.