HunAlign To employ HunAlign, we use an approach similar to the three-step workflow used in the JRC-Arcquis corpus (Steinberger et al., 2016) and DCEP corpus (Hajlaoui et al., 2014), which consists of an initial alignment using length similarity, automatic dictionary construction from the initial alignments, and a second alignment using lexical similarity calculated with the constructed dictionary in the second step.
we first run HunAlign to obtain the initial alignment without dictionary, randomly sample 10,000 sentence pairs from the 1-to-1 segments in the initial alignment, build a dictionary with minimum occurrence score of 2 and minimum association score of 0.2, and finally, re-align all sentences with the constructed dictionary.