Here, we use a very simple rule: in the English snippet, we regard all characters within (and including) the first and the last English letter in the snippet as the English content; similarly, in the Chinese snippet we regard all characters within (and including) the first and the last Chinese character in the snippet as the Chinese content; b) Word segmentation of the Chinese content.
our approach contains four steps: 1) Preprocessing: parse the web page into a DOM tree and segment the inner text of each node into snippets; 2) Seed mining: identify potential translation pairs (seeds) using an alignment model which takes both translation and transliteration into consideration; 3) Pattern learning: learn generalized patterns with the identified seeds; 4) Pattern based mining: extract all bilingual data in the page using the learnt patterns.