At this stage, we took advantage of the fairly homogeneous HTML structure of the crawled web documents to identify the actual content, remove boilerplate, and export text segmented into paragraphs.
we obtained a total of 2.50M paragraphs, of which approximately 293K were annotated in the exported files as in a language other than the main language of the document.