Parallel Sentences occur naturally, and provide training examples that are more consistent with downstream applications.
they can be noisy due to automatic sentence alignment and one-to-many mappings, and bag-of-word representations of sentence meaning are likely to be increasingly noisier as segments get longer.