They find that language models trained on a number of different languages assign higher probabilities to valid long-distance dependencies than to incorrect ones.
lMs seem able to induce syntactic information despite being provided with no linguistic annotation.