For Skip-gram and CBOW, a 5-word window size is used to allow them to get the same amount of raw information, also words appearing 5 times or fewer are filtered out.
both full English Wikipedia and Simple English Wikipedia are used as training corpora with minimal preprocessing procedures: XML tags are removed and infoboxes are filtered out, thus yielding four models: Full English Wikipedia -CbOW(FW-CbOW), Full English Wikipedia -Skip-gram(FW-SG), Simple English Wikipedia -CbOW(SW-CbOW) and Simple English Wikipedia -Skip-gram(SW-SG).