Even though listeners are known to excel at the task of native speaker versus non-native speaker distinction (McCullough and Clopper, 2016), we acknowledge that perception of nativelikeness remains a somewhat subjective judgement.
we were cautious in our assessment, and decided to ask for further judgements from CrowdFlower workers which we could refer to if in doubt (section 3.2).