This is because it is common for English words to act as either a noun or a verb and, since these are the most polysemous grammatical categories, the average polysemy count becomes large due to the cumulative effect of polysemy across grammatical categories.
words that can act as adjectives or adverbs are unlikely to be nouns or verbs.