Part-of-speech tags can be easily obtained for unannotated data using off-the-shelf POS taggers or PCFG parsers.
the amount of information these tags typically provide is very limited, Figure 1: A parse tree example e.g., while it is helpful to know whether fly is a verb or a noun, knowing that you is a personal pronoun does not carry the information whether it is a subject or an object (given the Penn Tree Bank tagset), which would certainly help to predict the following word.