The simplest phonetic recognizer is a regular recognizer with the vocabulary replaced by the list of phonemes of the language, and the language model replaced by a phoneme M -gram.
such phonetic language model is much weaker than a word language model.