Best results are obtained when taking into account both the probability of a form given its dediacritised version and the probability of the form in the given context (TM+LM).
overall, the best type of data for training diacritic restorers of both standard and non-standard texts is Web data.