Given two probability mass function and ( ), the KL divergence (or relative entropy) between and is defined as: In terms of text sequences (either queries or documents), the probability distribution can be regarded as a probabilistic language model or from each document or each query .
it assumes that there is an underlying language model which "generates" a term (sequence) (Ponte and Croft, 1998).