To estimate the parameters in (5), we create a dataset of ranked pairs from M in (4): for each utterance u and each observed fact f + = u, x + , where M u,x ≥ δ, we choose each word pattern/slot x − such that f − = u, x − , where M u,x < δ, which refers to the word pattern/slot we have not observed to be in utterance u.
we construct the observed data O from M .