Ideally the third item can be estimated by using the forward-backward algorithm (Rabiner 1989) recursively for the first-order (Rabiner 1989) or second-order HMMs (Watson and Chunk 1992).
several approximations on it will be attempted later in this paper instead.