We are relying on information available from e to get the best tokenization for f. However, the parallel sentences will not be available for new data we want to translate.
for the new data, we have to rely only on P (f ) to tokenize any new data, which can be obtained by calculating P (f ) = e P (f | e)P (e) With P (f ) from the bilingual model, we can run the Viterbi algorithm in the same manner as monolingual tokenization model for monolingual data.