This is unusual in the field of text processing which has generally dealt with well-punctuated text: some of the most commonly used texts in NLP are machine readable versions of highly edited documents such as newspaper articles or novels.
there are many types of text which are not so-edited and the example which we concentrate on in this paper is the output from ASR systems.