The metric described above only considers word overlap and ignores other semantic relations (e.g., synonymy, hypernymy) between words.
annotators write labels of their own and may use words that are not directly from the conversation but are semantically related.