The former NER output is clearly preferable to the latter; because, in the former case, lung and cancer are semantically related to the reference annotation; the meaning of the the top level entity (lung cancer) can be composed from the annotated entities.
only recognizing cancer should arguably be penalized, since it is not as informative.