We also measure the quality of sentences at a syntactic or shallow semantic level that operates at the level of dependency triples by a measure that we call Pyramid BE.
we extract all of the dependency triples of the form t = (h, r, a) from the sentence under evaluation and the gold-standard summaries, where h and a are the lemmata of the head and the argument, and r is the syntactic relation, normalized for grammatical voice and excluding the collapsed edges which are mostly noun-phrase-internal elements and grammatical particles.