Since it is impossible for the model to predict for classes out of the training set, we thus only keep the overlapping classes between a pair of source and target buildings in evaluation.
given a pair of buildings, if a class exists only in either of the two buildings, we will mark it as an "other" class.