The HICO dataset also has multiple annotations per object and it incorporates the information that certain interactions such as riding a bike and holding a bike often co-occur.
it fails to include annotations to distinguish between multiple senses of a verb.