We create the diverse queue by sampling instances with high prediction confidence from all queues.
at every episode, we rank instances of each queue based on their prediction confidence and create a diverse queue by combining top r% instances of each queue, where r indicates the rate of adding new instances and set to r = 10%.