In ensembling, multiple models are trained and their predictions are averaged during inference.
if α l denotes the output of model l ∈ [1, L], the output probability is defined as: Since ensembling reduces the model variance while maintaining low bias, it is most effective when the models are diverse and each model excels at a particular type of input.