A common technique for learning such experts in the Weighted Majority algorithm (Littlestone and Warmuth, 1994), which weighs a mixture of experts (classifiers).
since we require a hard assignment -pick a single shared parameter set s -rather than a mixture, the algorithm reduces to picking the classifier s with the fewest mistakes in predicting domain d. This requires tracking the number of mistakes made by each shared classifier on each domain once a label is revealed.