Instead of selecting the top K% utterances, we define a threshold θ: u is selected if R(u) >= θ. θ and α k become hyperparameters which can be optimized using Bayesian Optimization with the score of the CLTL model on a development set as the objective.
while this could improve performance, it is expensive, especially in large-scale settings, i.e.