To simulate a case where this independence is broken, we experiment with adding an additional feature that is correlated with the bias feature, but is not treated as being part of the bias (i.e., it is not used by the bias-only model).
80% of the time a token is added to the start of the hypothesis that matches the label with 90% probability, and the "0" token is appended to the end of the hypothesis.