Then, the algorithm iteratively merges the most frequently co-occurring tokens into a new token until a predefined vocabulary size is reached.
the distribution over the target tokens is modeled using the previously defined transformer model as follows: where h L is the sequence of states after the final layer L, W e is the embedding matrix, and θ are the model parameters that are optimized by stochastic gradient descent.