It is important to recall that the main goal of our work is to develop fast and efficient on-device neural text classification approach, which can achieve near state-of-the-art performance while satisfying the on-device small size and memory resource constrains.
we train the network with cross entropy loss and apply softmax over the output layer to obtain predicted probabilities y C k for each class C during inference.