Our work thus leverages this computational efficiency of TMs for efficient and scalable fine-tuning for BERT in document classification.
our contributions(1) Complementary Fine-tuning: We present a novel framework: TopicBERT, i.e., topic-aware BERT that leverages the advantages of both neural network-based TM and Transformer-based BERT to achieve an improved document-level understanding.