All the models follow the BERT-Base setting, i.e., L = 12, H = 768, A = 12, and are trained on the same corpus -BooksCorpus and English Wikipedia with uncased word piece tokens.
all models use the same vocabulary as Google BERT (uncased version) with vocabulary size 30,522.