Koç University, CS – PhD Student
Thanks to Google for providing free TPU for the training process and for Huggingface for hosting these models on their servers
The models were pretrained on ~4.4 Billion words:
Notes on training data:
Note: More results on other downstream NLP tasks will be added soon. if you use any of these models, we would appreciate your feedback.
You can use these models by installing
tensorflow and Huggingface library
transformers. And you can use it directly by initializing it like this: