
Microsoft releases code to enable training BERT at large scale on commodity GPUs - TheIronYuppie
https://azure.microsoft.com/en-us/blog/microsoft-makes-it-easier-to-build-popular-language-representation-model-bert-at-large-scale/
======
nonfamous
The actual code is in this repo: [https://github.com/microsoft/AzureML-
BERT](https://github.com/microsoft/AzureML-BERT)

------
TheIronYuppie
Disclosure: I work at Microsoft/Azure on ML

Really excited to announce the release of the code we use to train/fine-tune
BERT at Azure & Bing. Please let me know if you have any questions!

------
kdatta1
seems like the scripts have a dependency on pytorch_pretrained_bert library.
Is there a way to run the dataprep/create_pretraining.py or train.py scripts
on-premise?

~~~
aagarg
The recipes are optimized for AzureML but the implementation can be adapted to
any cluster. --aagarg [MSFT]

------
maxluk
If you have any questions, please let us know. maxluk [MSFT]

