
Finetune – Scikit-learn style model finetuning for NLP - madisonmay
https://github.com/IndicoDataSolutions/finetune/tree/master
======
ovi256
This is another good result of applying transfer learning to NLP.

Transfer learning works great for vision problems (just reuse one of the big
SoTA trained on ImageNet networks - I like resnet50). This was enabled by the
amazingly shared structure of vision problems. There was nothing similar for
NLP, besides pre-trained first layers like word2vec. If you want to learn
more, check out the fast.ai DL course, it features transfer learning a lot.

But this model and ULMFiT (nlp.fast.ai) show that deeper nets can be
pretrained for NLP, and achieve good results when transfered to other datasets
and problems.

This enables not just the obvious use case of "I don't have N GPUs to train a
deep net from scratch but I can now finetune a pre-trained model" but more
subtle and interesting cases like fine-tuning on a very small dataset
(compared to ImageNet or 100000 samples NLP data sets) and cheap training on
demand. Training a new model for every user was way too expensive if training
from scratch, but if fine-tuning a pre-trained net takes just a few minutes,
why not ?

~~~
mikert5671
more marketing from fast.ai. Its in every single ML thread

~~~
joshgel
I found fast.ai the best resource for learning ML/DL out there. So, while I'm
not associated with them in any way, I wouldn't hesitate to recommend it as a
course for anyone interested in learning more about ML/DL. Not sure that would
count as marketing if I were to do so...

------
Tarq0n
*a very specific implementation of NLP.

Not that this library isn't promising, but the name and presentation makes it
seem far more general than it really is.

------
stared
In that spirit, and most likely much more general, for PyTorch:

[https://pytoune.org/](https://pytoune.org/) (Keras-like interface for
PyTorch) and
[https://github.com/dnouri/skorch](https://github.com/dnouri/skorch) (Scikit-
learn interface for PyTorch).

As a side note, a project of mine: super-simple Jupyter Notebook training
plots for Keras and PyToune:
[https://github.com/stared/livelossplot](https://github.com/stared/livelossplot)
(with bare API, so you can connect it to anything you wish)

