
How TensorFlow outdid itself in handling Overfitting in DNNs. - salma-ghoneim
https://heartbeat.fritz.ai/5-tensorflow-techniques-to-eliminate-overfitting-in-dnns-281590cc2eb
======
johnmoberg
Misleading title. These techniques are standard and not particular to
TensorFlow.

------
jphoward
Ignoring the fact that the title of this post is misleading and also not used
by the linked article, I was interested to read that MaxNorm was _so_
effective. In my experience it is rarely used in the state-of-the-art
ConvNets, though maybe that is because they are trained on large ImageNet
datasets where overfitting is less of an issue? Weight decay/L2 norm seems
almost ubiquitous is comparison.

Have other HN readers found MaxNorm to be that useful? Am I missing out?

------
kuu
It's interesting to see all the techniques listed but there is no indication
about when and how use any of them. Is it better to start by dropout or by
regularization? Can I use both? (These are rhetorical questions...)

And as another commenter said, this is not unique from TF.

