@machinelearningtogo: ➡️Machine Learning from Scratch, part 51: dropout. At every training step each hidden neuron flips its own coin and about half of them are switched off, a new random half every step. No neuron can lean on its neighbours, so the knowledge spreads over many neurons, and only patterns that survive these small tests get stronger. After training, dropout is switched off and all neurons work together, like the average of many smaller networks that share the same weights. The code (dropout_reel.py, 16 lines of numpy) trains the same 1-64-1 tanh network twice on the same 12 noisy points of sin(3x), 60,000 steps of plain gradient descent, once the plain way and once with inverted dropout (chance one half, survivors multiplied by two). On 99 new points of the same curve the plain network ends at an error of 0.135 (its training error is almost zero, and its test error grows from 0.026 at step 100), the dropout network at 0.045: three times smaller, with a training error of 0.069. Sources: Hinton et al. 2012, arXiv:1207.0580; Srivastava et al. 2014, JMLR 15. #machinelearning #ai #coding #neuralnetworks #deeplearning

MachineLearningToGo
MachineLearningToGo
Open In TikTok:
Region: DE
Sunday 08 November 2026 11:40:30 GMT
17633
1270
10
68

Music

Download

Comments

jkyan01
josh k :
how about for traditional ML
2026-11-09 08:12:34
0
jack.chen3998
Jack chen :
The coin-flip analogy finally made dropout click for me — does it help as much on small datasets too?
2026-11-18 18:25:03
0
insipid.programmer
insipid.programmer :
Overfitting is such an annoying waste of time 😳
2026-11-17 22:49:00
0
ja29704121009
JA 💙 :
does this slow learning down
2026-11-17 23:05:46
0
gatis122
Yanmar :
Do you have a YouTube Channel?
2026-11-15 00:56:39
0
anxiousandcringey
Alex12345 :
👍👍👍
2026-11-08 12:27:09
0
larry_labs
Larry Labs :
😁😁
2026-11-19 13:33:41
0
To see more videos from user @machinelearningtogo, please go to the Tikwm homepage.

Other Videos


About