Backpropagation calculus

Excellent!!! It's really helpful!

Thanks for the reference

I highly recommend Michael Nielsen's (free) book on the topic. You'll find that it aligns with these videos, and he talks more about the notation to make this algorithm more compact, and how it looks when you implement it yourself.

3blue1brown

Thank you so much for your wonderful video. I wonder if you are also interested in explaining back propagation in tensor notation, which I would like to get familiar with. (Actually, I wish I could learn tensor in general, but could not find a good material to study. So, if you could just let me know some good textbooks, it would also be much appreciated. I study some graduate level statistics and machine learning classes, but I did not have good undergrad math prerequisites.)

you made me, a 12 year old kid, understand calculus. here, one month later, i'm teaching myself some multivariable calculus at 13. thanks.

Really good feedback to hear, thank you!

3blue1brown

You're talking completely over my head on this one so I think your beginning disclaimer should be: "If you don't have a very good handle on calculus via my calculus track, then this video is going to annoy you". My eyes went full-glazed over at the 5 minute 30 mark because none of the equations are clicking. The one part of the whole video where I felt I got a small foothold on understanding is at 7:25 where you're talking about the sum of squared differences between "alj" and "yj". I have a little bit of experience with this because I thoroughly understand RMSE, as described very clearly here: <a href="https://stackoverflow.com/questions/17197492/root-mean-square-error-in-python" rel="nofollow noopener" target="_blank">https://stackoverflow.com/questions/17197492/root-mean-square-error-in-python</a> The minus gives you the distance-cost, the squared throws it all into the positive, the sum totals up all the costs. I think you're doing the listeners a dis-service when you're reading the equations verbatim, like when you say "Sum" of "ylj" and "yj" squared. We can read the equation, because you make it pop out as you say it. I'd prefer you say: "The sum of "What the network produced" minus "The correct targets". Rather than reading out the letters you've selected, give them a little mnemonic so we know what kind of data the "YLJ" holds. I think a bottom up approach might be better, where you begun at the cost function, because that's where the first concrete numbers are found. Start at the beginning, which is the cost, then percolate out towards what we need to do, plot a tangent to the curve and roll downhill, fiddling the weight. It's all still pretty fuzzy. It hasn't clicked yet. Still, good job, I like it, but a lot of listeners are going to scratch their heads and say: "Sounds real cool, but I don't get it".