Lecture 1 | Introduction to Convolutional Neural Networks for Visual Recognition

For studying? Mainly watching and re-watching Karpathy's 'Zero To Hero'[1] and Stanford's 'Introduction to Convolutional Neural Networks for Visual Recognition'[2], also a lot of transformers from scratch videos like Umar Jamali's videos[3], and I also study backwards to McCulloch and Pitts. Reading the 30 papers [link] and so on. And of course Yannic Kilcher[4], and also listening in on the paper discussions they do on discord. Practicing a lot with just doing backpropagation by hand and making toy models by hand to get intuition for the signal flow, and building all kinds of smallish systems, e.g. how far can you push whisper, small qwen3, and kokoro to control your computer with voice? People think that deepseek/mistral/meta etc are democratizing AI, but its actually Karpathy who teaches us :) so we can understand them and make our own. [1] [link] [2] [this video] [3] [link] [4] [link]

See lecture 15 of cs231n 2017: [this video] Further, while all these tricks are amazing, they are not currently of much use to fungible data scientists at FAANG and in the greater Fortune 500 because they haven't been added to any of the major frameworks in a manner that is braindead/automagic. Further, most of these are but one GPU generation away from general availability if they become popular like FP16/32 mixed precision did. That said, the acquisition of DeePhi by Xilinx ought to be a sufficiently strong signal to add them to said frameworks, no? [link] Finally, my gut instincts (based on the monotonically decreasing speech recognition quality of my increasingly powerful Android phones over time, and the bizarre spelling errors it now makes such as "tech kneeq" as one example) is that one cannot rate the quality of these techniques from a single metric, and it seems like the decision to foist them on users may be based on one. #ItsAdversariesAllTheWayDown