Neural Networks: Zero to Hero (Andrej Karpathy)

About this course

Andrej Karpathy builds neural networks from scratch in code, from a tiny autograd engine to a GPT model, explaining every step as he types. Who it's for: programmers comfortable with Python and basic calculus who want to truly understand deep learning and LLMs. These are the free, official videos of the "Neural Networks: Zero to Hero" playlist from the Andrej Karpathy YouTube channel (youtube.com/@andrejkarpathy). All videos belong to Andrej Karpathy; Sikhami is not affiliated with Andrej Karpathy and simply arranges the public playlist into a course, so you can learn one lesson at a time and keep track of your progress.

What you will learn

• Backpropagation, by building the micrograd autograd engine • Language modelling with bigrams and multilayer perceptrons (makemore) • Activations, gradients and batch normalisation • Backpropagating by hand through a network • A WaveNet-style model • Building a GPT from scratch and reproducing GPT-2 • How assistants like ChatGPT are trained

Course content

1 section · 10 lectures · 19h 21m

1. 1. Neural Networks: Zero to Hero

   • The spelled-out intro to neural networks and backpropagation: building micrograd2h 25m
   • The spelled-out intro to language modeling: building makemore1h 57m
   • Building makemore Part 2: MLP1h 15m
   • Building makemore Part 3: Activations & Gradients, BatchNorm1h 55m
   • Building makemore Part 4: Becoming a Backprop Ninja1h 55m
   • Building makemore Part 5: Building a WaveNet56m
   • Let's build GPT: from scratch, in code, spelled out.1h 56m
   • State of GPT | BRK216HFS42m
   • Let's build the GPT Tokenizer2h 13m
   • Let's reproduce GPT-2 (124M)4h 1m