Hi Reddit, I posted my Build Your Own LLM workshop to Youtube teaching ML, LLM and math intuition [P]

Hi internet friends, I recorded a workshop about building your own LLM without any math / ML prerequisites. It covers everything from machine learning fundamentals, deep neural networks, transformer architecture, and pre/post-training.

The only prerequisite is being comfortable with learning through code & excel examples.

1. Sampling Large Language Models
2. Reverse Engineering Large Language Model
3. Perceptrons: wx+b
4. Activation Functions: ReLU, GELU, SwiGLU
5. GPU Coding: PyTorch, torch.compile(), fused kernels, CUDA, Triton
6. MLPs/FFNs: Multi-input, Multi-Layer Perceptrons, Feed-Forward Networks
7. Loss Functions: Residual errors, RMSE, Cross Entropy, Loss Landscapes
8. Backpropagation: Training loops, Optimizers, Learning Rate, Batch Size
9. Saving & Loading Models
10. Initialization: Kaiming, Glorot
11. Residuals: Addition, Scaling, Gated, Concatenation
12. Normalization: Pre-norm vs. Post-norm, RMSNorm, BatchNorm, LayerNorm
13. Regularization: Dropout, Gradient Clipping, Weight Decay
14. SoftMax
15. Tokenizers: By Character, By Word, BPE, SentencePiece
16. Embeddings: Absolute vs. Learned, Sinusoidal vs. RoPE
17. Attention: MHA, GQA, MQA, MLA
18. Transformers
19. Pre-training: Data Sources, Datasets, HTML Cleaning, Quality Filtering, Sharding
20. Evaluation: Leaderboards, Benchmarks, Verifiers vs LLM-as-Judge
21. Instruction Tuning: Alpaca & Other Formats, Self Instruct, Capabilities
22. Reinforcement Learning: Policy Optimization, SimPO
23. What We Didn’t Cover: Scaling

Each section has slides teaching the concepts, followed by excel-by-hand developing intuition for the math, and then coding examples. The goal is able to grok all parts of modern LLM development.

We did this workshop in-person in San Francisco last month and hopefully the spaciousness of watching online works for everyone. If don’t like watching videos, you can get the slides and exercises and work self-paced.