[P] How a Deep Learning Library Enables a Model to Learn

A lot of us know that a model is “learning” when the loss goes down, and that the loss is computed from the prediction and the target. The less obvious part is what a deep learning library is actually doing internally to turn that loss into parameter updates that improve the model. I wrote a short post [0] breaking that down: how the forward pass builds a computation graph, how loss.backward() applies the chain rule across it, and how the resulting gradients become parameter updates via optimizer.step(). I used a from-scratch numpy library I built [1] as a concrete reference point, but the main goal is to build intuition for what happens under the hood.

[0]: https://www.henrypan.com/blog/2026-03-14-how-deep-learning-library-enables-learning/
[1]: https://github.com/workofart/ml-by-hand