April 2021

Few days ago we had a talk by Gergely Neu, who presented his recent work: Gergely Neu Information-Theoretic Generalization Bounds for Stochastic Gradient Descent I’m writing this post mostly to annoy him, by presenting this work using super hand-wavy intuitions and cartoon figures. If this isn’t enough, I will even find a way to mention GANs in this context. But truthfully, I’m just excited because for once, there is a little bit of learning theory that I half-understand, […]

Ver mais

Recently I have been interested in obtaining a very particular skill. I have seen this ability demonstrated by many excellent individuals – not in only in the tech world, but also in finance or consulting industry. I am talking here about the art of back-of-the-napkin (or back-of-the-envelope) calculation, also known as guesstimation or order-of-magnitude analysis. In your professional and private life you may be often presented with difficult question, where the insight is much more important than the […]

Ver mais

I wanted to highlight an intriguing paper I presented at a journal club recently: Samuel L Smith, Benoit Dherin, David Barrett, Soham De (2021) On the Origin of Implicit Regularization in Stochastic Gradient Descent There’s actually a related paper that came out simultaneously, studying full-batch gradient descent instead of SGD: David G.T. Barrett, Benoit Dherin (2021) Implicit Gradient Regularization One of the most important insights in machine learning over the past few years relates to the importance of […]

Ver mais