By Michael Nielsen, September 26 2019 Note: Hastily written, a few observations based on writing I happened to find of interest The opening of Chapter II of Carl Sagan’s “Cosmos”: All my life I have wondered about the possibility of life elsewhere. What would it be like? Of what would it be made? … There was once a time before life, when the Earth was barren and utterly desolate. Our world is now overflowing with life. How did […]
This guide would not have been possible without the help and feedback from many people. Special thanks to Prof. Joan Bruna and his class at NYU, Mathematics of Deep Learning, and to Cinjon Resnick, who introduced me to DFL and helped complete this guide. Thank you to Avital Oliver, Matt Johnson, Dougal MacClaurin, David Duvenaud, and Ricky Chen for useful contributions to this guide. Thank you to Tinghao Li, Chandra Prakash Konkimalla, Manikanta Srikar Yellapragada, Shan-Conrad Wolf, Deshana […]
I got interested in ASP.NET Core 3.0 since the first preview and followed it very closely. I started using it since preview2 and will now soon go to production with it (preview9). I compiled this list to have all the new features, improvements and breaking changes that happened during this time for easy access and accessibility.
Stochastic gradient descent is a universal choice for optimizing deep learning models. However, it is not the only option. With black-box optimization algorithms, you can evaluate a target function $f(x): mathbb{R}^n to mathbb{R}$, even when you don’t know the precise analytic form of $f(x)$ and thus cannot compute gradients or the Hessian matrix. Examples of black-box optimization methods include Simulated Annealing, Hill Climbing and Nelder-Mead method.