July 2021

Reflections on ‘The Bitter Lesson’

⋅

By Michael Nielsen Attention conservation notice: Rough notes, very rapidly written, by an interested non-expert outsider. Rich Sutton is an expert on neural networks at the University of Alberta and DeepMind. He’s written a stimulating essay describing what he calls “the bitter lesson”: in AI research it’s extremely seductive to try to build expert domain knowledge into the systems you’re creating, but, according to Sutton, this approach gets beaten again and again by methods leveraging brute force computation, […]

Ver mais

Liked Liked

ICML 2021 Invited Speakers — ML for Science

⋅

By: Stefanie Jegelka and Ameet Talwalkar (ICML21 Communication Chairs) With ICML 2021 underway, we wanted to briefly highlight the upcoming invited talks. A general theme of the invited talks this year is “machine learning for science.” The Program Chairs (Marina Meila and Tong Zhang) have invited world-renowned scientists from various disciplines to discuss their problems and the corresponding machine learning challenges. By exposing the machine learning community to these fascinating problems, we hope that we can help to […]

Ver mais

Liked Liked

What are Diffusion Models?

⋅

[Updated on 2021-09-19: Highly recommend this blog post on score-based generative modeling by Yang Song (author of several key papers in the references)]. [Updated on 2022-08-27: Added classifier-free guidance, GLIDE, unCLIP and Imagen. [Updated on 2022-08-31: Added latent diffusion model. [Updated on 2024-04-13: Added progressive distillation, consistency models, and the Model Architecture section.

Ver mais

Liked Liked
Implicit Regularization in Tensor Factorization: Can Tensor Rank Shed Light on Generalization in Deep Learning?

Implicit Regularization in Tensor Factorization: Can Tensor Rank Shed Light on Generalization in Deep Learning?

⋅

In effort to understand implicit regularization in deep learning, a lot of theoretical focus is being directed at matrix factorization, which can be seen as linear neural networks. This post is based on our recent paper (to appear at ICML 2021), where we take a step towards practical deep learning, by investigating tensor factorization — a model equivalent to a certain type of non-linear neural networks. It is well known that most tensor problems are NP-hard, and accordingly, […]

Ver mais

Liked Liked
When Curation Becomes Creation

When Curation Becomes Creation

⋅

Algorithms, Microcontent, and the Vanishing Distinction between Platforms and Creators Authors: Liu Leqi, Dylan Hadfield-Menell, and Zachary C. Lipton To appear in Communications of the ACM (CACM) and available on arXiv.org. Ever since social activity on the Internet began migrating from the wilds of the open web to the walled gardens erected by so-called platforms (think Myspace, Facebook, Twitter, YouTube, or TikTok), debates have raged about the responsibilities that these platforms ought to bear. And yet, despite intense scrutiny […]

Ver mais

Liked Liked

↑

Digitado © 2025