[D] Why are serious alternatives to gradient descent not being explored more?

It feels like there’s currently a massive elephant in the room when it comes to ML, and it’s specifically around the idea that gradient descent might be a dead end in terms of a method that gets us anywhere near solving continual learning, casual learning, and beyond.

Almost every researcher, whether postdoc, or PhD I’ve talked to feels like current methods are flawed and that the field is missing some stroke of creative genius. I’ve been told multiple times that people are of the opinion that “we need to build the architecture for DL from the ground up, without grad descent / backprop” – yet it seems like public discourse and papers being authored are almost all trying to game benchmarks or brute force existing model architecture to do slightly better by feeding it even more data.

This causes me to beg the question – why are we not exploring more fundamentally different methods for learning that don’t involve backprop given it seems that consensus is that the method likely doesn’t support continual learning properly? Am I misunderstanding and or drinking the anti-BP koolaid?