[D] Are we prematurely abandoning Bio-inspired AI? The gap between Neuroscience and DNN Architecture.

We often hear that “neurons” in DNNs are just a loose analogy for biological neurons. The consensus seems to be that while abstract ideas (like hierarchies) match, the actual architectures are fundamentally different, largely because biological mechanisms are seen as either computationally expensive or incompatible with current silicon hardware.

However, as I’ve recently begun bridging the gap between my PhD in applied math and a BS in Neuroscience, I’ve started to question if we are moving away from biological concepts too soon for two main reasons:

1. Under-utilization of Bio-concepts: When we do successfully port a biological observation—like ReLU activation functions mimicking the “all-or-nothing” firing of human neurons—the performance gains are massive. We are likely leaving similar optimizations on the table.
2. The “Saturation” Fallacy: Many in ML treat the brain as a “solved” or “static” inspiration source. In reality, neuroscience is nowhere near a saturation point. We don’t actually understand the brain well enough yet to say what is or is not useful for AI.

Are we optimizing for what works on semiconductors rather than searching for better fundamental architectures? I’d love to hear from folks working in Neuromorphic computing or those who believe the “Black Box” of the brain is no longer a useful map for AI development.