Anyone else looking back at energy-based models for continuous reasoning? [D]

been re-reading some literature on search and planning lately, and it’s getting harder to ignore how brute-forcing next-token prediction is kind of hitting a wall when it comes to strict logic.

we keep throwing millions of dollars of compute at scaling transformers, and yeah they get marginally better at standard benchmarks, but the underlying mechanism is still just a massive probability distribution over a discrete vocabulary. when you need absolute mathematical certainty like for formal code verification or critical systems – it really feels like we are trying to force a probabilistic peg into a deterministic hole. you cant just prompt-engineer your way out of the fundamental architecture

I remember LeCun talking a while back about operating in continuous mathematical spaces and satisfying constraints instead of generating tokens. it seemed super abstract at the time, but you actually see it moving into applied architecture now. like I was looking at how Logical Intelligence is structuring their models around energy-based architectures specifically to bypass the hallucination problem. basically finding a low-energy state that satisfies logical constraints rather than guessing the next syntax element.

it makes me wonder if the whole “System 2” thinking everyone is chasing right now won’t actually come from scaling LLMs, but from hooking up an LLM interface to a dedicated EBM solver under the hood.

Curious what the folks working on architecture here think about the viability of EBMs for this kind of reasoning. last I checked training stability for EBMs was a massive headache in practice, but maybe the tooling and compute approaches have caught up?