[D] The engineering overhead of Verifiable ML: Why GKR + Hyrax for on-device ZK-ML?

The idea of ​​”Privacy-Preserving AI” usually stops at local inference. You run a model on a phone, and the data stays there. But things get complicated when you need to prove to a third party that the output was actually generated by a specific, untampered model without revealing the input data.

I’ve been looking into the recently open-sourced Remainder prover (the system Tools for Humanity uses for World). From an ML engineering perspective, the choice of a GKR (Goldwasser-Kalai-Rothblum) + Hyrax-based proof system is an interesting case study in balancing prover time vs. mobile hardware constraints.

Most ZK-ML implementations (like those using Plonky2 or Halo2) struggle with the sheer scale of circuit depth when you start mapping even mid-sized neural networks. GKR is theoretically “doubly-efficient”, but implementation-wise, it’s a nightmare to make it work on consumer-grade mobile GPUs.

The hardware-heavy approach (relating on physical Orb sensors for every state update) was always the biggest scaling bottleneck. Shifting the compute to client-side ZK-SNARKs means the “trust” moves from the hardware’s physical security to the mathematical integrity of the prover.

We often talk about Edge AI in terms of latency, but we rarely talk about verifiability. If we want a future where “Proof of Personhood” or “Proof of Model” is decentralized, we need provers that don’t melt a smartphone battery. Seeing a production-grade GKR prover that handles ML layers locally is a solid benchmark for the field, regardless of how you feel about the project itself.

I’m curious if we’re reaching a point where the prover overhead is finally low enough for real-time applications, or if we’re still just scratching the surface of what mobile GPUs can handle in terms of ZK-proof generation.