[D] Sim-to-real in robotics — what are the actual unsolved problems?

Been reading a lot of recent sim-to-real papers (LucidSim, Genesis, Isaac Lab stuff) and the results look impressive in demos, but I’m curious what the reality is for people actually working on this.

A few things I’m trying to understand:

  1. When a trained policy fails in the real world, is the root cause usually sim fidelity (physics not accurate enough), visual gap (rendering doesn’t match reality), or something else?
  2. Are current simulators good enough for most use cases, or is there a fundamental limitation that better hardware/software won’t fix?
  3. For those in industry — what would actually move the needle for your team? Faster sim? Better edge case generation? Easier real-to-sim reconstruction?

Trying to figure out if there’s a real research gap here or if the field is converging on solutions already. Would appreciate any takes, especially from people shipping actual robots.

submitted by /u/kourosh17
[link] [comments]

Liked Liked