Roadmap for learning RL for robot control beyond just using existing frameworks?

Hi everyone,

I’m looking for advice on how to properly learn Reinforcement Learning for robot control, not just at the “using a framework” level.

A bit of background: I have some basic academic knowledge in robotics and RL. Recently, I used MJLab to successfully train both locomotion and imitation/mimic motion policies for an existing robot model. The results worked, but honestly, most of the hard work was already implemented by MJLab: environment setup, reward structure, PPO training pipeline, robot interfaces, logging, etc.

So even though I managed to train something successfully, I still feel like I was mostly applying an existing pipeline rather than deeply understanding what is happening under the hood.

I’m not a complete beginner, but I’m also not yet confident enough to implement a full robot RL training pipeline myself.

Could you recommend a good learning roadmap for this?

I’m especially interested in:

• Coding-oriented RL courses
• Robotics-focused RL courses
• Papers that are important for robot locomotion / imitation learning
• Open-source codebases that are good for learning, not just using
• Practical projects I should implement step by step
• Any advice on how to move from “running existing frameworks” to actually understanding and modifying them confidently

For context, my goal is to work more seriously on RL-based robot control, especially locomotion and motion imitation for legged/humanoid robots.

Thanks a lot!