RL Air Hockey Agent Zero shot sim to real transfer

This is my undergraduate capstone project from Engineering Physics at UBC. We trained a policy in simulation and then directly deployed in onto our robotic air hockey table. The project involved designing a physical robotic air hockey table, computer vision system, reinforcement learning pipeline, simulation, embedded systems, and controls.

The agent was trained with SAC against itself and another defender agent. To get a successful sim to real transfer, instead of just throwing in lots of domain randomization, we ended up modelling everything in the system to put into the simulation. This involves all sensor noise or offsets, puck collision dynamics, communication delay between sensor data and action, controller accuracy, etc.

The agent isn’t directly outputting PWM signals to the motors, instead we hook up its output to a controller which gives an extra layer of safety (as the controller is transparent allowing for theoretical guarantees over its motion).

For more detail you can check out the github repo: HudsonNock/Air-Hockey-Sim