I spent 15 months building RL3 a no-code reinforcement learning playground

After ~15 months of development, my final year project is finally finished.

RL3 is a no-code reinforcement learning playground that runs entirely in the browser for authoring environments and supports server-side deep RL training.

The goal was simple: make reinforcement learning easier to learn without requiring users to write code or set up complicated environments.

The workflow is:

– Design an environment using a drag-and-drop editor.

– Create the reward function visually using a behavior graph.

– Assign graphs to one or more agents.

– Select the training algorithm and start training.

For browser-based training, RL3 supports tabular Q-learning (using state discretization to keep the Q-table manageable).

For deep RL, the authored environment is recreated on the server in PyBullet, where PPO training is available. I’m currently working toward MAPPO support. Multiple-agent inference already works today through shared policies.

Some features include:

– Save and resume training checkpoints.

– Modify environments or reward graphs and continue training (curriculum learning).

– Share environments and trained models with friends.

– Reuse someone else’s environment or policy as a starting point.

– Multiple built-in behaviors such as:

– Navigation

– Object collection

– Holding items

– Depositing objects

– Destroying obstacles

– Opening gates with collected keys

– Obstacle avoidance

– Football behaviors

One thing I’m particularly excited about is the football environment. My long-term vision is to let people publish their trained agents and challenge others to either:

– play against them manually, or

– train their own agents to compete.

Eventually I’d like to expand this to other game-like environments (basketball, hide-and-seek, maybe even cricket).

Internally, RL3 uses a behavior/state machine so that only the observations relevant to the current skill are exposed during training. For example, an agent learning to collect objects isn’t distracted by observations needed for later tasks like opening gates. This makes learned skills much more reusable when building longer behavior chains.

The project was developed over the past 15 months, mostly by myself, as my final year project. One thing that motivated me was realizing how inaccessible reinforcement learning still feels compared to other areas of AI. Here in Pakistan, RL isn’t commonly taught, and most AI discussions revolve around LLMs. I wanted to build something that lowers the barrier to entry and hopefully encourages more people to experiment with RL.

I’ll include the deployment link below. The public deployment currently supports Q-learning. PPO requires GPU-backed training pods, which I can’t afford to keep running continuously, but if anyone wants to try PPO, feel free to DM me and I’ll spin up the training service.

YouTube: https://youtu.be/V5d99hOM5ew?si=JhhZLP3hO-pb69rr

Application Link: https://rl-playground-beta.vercel.app/signing-in

I’d genuinely appreciate feedback from the RL community especially on the overall idea, architecture, and where you think a platform like this could be improved.