RL Environments for Language Models: free hands-on course

🌱 Course: https://github.com/anakin87/llm-rl-environments-lil-course
🎥 Video: https://www.youtube.com/watch?v=71V3fTaUp2Q

I’ve been deep into RL for LLM post-training lately, especially the shift from Supervised Fine-Tuning to Reinforcement Learning with Verifiable Rewards.

Previously, most of the focus was on SFT: learning from curated QA pairs.
Now, with approaches like GRPO, we can treat generation as an RL problem where models improve via trial and error in programmatically defined environments.

But what actually are these environments in practice? And how do you build them effectively?

Fascinated by these concepts, I spent time exploring this space through experiments, post-training Small Language Models.
I’ve packaged everything I learned into this short course.

What you’ll learn

🧩 Mapping RL concepts (agents, environments) to LLMs
🔧 How to use Verifiers (open-source library) to build RL environments as software artifacts
🔁 Common patterns: single-turn, multi-turn, and tool-use environments

🎮 Hands-on: turn a small language model (LFM2-2.6B by LiquidAI) into a Tic Tac Toe master that beats gpt-5-mini

• Build the game Environment
• Use it to generate synthetic data for SFT warm-up
• Group-based Reinforcement Learning

If you’re interested in building “little worlds” where LLMs can learn, this course is for you.

—

🕹️ Play against the trained model: https://huggingface.co/spaces/anakin87/LFM2-2.6B-mr-tictactoe