I’m working on an RL project using two algorithms: SAC and PPO. The project consists of a robot made up of 3 arms (controlled by 3 servos) attached to a plate on which a ball will roll. Through infrared sensors placed under the plate, I can detect the ball’s position (observations) and move the plate (pitch and roll actions) to bring the ball to the center and stabilize it. The reward is defined based on the distance from […]
If you walk across the open yard in front of the Physics, Math, and Astronomy building at the University of Texas at Austin, you’ll see a 17-story tower and a huge L-shaped building. What you won’t see is what’s underneath you. Two floors below ground, behind heavy double doors stamped with a logo that most students have never noticed, sits one of the most powerful lasers in the United States. I was the lead laser scientist on the […]
I am trying to convert XQuery statements into SQL queries within an enterprise context, with the constraint that the solution must rely on locally run LLMs. A key challenge is the limited availability of training data (pairs of XQueries and their corresponding SQL queries), especially with enough diversity to cover different patterns. I initially experimented with a parsing-based approach. The idea was to extract elements such as table names, columns, and conditions from the XQuery (using a […]
Read Online | Sign Up | Advertise Good morning, {{ first_name | AI enthusiasts }}. Most AI design tools give you a result and stop there. You get a finished image, and then you’re on your own, prompting in circles to get it closer to what you actually meant. Canva is betting the real value isn’t in generation — it’s in what comes after. The company just launched Canva AI 2.0, turning its platform into an AI-native environment […]
I went through a bunch of older posts here about research vs dev roles, but most of them were either very general or not really in a similar situation, so posting this. I’m a final year ISE student from a tier-3 college. Over the past 1.5–2 years I’ve been focusing quite a bit on ML research instead of just the usual DSA + dev route. Current situation: 1 paper in TMLR (reviews done, waiting on decision) 1 in […]
Training Qwen2.5-0.5B-Instruct on Reddit post summarization with GRPO on my 3x Mac Minis — trying combination of quality rewards with length penalty! So, with this project I want to see if a length constrained (like 64 tokens only) quality summarization can be done by tiny LLMs using GRPO! Why combination of quality rewards? ROUGE-L only cares about the longest common subsequence — it misses synonyms and paraphrases entirely. METEOR handles both: it aligns tokens with synonym matching via […]
This paper investigates the length problem in sequence-level relative reinforcement learning. We observe that, although existing methods partially alleviate length-related phenomena, a more fundamental issue remains insufficiently characterized: the comparison units used during training lack inherent comparability. Building on this observation, we propose a new perspective: the length problem should not be viewed merely as a loss-scaling or normalization bias, but rather as a emph{comparison unit construction} problem. We further establish a sample-construction-based training framework that, instead of […]
From confused hallucinations to accurate answers in one week. The complete guide to building smarter AI — no code required AGENTIC RAG It was demo day. Fifty people in the room. My manager nodding at me from the back row. I typed the question live: “What’s our employee vacation policy?” The AI answered with three paragraphs about GDPR compliance, two sentences about industry salary benchmarks, and — buried in paragraph five — a vague mention that “time-off entitlements may vary.” The actual answer was fifteen days PTO. Seven […]
Hubo un tiempo en que comprar un coche eléctrico podía considerarse una apuesta, una declaración de intenciones o incluso una extravagancia tecnológica. Ese tiempo ha pasado. En 2025 se vendieron más de 20 millones de coches eléctricos en el mundo, más del 25% de todas las ventas de automóviles nuevos, una cifra que ya no describe una rareza, sino una transición industrial en toda regla. Quien siga hablando del vehículo eléctrico como si fuese una curiosidad está discutiendo […]
Previous LLMs-based RL studies typically follow either supervised learning with high annotation costs, or unsupervised paradigms using voting or entropy-based rewards. However, their performance remains far from satisfactory due to the substantial annotation cost and issues such as model collapse or reward hacking. To address these issues, we introduce a new perspective inspired by cognitive learning theory and propose a novel approach called EasyRL. The core of EasyRL is to simulate the human cognitive acquisition curve by integrating […]
