digitado

Safe Reinforcement Learning from Human Feedback (Safe RLHF) has recently achieved empirical success in developing helpful and harmless large language models by decoupling human preferences regarding helpfulness and harmlessness. Existing approaches typically rely on fitting fixed horizon reward models from human feedback and have only been validated empirically. In this paper, we formulate safe RLHF as an infinite horizon discounted Con- strained Markov Decision Process (CMDP), since humans may interact with the model over a continuing sequence of […]

Ver mais

Diffusion distillation, exemplified by Distribution Matching Distillation (DMD), has shown great promise in few-step generation but often sacrifices quality for sampling speed. While integrating Reinforcement Learning (RL) into distillation offers potential, a naive fusion of these two objectives relies on suboptimal raw sample evaluation. This sample-based scoring creates inherent conflicts with the distillation trajectory and produces unreliable rewards due to the noisy nature of early-stage generation. To overcome these limitations, we propose GDMD, a novel framework that redefines […]

Ver mais

Here are two beliefs that are sort of haunting me right now: Folk who try to push people to uphold principles (whether established ones or novel ones), are kinda an important bedrock of civilization. Also, those people are really annoying and often, like, a little bit crazy And these both feel fairly important. I’ve learned a lot from people who have some kind of hobbyhorse about how society is treating something as okay/fine, when it’s not okay/fine. When […]

Ver mais

I keep coming back to this idea that reinforcement learning isn’t really being used the way people think it is. A lot of systems right now feel like they’re really good at pattern recognition but not actually getting better from outcomes. They can predict what should work, but they don’t really remember whether something did work in a good way that changes future decisions. Does anyone have thoughts on this? submitted by /u/TaleAccurate793 [link] [comments]

Ver mais

Scaling critic capacity is a promising direction for enhancing off-policy reinforcement learning (RL). However, larger critics are prone to overfitting and unstable in replay-buffer-based bootstrap training. This paper leverages Low-Rank Adaptation (LoRA) as a structural-sparsity regularizer for off-policy critics. Our approach freezes randomly initialized base matrices and solely optimizes low-rank adapters, thereby constraining critic updates to a low-dimensional subspace. Built on top of SimbaV2, we further develop a LoRA formulation, compatible with SimbaV2, that preserves its hyperspherical normalization […]

Ver mais

Ver mais

For years, the way large language models handle inference has been stuck inside a box — literally. The high-bandwidth RDMA networks that make modern LLM serving work have confined both prefill and decode to the same datacenter, sometimes even the same rack. A team of researchers at Moonshot AI and Tsinghua University is making the case that this constraint is about to break down — and that the right architecture can already exploit that shift. The research team […]

Ver mais

Cybersecurity has always had a dual-use problem: the same technical knowledge that helps defenders find vulnerabilities can also help attackers exploit them. For AI systems, that tension is sharper than ever. Restrictions intended to prevent harm have historically created friction for good-faith security work, and it can be genuinely difficult to tell whether any particular cyber action is intended for defensive usage or to cause harm. OpenAI is now proposing a concrete structural solution to that problem: verified […]

Ver mais

  Featuring the roots of various polynomials whose coefficients are the binary digits of numbers like π. Complex roots cluster around the circle, and the few real ones are on the horizontal X-axis (positive on the right, negative on the left). Download the free paper to see how to interpret the colors and get the code and full explanations, with state-of-the-art material on the topic. This research is part of a project at BondingAI to build an AI […]

Ver mais