digitado

Following up on some thoughts around RLHF and LLM training. Most current RLHF pipelines can be framed as optimizing a policy πₜ (the LLM) against a learned reward model r_φ that approximates human preference distributions over outputs. In practice, this is often implemented with PPO-style updates under KL constraints relative to a reference policy. This setup works well for alignment and helpfulness, but it has a few structural properties that seem limiting: 1. Static reward modeling The reward […]

Ver mais

Hi, I have been studying RL for the last 3 months. I wanted to create a project on a recommendation system. I have been a little lost on this path and wanted to ask for suggestions on the following: 1) Any basic research papers that I should read that can describe the process and problems faced? 2) Any beginning structure that you would recommend? 3) Any thoughts on problems like cold starts? 4) Anything else that you would […]

Ver mais

Test-Time Learning (TTL) enables language agents to iteratively refine their performance through repeated interactions with the environment at inference time. At the core of TTL is an adaptation policy that updates the actor policy based on experience from previous episodes, thereby improving future behavior. Existing methods rely on fixed, hand-crafted adaptation policies rather than optimizing them for downstream improvement. We argue that optimal adaptation policies should be learned from task environments, not hand-engineered based on human intuition. To […]

Ver mais

Overlap, also known as positivity, is a key condition for causal treatment effect estimation. Many popular estimators suffer from high variance and become brittle when features differ strongly across treatment groups. This is especially challenging in high dimensions: the curse of dimensionality can make overlap implausible. To address this, we propose a class of feature representations called deconfounding scores, which preserve both identification and the target of estimation; the classical propensity and prognostic scores are two special cases. […]

Ver mais

Plant phenology modelling aims to predict the timing of seasonal phases, such as leaf-out or flowering, from meteorological time series. Reliable predictions are crucial for anticipating ecosystem responses to climate change. While phenology modelling has traditionally relied on mechanistic approaches, deep learning methods have recently been proposed as flexible, data-driven alternatives with often superior performance. However, mechanistic models tend to outperform deep networks when data distribution shifts are induced by climate change. Domain Adaptation (DA) techniques could help […]

Ver mais

In this quiz, you’ll test your understanding of Exploring Keywords in Python. By working through this quiz, you’ll revisit how to identify Python keywords, understand the difference between regular and soft keywords, categorize keywords by purpose, and avoid common pitfalls with deprecated keywords. [ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Ver mais