digitado

Hi! Recently, I have tried to implemented a number of RL algorithms such as PPO for Mujoco and reduced versions of DQN for Pong and MuZero (only for CartPole…) and I wanted to share some impressions from debugging these implementations. Many points have already been written up in other posts (see some links below), so I’ll focus on what I found most important. Approach I found it best to implement the related simpler version of your algorithm first […]

Ver mais

Machine unlearning enables models to selectively forget training data, driven by privacy regulations such as GDPR and CCPA. However, its fairness implications remain underexplored: when a model forgets a demographic group, does it neutralize that concept or redistribute it to correlated groups, potentially amplifying bias? We investigate this bias redistribution phenomenon on CelebA using CLIP models (ViT/B-32, ViT-L/14, ViT-B/16) under a zero-shot classification setting across intersectional groups defined by age and gender. We evaluate three unlearning methods, Prompt […]

Ver mais

Federated Learning enables collaborative model training without centralising data, but its effectiveness varies with the selection of the aggregation strategy. This choice is non-trivial, as performance varies widely across datasets, heterogeneity levels, and compute constraints. We present an end-to-end framework that automates, streamlines, and adapts aggregation strategy selection for federated learning. The framework operates in two modes: a single-trial mode, where large language models infer suitable strategies from user-provided or automatically detected data characteristics, and a multi-trial mode, […]

Ver mais

Most discussions around AI in crypto focus on agents interacting with wallets or apps. That’s not the only interesting part. Blockchains are more than execution environments. They are competitive systems where participants bid for inclusion, ordering, and ultimately value. As agents become more capable, they won’t just participate in these systems. They will optimize within them. This starts with a simple constraint: blockspace. Blockspace is the limited room available in each block for transactions. Every block has constraints […]

Ver mais

Introduction Accounts Payable (AP) teams in many organizations still rely on manual data entry to process supplier invoices. This approach does not scale well in high-volume environments and introduces risks related to data accuracy, processing delays, and compliance. During multiple ERP implementations, I observed that Accounts Payable teams often rely on manual entry of invoice data from PDFs into the system. This inefficiency highlighted an opportunity to design an AI-driven solution to automate invoice processing. The approach presented […]

Ver mais

This paper addresses the problem of training a reinforcement learning (RL) policy under partial observability by exploiting a privileged, anytime-feasible planner agent available exclusively during training. We formalize this as a Partially Observable Markov Decision Process (POMDP) in which a planner agent with access to an approximate dynamical model and privileged state information guides a learning agent that observes only a lossy projection of the true state. To realize this framework, we introduce an anytime-feasible Model Predictive Control […]

Ver mais

Axios was compromised in a supply chain attack that injected malware into widely used versions, exposing developers and CI pipelines. The incident highlights growing risks in JavaScript dependencies. axios-fixed offers a secure, zero-dependency drop-in replacement built on native fetch, allowing teams to migrate in minutes without rewriting code while reducing attack surface and restoring trust.

Ver mais

Reinforcement learning is typically treated as a uniform, data-driven optimization process, where updates are guided by rewards and temporal-difference errors without explicitly exploiting global structure. In contrast, dynamic programming methods rely on structured information propagation, enabling efficient and stable learning. In this paper, we provide evidence that such structure can be recovered from the learning dynamics of distributional reinforcement learning. By analyzing the temporal evolution of return distributions, we identify signals that capture when and where learning occurs […]

Ver mais

Machine learning competitions (MLCs) play a pivotal role in advancing artificial intelligence (AI) by fostering innovation, skill development, and practical problem-solving. This study provides a comprehensive analysis of major competition platforms such as Kaggle and Zindi, examining their workflows, evaluation methodologies, and reward structures. It further assesses competition quality, participant expertise, and global reach, with particular attention to demographic trends among top-performing competitors. By exploring the motivations of competition hosts, this paper underscores the significant role of MLCs […]

Ver mais