Federated learning (FL) enables collaborative model training across distributed clients without sharing raw data, yet its stability is fundamentally challenged by statistical heterogeneity in realistic deployments. Here, we show that client heterogeneity destabilizes FL primarily by distorting local gradient dynamics during client-side optimization, causing systematic drift that accumulates across communication rounds and impedes global convergence. This observation highlights local gradients as a key regulatory lever for stabilizing heterogeneous FL systems. Building on this insight, we develop a general […]

Ver mais

arXiv:2602.00866v1 Announce Type: new Abstract: The Controller Area Network (CAN) bus provides a rich source of vehicular signals increasingly leveraged for applications in automotive and auto insurance domains, including collision detection, predictive maintenance, and driver risk modeling. Despite this potential, existing pipelines largely train isolated task-specific models on raw CAN data, with only limited efforts exploring decoded signals. Such fragmentation prevents shared representation learning and limits cross-task generalization. By contrast, natural language processing (NLP) and computer vision (CV) […]

Ver mais

A few weeks ago I posted here asking for advice about a Dueling DDQN chess agent that completely collapsed after I pretrained it with supervised learning. Several people pointed out that the issue might be the transition from supervised learning to value-based RL, and that actor-critic methods might be a better fit. They were right. I had been treating the Q-values as logits. Using cross-entropy loss during supervised learning meant that the “correct” Q-value (the expert move) was […]

Ver mais

arXiv:2601.00172v1 Announce Type: new Abstract: Forecasting high-dimensional spatiotemporal systems remains computationally challenging for recurrent neural networks (RNNs) and long short-term memory (LSTM) models due to gradient-based training and memory bottlenecks. Reservoir Computing (RC) mitigates these challenges by replacing backpropagation with fixed recurrent layers and a convex readout optimization, yet conventional RC architectures still scale poorly with input dimensionality. We introduce a Sequential Reservoir Computing (Sequential RC) architecture that decomposes a large reservoir into a series of smaller, interconnected […]

Ver mais

We built a synthetic evaluation framework (LOLAMEME) to systematically compare Transformer (GPT-2), convolution-based (Hyena), and hybrid architectures on tasks requiring logic, memory, and language understanding. The gap we address: Most mechanistic interpretability work uses toy tasks that don’t capture real-world complexity like variable naming conventions, persistent memory (global variables), latent type systems, or mixed-language syntax. What we did: Created two configurable programming languages (LoLa and MeMe) with different syntax (camelCase vs snake_case, different operators) Built a hybrid architecture […]

Ver mais

Mixture-of-experts variants of parameter-efficient fine-tuning enable per-token specialization, but they introduce additional trainable routers and expert parameters, increasing memory usage and training cost. This undermines the core goal of parameter-efficient fine-tuning. We propose Monkey Jump, a method that brings mixture-of-experts-style specialization to parameter-efficient fine-tuning without introducing extra trainable parameters for experts or routers. Instead of adding new adapters as experts, Monkey Jump treats the adapters already present in each Transformer block (such as query, key, value, up, and […]

Ver mais

arXiv:2602.19691v2 Announce Type: replace Abstract: Smooth activation functions are ubiquitous in modern deep learning, yet their theoretical advantages over non-smooth counterparts remain poorly understood. In this work, we study both approximation and statistical properties of neural networks with smooth activations for learning functions in the Sobolev space $W^{s,infty}([0,1]^d)$ with $s>0$. We prove that constant-depth networks equipped with smooth activations achieve smoothness adaptivity: increasing width alone suffices to attain the minimax-optimal approximation and estimation error rates (up to logarithmic […]

Ver mais

Modern social networks incorporate AI technologies throughout their platforms because they use it for content recommendations, ad targeting, content moderation, and analytics. Artificial intelligence functions as a secondary element in these platforms because it does not serve as the main component for operation. Content creation serves as the main method through which people establish social connections and make digital content interact with their audience. The Moltbook AI social network introduces an entirely different system to its users. The […]

Ver mais