February 2026

Grokking — the abrupt transition from memorization to generalization after extended training — has been linked to the emergence of low-dimensional structure in learning dynamics. Yet neural network parameters inhabit extremely high-dimensional spaces. How can a low-dimensional learning process produce solutions that resist low-dimensional compression? We investigate this question in multi-task modular arithmetic, training shared-trunk Transformers with separate heads for addition, multiplication, and a quadratic operation modulo 97. Across three model scales (315K–2.2M parameters) and five weight decay […]

Ver mais

With fault-tolerant quantum computing on the horizon, there is growing interest in applying quantum computational methods to data-intensive scientific fields like remote sensing. Quantum machine learning (QML) has already demonstrated potential for such demanding tasks. One area of particular focus is quantum data fusion — a complex data analysis problem that has attracted significant recent attention. In this work, we introduce an automated QML (AQML) approach for addressing data fusion challenges. We evaluate how AQML-generated quantum circuits perform […]

Ver mais

World models learned from high-dimensional visual observations allow agents to make decisions and plan directly in latent space, avoiding pixel-level reconstruction. However, recent latent predictive architectures (JEPAs), including the DINO world model (DINO-WM), display a degradation in test time robustness due to their sensitivity to "slow features". These include visual variations such as background changes and distractors that are irrelevant to the task being solved. We address this limitation by augmenting the predictive objective with a bisimulation encoder […]

Ver mais

Taalas serves Llama 3.1 8B at 17,000 tokens/second This new Canadian hardware startup just announced their first product – a custom hardware implementation of the Llama 3.1 8B model (from July 2024) that can run at a staggering 17,000 tokens/second. I was going to include a video of their demo but it’s so fast it would look more like a screenshot. You can try it out at chatjimmy.ai. They describe their Silicon Llama as “aggressively quantized, combining 3-bit […]

Ver mais

A Digital Twin (DT) may protect information that is considered private to its associated physical system. For a mobile device, this may include its mobility profile, recent location(s), and experienced channel conditions. Online schedulers, however, typically use this type of information to perform tasks such as shared bandwidth and channel time slot assignments. In this paper, we consider three transmission scheduling problems with energy constraints, where such information is needed, and yet must remain private: minimizing total transmission […]

Ver mais