The Key to AI Intelligence: Why Transformer Width Matters More Than Depth

The Key to AI Intelligence: Why Transformer Width Matters More Than Depth

⋅

“Sweet are the uses of adversity, Which, like the toad, ugly and venomous, Wears yet a precious jewel in his head” — Shakespeare, As You Like It (Act 2, Scene 1) Hyper-dimensional geometry is at the very heart of the transformer architecture. It is forbidding. It’s counterintuitive, odd, even hostile to intuition. It looks nothing like the 3D world we live in. Step into this alien world and you will find not comfort but a complete disorientation: nearly everything is the […]

Ver mais

Liked Liked

FedNSAM:Consistency of Local and Global Flatness for Federated Learning

⋅

In federated learning (FL), multi-step local updates and data heterogeneity usually lead to sharper global minima, which degrades the performance of the global model. Popular FL algorithms integrate sharpness-aware minimization (SAM) into local training to address this issue. However, in the high data heterogeneity setting, the flatness in local training does not imply the flatness of the global model. Therefore, minimizing the sharpness of the local loss surfaces on the client data does not enable the effectiveness of […]

Ver mais

Liked Liked
Hi Reddit, I posted my Build Your Own LLM workshop to Youtube teaching ML, LLM and math intuition [P]

Hi Reddit, I posted my Build Your Own LLM workshop to Youtube teaching ML, LLM and math intuition [P]

⋅

Hi internet friends, I recorded a workshop about building your own LLM without any math / ML prerequisites. It covers everything from machine learning fundamentals, deep neural networks, transformer architecture, and pre/post-training. The only prerequisite is being comfortable with learning through code & excel examples. Sampling Large Language Models Reverse Engineering Large Language Model Perceptrons: wx+b Activation Functions: ReLU, GELU, SwiGLU GPU Coding: PyTorch, torch.compile(), fused kernels, CUDA, Triton MLPs/FFNs: Multi-input, Multi-Layer Perceptrons, Feed-Forward Networks Loss Functions: Residual […]

Ver mais

Liked Liked

Diffusion models with physics-guided inference for solving partial differential equations

⋅

arXiv:2604.01242v1 Announce Type: new Abstract: Diffusion models have recently emerged as powerful stochastic frameworks for high-dimensional inference and generation. However, existing applications to partial differential equations (PDEs) predominantly rely on physics-informed training strategies, which tightly couple learning with specific governing equations and limit generalization across problem settings. In this work, we propose a diffusion model with physics-guided inference for solving PDEs, in which the diffusion model is trained using standard data-driven procedures, while physical laws are incorporated exclusively […]

Ver mais

Liked Liked

Rethinking Reinforcement fine-tuning of LLMs: A Multi-armed Bandit Learning Perspective

⋅

A large number of heuristics have been proposed to optimize the reinforcement fine-tuning of LLMs. However, inconsistent claims are made from time to time, making this area elusive. Reflecting on this situation, two fundamental questions still lack a clear understanding: 1) what is the role of each optimizing choice? 2) which ones are the bottlenecks? This paper aims to shed light on them, and it faces the challenge of several entangled confounding factors in the fine-tuning process. To […]

Ver mais

Liked Liked
Meta wants to block data about social media use, mental health in child safety trial

Meta wants to block data about social media use, mental health in child safety trial

⋅

As Meta heads to trial in the state of New Mexico for allegedly failing to protect minors from sexual exploitation, the company is making an aggressive push to have certain information excluded from the court proceedings. The company has petitioned the judge to exclude certain research studies and articles around social media and youth mental health; any mention of a recent high-profile case involving teen suicide and social media content; and any references to Meta’s financial resources, the […]

Ver mais

Liked Liked

Ground truth is a process, not a dataset

⋅

Today, the key challenge in AI isn’t only how to build better models; it’s how to build evaluation systems that can keep up. Search-augmented AI systems can now produce deep research reports — long, polished syntheses of many sources that increasingly resemble expert analysis. But those reports are useful only if their claims are supported by the underlying literature. Most existing fact-checking tools work best when a claim can be matched to a short quote or a single […]

Ver mais

Liked Liked
The AI Readiness Myth: Why One Size Fits No One

The AI Readiness Myth: Why One Size Fits No One

⋅

“The test of a first-rate intelligence is the ability to hold two opposed ideas in the mind at the same time, and still retain the ability to function.” — The Crack-Up by F. Scott Fitzgerald The debate on AI feasibility, adoption rates, and baseline capabilities has effectively concluded. AI is now a structural force reshaping both enterprise operations and individual workflows. The central strategic question has shifted: will organizations default to the Tyranny of the OR — forcing a binary choice between […]

Ver mais

Liked Liked

KAPLAN: Kolmogorov-Arnold Prognostic Learnable Activation Networks for Survival Analysis

⋅

arXiv:2605.23082v1 Announce Type: new Abstract: Survival analysis aims to model how covariates and time jointly shape the time-to-event distribution under right censoring. Classical methods such as the Cox model and generalised additive models (GAMs) require interactions and time-varying effects to be manually specified, which is increasingly impractical on rich clinical datasets. We introduce KAPLAN-HR, a B-spline Kolmogorov-Arnold Network (KAN) for nonparametric estimation of the conditional hazard as a joint function of covariates and time. A single-layer KAPLAN-HR model […]

Ver mais

Liked Liked

Toward a Uniform Algorithm and Uniform Reduction for Constraint Problems

⋅

arXiv:2604.06335v1 Announce Type: new Abstract: We develop a unified framework to characterize the power of higher-level algorithms for the constraint satisfaction problem (CSP), such as $k$-consistency, the Sherali-Adams LP hierarchy, and the affine IP hierarchy. As a result, solvability of a fixed-template CSP or, more generally, a Promise CSP by a given level is shown to depend only on the polymorphism minion of the template. Similarly, we obtain a minion-theoretic description of $k$-consistency reductions between Promise CSPs. We […]

Ver mais

Liked Liked

Load More

↑

Digitado © 2026