arXiv:2601.02530v1 Announce Type: new Abstract: We present Connection-Aware Motif Sequencing (CamS), a graph-to-sequence representation that enables decoder-only Transformers to learn molecular graphs via standard next-token prediction (NTP). For molecular property prediction, SMILES-based NTP scales well but lacks explicit topology, whereas graph-native masked modeling captures connectivity but risks disrupting the pivotal chemical details (e.g., activity cliffs). CamS bridges this gap by serializing molecular graphs into structure-rich causal sequences. CamS first mines data-driven connection-aware motifs. It then serializes motifs via […]

Ver mais

Scaling test-time compute via parallel sampling can substantially improve LLM reasoning, but is often limited by Best-of-N selection quality. Generative selection methods, such as GenSelect, address this bottleneck, yet strong selection performance remains largely limited to large models. We show that small reasoning models can acquire strong GenSelect capabilities through targeted reinforcement learning. To this end, we synthesize selection tasks from large-scale math and code instruction datasets by filtering to instances with both correct and incorrect candidate solutions, […]

Ver mais

arXiv:2602.12295v1 Announce Type: new Abstract: This study aims to ensure consistency in accuracy throughout the entire design flow in the implementation of edge AI hardware for few-shot learning, by implementing fixed-point data processing in the pre-training and evaluation phases. Specifically, the quantization module, called Brevitas, is applied to implement fixed-point data processing, which allows for arbitrary specification of the bit widths for the integer and fractional parts. Two methods of fixed-point data quantization, quantization-aware training (QAT) and post-training […]

Ver mais

Successful comic book adaptations have been a thing since forever, but we can all agree that it’s only been in the last 15 years or so that comic book movies have really become a juggernaut at the box office. And the numbers back this up; multiple comic book movies have crossed the billion-dollar mark in the last decade and a half. This got me thinking, however. Out of all the comic book movies that have been released, which […]

Ver mais

arXiv:2511.08401v4 Announce Type: replace Abstract: We prove that training a source model optimally for its own task is generically suboptimal when the objective is downstream transfer. We study the source-side optimization problem in L2-SP ridge regression and show a fundamental mismatch between the source-optimal and transfer-optimal source regularization: outside of a measure-zero set, $tau_0^* neq tau_S^*$. We characterize the transfer-optimal source penalty $tau_0^*$ as a function of task alignment and identify an alignment-dependent reversal: with imperfect alignment ($0<rho<1$), […]

Ver mais

This note introduces Projected Microbatch Accumulation (PROMA), a proximal policy update method for large language model fine-tuning. PROMA accumulates policy gradients across microbatches by projecting out sequence-wise gradient components before microbatch aggregation. The projection is applied layer-wise during the backward pass, enabling efficient implementation without additional forward or backward passes. Empirically, PROMA enforces tighter control of local KL divergence than GRPO, resulting in more stable policy learning. Unlike PPO and GRPO, PROMA achieves proximal updates without inducing entropy […]

Ver mais

Every few years, the SaaS industry collectively loses its mind and decides that sales is a magic wand. The logic—if you can call it that—goes like this: We have $10M in the bank, our board wants 100% growth, and our current reps are at 70% of quota. Ergo, if we double the headcount, we’ll double the revenue. It’s a beautiful, linear, spreadsheet-friendly fantasy. It’s also a recipe for a massive burn rate and a demoralized sales force. If […]

Ver mais

Training large language models (LLMs) is constrained by memory requirements, with activations accounting for a substantial fraction of the total footprint. Existing approaches reduce memory using low-rank weight parameterizations or low-rank gradient subspaces for optimizer states, while activation memory is addressed through architectural modifications or compression schemes based on periodically updated projections. We propose OASIS, an online activation subspace learning algorithm for memory-efficient training that tracks and continuously updates a low-dimensional activation subspace during training. Intermediate activations are […]

Ver mais