May 2026

arXiv:2605.06672v1 Announce Type: new Abstract: Chain-of-thought (CoT) reasoning and reasoning-tuned models such as DeepSeek-R1 are commonly assumed to reduce shallow heuristic biases by thinking carefully. We test this on position bias in multiple-choice QA and find a different story: within any reasoning-capable model, per-question position bias scales with the length of the reasoning trajectory. Across thirteen reasoning-mode configurations (two R1-distilled 7-8B models, two base models prompted with CoT, and DeepSeek-R1 at 671B) on MMLU, ARC-Challenge, and GPQA, twelve […]

Ver mais

arXiv:2605.06671v1 Announce Type: new Abstract: Large Language Models (LLMs) have demonstrated strong potential for many mathematical problems. However, their performance on graph algorithmic tasks is still unsatisfying, since graphs are naturally more complex in topology and often require systematic multi-step reasoning, especially on larger graphs. Motivated by this gap, we propose GraphDC, a Divide-and-Conquer multi-agent framework for scalable graph algorithm reasoning. Specifically, inspired by Divide-and-Conquer design, GraphDC decomposes an input graph into smaller subgraphs, assigns each subgraph to […]

Ver mais

arXiv:2605.06669v1 Announce Type: new Abstract: Educational LLM tutors face a core AI alignment challenge: they must follow user intent while preserving pedagogical constraints and safety policies. We present an evaluation methodology for prompt-injection defenses in this setting, showing that guardrail design entails explicit trade-offs among adversarial robustness, benign-task usability, and response latency. We evaluate a domain-specific multi-layer safeguard pipeline combining deterministic pattern filters, structural validation, contextual sandboxing, and session-level behavioral checks. On a controlled holdout benchmark with 480 […]

Ver mais

GitLab Act 2 There’s a lot going on in this announcement from GitLab about the “workforce reduction” and “structural and strategic decisions” they are making with respect to the agentic era. They’re “planning to reduce the number of countries by up to 30% where we have small teams”. One of the most interesting things about GitLab is that they have employees spread across a large number of countries – 18 are listed in their public employee handbook but […]

Ver mais

Home Table of Contents Building and Training a Kimi-K2 Model Using DeepSeek-V3 Components Kimi-K2 vs DeepSeek-V3: Key Architecture Differences in LLM Design Mixture of Experts Scaling in Kimi-K2: Model Size, Sparsity, and Efficiency Attention Head Optimization in Kimi-K2 for Efficient Long-Context LLMs MuonClip Optimizer: Stabilizing Large-Scale LLM Training in Kimi-K2 Token Efficiency in LLM Training: Why It Matters for Kimi-K2 Attention Logit Explosion in LLMs: Training Instability and Challenges QK-Clip: Preventing Attention Logit Explosion in Kimi-K2 Training Training […]

Ver mais

Scaling large language models (LLMs) is expensive. Every token processed during inference and every gradient computed during training flows through feedforward layers that account for over two-thirds of model parameters and more than 80% of total FLOPs in larger models. A team researchers from Sakana AI and NVIDIA have worked on a new research that directly targets this bottleneck — not by changing the architecture, but by making the computation inside feedforward layers significantly cheaper through unstructured sparsity. […]

Ver mais

A team of researchers from Meta, Stanford University, and the University of Washington have introduced three new methods that substantially accelerate generation in the Byte Latent Transformer (BLT) — a language model architecture that operates directly on raw bytes instead of tokens. Byte-Level Models Are Slow at Inference To understand what this new research solves, you need to understand the tradeoff at the center of byte-level language modeling. Most language models today work on tokens — chunks of […]

Ver mais