Naihe Feng, Yi Sui, Shiyi Hou, Ga Wu, Jesse C. Cresswell
When multi-agent systems (MAS) fail, identifying where the decisive error occurred is the first step for automated recovery to an earlier state. Error attribution remains a fundamental challenge due to the long…
HuggingFacedaily curated papers
Tz-Huan Hsu, Jheng-Hong Yang, Jimmy Lin
Does a lexical retriever suffice as large language models (LLMs) become more capable in an agentic loop? This question naturally arises when building deep research systems. We revisit it by pairing BM25 with frontier…
HuggingFacedaily curated papers
Hamid Kazemi, Atoosa Chegini, Maria Safi
Safety alignment in language models operates through two mechanistically distinct systems: refusal neurons that gate whether harmful knowledge is expressed, and concept neurons that encode the harmful knowledge itself.…
HuggingFacedaily curated papers
Diancheng Kang, Zheyuan Liu, Ningshan Ma, Yue Huang, Zhaoxuan Tan
Activation steering controls language model behavior by adding directions to internal representations at inference time, but standard residual-stream steering can fail in stateful dialogue. We identify KV-cache…
HuggingFacedaily curated papers
Daniel Goldstein, Eugene Cheah
We present Key-Value Means ("KVM"), a novel block-recurrence for attention that can accommodate either fixed-size or growing state. Equipping a strong transformer baseline with fixed-size KVM attention layers yields a…
HuggingFacedaily curated papers
Zixi Li, Youzhen Li
Learning-rate steps are usually treated as hyperparameters. This paper isolates a local beliefspace calculation: when an update is modeled as a projected forward step on the probability simplex, admissibility means…
HuggingFacedaily curated papers
Keya Hu, Linlu Qiu, Yiyang Lu, Hanhong Zhao, Tianhong Li
Diffusion and flow-based models have become the de facto approaches for generating continuous data, e.g., in domains such as images and videos. Their success has attracted growing interest in applying them to language…
HuggingFacedaily curated papers
Zhengyang Tang, Yi Zhang, Chenxin Li, Xin Lai, Pengyuan Lyu
When a phone-use agent avoids harm, does that show safety, or simply inability to act? Existing evaluations often cannot tell. A harmful outcome may be avoided because the agent recognized the risk and chose the safe…
HuggingFacedaily curated papers
Omatharv Bharat Vaidya, Connor T. Jerzak, Nhat Ho, Chandrajit Bajaj
We present a data-adaptive method for parameter-efficient fine-tuning of large neural networks. Standard low-rank adaptation methods improve efficiency by restricting each layer update to a fixed low-rank form, but this…
HuggingFacedaily curated papers
Yihong Liu, Raoyuan Zhao, Michael A. Hedderich, Hinrich Schütze
Large language models (LLMs) have achieved remarkable progress in mathematical reasoning, but this ability is not equally accessible across languages. Especially low-resource languages exhibit much lower reasoning…
HuggingFacedaily curated papers
Xingyu Qu, Peigeng Huang, Samuel Horvath
Muon has emerged as an efficient alternative to Adam for pretraining, yet remains underused for fine-tuning. A key obstacle is that most open models are pretrained with Adam, and naively switching to Muon for…
HuggingFacedaily curated papers
Víctor Gallego
We present Metal-Sci, a 10-task benchmark of scientific Apple Silicon Metal compute kernels spanning six optimization regimes (stencils, all-pairs in n-body problems, multi-field Boltzmann, neighbor-list molecular…
HuggingFacedaily curated papers
Bian Sun, Kevin Zhai, Mubarak Shah, Zhenyi Wang
Diffusion language models (DLMs) have recently emerged as a promising alternative to autoregressive models, primarily due to their ability to enable parallel decoding. Despite this advantage, most existing DLMs rely on…
HuggingFacedaily curated papers