Continual Learning (CL) aims to enable models to sequentially learn multiple tasks without forgetting previous knowledge. Recent studies have shown that optimizing towards flatter loss minima can improve model generalization. However, existing sharpness-aware methods for CL suffer from two key limitations: (1) they treat sharpness regularization as a unified signal without distinguishing the contributions of its components. and (2) they introduce substantial computational overhead that impedes practical deployment. To address these challenges, we propose FLAD, a novel optimization […]

Ver mais

After DeepSeek’s mHC paper, the Sinkhorn–Knopp algorithm has attracted a lot of attention because it turns $$mathcal{H}^{mathrm{res}}_{l}$$ at each layer into a doubly stochastic matrix. As a result, the layerwise product remains doubly stochastic, and since the L_2 (spectral) norm of a doubly stochastic matrix is 1, this helps prevent vanishing or exploding gradients. This makes me wonder why such an apparently straightforward idea wasn’t discussed more during the era of recurrent neural networks, where training dynamics also […]

Ver mais

arXiv:2504.14772v2 Announce Type: replace-cross Abstract: The exponential growth of Large Language Models (LLMs) continues to highlight the need for efficient strategies to meet ever-expanding computational and data demands. This survey provides a comprehensive analysis of two complementary paradigms: Knowledge Distillation (KD) and Dataset Distillation (DD), both aimed at compressing LLMs while preserving their advanced reasoning capabilities and linguistic diversity. We first examine key methodologies in KD, such as task-specific alignment, rationale-based training, and multi-teacher frameworks, alongside DD techniques […]

Ver mais

The observations for an environment I’m working with are large, nested json objects. Right now it’s infeasible to flatten them into consistent vectors. My initial thought is to use a text embedding model to convert them to vectors. What other approaches have people used when they encounter problems like this? submitted by /u/m_js [link] [comments]

Ver mais

arXiv:2601.09808v1 Announce Type: new Abstract: Variable scoping dictates how and where variables are accessible within programming languages, playing a crucial role in code efficiency and organization. This paper examines the distinct scoping rules in SAS and R, focusing on SAS’s dynamic scoping and R’s lexical scoping. In SAS, dynamic scoping utilizes symbol tables, resolving variables at runtime by dynamically searching through active macro layers. R, in contrast, employs lexical scoping, using environments to resolve variables based on the […]

Ver mais

arXiv:2408.01517v3 Announce Type: replace-cross Abstract: We prove that the standard gradient flow in parameter space that underlies many training algorithms in deep learning can be continuously deformed into an adapted gradient flow which yields (constrained) Euclidean gradient flow in output space. Moreover, for the $L^{2}$ loss, if the Jacobian of the outputs with respect to the parameters is full rank (for fixed training data), then the time variable can be reparametrized so that the resulting flow is simply […]

Ver mais

Oxide and Friends Predictions 2026, today at 4pm PT I joined the Oxide and Friends podcast last year to predict the next 1, 3 and 6 years(!) of AI developments. With hindsight I did very badly, but they’re inviting me back again anyway to have another go. We will be recording live today at 4pm Pacific on their Discord – you can join that here, and the podcast version will go out shortly afterwards. I’ll be recording at […]

Ver mais

arXiv:2601.05293v1 Announce Type: new Abstract: Agentic AI marks an important transition from single-step generative models to systems capable of reasoning, planning, acting, and adapting over long-lasting tasks. By integrating memory, tool use, and iterative decision cycles, these systems enable continuous, autonomous workflows in real-world environments. This survey examines the implications of agentic AI for cybersecurity. On the defensive side, agentic capabilities enable continuous monitoring, autonomous incident response, adaptive threat hunting, and fraud detection at scale. Conversely, the same […]

Ver mais

arXiv:2601.00205v1 Announce Type: new Abstract: Package dependencies are a critical control point in modern software supply chains. Dependency changes can substantially alter a project’s security posture. As AI coding agents increasingly modify software via pull requests, it is unclear whether their dependency decisions introduce distinct security risks. We study 117,062 dependency changes from agent- and human-authored pull requests across seven ecosystems. Agents select known-vulnerable versions more often than humans (2.46% vs. 1.64%), and their vulnerable selections are more […]

Ver mais