arXiv:2602.08927v1 Announce Type: new Abstract: We study the problem of online monotone density estimation, where density estimators must be constructed in a predictable manner from sequentially observed data. We propose two online estimators: an online analogue of the classical Grenander estimator, and an expert aggregation estimator inspired by exponential weighting methods from the online learning literature. In the well-specified stochastic setting, where the underlying density is monotone, we show that the expected cumulative log-likelihood gap between the online […]

Ver mais

A few years ago, I found that duplicating a specific block of 7 middle layers in Qwen2-72B, without modifying any weights, improved performance across all Open LLM Leaderboard benchmarks and took #1 place. As of 2026, the top 4 models on that leaderboard are still descendants. The weird finding: single-layer duplication does nothing. Too few layers, nothing. Too many, it gets worse. Only circuit-sized blocks of ~7 layers work. This suggests pre-training carves out discrete functional circuits in […]

Ver mais

Table of Links Abstract and 1. Introduction Related Work and Background Analysis 3.1 Limitations about Existing ReLUficatio 3.2 dReLU Are Neurons in Expert still Sparsely Activated? dReLU Sparsification Experiments Results 6.1 Downstream Tasks Performance 6.2 Sparsity of Sparsified Models Practical Inference Speedup Evaluation 7.1 Experiments Setting 7.2 Pure CPU Inference and 7.3 Hybrid GPU-CPU Inference 7.4 Deploy LLMs on mobile phones Conclusion and References A. Appendix / supplemental material B. Limitation C. Broader Impact 7 Practical Inference Speedup […]

Ver mais

Institutional investors aren’t just chasing the next big Bitcoin rally anymore. Lately, the conversation has shifted. Now, it’s about how Bitcoin fits into a bigger portfolio—not just how much its price can jump in a day or two. Asset managers are thinking hard about risk and how Bitcoin might actually help smooth things out not just add more chaos. Cathie Wood, the CEO of Ark Invest, talked about this in her 2026 market outlook. She called Bitcoin a […]

Ver mais

arXiv:2603.00105v1 Announce Type: cross Abstract: Large language models (LLMs) have gained significant attention by many researchers and practitioners in natural language processing (NLP) since the introduction of ChatGPT in 2022. One notable feature of ChatGPT is its ability to generate summaries based on prompts. Yet evaluating the quality of these summaries remains challenging due to the complexity of language. To this end, in this paper we suggest a new method of LLM summary inference with BERT-SVD-based direction metric […]

Ver mais

arXiv:2601.21739v1 Announce Type: cross Abstract: Adam has been at the core of large-scale training for almost a decade, yet a simple empirical fact remains unaccounted for: both validation scores and the qualitative behaviour of the training runs improve when the momentum parameters satisfy $beta_{1}=beta_{2}$. Some recent studies have reported this pattern, but there is still no explanation for why this choice helps. We show that this choice is closely tied to a structural property that we refer to […]

Ver mais

arXiv:2602.15059v1 Announce Type: new Abstract: Let $(u,p)$ solve the incompressible Navier–Stokes equations in a regime in which an energy inequality is available and each constant in that inequality is computable from declared data. We construct a reduced-order model $u_n$ constrained so that its discrete evolution satisfies a certified energy inequality. This certificate yields global-in-time boundedness of the ROM energy and a regime-of-validity test that fails when a stated hypothesis fails. It follows that one can attach a computable […]

Ver mais

arXiv:2601.00090v1 Announce Type: new Abstract: Contemporary text-to-image models exhibit a surprising degree of mode collapse, as can be seen when sampling several images given the same text prompt. While previous work has attempted to address this issue by steering the model using guidance mechanisms, or by generating a large pool of candidates and refining them, in this work we take a different direction and aim for diversity in generations via noise optimization. Specifically, we show that a simple […]

Ver mais

In one of his earliest writings on the topic, Teilhard de Chardin explores humanity’s unique place in evolution. He argues that humans represent an entirely new phase of life on Earth—the , or sphere of conscious thought. While physically similar to other primates, humans are revolutionary in their ability to use tools, form global connections, and reflect on their own existence. This self-awareness comes with both great power and great risk, as humans can choose to either advance […]

Ver mais

Deep learning models trained in a supervised setting have revolutionized audio and speech processing. However, their performance inherently depends on the quantity of human-annotated data, making them costly to scale and prone to poor generalization under unseen conditions. To address these challenges, Self-Supervised Learning (SSL) has emerged as a promising paradigm, leveraging vast amounts of unlabeled data to learn relevant representations. The application of SSL for Automatic Speech Recognition (ASR) has been extensively studied, but research on other […]

Ver mais