WW-PGD: Projected Gradient Descent optimizer

WW-PGD: Projected Gradient Descent optimizer

⋅

Announcing: ๐—ช๐—ช-๐—ฃ๐—š๐—— โ€” ๐—ช๐—ฒ๐—ถ๐—ด๐—ต๐˜๐—ช๐—ฎ๐˜๐—ฐ๐—ต๐—ฒ๐—ฟ ๐—ฃ๐—ฟ๐—ผ๐—ท๐—ฒ๐—ฐ๐˜๐—ฒ๐—ฑ ๐—š๐—ฟ๐—ฎ๐—ฑ๐—ถ๐—ฒ๐—ป๐˜ ๐——๐—ฒ๐˜€๐—ฐ๐—ฒ๐—ป๐˜ I just released WW-PGD, a small PyTorch add-on that wraps standard optimizers (SGD, Adam, AdamW, etc.) and applies an epoch-boundary spectral projection using WeightWatcher diagnostics. Elevator pitch: WW-PGD explicitly nudges each layer toward the Exact Renormalization Group (ERG) critical manifold during training. ๐—ง๐—ต๐—ฒ๐—ผ๐—ฟ๐˜† ๐—ถ๐—ป ๐˜€๐—ต๐—ผ๐—ฟ๐˜ โ€ข HTSR critical condition: ฮฑ โ‰ˆ 2 โ€ข SETOL ERG condition: trace-log(ฮป) over the spectral tail = 0 WW-PGD makes these explicit optimization targets, rather than […]

Ver mais

Liked Liked
Rage Against the Machine. Or Donโ€™t.

Rage Against the Machine. Or Donโ€™t.

⋅

Itโ€™s unusual to be in the situation of reviewing a book no one will like. I donโ€™t mean that literally; a handful of people will appreciate Paul Kingsnorthโ€™s new book, Against the Machine, probably the same people who have followed his work for the past decade. Continue Reading…

Ver mais

Liked Liked
From Transformers to Associative Memory, How Titans and MIRAS Rethink Long Context Modeling

From Transformers to Associative Memory, How Titans and MIRAS Rethink Long Context Modeling

⋅

What comes after Transformers? Google Research is proposing a new way to give sequence models usable long term memory with Titans and MIRAS, while keeping training parallel and inference close to linear. Titans is a concrete architecture that adds a deep neural memory to a Transformer style backbone. MIRAS is a general framework that views most modern sequence models as instances of online optimization over an associative memory. Why Titans and MIRAS? Standard Transformers use attention over a […]

Ver mais

Liked Liked
Teaching robots to map large environments

Teaching robots to map large environments

⋅

A robot searching for workers trapped in a partially collapsed mine shaft must rapidly generate a map of the scene and identify its location within that scene as it navigates the treacherous terrain. Researchers have recently started building powerful machine-learning models to perform this complex task using only images from the robotโ€™s onboard cameras, but even the best models can only process a few images at a time. In a real-world disaster where every second counts, a search-and-rescue […]

Ver mais

Liked Liked

Load More

↑

Digitado ยฉ 2025