October 2025

TL;DR Dead neurons silently waste compute and reduce effective model capacity in foundation models. Simple visualizations of the activation frequency make neuron health measurable. Dead neurons can be brought back to life by swapping activation functions or implementing synaptic stripping. It is crucial for foundation model training success to proactively monitor neuron health with audits and alerts. In neural networks, some neurons end up outputting near-zero activations across all inputs. These so-called “dead neurons” degrade model capacity because […]

Ver mais

I’ve been experimenting with getting a tiny circa-2015 recurrent neural network to generate Halloween costumes. Running on a single cat hair-covered laptop, char-rnn has no internet training, but learns from scratch to imitate the data I give it. A little while ago I revisited a dataset from 2018, over 7100 user-submitted Halloween costumes (3173 with exact duplicates removed). Char-rnn generated some pretty intriguing costumes. But because its training data was old, it was missing out on more recent […]

Ver mais

How can you use science to build a better gingerbread house? That was something Miranda Schwacke spent a lot of time thinking about. The MIT graduate student in the Department of Materials Science and Engineering (DMSE) is part of Kitchen Matters, a group of grad students who use food and kitchen tools to explain scientific concepts through short videos and outreach events. Past topics included why chocolate “seizes,” or becomes difficult to work with when melting (spoiler: water gets in), and […]

Ver mais

TL;DR Standard LLM evaluation metrics fail to distinguish between a plausible-sounding text and a response that genuinely follows task instructions. Specialized metrics assess the relevance, fidelity, and multi-turn coherence of instruction-tuned LLMs, relying on techniques like LLM-as-a-Judge. More comprehensive evaluation approaches look beyond individual instruction-response pairs to assess a model’s ability to fulfill tasks not seen during training. Since Instruction Fine-Tuning (IFT) is aligning a model to a given goal, rather than imprinting new knowledge, training approaches that […]

Ver mais

On Oct. 20 during its annual meeting, the National Academy of Medicine announced the election of 100 new members, including MIT faculty members Dina Katabi and Facundo Batista, along with three additional MIT alumni. Election to the National Academy of Medicine (NAM) is considered one of the highest honors in the fields of health and medicine, recognizing individuals who have demonstrated outstanding professional achievement and commitment to service. Facundo Batista is the associate director and scientific director of […]

Ver mais