AI Interpretability

How do LLMs work?

Large language models offer what every neuroscientist can only dream of having: perfect knowledge of the connectome, with every unit’s weight known exactly. And yet we still don’t fully understand how they work. We can record from every unit without noise, replay any input stimulus, and lesion or stimulate any part of the network at will, but knowing the wiring does not tell us what computation it carries out. Working that out has become a field of its own, mechanistic interpretability, and many of its tools would be familiar in a systems neuroscience lab: population analysis, tracking changes across learning, lesions, stimulation, and the crucial search for circuit function. And maybe, just maybe, there are insights that a neuroscientist can bring to the interpretability field. But first, what are LLMs, and what tools are there for studying them? These lessons on GitHub introduce neuroscientists to how LLMs are built and how we can look under the hood.