A new survey of the activity of nearly 60,000 neurons in the mouse visual system reveals how far we have to go to understand how the brain computes. Published today in the international journal Nature Neuroscience, the analysis led by researchers at the Allen Institute reveals that more than 90% of neurons in the visual cortex, the part of the brain that processes our visual world, don’t work the way scientists thought — and it’s not yet clear how they do work.

“We thought that there are simple principles according to which these neurons process visual information, and those principles are in all the textbooks,” said Christof Koch, Ph.D., Chief Scientist and President of the Allen Institute for Brain Science, a division of the Allen Institute, and co-senior author on the study along with R. Clay Reid, M.D., Ph.D., Senior Investigator at the Allen Institute for Brain Science. “But now that we can survey tens of thousands of cells at once, we get a more subtle — and much more complicated — picture.”

Nearly 60 years ago, two neuroscientists, David Hubel and Torsten Wiesel, made groundbreaking discoveries about how mammals’ brains perceive the visual world around us. Their work uncovered individual neurons that switch on only in response to very specific kinds of images.

Hubel and Wiesel made their discoveries by showing simple pictures — things like a black bar or dot on a white background — to cats and monkeys. The general principle they uncovered says that as you view the world around you, specific neurons in your brain are responsible for recognizing exact parts in a particular region of that scene and the recognition gets more specialized and fine-tuned in higher-order parts of the brain.

Say you’re in a park: One set of neurons will fire a rapid electrical response to a dark tree branch in a precise spot in your line of sight. Other neurons switch on only when a bird flies across your field of vision from left to right. Your brain would then stitch together information from the “tree branch” neurons and the “moving bird” neurons to get a complete picture of the world around you, or so the theory goes.

Hubel and Wiesel’s findings were recognized by a Nobel Prize in Physiology or Medicine and formed the backbone of the neural networks that underlie most computer vision applications. In the past decade, with the advent of new neuroscience methods that enable the study of more and more brain cells at once, scientists have come to understand that this model of how our brains see is likely not the whole story — some neurons clearly don’t follow the classic model of tuning into specific features.

But it wasn’t clear just how incomplete the story was.

Brain activity variability

The new study is the first large-scale analysis of the publicly available data from the Allen Brain Observatory, a broad survey that captures the activity of tens of thousands of neurons in the mouse visual system. The researchers analyzed the activity of nearly 60,000 different neurons in the visual parts of the cortex, the outermost shell of the brain, as animals see different simple images, photos and short video clips — including the opening shot from the classic Orson Welles movie “Touch of Evil” (chosen because it has a lot of movement and is a single shot with no cuts).

1950s and 60s neuroscience studies, by necessity, were like fishing expeditions — researchers hunted through the brain with a single electrode until they found a neuron that reliably responded to a certain image. It’s akin to trying to watch a widescreen movie through a few scattered pinholes, Koch said — it would be impossible to get a complete picture. The Allen Brain Observatory dataset doesn’t capture the activity of every neuron under every scenario, but it allows researchers to study more neurons at once, including those with more subtle responses.

The researchers’ new analysis found that less than 10% of the 60,000 neurons responded following the textbook model. Of the rest, about two-thirds showed some reliable response, but their responses were more specialized than the classic models would predict. The last third of neurons showed some activity, but they didn’t light up reliably to any of the stimuli in the experiment — it’s not clear what these neurons are doing, the researchers said.

“It’s not that the previous studies were all a big mistake, it’s just that those cells turn out to be a very small fraction of all neurons in the cortex,” said Saskia de Vries, Ph.D., an Assistant Investigator at the Allen Institute for Brain Science who led the study along with Jérôme Lecoq, Ph.D., and Michael Buice, Ph.D. “It turns out that the mouse visual cortex is much more complex and richer than we previously thought, which underscores the value of doing this type of survey.”

That these more variable, less specific neurons exist is not news. But it was a surprise that they dominate the visual parts of the mouse brain, the researchers said.

How the brain computes

It’s not yet clear how these other neurons contribute to processing visual information. Other research groups have seen that locomotion can drive neuron activity in the visual part of the brain, but whether the mice were running or still only explains a small amount of the variability in visual responses, the researchers found.

Their next steps are to run similar experiments with more natural movies, offering the neurons a larger set of visual features to respond to. Buice has made a 10-hour specialized reel of clips from pretty much every nature documentary he could get his hands on.

The researchers also point out that the classic model came from studies of cats and primates, animals which both evolved to see their worlds in sharper focus at the center of their gaze than did mice. It’s possible that mouse vision is just a completely different ballgame than ours. But there are still principles from these studies that might apply to our own brains, said Buice, an Associate Investigator at the Allen Institute for Brain Science.

“Our goal was not to study vision; our goal was to study how the cortex computes. We think the cortex has a structure of computation that’s universal, similar to the way different types of computers can run the same programs,” Buice said. “In the end, it doesn’t matter what kind of program the computer is running; we want to understand how it runs programs at all.”

The material described in this article is based on work supported in part by the National Science Foundation under grant CCF-1231216. Any opinions, findings, and conclusions or recommendations expressed here are those of the author and do not necessarily reflect the views of the NSF. Research reported in this article was supported in part by the National Institute of Neurological Disorders and Stroke and the National Institute of Biomedical Imaging and Bioengineering at the National Institutes of Health under Award Number R01EB026908. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.