Paper Review: Could a neuroscientist understand a microprocessor?
Eric Jonas and Konrad Kording
http://biorxiv.org/content/early/2016/05/26/055624

The Atari Brain

Do neuroscientific techniques really tell us how the brain works? Are neuroscientists just barking up the wrong tree? Jonas and Kording think we might be. However, Alona and Nicole disagree.

Alona: Nicole, you found this paper a few weeks ago and we read it as a Mitchell Lab alumni reading group.  It really got everyone thinking, so I thought it would be a great first paper review for the blog.

For our readers who have not read the paper, the premise is: let’s pretend the microprocessor is a brain and perform the sorts of experiments on it that people have used to study the brain.  The experiments include lesions studies, quantifying tuning curves, local field potential (LFP) analysis, granger causality and more.  I’d say many of the experiments are unsuccessful and inconclusive.  In the end, not a lot is “learned” about microprocessors by applying neuroscience analysis techniques.  Nicole and I, however, feel like neuroscience wasn’t given a fair chance. I’ll let her start the discussion.

In the end, not a lot is “learned” about microprocessors by applying neuroscience analysis techniques.  Nicole and I, however, feel like neuroscience wasn’t given a fair chance.

Nicole: I want to start with the choice of a “model organism.” Sure, they can observe the voltage at each transistor in the microprocessor, but their notion of “behavior” in this organism is extremely weak. Neuroscience has always been closely related to behavioral Psychology – we care very deeply about the behaviors that we want to associate with the brain. Organisms are built to respond to things in their environment – let’s call them inputs. Psychologists have spent centuries measuring the behavioral output in response to input , and neuroscientists seek to measure the brain’s response to these inputs. By carefully crafting the inputs, we can test hypotheses about how the brain works.

Our hypotheses typically result from the confluence of our understanding of behavior (e.g., humans tend to respond to stimulus X with behavior Y) and our understanding of the brain (e.g., humans with a lesion in area A produce behavior Z instead of Y in response to stimulus X). When we measure neural activity, it is almost always to see if we can relate our knowledge of the brain to our knowledge of behavior, and thereby improve our understanding of both.

The system in the paper, namely a microprocessor pre-loaded with classic video games, lacks the ability to receive input. Or rather, it could receive inputs but the authors fail to provide it with any. The “behaviors” that they use are just the intro sequences from these games. It’s hard for me to find an analogy for such a thing in humans.

When we measure neural activity, it is almost always to see if we can relate our knowledge of the brain to our knowledge of behavior, and thereby improve our understanding of both.

Alona: Yes, I was wondering what the human parallel would be to this Atari brain.  Is it resting state?  Even without getting into the controversiality of resting state studies, I think we can all agree that brains are not on an imperturbable loop during resting state.  Even without input, our brains are marching along, having thoughts, making plans.  That is certainly not what the opening screen of a game is doing.  Maybe the closest parallel is an anesthetized animal experiment.  Is that what the authors were going for here?  A lot of the seminal vision research was carried out on anesthetized cats, for example.  But I think they are trying to draw parallels to the study of cognition, not vision.

Nicole: Even experiments in anesthetized animals typically involve some sort of input, often  images. I would agree that the microprocessor is not even at that level. Your analogy to resting state is good, although that makes their “lesion” study a little strange. The authors would maintain that pixel luminance is some sort of behavior, though I’m not sold.

But let’s give them the benefit of the doubt, and go with their argument that pixel output is something like “free behavior.” Even that explanation lacks substance. When we measure freely behaving mice, we typically understand their environment and their goals. The authors make no attempt to parameterize the output space – what kind of structure can we find in the pixels? That would have been much more convincing, especially in the lesion study case. For example, is there a lesion that causes a more specific change than just “the program no longer runs?”

Alona: to be fair, they were asking “Is there a lesion that makes only one program stop running?”.  So if each of the games is a behavior, they are trying to knock out one, and only one, of those behaviors.

Nicole: But no attempt is made to analyze the similarities and differences in those behaviors. All three game behaviors rely on similar functions. Depending on the level of similarity between the behaviors, you might think of it as trying to find a lesion that only knocks out your ability to read words that start with “k” versus words that start with “s.” That’s an experiment that’s unlikely to succeed. But if the behaviors are more like “speaking” vs “understanding spoken words” vs “understanding written words” then it’s a more reasonable experiment.

The authors argue that neuroscientists make the same mistake all the time; that we are operating at the wrong level of granularity for our behavioral measures and don’t know it. That argument denies the degree to which we characterize behaviors in neuroscience, and how stringent we are about controls.

The authors point to the fact that transistors that eliminate only one behavior are not meaningfully clustered on the chip. But what they ignore are the transistors that eliminate all three behaviors. Those structures are key to the functioning of the device in general.  To me, those 1560 transistors that eliminated all three behaviors are more worthy of study than the lesions that affect only one behavior, because they allow us to determine what is essential to the behavior of the system. You can think of those transistors as leading to the death of the organism, just as damage to certain parts of the brain cause death in animals.

Jonas and Kording argue that neuroscientists make the same mistake all the time; that we are operating at the wrong level of granularity for our behavioral measures and don’t know it. That argument denies the degree to which we characterize behaviors in neuroscience, and how stringent we are about controls.

Alona: The obvious extension then is to also study the lesions that affect only two “behaviors”, and quantify their behavioral overlap with respect to the third unaffected behavior.

I have to say, I was also surprised that the authors couldn’t do more with pixel luminance.  Presumably there’s a set of transistors that are actually driving the pixel output to the screen?  Certainly, if they had done multivariate pattern analysis (MVPA) here, I think they would have been able to get really good prediction accuracy for some pixels, because I’m sure some pixels have exactly the same value throughout the first few seconds of the game, for example.

Nicole: I was really surprised by that too, because I feel like people love to harp on MVPA and other uses of machine learning for neuroscience. It would have made a great addition to the battery of techniques they used here.

Alona: And while we’re on the topic of machine learning: this sentence made me laugh: “The brain is clearly far more complicated and our difficulty at understanding deep learning may suggest that the brain is hard to understand if it uses anything like gradient descent on a cost function.”  I have trouble even parsing that sentence. Are the authors saying that deep learning is hard to understand because it uses gradient descent?  Linear regression can also be fit using gradient descent, and there’s a long history of using the learned weights to understand the underlying data.   Deep models are opaque because they are nonlinear and multi-layered.  I’d say the brain’s extreme size, layering and interconnectedness are some of the reasons it is so difficult to understand.  Gradient descent is just a method for fitting a model, and I disagree that there’s anything in particular about gradient descent that would make the final model difficult to interpret.

Nicole: I also just found that statement to be really out of place and not connected with the rest of the paper. I wonder what would have happened if they had tried these techniques on a deep network? I guess that’s for another day.

Alona: Right, maybe that’s enough complaining for one paper review. We should try and be constructive (we aren’t reviewer #2, after all).  

The reason we wanted to talk about this paper was not because we think it’s crazy, but because it made us wonder if we could do it better.  And by “it” I mean 1) the study of the microprocessor itself, and 2) neuroscience more generally.

The reason we wanted to talk about this paper was not because we think it’s crazy, but because it made us wonder if we could do it better.

Nicole: You’re right. It definitely did make me wonder. Let’s start by discussing how we would have studied the microprocessor differently. The main overall difference between my approach and theirs would have been that my experiments would have built on each other.

The history of neuroscience starts clinically – sometimes brains get damaged, whether it be by trauma, stroke, or surgery (think patient HM). Those damaged brains exhibited altered behavior in response to stimuli. Thanks to a solid understanding of expected behavior, we were able to learn a lot about the brain from these unfortunate lesions. Neuroscientists rely on this early lesion work to develop theories of how the brain might work, and then test those theories with newer techniques.

If I were trying to understand a microprocessor, I would have done follow-ups on the lesion study in which I further examined the transistors in each group. The Granger causality analysis would then have been informed by my results, as would the tuning curves. It seems unfair to try to make every technique/experiment stand alone. And of course, I would have applied MVPA to try to tease apart what underlies each “behavior.”

In this paper, they perform a pretty cool lesion study that none of us could do in an organism more complex than say, drosophila. Then, because the lesion results  didn’t take the shape they were expecting, they threw out those results and proceeded to try another technique, selected seemingly randomly.

The title of the paper is “Could a neuroscientist understand a microprocessor?” but the content of the paper answers the question “Do neuroscientific techniques, applied essentially at random, lead to an understanding of microprocessors?”

The title of the paper is “Could a neuroscientist understand a microprocessor?” but the content of the paper answers the question “Do neuroscientific techniques, applied essentially at random, lead to an understanding of microprocessors?”

Alona: Unsurprisingly, no.  But, these techniques, applied in a non-random way to actual brain images of people performing well defined tasks, do help us understand brain structure and function.  But that doesn’t mean we can’t do it better.

Nicole: Fair enough, let’s have the conversation that the authors want us to have: are we doing the right thing when we study the brain?

Alona: So, let’s use language in the brain as our example, since that’s what we both study.  Let’s say we had perfect data that told us the exact areas of the brain that were more or less active in response to particular stimuli, or we had groups of neurons, that, when taken together, were good predictors of word meaning. Would that be enough to understand language in the brain?

Nicole: I think that even in that case, you still can’t escape from the principle of hypothesis formation. No matter how nice the signal and how sophisticated the model, you still need to extend the understanding of the human brain in a principled fashion. Saying “I can decode the noun this person is reading with 99% accuracy” is not helpful by itself. Can you decode other words that well? What if I put the nouns in different contexts? What are the brain regions allowing you to decode? In the end, you want a theory of how someone went from reading the word to knowing its meaning.

Alona: So what does that mean for Leila Wehbe’s Harry Potter paper?  That was a completely uncontrolled experiment that used a natural reading paradigm.  Can we learn nothing from that?

Nicole: I think naturalistic, unstructured studies are useful, but in the context of prior controlled work. For example, we know a lot about language organization in the brain from studies that contrast word lists to sentences. However, when we go out into the real world, people read word lists much less frequently than sentences.. Many controlled language experiments are very unnatural, and it’s been demonstrated that many controlled experiments don’t  generalize well in the real world. Leila’s paper answers the question: “How many of these controlled studies stand up to the test of natural story reading?” It requires some extra legwork on her part: she needed to situate the language map that she discovered in the context of the literature of controlled studies and lesion studies.

Alona: But what can it tell us about the function of the brain?

Nicole: It depends largely on how well her results fit in the literature. If there is a brain region that lights up when you read sentences but not word lists, and it’s not one of her “syntax” regions, why is that? What about regions she labelled as “syntax” that don’t appear in the syntax literature at all? This helps you answer the question: “what kind of processing is necessary for story reading but not for simpler reading tasks?” And then you can dig deeper at that question. This touches again on the first thing I brought up – that understanding behavior, even “free behavior” is important for successful analysis.

Anyway, let’s bring it back to the original paper under discussion: could a neuroscientist understand a microprocessor?

Alona: I think so. To summarize, I think our main issue with the microprocessor paper is that you can’t study a system without understanding the input and outputs of that system.  A well-constructed experiment is built with overlaps or similarities between the inputs which allows you to draw conclusions about the system’s function.  A real neuroscientist would have cared a lot more about the behavior of the system, and would likely have been much more successful.

Though we’ve put this paper through the ringer, I think we both found it to be very thought-provoking and worth the read.   It’s not every day that a paper really makes me reflect on the way I approach my research questions.  So kudos to Jonas and Kording!