The Stories Our Brains Tell

Rena Dorph: Ehud Isacoff, who you may know as a member of Netivot Shalom, is a professor of neurobiology in the Department of Molecular and Cell Biology at UC Berkeley. He’s a biophysicist and a neuroscientist recognized for his work on membrane-signaling proteins, synapses, and neural circuits. Ehud received his bachelor’s degree in biology and a PhD in physiology at McGill University in Montreal and was a postdoctoral fellow at UC San Francisco. 

Over the course of his career, Ehud has served as the head of the neurobiology division, the director of Molecular Imaging center, and the director of Nanomedicine Development center in the Optical Control of Biological Function, and Ehud has received multiple honors and award for his work. He also is a member of the National Academy of Sciences and serves on multiple scientific advisory boards, including for the Max Planck Institute for Medical Research in Heidelberg, the Sagol School of Neuroscience in Tel Aviv University, and on the board of scientific governors of the Lowy Medical Research Institute. 

The great thing about being a member of Netivot Shalom is you actually never know who’s sitting next to you and all the amazing things that they’ve done. So I’m happy to have learned all this about Ehud in preparing to bring you his talk today. So with that, I’ll welcome Ehud.

Ehud Isacoff: This is my third time giving a talk, through many, many years, to an audience that’s not just scientists. You heard the kind of work that I do is very much from the molecular and biophysical end of neuroscience, meaning how tiny things work – the tiniest things, proteins, even single proteins, one at a time, how they signal in the nervous system at synapses to communicate information between neurons, and how they make cells excitable and do what we call fire action potentials when they’re active, and transfer information over long distances.

However, some of the methodologies that we use are also things that are very applicable to being able to use in the living system. And so as a result, we do some circuit and behavioral neuroscience. And of course, that’s, in a way, the thing that always captures the imagination in neuroscience, including mine – because when I started with neuroscience, I thought I was going to be studying literature.

So what I want to tell you about is the mind, right? And this is also when I give my first lecture to juniors and seniors at Berkeley, the first lecture is always what I call the “bait,” and the next lecture is the “switch” – so you’re getting just the bait, which is the best part. You don’t have to worry about the biophysics, but you’ll get the bait.

Neuroscience, Vision and Interpretation

So I’ll start off with this, and it’s another story. And it’s about somebody who is an example of a great intellectual at Berkeley, who some of you may have heard of, partly very relevant to what’s happening today, which – we will not be talking about the politics of today. And her name is Judith Butler. So I had the pleasure at Berkeley – scientists, normally we kind of live in our own world, partly because just we’re in a lab, and the lab is embedded among other labs, and we talk to one another in gobbledygook, right? And so, we rarely get the real pleasure of being able to talk to other people who are doing really fascinating intellectual work of different classes on the campus. But the Townshend Center for the Humanities at Berkeley sponsored something that was bringing together people from different disciplines. So there would be a lead, and then the lead would find a sidekick, and the sidekick would be in another discipline, and the lead would get a semester off from teaching – or two – and the sidekick would get the pleasure of just, like, being a sidekick.

I was a sidekick to someone named Anne Nesbet, who’s a Professor in Rhetoric and in Slavic Studies. Anne teaches film, and in teaching film, she wanted to know, really, how the brain processes visual information, and, from individual frames in pictures, can produce a continuous feeling for a real narrative and a natural world.

And so she asked me to participate, because I could give her sort of a view of how the visual system works from the scientific point of view, and that would enrich, in a way, what she was doing.

So the sidekicks just kind of sat there and would ask questions once in a while around this beautiful oak table. But then we all had one time when we gave a presentation – so I gave a presentation. And in my presentation, I showed slides.

And this one slide – you may have even seen this image. It’s an image of a woman with a very, very white complexion. You barely see her. She has very dark hair and dark eyes, and kind of scratched over the surface of the picture, you see a black trace that shows where a person viewing her, where their eyes rest – and how the eye, the gaze of the looker at the picture – go over the face.

And so what you see is that they spend a lot of time on the eyes and on the border, that’s very, very high contrast, between the black hair and the white face, and on the mouth, right?

And so I said, “You know, vision is not simply a passive process. We’re not just like in the Nobel Prize-winning work of Hubel [and] Wiesel, paralyzed cats looking on a screen. We are constantly probing, and we probe with our eyes, but we probe with our – if we’re mice, with our whiskers, we probe with our hands. We’re looking around all the time. The world is something we probe, and this is the way that the gaze kind of probes the image.”

Judith Butler is about three feet tall in a leather jacket – so she seems four feet tall. She slams her hand on the table and says, “No, that’s not what I see!” And I say, “What do you see?” She says, “I see someone who could be my grandmother in the Warsaw Ghetto.” And I said, “That’s very interesting, that association, but that’s not what I’m talking about. That’s very interesting. That informs how you react to it, what associations it has, what kind of mental processes it triggers in your mind, what you bring to it. But all I’m saying is how your eyes travel. And we all do this – our eyes travel over the picture. That’s not how we see. That’s not what we see. That’s how we look.”

“No!”

So a year later, my wife Jen and I are in Paris, among other things, on sabbatical and making our first child, Yael, who had her bat mitzvah a year and a half ago here. And when we’re there, we run into people who are professors at Berkeley in the humanities. And one of them, I recognize – Shannon Jackson. And we’re waiting at the Prefecture de la Police for a work permit, which takes forever. So we get to be really good friends. And they invite us to their house, near the Jardin des Plantes.

We go, and over dinner with a lot of wine and all sorts of other guests – it’s really, really nice. It’s all people in the humanities and philosophers, including this philosopher from the Collège de France, right? I tell the story, the same story I just told you, and I say, “Isn’t it funny that Judith Butler said, ‘No, you’re wrong’?”  They all look at each other, and then the philosopher from the Collège de France says, “Mais c’est la raison.” – “She’s right!” That tells you something about Judith Butler, and tells you something about what we do all the time, which is interpretation. 

Parshat Balak: Interpreting When the Story Changes

So, from [Rabbi] Carol [Caine], you heard several different possible stories, and one element that she had to do in the way that she described things was impose a logic, meaning she was going to accept the sequence in which the story is told in the parsha. I mean, you don’t have to have read The Book of J to know just from everything you’ve seen, reading parsha after parsha after parsha – that the parshah, the Torah, is not one item. It’s like several items that were just in scraps somewhere that were merged together into the best copy possible. And because it’s the word of God, you can’t throw pieces away.

So often, the story is told again and again from slightly different angles, sometimes out of order. And that’s okay, right? We just deal with that. But when we deal with that, we’re often dealing with contradictions. So she deals with the contradiction of – one minute, God is saying, “Go, Balaam, go to Balak.” And the next minute, God is mad. Well, had Balaam been paying attention before his eyes were opened, he would have realized there was a message to tell him that God had changed something in the game, and he just missed that, right? 

But the truth is that God just changed the story. But God changed the story simply because we had snippets that we pieced together. Those snippets are – that’s what our neurological experience is like. We’re all snippets. We have fragments of experiences. The experiences that you’re currently having now, where you’re integrating information that you get through your hearing and processing of language, and through seeing me wave my arms around, and from the rumbling – that’s the after-effect of what you ate in the kiddush… and just from the fact that you’re a little tired because you watched the late movie last night. And all these things, they’re all kind of merged together into something that has continuity in it. A continuity is – without that, without the tyranny of that organization, we have trouble.

But the question is, where does that tyranny come from? What does that have to do with the normal brain?

And so, today, I think we’re much more advanced than we’ve ever been, both in the theory of neuroscience and in the kind of measurements and manipulations that we can make. So that’s great. But we’re still standing in front of this sort of universe of complexity, that we really don’t even know exactly what questions to ask. It’s very similar to the situation in cosmic physics right now, where we have these new, amazing telescopes, like the Webb Telescope, and all sorts of devices that can measure gravitational waves and so on, and they can tell us something about what is 90% of the universe that we don’t know what it is. So we call it dark matter. And in the dark matter, there may be the secrets of everything about the universe that we currently don’t understand, which would completely change how we would think about it.

In neuroscience, I can’t even give you something as coherent as that as an explanation for what we don’t know.

Carol Caine: Thank you. That has never happened – that I’ve given a drash and the after-kiddush speaker referenced it. And you were talking about – it’s really different. I was imposing meaning where it’s not really there. And my personal opinion, intellectually, is that the Torah was put together by various authors, and the most logical reason for this bizarre “God changing God’s mind” – it might well be different texts that were mushed together. But if I got up and said that, it would be boring and it would be short, which I suppose that could be some benefit. 

But also, the second aspect of that is the meaning I’ve imposed on it before, either when I’ve given drashot about it or studied it with other people, also is added to sort of – even though I kind of know it’s a bunch of different texts, that also becomes, in my brain, “this is what it means.”

Neuroscience, Interpretation and Seeing

Ehud Isacoff: What I’m trying to get at is the thing that’s motivating you, which is the interpretation, or what I was calling “the tyranny of order,” or whatever – that is what it is to be a conscious being. And the reality is, I don’t think we can really limit it to things that we think as conscious, because we do things automatically that are also like that, that don’t involve consciousness in the way we think of it. And animals do it, various kinds of animals, down to the smallest animals. They do that. Everybody organizes the universe around them because of what we call priors. Right?

I don’t know how many of you remember reading Kant, but I thought that the worst thing I could ever do to one of my colleagues was accuse her of having stolen something from Kant, because in neuroscience, there are people who do philosophy of neuroscience, and we make sure to never let them near our faculty, right?

And the real neuroscientist, and the one I’m describing to you, Doris Tsao, who’s on our campus – who’s amazing, she’s one of the best neuroscientists in the world – I thought, “If I tell her about Kant, she’s going to tell me ‘Huh?’” But she said that’s exactly what it is. And what she was talking about – she studies vision. And what she is dealing with are two opposing views of how we organize our visual experience – our visual perception.

One is a very standard and kind of mechanical idea, that you have sensory information come in and you process it in all sorts of ways. I mentioned Hubel and Wiesel before. You recognize things that are lines, and that makes edges, and the edges come together in various angles, and you have texture and you have motion, and so on. And from that, you somehow get enough information to compose an image. And then to even be able to somehow recognize “this one is the same as this one. This one is different from that one,” and so on – somehow, you can gather that kind of information together.

But in philosophy, there’s – in Kant – a very strong idea that that’ll never work. It’s too complicated. There’s too much stuff out there, too many possibilities. And if you don’t have some pre-organized conception of what it is that’s there outside, you will never be able to interpret what’s coming in.

And so we call one of these “bottom-up” – “bottom-up” meaning “from the sensory world.” Another one is “top-down,” which is kind of like “from intelligence,” right? And the two meet at some point.

And in fact, what Doris does is she studies the timing of those two events. So, for example, for a long time, it was suspected – and she and others have shown – that when you have a visual experience, let’s say, of a famous face, or a face that is known to you, a recognizable face – when you’ve experienced that face, a certain part of your brain is active, and it’s that same part of your brain that’s active when you think of that face, not when you are just observing the face. And the memory of the face, when you call it up to memory, is active in the same part of the brain, but at a delay. So it’s as if the same part is encoding the face when it comes in and creating the pattern of the face when you’re conscious of it, meaning it’s both a site of experience and of memory. And actually, when you think about it, if it weren’t like that – it’s not clear how it could possibly be.

But the stuff that is coming top-down is – in science, we try to make things sound like our own, so it’s called the “prior.” Instead of saying “a priori” the way that Kant did, we just forget about Kant. Most people, most neuroscientists, have never heard of Kant, right? But we call it a prior, “prior” meaning some kind of initial nervous system organization – which, by the way, the nervous system gets organized in part by experience, and the experience of seeing faces.

So, it’s complicated. It gets wired up by experience, and yet then it’s programmed in a way to be able to interpret the world and impose on the world things that are recognizable and separate out the stuff that is not. Before, I was talking about narrative, and I’m talking about vision, but you can see how that would, in fact, be related, right – the imposition of order.

So, with that in mind, I just want to read you something. And anybody who is interested in neuroscience should read Oliver Sacks. This is a great story about vision and Oliver Sacks. I’ll just read you a little segment.

“I am a rather successful artist, just past 65 years of age. On January 2nd of this year I was driving my car and was hit by a small truck on the passenger side. When visiting the emergency room of a local hospital, I was told I had a concussion. While taking an eye examination, it was discovered that I was unable to distinguish letters of the alphabet. They all looked like Greek to me. And I couldn’t see color. Within days, I could distinguish the letters, and my vision became that of an eagle. I could see a worm a block away. But I’m absolutely colorblind.”

This guy’s a painter who becomes colorblind, not because of damage to the eye, but because of an insult to the brain. So, it’s kind of a mystery how that could happen – it’s still not explained. But the thing that’s amazing is that a few months later, he can still imagine color – he just can’t experience color.

And a few months after that, he can no longer imagine color. And a few months after that, things that are colored – he can kind of tell that there’s something wrong with them, but he can’t tell what it is. They make him kind of nauseated. And so, he changes his entire wardrobe, all the paintings in his house, all the colors of the walls and the furniture, to black and white. Then he feels okay and serene. But anytime he sees something that is not falling into simple black and white, it just makes him ill.

So, there’s at least a point where he thinks he can remember the experience of color without being able to see it. And at a certain point, even that goes away. And you can see that this is the right question to ask – “What is going on here?” We don’t have an answer to it yet. But we’d like to be able to understand that, because if we could, we’d understand what coding of information is, interpretation of the world, how we read the world, and how we store the world in our minds, and then when things get disrupted, what that would mean, and then maybe how to fix those disruptions.

And this is a very dramatic disruption. So even if we can never fix this one, if we can fix the kind of disruptions that you get cognitively with a neurodegenerative disease like Parkinson’s or Alzheimer’s and so on, we would obviously be doing something really amazing. So just to give you one neuroscience – actually, the world of neurology – is full of these fantastic stories that just make you all want to either read books or be a neuroscientist, or both.

Understanding the Brain: Physical Structure

Okay, so how do we understand how the brain is constructed? And what do we know now? Where are the challenges lying and what’s the outlook? I’d like to touch just a little on these things. And so remember, this is the “bait,” right? And by “bait,” [I mean] there’s no follow up, aside from getting you curious enough so that you tear into the local bookstores and start reading all sorts of neuroscience books.

So in the classical literature, the way you knew what different parts of the brain did was because there would be brain damage due to some kind of injury. War is really, unfortunately, very good for that. 

Typically, the thing about some shrapnel or a small object like a bullet is that it may make damage in a very local place. If you have damage brain-wide, you just don’t know what to say. But if it’s damaged in a very local place, you can try to say, “Well, maybe the deficiency I currently have is associated with the function of the damaged part of the brain.”

Wilder Penfield, in Montreal, he got a lot of patients from the war in the early part of the Second World War, and when they came, they had damage to specific parts of their brain. He would often have to do surgeries on them. And then those surgeries continued afterwards when he worked on epileptic patients. And he got into the habit, not simply of trying to say, “Where’s the hole?”, but when the brain was open to do the surgery, he would record from different parts of the brain, and he would ask the patient to do something – show him a picture, say a word, move an arm, or something. And he would then also sometimes stimulate and see “Was there a response in you?” Could you say, “Oh, that changes how I see, or that makes my finger move,” or something?

And from doing that, he was able to create something called the sensory and motor homunculi – homunculus is the singular – meaning the little person inside you, inside your brain, who is the brain map of everything that part of the brain is connected to. The thing that’s remarkable about it is that there’s an orderly topographical map, [as] we call it. So that if you go across the motor cortex, for example, which is kind of a strip, right, or the sensory cortex from one end to the other, you actually are going up, down the body of the person in an orderly manner. So, it’s really a map, although its shape is different. And you can imagine, you’re very sensitive in your fingers. The motions that we make in our mouths when we talk are extremely complex. There are many, many muscles, in the mouth, lips, larynx, to produce speech – those are very overrepresented. The homunculus looks like it has very big lips and very big cheeks that are very sensitive, and a very short back because you can’t tell where somebody’s poking you in the back, and so on. So that gave you just an initial map. And the question is, can we take that further to understand how the activity of neurons in those parts actually mediate experience or behavior?

We all have heard about MRI. Many of us maybe have taken it and they’re still feeling the “ding, ding, ding, ding, ding, ding…” MRIs can be used either for imaging the structure of the brain, but when you do it under certain ways, what you’re measuring is the blood flow in the brain, and the blood flow gets higher to regions where the neurons are active. And so, even though you’re not actually watching the neural activity, you’re watching where the neurons have been active, and because you have so many little capillaries, you can localize things actually very nicely in very, very small regions if you have a very good functional MRI – we have the best one in the world just a few blocks away from here. 

Understanding the Brain: Language

So, in functional MRI, you can actually visualize activity in the brain. And I’m going to read you something. This is a student who was at Berkeley who’s moved to University of Texas, and we’re trying to recruit him back to Berkeley. And using functional MRI, he was trying to develop some very fancy computer, computational, interpretations using artificial intelligence in order to read the activity and see whether he could turn what he sees in activity into interpretable language.

Now, the idea was that some people who are unable to communicate – if you could only read the activity in their brains, you’d know what they’re trying to say. So the beautiful functional MRI is that you’re recording from the entire brain – so, any part of the brain that may be relevant is something you can pay attention to.

And so here I’m going to read you the original transcript. They hear things from the Moth Radio Hour. I don’t know if you ever listen to that, right – it’s sort of these nice stories. And so the subject – it could be a normal subject or somebody who’s injured in some way, they hear a story told, and they hear many, many stories. That gives all the information about how the story corresponds to the recording of the brain activity. And then you put this through a very heavy computational model, and now you let them hear a new story, except you, the investigator, don’t know what that story is. And the question is, from the brain activity that’s elicited by that, can you now recreate the story you haven’t heard, just by looking in their brains? 

And the answer is: you can if you’ve trained a lot, but only within the same subject, because our brains, from person to person, structurally vary and the patterns of activity vary so you can’t take my brain pattern, learn a lot about it, and then try to read your brain from it. But to read my brain, you can, to some degree, just listen to this. Original transcript: “I got up from the air mattress and pressed my face against the glass of the bedroom window, expecting to see eyes staring back at me, but instead finding only darkness.”

The decoded activity – that’s the stimulus, which the investigator doesn’t know what it is, but the investigator decodes the brain activity and produces this sentence: “I just continued to walk up to the window and open the glass. I stood on my toes and peered out. I didn’t see anything and looked up again. I saw nothing.”

Now, you wouldn’t say that’s perfect, right? The grammar is different, but it captured a lot about the meaning, which is really remarkable. Here’s another example – that’s where the patient just imagines a sentence, right? And then the patient writes it down. And then the question is, can you read that? This is the patients or the participant’s sentence that they created: “I look for a message from my wife saying that she changed her mind and that she was coming back.” This is the decoded version: “To see her, for some reason, I thought she would come to me and say, she misses me.” Not perfect, but it captures something about the meaning.

And so you say, which part of the brain – is there a particular part of the brain, like language parts of the brain that should be doing this? All the parts of the brain are contributing to this. All parts, except for the part that produces speech, which is weird. That’s the language part. But the part that produces speech cares about grammar and not meaning.

There’s a neuroscientist who’s an engineer at Berkeley who trained with Eddie Chang at UCSF. Those of you who a couple of years ago would be driving through the city and see a big bus with a smiling scientist on it, that was Eddie Chang.

So, they came up with this thing where they do epileptic patient surgeries, and just like Penfield, they insert electrodes, except not just single electrodes now, arrays of electrodes. Many, many electrodes. And they ask patients to do things that are linguistic, and this is what they see. This is what they can recreate. So here: “you should have let me do the talking.” Right? That’s the real thing. The interpreted one is, “you should have let me do the talking.” “I think I need a little air”/ “I think I need a little air.”

“Is she a friend of yours?”/“I see afraid of yours.”

“How’s your cold?”/“You’re old.”

You can see already that some of it sounds like the errors are phonetic errors. So, this area is important for phonetics particularly, but also for grammar. But you can see it’s getting the grammatical structure better, and the meaning, if it were obeying the meaning, it would not be making these kinds of mistakes – it would be making other kinds of mistakes. 

So, this is quite remarkable. And the idea behind these is that maybe you can produce, actually, a speech prosthetic for someone who’s had brain damage that prevents them from actually being able to control their mouth – they want to be able to speak. Now, maybe there’s a way to be able to enable that.

This is one of the powers of AI. But I know many of you are probably a little nervous about AI. HI, fine, because it’s okay, it makes for good drash. But AI is kind of weird, and is maybe nervous-making. We already know that it has these great limitations. For example, can you imagine creating a ChatGPT based on all texts written before Shakespeare that would come up with Shakespeare? No. Or come up with Kant, for that matter, or Proust, or Joyce? No.

But, given that writing has changed, what about the future great writers? Is it going to become – because, of course, Shakespeare is, in some way, gathering everything that’s been written before that he’s experienced, and turning that, and many other things, into his beautiful plays. Is it [at] some point that we, as humans, are absorbing many, many things that the machines can also absorb and maybe create? So are we, at some point, going to get to a situation where it’s not just like the only way for these monkeys to write Shakespeare is that they have to have a hundred of them, and they have to have infinite time? (Because by definition, even a single monkey, given infinite time, would come up with Shakespeare, not that he’d appreciate it – or she.)

Understanding the Brain: Rehearsal and (Dis)inhibition

So, I have two questions.  One, if you could just talk about mental rehearsal, and the other is if you could just talk about the idea of inhibition and disinhibition in the role of neurology.

Okay, so – in terms of rehearsal and inhibition, let me first say this. In our brains there are about 100 billion neurons, and they make, on average, a thousand connections each. That’s 100 trillion connections. At those connections, there are many, many different kinds of neurotransmitter – dozens and dozens. Each one of those has multiple receptors that it acts on. It’s extremely complicated, and often the same cell is releasing, at the same synapse, more than one transmitter type and acting on multiple receptor types, both in the follower cell and on itself, and in neighboring cells that are parallel to it – super-complex. But we break things up into excitation and inhibition because there is that, right? And often we think, “Okay, the information is really going through excitatory systems, and what the inhibitory systems do is they in some way sculpt the excitatory systems.” 

But the truth is that the strongest signals in the brain are inhibitory. And the way that one version that makes us know this is that for anyone who has Parkinson’s, then, hopefully the L-dopa, when the Parkinson’s gets severe, is something that can work. The dopaminergic neurons are dying off, but the L-dopa is a precursor for dopamine, and the remaining dopaminergic neurons take it up and then can release the dopamine at larger levels than they usually do and make up the deficit of there being fewer dopamine neurons that are releasing.

But that eventually stops working. And under severe conditions, one of the things you do is you go to UCSF, which is the best place in the world for doing this, and you go to the clinic there, where they do deep-brain stimulation and implants of electrodes. They’re stimulating in your brain, and so you naturally think they’re stimulating your brain, and that’s going to be activating some neurons, and if you activate the right ones, maybe you’re doing something.

But the reality is, you’ve taken a wire and you’re sticking a wire into the brain. It’s activating many, many neurons all around it. What is the dominant effect? The dominant effect is inhibition. So even though you’re stimulating, you’re actually inhibiting that part of the brain. And that’s simply because inhibition is so strong. So, inhibition, as just a sign and power in the brain is incredibly dominant over excitation.

But when you say “disinhibition,” you are probably thinking at the level of, more, brain perception and psychology. And there it really depends on what circuit you’re thinking about. It’s standard that if you have damage to the frontal lobe, then people become disinhibited. And the reason is that what the frontal lobe is doing is it’s helping with executive function that is doing part of what I started with, which is the tyranny of the narrative. It’s making everything quiet down so you can focus and control, in a controlled way, on one thing at a time.

And so the other version of loss of that is with psychosis. It’s a very different class of thing, because now you’re hallucinating. But in a sense, it’s the same thing, which is the brain is running freely. You have no way of suppressing the activity. 

And it is thought, actually, with autism, that in a very different way, again, it’s the same thing. There’s a change in the balance between excitation and inhibition, with a loss more on the inhibition side. And that loss actually produces kind of an overstimulation, which is what people with autism shy away from, which is why they try to find simple situations – the most terrifying ones being the social situations [because] we have to pick up on so many different cues, and so on. 

Of course, the way that the brain normally works is not that whole regions of the brain are activated simultaneously – every cell in a very large region – or inhibited simultaneously. They work in patterns, and very few cells, very few neurons, are active at any one time. We call that a “sparse code,” and it’s the sparseness of that code that enables us to have so much different kinds of information in our brains, because very few neurons participate in any one of them. And even though there’s some overlap, it’s rare.

And so, you can actually capture, you can remember, you can experience, and you can control, many, many different things – even with the same part of the brain –  because very small groups of cells are responsible.

Optogenetics

When they do these kinds of manipulations, that’s not possible yet, except it is something that we hope will become possible with this new field called optogenetics. In optogenetics, instead of using a big wire, what you try to do is individually control many, many cells. In fact, one of the things that we hope to discover soon, although it’s been actually not established yet, is if you record from part of the brain with functional MRI or with electrodes, the way I was describing from Eddie Chang and Gopala, and you see certain activity patterns, and you say those activity patterns happen every time that you do something – that must mean that that’s the code for that something, have you understood what the system is actually doing?

Well, the problem is that many other parts of the brain are also active. So is this the whole of it? You don’t know yet. The way that you could, as an engineer, fantasize about testing it is, you say, “If I could turn those same neurons on in that same pattern – this group, and then that group, and then that group – then I should be able to produce the same sensory experience or behavior output or something.” And at that point, you’d be able to account for it.

Hillel Adesnik is a person who is trying to do this – he’s a professor here. We’ve not been able to address large enough numbers of the brain, areas of the brain, and large enough numbers of neurons yet, to have that succeed. So that’s still fleeting. We actually don’t know. We don’t know yet.

So it’s basically every organism, from the simplest organism to the most complex, in terms of the nervous system and whatever. And you may think of tyranny as being something kind of purposeful, right? Because we think of tyrants as being – they do it on purpose. But this is a tyranny that is just built into how the system works. But it has levers by which it can adjust itself in order to ensure that the operation is good. And when the operation breaks down, then you have problems.

So what I’m describing to you in terms of visual perception – there’s a beautiful example of two vases that are next to you. You’re familiar with this, right? Two vases, just black vases on a white background. They’re next to each other. And if you look at it one way, it looks like two faces facing each other. And if you look at it another way, you’re just looking at two vases. Right. And it’s vases and faces. So, yeah, my mother could not distinguish between the interpretation, saying those two things.

But you never see both at the same time. Your brain switches back and forth between these. It turns out that because of that, that’s one of the best tools for understanding visual perception and consciousness of vision. Because a person, a subject – which could be an animal or it could be a human – can actually, with a push of a button, indicate when they are observing the faces and when they’re observing the vases, and you can look at their brain activity. It’s really, really fascinating. But that order is imposed from the inside.

One of the most remarkable versions of that, which is much earlier in the visual system, was – forever, people have been trying to record, for many, many neurons to see the general patterns of activity. And Amiram Grinvald, who was a professor at Weizmann, invented these chemical dyes that got brighter when neurons were active. So he could open up the brain of a monkey and pour the dye on, and then turn all the lights off on the room and just shine one color of light, and the dyes would fluoresce and the fluorescence’s strength, the brightness, would change when the neurons were active. And so he did this many years after Hubel and Wiesel, and he saw exactly what they saw. They saw with one electrode recording from one neuron at a time; he saw the entire brain pattern instantaneously – which is that you have the part of your visual system that sees this, [which] is right next to the one that sees this, which is right next to one that sees this, at every angle, all for that one part of space. 

And then you go to the next part of space, and it also has these things, and it makes a shape of pinwheels on the brain – he saw that whole thing when he would give the monkey a visual stimulus.

And then one day, his student, his postdoc, I think, made him really mad because he just turned the light off in the middle of an experiment, and the experiment kept running. And Amiram (who was also wearing a leather jacket, but not only 4 feet tall) started yelling at the guy. And Amiram, when he started yelling, it would take an hour. So an hour later, Amiram decides, “Okay, I’ve had enough yelling, let’s turn the lights back on.” 

But the postdoc, without telling him, had analyzed what happened during that hour and discovered the same patterns of activity that were there when they were giving visual stimuli were there in the dark, when there was no visual stimulus. There’s a wiring in the brain that makes neurons active at the same time, already such that neurons tend to be active at the same time – the same ones that would be activated at the same time by visual stimulus. There’s another version of the same thing.

The brain is designed to impose this order. But that version of tyranny, of course, is a very different meaning of tyranny than something that you think of more at the conscious level, the very high level. But it operates at every level.

Genes and Neurons

Okay, so I’ll tell you one interconnected story. I mentioned optogenetics. It’s a beautiful example of how to study the nervous system or biology in general, because the reason it’s so beautiful is that it reminds us how we’re all related evolutionarily – meaning, every organism, you can take a gene from one organism and take that gene and put it into another organism, and it will still function in the same way as it did in the original organism.

Now, mind you, in many cases, the genes that we have, and the genes that mice have and that flies have – it’s the same gene. Evolutionarily, it just got changed a little, so that wouldn’t be so surprising. But there are genes that are present in these single-cell microbes that live in saltwater, in Egypt, for example, or in the Dead Sea or in Salt Lake, or who live in other weird environments. And they have genes that make proteins that are signaling proteins in the cell and that are sensitive to light. These single-cell organisms use them in order to be able to detect where light is coming from and to go toward it, because then they can photosynthesize for energy and use that energy, then, to metabolize.

So for them, they use it for one purpose. We take that same gene, put it into a neuron, shine light on that neuron, that neuron fires an action potential. So, that’s really amazing. That’s how optogenetics works. And there are versions of this that are excitatory, and for you, versions that are inhibitory. Those are great, but those are imposing something on the nervous system the same way that an electrode will. The electrode is not – it’s just overriding what the brain would normally be doing. This is overriding. It tells you a lot, but it’s not the native thing. So, what we want to do is develop a version of this that would actually be controlling, with light, the natural signaling systems of the cell.

And we focused on several different kinds of systems, one of them being dopamine. So dopamine, as we’ve said, is relevant to movement disorders, but it’s relevant to many, many other things – sleep, appetite, reward, aggression, and so on. And the problem with the medications that we take for it is that they boost dopamine signaling everywhere in the nervous system, but when Parkinson’s hits, the neurons that make dopamine don’t die off at the same rate. Some die off earlier than others. And so you get an imbalance in the dopamine signaling. And then taking the drug then over-makes it – too much dopamine signaling in some areas, too little in other areas. It could be a real problem.

We would love to be able to have something where you can just activate the dopamine receptors where you need to. You can’t take dopamine as a drug. That’s too dangerous. And you can’t inject dopamine and get it to work when you want it to, because it’ll just go everywhere. So we developed a way that’s basically – imagine a dopamine molecule that’s on a string. The dopamine is inert. It can’t do anything until you shine light at it. You shine one color of light, it becomes active. You shine another color of light – it’s inactive. And then you target this thing just to the cells you’re interested in, because in the cells, there’s a protein that grabs that thing at the opposite end of the dopamine.

And so you can light-control dopamine – maybe just in the part of the brain that’s important for, let’s say, the movement disorders, and not touch the other parts. So, this was a hope – we developed it. It works beautifully in animals – beautifully.

And then I went to talk to my buddy Phil Starr, who, at UCSF, is one of the lead people putting in electrodes for DBS  (deep-brain stimulation). And I said, “This is what you should be doing.” And he said, “What are you, nuts? You’re telling me that the thing that I do – that has all sorts of problems but works, and it’s just a wire – is going to be replaced by me sticking into this person’s head a foreign gene? I still have to stick something in there, but now it’s not a wire, it’s a fiber-optic cable. And I have to supply the chemical, which we call the photoswitch, this dopamine that’s light-sensitive – all three of those things. And you want me to do that? You’re nuts. I still think it’s a good idea, but it’s not going to happen.”

So, what does a scientist do when crushed? You do basic science. One question that we have, when we study the nervous system is: how do we do anything? When we have a volitional act, how do we learn to do that? You know that if you’re learning how to play the piano, or to ski, or to play baseball, or whatever, you have to kind of repeat the process, and so on. The memory part of the brain that’s involved in that is different from the cognitive part. It’s in the cerebellum, rather than in the hippocampus. And so somehow that all happens, and it’s nice, but that tells you the brain region, it just doesn’t tell you how the operation works.

So basically, what is happening – if you think about it, this is what must be happening, first of all, and it is – is that your cortex, which is the place that you do volition, has all these neurons, and in the motor cortex, the neurons are the ones that control motor behavior. And those neurons are just active. They’re active in various combinations, kind of randomly.

So the question is, can you plug that random activation into something that can produce what you want? So, for example, let’s say somebody has a spinal injury and can no longer move their arm. Can you make a prosthetic arm that then your mind can control, your brain can control?

And so in recent years, that’s been developed; it’s called Brain-Machine Interface. But it doesn’t have to be so complicated. It really is just: you record from some neurons, and whenever a pattern of neurons fires in a certain way, you encourage it in some way, and then you make that be the thing that’s controlling, initially a cursor on a screen, but later, the artificial limb. It’s a complicated artificial limb, you have to control it in many, many different ways, so it takes many neurons to do different kinds of operations. But little by little, you can learn this. And the reward that you get that encourages learning this pattern is: you’re conscious, you want to do it – you get the pleasure of that.

First, these experiments were done in animals. So, animals, you can’t encourage them because you’re saying, “Yay, you’re doing the right thing.” So what you do is you take an animal that’s thirsty, and you give it a squirt of water. Then, alternatively, what you can do is you can give it a shot of dopamine. The dopamine can do it.

Our system that I just described to you – that is never going to work clinically for Parkinson’s, according to Phil [Starr] – actually works to select groups of cells that are taken randomly. And actually, they’re not even doing anything. They’re not plugged [in] to behavior. This is really showing you how patterns of neurons can be connected, established, and reinforced at the most fundamental level by a dopamine’s activation of a particular receptor in particular cells in the middle of the brain that makes the neurons in the cortex actually, now, learn the new operation and perform it.

So that’s telling us a lot about how, normally, reinforcement learning happens. It’s really powerful because it’s unplugged from, actually, the behavior. So that many things that otherwise are complicated aren’t there. So, we’re very excited about that. Phil is not. Phil has no interest in that whatsoever.

The Aging Brain

In the aging brain, do we lose the capacity of accessing the priors? And why can’t we access information that is there, but it doesn’t come forward?

Ehud Isacoff: Okay, so I think that’s a really interesting question. There is no indication that the priors break down in the aging brain, but the pathways get disrupted, and it is still unclear whether, when you’re searching for a word and struggling, or a concept, or you get stuck cognitively in the middle of something complex, and you just can’t work your way out of it – how much of that is due to a storage issue, something that was stored, as the storage is not good anymore, and it’s fallen apart in the storage site? And how much of it is because you cannot access it as readily as before?

 The experience, which you don’t have to be very aging to start getting that experience, is, “I know it’s there. I know what it is. I know her name. I just can’t come up with it.” And then moment later, you come up with it, you feel like. It feels like more an access problem than a storage problem. But that’s not necessarily true, that experience, because the storage is not necessarily just the unique area. It’s distributed, and access is a way of assembling that, and storage and assembly are, in a way, inter-wound. So it’s actually very hard. I don’t think we can answer that question yet. I don’t think we can answer it. It definitely is an issue, right. With all cognitive decline, that’s an issue – which is, I think, really fascinating, right?

AI and HI (Human Intelligence)

So, I did promise to say something just about AI, right, so we’ll just finish with this. And that is: we think of this awesome power of these things like ChatGPT. I don’t know how many of you have actually tried to use it – Yes, okay, so some of you  When you hear it described, it sounds fantastic, and disheartening, if you’re a teacher and you’re afraid, “who wrote that?” And so on. Right? Now when you publish a scientific article, you have to acknowledge ChatGPT as a co-author if ChatGPT wrote any part of your thing. And by the way, you can’t get away with it, they have a way of figuring it out, so you have to admit it.

So ChatGPT is run by 3,000 super-fast computers called graphic processing units, GPU’s. This is why Nvidia, the company which makes these, right, is the richest company in the world now, right? That uses – the 30,000 GPUs use 520 million watts of power per hour.

Our brain – the brain that you’ve been using to listen to all this – I know you feel it’s not operating the way it used to. It’s not quite as good as when you were less than 40 for holding many, many things in memory and so on, right? But it’s still an amazing thing – 30 watts, as opposed to 522 million watts. Now, mind you, that’s ChatGPT. That’s in the service of everybody who’s a client. There are many, many clients out there. But still, we’re going to have a major problem of energy. We already have a problem of energy and greenhouse warming. It’s going to be much worse because of these things that are so computationally heavy. The reason is that the way that the systems work is that they take very simple things that are actually based on the nervous system, like a combination of layers of different neuron-like things (although they’re nothing like neurons, they’re very stupid, right?) And they have a few layers and they have a few elements. They talk to one another in these mysterious kinds of ways. Nobody really knows how they work. They gobble in all sorts of data, and then they can construct things that are basically synthetic from all the data out there. That’s a very wasteful way to work, but a computer, a massive computer or network, can do that. We do that with our little 30-watt bulbs in here, with our 100 billion neurons and 100 trillion synaptic connections, that every one of us has – from the person who is doing this kind of work, to that kind of work, everything – we each have that. And of course, many of us have experienced that, whether it’s ChatGPT, or any other thing, that it never finds the thing you’re looking for.

We can only comfort ourselves with that for now. But it’ll get better, the systems will be much more powerful, and we have to face up to it, and it’s actually not necessarily a bad thing, although it can be a very wasteful thing. The thing to look forward to in the future is that AI will help us understand ourselves and our brains in a way that currently is just very, very difficult, and that the systems that do AI will operate more intelligently, more like human intelligence than artificial intelligence, and hopefully will be designed in such a way that they won’t do some of the terrible things that we fear from computers – although, seeing what some of the humans are like, I don’t know.

The problem is that humans program the AI.

Ehud Isacoff: The fear is that when they start programming themselves. But anyway, thank you. Thank you very much.

Rena Dorph: Thank you all for being here, and thank you for your generous amount of time and all your shared wisdom and knowledge. We so appreciate it. Thanks for coming, and Shabbat Shalom.