0:00 um what I'll be talking about today, I'm a neuroscientist at at MIT and our lab has been um examining the role of electric field effects in uh in in um cognition and consciousness and it's electric fields are something that's been um it's a signal it's a mechanism that's been underststudied in neuroscience and we're trying to change that. So let me dive right in. So I think the the best way to tell you how our think where our thinking about the brain is right now is to tell you where we came from. So I'm going to give you a little bit of an historical overview of our how our thinking in the brain has changed in the past I don't know hundred years and then you'll you'll you'll have a better understanding of of where where we why we're thinking the way we do now. So sorry let me get my screen going.
0:46 There we go. What is thought? Well we know thought is composed of neurons. Neurons are brain cells. They connect to one another at junctions, connections called synapses. And they signal each other by brief electrical impulses that we call spikes. And I'll be referring to spikes for the rest of my talk in addition to electric field. But spikes are the electrical impulses from individual neurons like individual voices in a large crowd. And the idea is that the neurons signal one another by by um sending these spikes along uh um axons and dendrites. the connections and across synapses and that's what's illustrated here spiking activity the brain signaling all taking place by these spiking moving down the brain's connections and the brains have a lot of
1:31 this stuff your cortex alone I'll be f since I'm interested in cognition and consciousness I'm uh going to focus mainly on the cortex today the cortex alone which is where a lot of the higher thinking in your brain goes on your cortex alone contains 20 billion neurons and they form uh 10 to the 14 synaptic connections. I don't know what number 10 to the 14 is. Let's just call it a gazillion. And at first, like um before the mid 20th century, so late 1800s, early early uh uh early 1900s, um neurons were poorly understood and sometimes they were regarded as like mere build building blocks or scaffolding, right? thought that that neurons were like the the somehow the stuff that
2:19 constructs the brain, but they didn't seem to have any distinct function. And why is that? Well, you know, there's limitations to technology. Until about the 1950s, we didn't have the technology to eased on the spikes from individual neurons. And also, if you're going to study something as complex as the brain, you got to start somewhere. You don't dive into the details immediately. You build up from the details by starting simple. And the simplest place to start is the brain's gross anatomy. But by the 1950s, we developed a technology to study electrical spiking of individual neurons. And a major discovery was that neurons are not bricks or scaffolding. They're little information processing units. And this is based on the Nobel prize the the watershed was the Nobel Prize winning work of Hub and Visel where
3:05 what's shown here is for example they were recording from individual neurons in the visual cortex which is at the very back of your brain. It's sort of the first v stage of visual processing goes on in your cortex and they showed the subject different stimuli different pictures different features on on a screen. In this case, this individual neuron they're recording from happened to give off spikes at the site of this leftleaning line appearing at this particular location on the screen. So this neuron, this individual neuron is like a geiger counter for this one visual feature detecting that one visual feature. And other neurons detect detected other features like right leaning lines or squares or colors. And the idea was that other neurons spike at different things. So they detect other
3:51 things. they're sending different messages and this led to the dominant model in the late 20th century of of the brain and that is connectionism. Uh the cortex was thought to be wired like a giant telegraph system that combines signals. So the spikes travel down these telegraph like wires and neurons signal one another. These little information processing units signal one another like a giant telegraph system. And this is captured here. If I get this right, here's here's the primary visual cortex at the back of the brain. And here's the um edge detectors like I showed you earlier at this first stage of visual processing. There's ner here's a visual scene here of these this kind of bent paper clip. And the idea was
4:37 that individual neurons back in primary visual cortex detect different features of this uh scene. In this case, the simplest case is the simple line segments of different orientations. And then those edge detectors combine to a second level of processing where they combine their activity to detectors of longer edges. And then those longer edge detectors combine their signaling and you get corner detectors. Then you combine more and more and eventually you get neurons that are that are only spike the whole images or whole things like faces or or hands and stuff stuff like that. And because there's 20 billion neurons in your cortex, the idea was that if you combine combine, combine enough, you eventually get neurons and
5:22 circuits specialized for peace, love, and understanding. That's the way we thought about the brain in the 20th century. This this connectionism and this combining of signals into higher levels of specialization and complexity. And this may be familiar to you because this is how AI kind of works. Large language models are trained on large data sets and they adjust their connections, their associations to capture statistical patterns in the data. This is where AI got this idea from from the from how we thought the brain worked. But starting about the um turn of this century, a discovery challenged connectionism. The discovery of multifunctional neurons. These neurons rather than being selective for one thing indicated by these single colors here, these neurons in cortex, they
6:09 combined different features in seemingly ways that didn't make sense from a from a neuron having a single function. Their spiking of these neurons reflected complex nonlinear combinations of different things in different situations. Okay, as a result, these neurons were multifunctional. They didn't just do one thing and send one message. They sent different message in different situations. And what that means is that cortex is a web of overlapping multifunctional networks that constantly interact. Not a collection of specialized parts like we thought or specialized circuits. So the cortex didn't seem to be specialized like this but we like we thought the 20th century. It was more like this. a collection of overlapping networks or individual neurons participate in
6:56 multiple different networks and they change their network participation in different situations. Now I'm just for a moment I'm going to take a little bit of a detour here and explain to you a little bit about the property of mix activity because it really is key. Your cortex is full of these neurons and they form a core computational strategy that of your cortex. So here's an example of mixed selectivity neurons. First, we'll start with a 20th century model, the specialized neurons. And here's a simple task. You have the subject. In our case, we use non-human primates, monkeys. We have the subject. I hope you guys can see my cursor. Can you give me a thumbs up if you can? Good. Um, in this simple task, the animal sees two objects in a row. This case, a butterfly and a and a um and a um padlock. Then after a short
7:43 memory delay, they're given three pictures sim three objects simultaneously and they have to demonstrate their memory for the order. These two pictures are in their order by choosing them in in sequence. So first choose the butterfly, then choose the padlock. That's uh that's because that's what the the animal just saw. Now according to the 20th century model, what should happen is you should find some neurons that are selective. They spike selectively to different objects. So here's a neuron that spikes the most to the padlock and the least to the this is neural activity spiking activity on the x- axis time on the x- axis. The gray is when you're presenting these objects. And according to 20th century model, you have a neuron. This is a single neuron here that responds most to the padlock responds um spikes at a
8:31 medium level to the flower and the least to the butterfly. And this neuron will always give the same message every time it sees a butterfly or a padlock or a flower. Never changes. And you might find some other neurons that just keep track of the order of the objects. They're pure some neurons that will respond more when things appear first and less when things appear second. Other neurons that do the opposite. That's what the 20th century model predicted. We did this experiment and what we found instead was these mixed selectivity multifunctional neurons. Many experiments have been done since in the past 20 years or so. And what you find is you find neurons are multipplexing combinations of factors. So this neuron we I've illustrated here is responding most to the padlock and least to the butterfly. Most spikes to
9:17 the padlock and least spikes to the butterfly when the but when when those objects appear first. But when when those objects appear second, they show different patterns of spiking. Now this neuron is spiking most to the butterfly and least to the padlock. And you see different combinations of these factors, these neurons seemingly giving different messages in diff in different contexts. And because they spike at different objects in different orders, different contexts, these neurons are doing this multipplexing of different factors. And it this gives them flexibility. It means they're spiking is context dependent. These neurons are doing different functions in different contexts. And if you had one word to describe cognition, it would be context dependent behavior. flexibility. Our brains don't do the same thing every time to the same input.
10:04 We're not like thermostats. We behave differently in restaurants than we might at a picnic. We we we we u behave differently when we're sitting down to a lecture versus a concert hall. Our our behavior is all context dependent. And that's what cognition is all about. And these neurons seem to be the embodiment of that cognitive flexibility, that context dependent behavior of cognition. And this skin this makes them multifunctional. And what my colleagues Madi Regati and Safano Fusi did using a data collected by Melissa Warden who's in my laboratory is they built computational models showing this mixed selectivity property is critical for cognition. It gives the brain more computational horsepower. They form a nonlinear highdimensional computational
10:50 space that allows the brain to solve more complex problems. And their flexibility allows the brain to learn things really quickly. And they increase the storage capacity of the brain because rather than all your memories and knowledge being stored in a few specialized neurons, they're distributed across many multifunctional neurons which makes the brain pretty much capacity unlimited. But this leads us to our biggest challenge. How does the brain control such a complex system with overlapping networks and these multifunctional mix selectivity neurons that change their membership in different networks depending on the situation at hand? How can the brain do that? And this is a challenge because we are again our brains don't just respond to inputs to
11:36 the out from the outside world. We have control over our own brains. We take charge of our own thoughts. We don't don't just react to inputs like a thermostat or like AI does. by way that's what AI does. AI just responds to inputs. It has no internal mild internal monologue like we do. We act differently depending on the situation. This is the context dependent behavior I was just telling you about. And we make goals and set plans. I can choose what goal I want to reach in the future and make plans to get there and then ignore things in the outside world that count that that counteract that plan in order to get to my goal. And we do this flexibly to navigate a fast-paced world. It seems impossibly complex to build a control system that works by connectionism. Connectionism may work
12:22 fine for storing information in the brain, for baking information into the brain, but in order to do all this computation allows us to take control of our own thoughts. How does that work if the brain only works by connections, right? It would require on the-ly subsecond control of gazillions of connections, gazillions of synapses between a gazillion neurons. And that just seems computationally intractable. So what's the solution? A number of us including our laboratory have come to the conclusion that that the electric fields are the solution to this problem. What we call brain waves. Brain waves are rhythmic fluctuations in the electric field surrounding neurons. When neurons spike, of course, they're existing in this electrical environment. When they spike, they cause ripples in the electrical fields around them. and
13:08 you get enough spiking, you get enough ripples in the rhythmic fluctuations in the electric fields and they turn around and they influence the spiking. A lot of you may be familiar with with brain waves. There's here's a recording here on the left is a cartoon showing EEG scalp recordings from the human scalp. And you get you get a range of fre the squiggly lines you see on an EEG reflect a summation of of um a variet a wide band of of of of rhythmic oscillations in the brain and they're grouped into frequency bands loosely associated with different behavioral cognitive states. So when your brain is giving high frequencies like 30 hertz and above gamma means you're awake and alert. Next down beta I'll talk about beta. This is when your mind is concentrating. when your brain is concentrating. I'll talk about that in a moment. And also alpha.
13:55 And then you get down to lower frequencies and now you're more relaxed. Eventually you get to delta low frequencies and that's when you're sleeping or anesthetize, which I'll also show you in a minute. So we know that these these um oscillations, these electric field oscillations, brain waves correlate pretty well with cognitive state and consciousness. And they were the first electrical signal measured from the brain before we could had the technology record individual neurons. But when the focus in the mid in the in uh mid 20th century shifted to neurons and connectism, brain waves were dismissed as non-functional. Kind of like the humming of a car engine. They don't make the engine run. These these electrical oscillations, they don't make the brain run. There is a byproduct of of the of the um of the brain running.
14:41 Now um first of all anybody who understands basic electromagnetic theory knows that can't be true. If the neurons exist in this electrical environment and their spiking is changing that electrical environment it's going to have an influence on on um sp on on spiking. But also I'm here to say that we need to banish the word epiphenomenon from neuroscience specifically in science in general because whenever I hear the word epiphenomenon that's what people said about this mean by non-functional. Brain waves are epifenomenal. When I hear the word epiphen whenever someone says epiphenomenon I hear doesn't fit my theory. Okay? You know be and if your theory is not explaining all the observations then the your theory is falling short. It's not that the observations are irrelevant. Saying the
15:27 observations are things are non-functional or epifenomenal is an eg crutch to dismiss ideas that don't fit your hypothesis. So we got to banish that from from from neuroscience particularly because we've only scratched the surface in how the brain works. And again we now know this is not true. One way we know this is not true is work I've done uh in my my lab with in collaboration with Emmy Brown's laboratory where we're studying how general anesthesia affects the brain. Now it may be um a little disturbing you know because anesthesia has been around since I believe 1869. Etherdome over here at MGH. I believe it's 1869 around there. Um where the first demonstration of med medical anesthesia was given. Um there's been this no doctors for all
16:14 this time doctors have really not known why anesthesia makes you unconscious. They just know it works. It's like a lot of things in medicine. It works. We don't know why. So we use it. And it was thought that anesthesia just kind of shut off your cortex. May just zero out your cortex. Hit the off button at your cortex. And we found in starting these studies about 10 years ago, this is absolutely not true. What general anesthesia does, it doesn't turn off your cortex. It it dramatically alters the brain wave dynamics. So here's a recording of what's called local field potentials. These are essentially electric field recordings from inside the the brain, inside the skull. And that's we're seeing these colored these are uh four different areas of the cortex. These two areas in the top are the frontal cortex, the front of your brain, the executive part of your brain. This is the parietal cortex in the
17:00 middle of your cortex and back here STG is is primary auditory cortex. So sensory cortex. So we're we're bracketing sensory to frontal cortex here. And these are electric field recordings on the top and the bottom are the spikes of individual neurons recorded from those same electrodes. And what you see in the conscious awake state is a lot of high frequency chatter. A lot of stuff is going on. There's a lot of uh waves superimposed on top of waves as all these waves do different things. So you see a lot of high frequency low amplitude chatter and a lot of spiking going on. It's very very very busy and complicated complex. Then once we anesthetize the animal with a commonly used anesthetic called propall we then find that that this shifts the brain to lower frequency a
17:45 lower frequency band to to to 30 hertz and to mainly 20 hertz and below mainly centered around delta one time a second. So your brain shifts to these low frequency oscillations. You can see here, here's the high frequency chatter when you're conscious. Now the the electric fields are now doing this slow frequency almost like a slowwave sleep kind of thing. And the spiking activity in different parts of cortex are now lining up to these slowwave dynamics. They were lined up to the high frequency stuff here. We just couldn't sort them out at least at first glance because there's a lot of fre lower and higher frequencies embedded in one another harmonizing one another in the conscious situation. But now that most of the frequencies have been replaced by slow frequencies, we can see it effect
18:31 spiking activity from individual neurons strongly phase locks to different um parts of of these low frequency waves. And the other thing anesthesia does is it seems to scramble the alignment of the brain waves. Okay. So before in in the awake state all these brain waves are highly synchronous and the peaks tend to line up one another because your your brains your cortex is talking to one another and when peaks of um of waves mean then the the is a highly excitable state and troughs are are low energy states. So that's how cortical communication works. When when waves are in sync with one another, the neurons are excitable at the same time. And then when they're and then they're in low energy states at the same time. So they can talk to one another because they're they're in high energy states, their
19:17 high energy states are synchronous. But after anesthesia, these waves become misaligned. They become almost 180 degrees out of phase of one another. So now when one set of neurons is in excitable state, another set of neurons is a low energy state. And you're essentially fragmenting cortex and breaking apart communication. And you can see that here. Here is the spiking activity from different areas. And within an area, the spikes line up, but you see the spikes are now misaligned across the different cortical areas. So that's what anesthesia does to your brain. It doesn't shut off your cortex. It misaligned and fragments the uh the the brain waves. Now, we did this first study. We did this with propall one common anesthetic. We've since then branched off of different anesthetics because medical um
20:03 practice uses a variety of different anesthetics. So we did too. And what we found is something kind of I think is kind of remarkable. We're finding a universal signature of unconsciousness. We saw these same effects with anesthetic doses of three different drugs that have different molecular actions. Propol, which I just told you about, ketamine, anesthetic doses of ketamine and dexatonine. All these things make you unconscious and they all act on different molecular receptors in different systems in your brain, different cells in your brain, but they all do the same thing to the brain waves. Now, if you think about that, that means we're seeing a convergent level effect at the brain wave level that can't be seen at the more reductionist level. So, to my reductionistic colleagues who say, you know, this brain wave stuff is just a humming of the car engine, we can
20:48 explain how all these different anesthetics um have the same effect making you unconscious. And this in principle cannot be explained on the reductionistic level. So a strong argument for function. And the other we know we also know from a large body of work is that these brain waves have a direct influence on on spiking activity in cortex and in fact all over your brain not just cortex. So if this buoy here is a spike in in your uh in the in your cortex, the spiking activity rolls the waves and the waves begin to knock the buoy up and down. This is it's a two-way street. is birectional. It's inevitable because of basic electromagnetic theory. And so brain waves are made of the same stuff, spikes, changes in voltage levels, just on a longer time scarger spatial and
21:35 longer time scale. And that means they exert a strong influence on when neurons spiking cortex. It's kind of like overlaying powerful radio waves onto a telegraph system with the waves modulating the impulses and wires. I know I'm probably speaking on a very elemental level to this particular group, but to a lot of people who haven't thought about these waves before, this kind of stuff is uh can be new to them. So, here's a crowd here doing the wave. You see the the wave is moving from right to left across the screen. What your brain does is this too. If these pe individual people in the crowd are individual neurons, what we see all over cortex, in fact all over the brain, your cerebellum, subcortical areas, even primitive brain structures like your youractory bulb, you see the ongo you see neurons spiking in
22:22 synchrony to peaks of ongoing uh uh brain waves. You see that here? Here's the brain waves going on. Your brain's doing the wave here. And spikes tend to occur. They phase lock to high energy states of the waves. Okay? Now for the last 10 years or so, our laboratory has amassed quite a bit of evidence showing that there's now the know it's a mistake to to to assign specific functions to individual frequencies or frequency bands because that's like saying a electrical impulse has a certain function. It doesn't. It depends on the circuit which is embedded in. But we're finding a general divide of function between higher frequencies and mid mid to lower frequencies. And what seems what what this evidence
23:08 suggests is that bottom up incoming sensory information which is associated with high energy states in your brain, lots of spiking activity of neurons. That's also associated with high frequency brain waves that feed sensory information forward from the back of your brain where it's first processed forward in your brain to your executive parts of your brain, the front of your brain. So that's so when you when you see something, when you hear something, when you're touched, there's a lot of spiking going on your sensory cortex, that's riding on top of these high frequency gamma waves, 30 Hz and above waves, feeding information forward in waves in your cortex. At the same time, your executive frontal cortex, which is what takes charge of your brain, it's feeding back signals in the opposite
23:54 direction that that provide that executive control. And those signals are at a lower to mid-frequency band, alpha, beta, 10 to 25 hertz. And we've shown that they have an inhibitory influence. They suppress they reduce energy on the on these excitatory gamma signals. Okay? And that allows your brain to control what you pay attention to. It allows your brain to filter out the bulk of sensory information. If your brain tried to process all the sensory information flooding into it right now, it would be overwhelmed because your brain doesn't have that capacity. And that's what you see in autism, by the way. You see the sensory overload. Well, these we've amassed evidence showing these top down these alpha beta control signals having this inhibitory influence on these excitatory signals feeding forward can
24:40 provide that filtering that prevents sensory overload and allows attention and executive control over the brain. Now your cortex has multiple layers. Your cortex is kind of like an onion. And we know from the anatomy that the superficial, the outer layers of cortex are the ones that are supposed to feed forward sensory information from the back of the brain to the front of the brain. And the deep layers are the ones that feed back information anatomically. We know for long time now, half a century more, that the deep layers are the ones feeding back information. So these these rhythms should in fact map onto these layers of cortex. Well, in fact, they do, or else I probably wouldn't be talking about it. We've mass evidence and and developed a model where we show that the feedback information
25:27 carried by alpha beta is feeding back information in cortex and that has an inhibitory influence on the gamma being and spiking being fed forward, sensory information being fed forward in the superficial layers of cortex. And I mentioned this because we've we first developed this this uh found made this discovery in the frontal cortex. We've since since branched out and studied all over cortex and we saw the same motif gamma feed forward bottom up sensory alpha beta um feedback ex executive control we've seen the same theme all over cortex 20 plus cortical layers and in four different species and when you see something this common and this ubiquitous across cortex and across species you know it's doing something
26:13 fundamental to cortical function and here's the way it works. And here's how can I can I ask a question? Uh so uh in the previous slide or two slides ago um I think the one before this uh yeah so I think uh what you uh said here is that the the waves and the spikes are phase loft right? >> Yeah. >> Uh and the next slide uh you showed that uh uh they have this sort of uh inhibitory effect. The waves have inhibited effect on the spiking activity. So that means that they cannot be faced off like they there has to be some sort of temporal separation between the >> well there's a lot of electrical there's a lot of electrical influences going on that that's that that's beyond spiking.
26:59 Okay spiking your average cortical neuron only spikes about 10 to 15 hertz and it gives little bursts of spiking then it's quiet most time little burst of spiking. So much of the activity going on in your cortex is actually electrical waves which are happening 100% of the time. So a lot of those influences are happening independ independent of spiking. >> Thank you. >> Can I ask another question? >> Uh I if if it's really important yes but but if I keep getting interruptions then we're >> okay I will ask later. >> Yeah please ask for later. Otherwise you know you when you give a talk somewhere you time at a certain time and then if you get interruptions then you go long and if you don't and if you don't get interruptions then you go short. So I'm not if you have to something something is really unclear please stop me. But other than that, I I hope we can save questions for the end, please.
27:47 But thank you for your interest. Okay. So, here's how it works. Um, and here's how we're going to how these this this idea of these these gamma and beta um influences can explain this prop. We explain this property of big selectivity I told you about earlier. We incorporate this model called spatial computing. And the idea is these alpha beta are having this inhibitory influence on spiking activity in cortex and they act as a negative network space that the brain uses as a coding dimension. So essentially what gamma gamma and spiking is kind of like spray paint spraying energy into into your cortex from the outside world sensory information. And these alpha beta signals act like stencils that prevent neurons from
28:33 spiking. And as a result, the spiking um um being um associated with sensory information can only be expressed in certain parts of your cortex, not everywhere. And your brain controls these alpha beta um influences like stencils to control where your brain cortex can become activated and it uses as a computational tool. So explain in terms of mixed selectivity. Remember we had these neurons that show this context nonlinear um context dependent spiking where a neuron will show one pattern of selectivity for these objects in one context first and another um pattern of selectivity to these objects spiking to these objects in another context that can be explained by these alpha beta stencils. The idea is there's one alpha
29:19 beta stencil a pattern that corresponds to the operation of being first and as a result these neurons can't spike and only these neurons can spike to the butterfly if that's the thing that happens to appear first. And then you have another pattern that corresponds to the operation the context of being second. And now only these neurons can respond. These neurons that are not being inhibited only they can respond to the padlock. and you just change the different objects and you get different patterns of neurons corresponding to the different being allowed to respond to the different objects in different contexts and that's how you get this context nonlinear dependent behavior. So the alpha beta patterns control which neurons can spike in the different behavioral contexts and this results in neurons with this context dependent spiking and all the
30:05 computational benefits of mix selectivity. So why why so what this brings us to is the idea that brain waves are ideal for control and organization your cortex for exerting this top down control over over cortex wave functions are simple and efficient imagine that I want to control all these neurons in my brain in order to do something like pay attention process what a speaker is saying whatnot how do you how do you do how do you do that you can't do it by this connectionism model as I mentioned earlier because connectionism would require that we shift around connections constantly induce. We have to change around all the connections in order to get these context dependent neurons. That seems computationally intractable. However, wave functions are
30:52 simple and efficient. Rather than controlling the individuals one by one, you can use a wave to guide an entire group. and try to if these if these uh people in this crowd here are individual neurons in my cortex I can produce this wave pattern under the old hypothesis the connectionism by trying to tell each individual person what to do but if you throw this wave function across the crowd now you get a simple function that controls the crowd and makes or organized behavior and sure enough your brain does this all the time so these oscillations these electrical waves I'm talking about they're not just jumping up and down in a like a like a standing wave, like a jump rope. They form patterns that travel and move across your cortex, much like fans doing the wave in a stadium. So, here's a recording from the frontal
31:39 cortex of a non-human primate. And what it shows is an array of electrodes. The arrays are separated by about a millimeter or two. Each of the squares is a different electrode in in the frontal cortex of the animal. And what the colors are are electric voltage values relative to a reference electrode. And this this is raw unprocessed data. And what you can see is the electrical voltages are forming this rotating wave that's rotating counterclockwise across the array. So much like much like the um a stadium crowd can organize using waves, your brain your brain is or organizing itself using waves. These waves, these oscillations aren't like pingpong balls
32:24 bouncing around your cortex randomly. They're highly highly organized. And whenever you see organization in the brain, organization equals function. Function invariably follows from organization because or organ organization means predictability. And waves are also natural way produced not just simple organization like this to produce complex organization. And for a metaphor for that, here's a a video of grains of sand on a speaker. And when you play different musical notes, you get you get the the sand forms different patterns. Now, obviously, I'm not talking about shifting around neurons in your cortex. But think of the sound waves as electrical brain waves, electrical field waves. I mean, like uh they're the same physics, waves are waves. And the sand instead is spiking
33:11 patterns of individual neurons. Now, what your brain needs to do to organize, it needs to somehow control all these neurons. So, they're doing something in an organized way. Imagine trying to make these patterns, this organization here. Imagine trying to make this organization by organizing the sand grain by grain. It just be possibly complicated. But organization is a natural outcome of the influence of wave patterns. You have wave patterns, they form, they create interference patterns, and you get you get this natural organization that falls out of it. So this is basic physics that evolution could exploit. And for further um demonstration of this of this for a demonstration of this principle, here's a recent paper um we published LA last year where we show a
33:57 direct connection between these traveling waves are rotating around the cortex and changing spiking patterns in cortex. Now what I'm showing here is not electrical field waves anymore. Not I'm not showing brain waves. I'm showing patterns of neural spiking from 200 individually recorded neurons of sorry 200 simultaneously recorded neurons but 200 individual neurons and what it shows is something called subspace coding. Now what subspace coding is it's a measure of the var of shared variance of the spiking patterns. Now if I record from 200 neurons we did record from 200 neurons I can create a 200dimensional space to explain how these neurons are spiking. That's self-evident. It's self-descriptive. 200 neurons. Each neuron has his own vector. And the spiking is the distance along the
34:42 vector. Bob's your uncle. You can do that. But with subspace coding, we ask the question, how many dimensions do I actually need to explain what these 200 neurons are doing? And it could be 2,000 neurons. How many dimensions do I need to explain what these 200 neurons are doing? And it turns out you can explain them what these neurons are doing in just three dimensions. Just three dimensions. And what that means is that spiking in cortex is highly coordinated. It's not like insects buzzing chaotically in the in the night sky. Cortical spiking is more like a murmmoration of birds. Birds flocking in the sky together in a highly organized fashion. That's what spiking actually is in cortex. Highly organized. Now you guys no doubt get to where I'm going with
35:29 this. How do you produce this kind of organization? Well, you can throw a wave across it, right? That's what I've been arguing. Well, that's exactly what we found. We this we when we record from the same electrodes that we record this spiking activity showing the this highly organized movement of spiking patterns, we see traveling waves rotating around the around the surface of cortex. So traveling so spiking activity rotate I'll explain what the rotation is in a moment. These spiking patterns are moving around this computational space used to represent how spiking is changing in cortex and we find a correspondence of a wave moving across the surface of cortex. Now brief explanation this is a working memory task and what we're looking at is the memory delay the task where the where
36:14 the non-human primate has to hold one of four objects in working memory in mind over the short memory delay. And what the four colors represent are the four different objects the animal's holding in mind. And you see separation between the lines, which means that the spiking patterns are conveying information about which object the animal's holding in working memory. Okay. Now, what's happening at time zero in the middle of the delay? We're distracting the animal. We're showing we're trying to distract the animal by showing it flashing a picture on the screen trying to knock it off its game. And that works about the knocks the animal's performance down. makes the animal forget about 10 to 15% of the time. So, what's happening at time zero? We're distracting the animal. And what you see is the neurons are being knocked out of where they should
37:00 be by the distraction and then something in the brain is gradually steering the neuron patterns back to where they were before. So, this rotating pattern here is a recovery from the distraction. And we see these rotating. So these rotating waves in cortex are we think are are producing you see a direct correspondence producing this recovery from distraction. Now let me emphasize here that what we're seeing here is a is a we're mapping a mathematical construction of neural computation that in principle doesn't have to have any relationship whatsoever to anything happening on the mapped on the surface of your cortex. But it does. we'd see a correspondence between a mathematical construction of neural spiking and and a physical
37:46 movement of waves across the surface of cortex. And what that suggests to me is somehow these waves are actually involved in computation in neural computation. And sure enough, waves can perform computation. You don't need digital computers digit perform digital computation to do computation. In fact, analog computation has been around for thousands of years. Digital is a bit of a an engineering convenience, a bit of a detour. And what you see here is I've illustrated analog computing by showing addition and subtraction using two sine waves. Here's one wave in blue moving um left to right. Another um um wave in this amber orange like color moving uh right to left. And the dotted line shows what
38:32 happens when you sum the two waves together. And if you sum it just this one, there's a space here on the on the uh x axis and amplitude in the y amplitude on the y- axis. If we sum right here, we could see we're doing addition subtraction just by the waves coming together. Okay, so how the waves interact models the equation. I've modeled addition and subtraction by just having two simple slide waves come straight at one another. And all the values are car all the values I might want to add and subtract are carried in carried in different parts of these two of this blue and amber traveling wave. Right? So when the waves interact the entire equation all the addition subtraction across all these different values all the addition subtraction across all the space all
39:18 happens simultaneously you get pure parallel computation inherently parallel computation whereas digital is inherently sequential. And this is just one dimension and one frequency. Okay, your brain could do this across multiple frequencies anywhere from one hertz to 100 hertz and can do it in three dimensions. That's a lot of space to do a lot of complex computation. And this isn't a conjecture. You could do complex math with analog computation. There's many modern examples of this, but one example from hundred years ago was MIT's own Vanir Bush, right across the street from where I'm sitting right now, built an analog differential analyzer that solved 18 variable differential equations using wheels and
40:04 discs. Wheels and discs and waves are the same thing. And what he did is he set up the wheels and disc to model which differential equation he wants to calculate. And then when he turns the crank the whole equation gets to sol solved in parallel. So the setup time the main time here was setting up the wheels and disc to model the equation. But the computation takes place instantaneously. In the brain we don't have to set up wheels and disc. We can just have waves are flexible. We we can model equations on on the on the fly and do parallel computing when waves come together. And analog computing as I've hinted at is a huge advantage over digital. Analog computing is is a way vastly more efficient. It's informationrich. You know the real world is made up of continuous signals. But if
40:50 you want to do digital computation then you got you got to arbitrarily divide up the world into ones and zeros which is already highly inefficient. But again the real advantage is that that analog computation is purely parallel. When the waves interact, all the variables compute simultaneously. Digital computation is sequential. That both these things are a huge advantage over over digital computation. Think about AI. AI based on digital computation and this connection as a model consumes enormous amounts of energy. They're building nuclear power plants to to um to power AI. They're building nuclear power plants to power AI. And here's a graph showing the total output of the of produced by the state of Virginia alone is about a quarter of its goes to these AI um
41:37 powered data centers which are causing brownouts and neighborhoods around them and stuff. By 2030 is projected as much as half of the power that one that Virginia produces will be devoted to AI data centers. Okay. the projection AI projection power demands outstrip the energy the ability of the energy supply to provide it in the future. This is not the way biology works. Biology does not solve problems by throwing huge amounts of energy at it. A pretty good definition of life is local reverses in entropy using as little energy as possible. Evolution favors energy efficiency and your far more capable brain runs on just 20 watts of power. The power of a dim
42:22 light bulb and your brain is more powerful than any AI system out there. So clearly our brains are doing something different. And I think it's not connectism and it's not not digital logic like comput computations. It's wave-based organization and wave-based analog computation. So in summary, brains we I think AI like connectionism connectionism is not wrong and AI is not wrong to use connectionism. It's a great way to store information. Okay, great way to store information in the long term. But expressing information and computing it may be a whole different game especially for the high level computation required by um you know intelligence and cognition. Waves operate on the right scale. You're not don't have to worry about
43:08 individuals and those you're you're you're you're operating on whole sections or crowds of neurons simultaneously by a wave function which is a simple function and they're mathematically easy way can gain control over a complex system and can they can perform highly efficient analog comp computation. So I want to leave you with the thought that cognition and consciousness is not digital or logic gate or connectionism. It's analog analog computation lets exist wave wave patterns in your brain. These wave patterns in the brain are patterns of excitation and inhibition. More energy less energy. That is how your brain works. That's how your brain works. Okay. So what analog computation allows is one pattern of excitation, one wave
43:55 pattern in your cortex to come together and interact with another wave pattern and generate a third wave pattern, a brand new wave pattern in a computationally principled predictable way. And if it's predictable and principle in that way, evolution could take advantage of it. Now consciousness virtually every biological theory of consciousness proposes that consciousness requires somehow your cortex be unified create this metaansemble meta network that unifies your cortex to create the unified experience of consciousness. That's what consciousness is. It's a unified experience. So somehow your your brain has to do this but they're very vague on how exactly your brain does this. Well, I want to suggest that when brain wave patterns produced by analog computation, when wave patterns come together and produce new wave patterns
44:41 using analog computation, when these wave patterns become large and organized, large and organized enough to bind the cortex into a unified state, the result is conscious experience. So, I thank you for your attention and I'm more than happy to take questions now. Thank you. >> Thank you. Who want to go first? >> Thank you. I have a I have a question that was very interesting. Um, so you mentioned that the uh waves are a
45:27 natural way to produce complex organization. And so we've been seeing traveling waves inside frog embryos, but also between them. And I'm wondering if these waves could play like a similar similar role in the brain during cognitive processing and maybe also do some analog computation. And I'm wondering what your opinion is like what does this non-neural wave need to call it cognitive or brainike? >> Oh, I mean waves are are useful physics that your brain could exploit for a wide variety of reasons. And much like you're seeing in the frog embryo, you're developing retina has ways moving across it. So before you can even see, there's these ways that move across your retina that allow the the retina to organize itself into into regular regular patterns. So yeah and I you know Michael
46:14 has shown that uh you know electric fields play a role in in um in development in uh in um so you know wave I I wouldn't say waves become conscious wave electrical biological systems produce electrical fields and your nervous nervous systems will naturally produce oscillations and waves because your nervous system is composed of things called recurrent connections. neurons loop back on themselves like snakes eating their own tail and this happens on a short in a short distance and also will cross long distance in your cortex. So your whole cortex your whole nervous system is set up to produce these oscillations and it's just another thing that evolution could exploit. So I think about the you know very this probably happened very
46:59 early in in evolution. So probably you know imagine your simplest nervous system all they're doing is responding to the outside world like a thermostat which means that the organization of the nervous system is depends on the outside world. Then you get a bunch of neurons hooked together into a loop and now the ner the the nervous system will begin oscillating. So far all it's doing is oscillating but right there there's an internal organization something separate from the outside world that evolution could now exploit push around to do more complex stuff with. So I I have no doubt that uh that um the um electric field waves play a wide variety of roles in biology not just in cognition. >> Hey uh thank you very much. I have a question. I want to relate this uh to
47:47 physics. So in physics uh we have a magnet mechanical waves and they can form a very interesting behavior like uh standing waves and uh and also uh so so I was wondering what's the difference or and dis uh uh similarity between these waves in brain and uh and physical physical waves. Yeah. Well, in the end waves are are waves. I mean like they they you see I mean you could do you could do computation for example with mechanical discs and wheels and also you know um sound waves and and electrical waves. So in terms of the physics of I'm
48:32 not sure what you mean by the question I mean the same wave function describe electrical waves as mechanical waves. So I'm not um sure what what >> yeah by the physics. So I I mean I mean exactly the the mechanical waves because in in like if you consider wave equation in physics they they're not directional but uh in neuron uh on under my understanding it is directional. So >> yeah yeah I see what you're saying. Yeah >> I see what you're saying. Yeah. Interesting. You know, in terms of brain function, you really wouldn't want a lot of standing waves in your brain because what it means is whole networks are going to be activated and deactivated like in unison. What you want to see is you want to see a lot of traveling waves. Traveling waves are a great way
49:17 to control um the computation and the traveling way. So that so the electrical field influences I should maybe I should have mentioned this earlier but electrical field influence one thing one thing they're very useful for is they transmit influences around your cortex 5,000 times faster than spiking but interestingly the they form these traveling waves. So these traveling waves the peaks of the traveling waves you know when you have an oscillating wave the peaks of them move around your cortex and they generally follow the anatomy of cortex and they generally move at the speed of spiking and synaptic transmission. So the peaks of the waves are kind of moving around following anatomy but then when something changed in the resonance of the system all the changes all the influences like propagate instantaneously in in in the surrounding uh circuits. So for the so and traveling
50:06 waves are useful for things like um again there I'm giving you biological not physical explanations but traveling are useful for things like if you want to bake information into your brain change connections in your brain there's something called spike timing independent plasticity. two neurons fire within a short temporal window of one another then they increase their connectivity their syn their synapses get stronger well traveling waves are a great way to do that because what the tra when many of these traveling waves do as I mentioned is a lot of them are rotating in fact the bulk of the waves some of them move across your cortex but a lot of them rotate and rotating waves are a great way to induce synaptic plasticity because you can move the center of the rotation around the surface of cortex and change that precise timing window for spike time dependent
50:51 plasticity. You can move that window around in a parametric fashion. Did that answer your question? I'm not a physicist. >> Yeah, that this is very very very interesting. Uh may I ask another question also inspired by physics. So yeah. So, so in in condes metaphysics uh people studying uh studying uh large scale uh oscillation in a system they call it phon uh it's a imaginary particle that uh uh basically when you have a global oscillation global wave you can transform it into wave spa uh into frequency space and it become a particle I wondering if neuroscience is
51:38 doing the same thing. >> Well, uh yes and no. There are like uh theories of uh consciousness that include like um quantum physics and and uh and um I wouldn't say it's the mainstream of neuroscience. Right now we're me and uh uh several others are are are trying to you know shift the focus of neuroscience. not shift but um um draw more attention to these um non-connectionist mechanisms that that are uh that we think play a major role in uh brain function. And back when I started selling this 20 years ago, people people a lot of people thought it was uh you know um why would you want to study waves when you could study neurons? And now people are uh you know generally people are coming around to it. But we haven't gotten to the uh the
52:24 um you know the quantum collapse kind of thing that although there is a quantum theory of consciousness involving microtubules you might want might want to check out. >> Yeah. Thank you. >> Sure. Um would you mind calling on people? >> Yeah, Chris. >> Uh yeah, thank you. This was this was a very nice talk. I appreciate it. Um on this topic of quantum effects, how reproducible are the context changes that you see with um sort of simple experimentally controllable behaviors?
53:12 Uh, can you control an experiment well enough to get the same wave patterns moving in the same way across a single brain? >> You absolutely can. Yeah. I mean, I wouldn't be able to show you this effect if that wasn't true because all you know, all the experimental work we do, it depends on reproducibility. We have to see the same thing across multiple renditions, multiple trials of the same context. Otherwise, we wouldn't uh even even see these effects. But you can in fact do that. So you have the same behavioral context. You see the same wave pattern every time or a similar wave pattern every time. That that it really would be it really couldn't work any other way because you need this negative cortical space negative um space to be a computational dimension. You got to have cons consistency and we
53:58 do see that consistency that you see it uh there's there's variability across context but consistency within a context. >> Okay great. Um, so whenever um I am a at least a part-time physicist, whenever a physicist hears context, the natural question is, is the context effect causal or non-causal? And um they're nice statistical tests for determining whether the context effect is causal or non-causal. And roughly speaking, if you can reproducibly get context to interfere, you can see non-causal context effects that express by uh statistics
54:44 of some other observable or observables that overlap between context >> violating bells in equality. So you can get very sensitive tests of >> u the non causality of the context effects and I just wonder if your data are immunable to running that kind of statistical test. >> Um yes we in fact do run those contestical tests and we show the causality going from the brain waves down to the spiking neurons. So we've done those analyses but also you can do it experimentally too. So not not work that I've done but work other people have done. For example, you do things like you record from inside the brain and from outside the brain, you create a fluctuating electric field, one that has the same strength and same frequency
55:30 range as as happens inside the brain. And when you record inside the brain, what you find is all the neurons are are are phase locked to to that fluctuating electric field. So that's just one example of many experiments that that's a causal manipulation experimentally but you can also do the math and show causalities moving in in the electric field the spiking direction too from the existing data. >> Do you do you have any data that display violations of Bell's inequality? >> I'm not I'm not a physicist and I'm not a quantum physicist. I know what Bell I'm a I know what Bell's Bells is but I'm not a expert in it. I'm I'm a neuroscientist. >> Yeah, it' really be worth looking for that. >> Experimental psychologists have been
56:17 able to demonstrate um very good statistics for violations of Bell's inequality on simple sort of linguistic trial data. Uh so it's pretty clear that that brains are using uh mutually interfering context to do computation uh even in civil kind of language games and it would be very cool if you could see it in the kind of data that you can collect and it sounds like it would be fairly straightforward. Well, I'll tell you what. My my lab has a vast library, a large library of of electrophysiological data. We've collected spiking neurons, electric fields, animals doing interesting
57:04 behaviors. I have a large library and we love to share data. So, if um you have an interest in in taking a look, any of you, anybody, write me and I'm happy to set up a collaboration. We're happy to share our data with you. We do this all the time. >> Well, thank you. >> Thank you. send um I have a a question about your stencil idea. So the idea being the electric fields as a act as a gating mechanism uh on the neurop right so a couple of questions on that uh so the idea of stencil is something something that's fixed or static right so how do we imagine traveling waves implementing that stencil the second question is who
57:50 is generating that uh stencil is it a different set of neurons than the one that's being constrained >> so here's the so think of it um as the as the stencil is not static stencils but moving stencils. Okay, that that's what the treers are. They're moving stencils moving across your cortex. Same concept as the stencils, just stencils, stencils in motion. Now, who is doing the the um organizing? That's the deepest question, isn't it? And you know um the way I think about it is that you don't don't need so all the information you need for for to engage in complex goal directed behavior intelligent behavior is baked into your brain's connections and the way I think about it is that when neurons start spiking they
58:36 express that information in the wave patterns then those wave patterns that are based on the information about goals you know your desires your plans baked into your brain they create a wave pattern that reflects that stored information and input from the outside world. Then those wave patterns come together and they create a new wave pattern in a computationally principled way that's dependent on that on that the initial wave patterns and that produces a new pattern of thoughts and they could turn around and then by influencing spiking bake information um back into your brain again. So the way I think about this is that you don't need a homunculus a little person in your brain interpreting what's going on. You don't need a you know neuroscientists like me we we they we use down we use decoders all the time on our neural activity can
59:22 I decode information from this activity and that's a very useful tool but some neuroscientists um there's this tacid assumption that something must be decoding activity well the way I think about it these wave patterns producing new wave patterns are patterns of brain excitation producing new patterns in a computationally principled way and you don't need a homunculus. You don't need a downstream decoder. It's just wave patterns producing wave patterns based on the um on the um knowledge your brain already has baked into it. So that's the way I think about the control happens. Control happens because your brain has the um information needed to exert this control and the wave patterns create those control because it's based on that information stored in your brain. That's all I got for for consciousness.
1:00:11 >> Thank you. Um I have a question about that. Um you're talking about consciousness but what about memory? >> Is wave as wave contribute to memory to storing or retrieving memory? Well, certainly I mean this when those information is encoded memories are encoded in the connections in your brain and the synaptic weights much like AI and connectors themselves probably and when that information gets expressed by spiking it produces those wave patterns and that that wave pattern is now the memory that you're now thinking about holding in your mind holding your conscious of. But then when they create new wave patterns because the wave patterns have a downward influence on spiking and spiking changes the
1:00:57 connections of the brain they can now exer um store new memories in your brain. >> Okay. So what kind what kind of speed I can um uh expect from that kind of um mechanism of wave mechanism >> speed of encoding information of >> encoding driving. >> Yeah. Well, it's the same speed as the speed as spiking. All it is is another level of organization on top of the way we traditionally thought about memories in terms of being expressed by spiking and induced by spiking. So, it's the same dynamics. It's same time course, same everything. It's just how the information is expressed at a higher level. >> Right? I'm trying to think about the the experiment of reacting to a surprise or or break pressing the brakes on the car.
1:01:45 uh um >> if you need to make a decision, see something and press the brakes, that's couple of milliseconds and and to do that what I understand you need pretty much a huge amount of speed in terms of the system. >> Yep. >> And I I I don't know how if it's possible to create that kind of speed. >> Uh sure it I mean, it's it's more possible to create it with waves than it is with spiking alone because the electrical field includes move around your brain 5,000 times faster than spiking. So, that produces a lot of uh that's a lot of speed gain right there. But the other thing to um if we're talking about consciousness, which I think is how we began this this question. Um first of all, when you put your foot to break, that's an
1:02:31 unconscious thing. A lot of what your brain does is unconscious. Okay? Consciousness is often the story your brain makes up to explain what it just
…
