What can physics tell us about the brain? - Part 1

I just love the two episodes that you're about to hear. We are joined again on the show by Christopher Lane, assistant professor of physics from Yale University. Now the last time Christopher was on the show we talked all about how we use network theory to try and understand how our brains work. One of these two episodes we take not one area physics but two areas of physics. And we apply them to the brain. So the first of these is statistical mechanics. The second of these is something that I find really difficult, which is information theory. So buckle up as we go on what I think is a relicking journey through two difficult areas of physics to try and understand a little better how our brains work. This is simplifying complexity. A podcast where we explore the underlying principles of complex systems. Systems that seem to defy our rational view of the world. Like economies, ecologies, or even you or me. I'm forensic engineer, Sean Brady and I'll be your host. Christopher, welcome back on the show. Thank you for having me, Sean. So the last time you were on we spoke about the brain and neurons in the brain and talked about trying to understand how the brain works. And specifically we used network theory and the concepts of network theory. This episode is quite different. You want to talk about both information theory and statistical mechanics and how we can use them to understand the brain. Yeah, they might seem like scary concepts, just like network science might seem like a scary concept. And indeed, I think statistical mechanics and information theory are a bit more scary than the network site. So I would say that in practice they're a bit more difficult and subtle to understand. But I do think that they can in combination tell us something interesting about how the brain works. And indeed, actually other complex living systems. So start with statistical mechanics. Obviously, with these topics are massive. So let's talk about the most useful when I'm going too far down that path. Yeah, statistical mechanics is this very old and interesting branch of physics that is older than general relativity, Einstein's theory, it's older than quantum mechanics. It's not quite as old as the classic Newtonian mechanics that you might learn in high school. But it's founded around the industrial revolution actually, so sort of mid-late 1800s. Actually, it was largely driven by practical considerations having to do with understanding how engines work. So we're starting to build our first steam engines, for example, and understanding how can you maximize the efficiency of a steam engine? Is there some maximum efficiency that you can get out of a steam engine for a given temperature and things, questions like this? So it actually derives from very practical engineering considerations. And people started realizing that there's these relationships between like the temperature of a gas and the pressure of that gas and the volume of the gas and how much so that the pressure being how much it pushes back when you push on it. And people realize that there were these quite universal laws. And this was sort of known as thermodynamics. This is what we would call thermodynamics. It's sort of large scale laws that govern gases and other random type materials like this. Fluids. But we didn't really know where they came from. So what determines the relationship between like temperature and pressure? So for example, if you take a balloon of any gas, it could be air, it could be helium, anything you want. If you heat that balloon up, the balloon will expand. It's kind of hard to do this because you might just burn the balloon or something. But if you took a balloon outside on a cold night, it should shrink a bit. You can see these relationships between these macroscopic quantities that we can see with our eyeballs. But it wasn't clear how you could get at these large scale properties from simpler rules as we talked about like on the network science version. How can we get at these larger scale rules about how things work from the rules of much simpler things? So if we think of gas as just a bunch of little billiard balls bouncing around and knocking into each other, then from that picture, just from thinking about little particles of gas, could I construct these larger scale laws about the relationship between temperature and pressure? And that is exactly the realm of statistical mechanics. So statistical mechanics is thinking about the statistical or random properties are very small things. And how when you get a bunch of them together, you can derive hopefully larger scale rules that govern a system at a larger scale. So this is exactly what statistical mechanics is all about. And it's a fair to say, you know, with Newtonian mechanics where we have real billier balls bumping into each other, shall we say? Where you say those is discrete elements. Statistical mechanics is saying, we've got loads of discrete elements, these molecules in this gas bouncing around. But we don't really need to know where each of them is and what each of them are doing because we get this emergent property. I call that emergence. Yeah. So this is the pressure or the temperature of this gas inside the balloon. But we don't need to know the positions of everything that we did with traditional Newtonian mechanics. That's exactly right. So you could, if you told me all the positions and velocities of all the little gas particles, you could derive from that the basically the temperature and the pressure and things like this. But it turns out that you can also derive rules governing the relationships between these larger scale or these emergent properties. And in fact, you can speak in a self-consistent way only using the language of these emergent properties. So to describe the relationships between temperature and pressure and volume, just like you said, you don't need to know the underlying details. So you can speak self-consistently about these things. But it turns out if you know the smaller scale things, you could derive these larger scale things. So statistical mechanics, you can think of as effectively, or at least at the beginning of statistical mechanics, it was basically connecting this Newtonian mechanic picture where you have billiard balls bouncing into each other with this larger emergent picture of how does temperature relate to pressure. And these rules that we were seeing just in experiments that we didn't know how to derive basically. So statistical mechanics bridges that gap. So the reason you might be tantalized by thinking about why statistical mechanics might be interesting for thinking about the brain is that you can immediately draw analogies to the question of understanding the brain from the story we just told about sort of classically how we we learned about the relationship between gas molecules and these emergent properties. So in the brain, we have a similar but much more complicated picture where we have a bunch of smaller things, which are these nerve cells interacting in this vastly complicated network. So you could speak at that level. You could speak about individual nerve cells, these neurons, and how they're interacting with one another, just like the billiard balls, like thinking about gas molecules as billiard balls bouncing into each other. But then when you zoom out in the brain, you see things that feel completely unrelated, like consciousness or vision or things that we can speak about self-consistently, which is perhaps the realm of psychology or cognitive science. So there's a whole field. Yeah, there's whole fields called cognitive science and psychology devoted to studying these perhaps what you might call an emergent property of the brain. And you don't actually need to say anything about neurons. Psychologists often go their whole career without needing to say the word neuron. You can speak self-consistently about these more emergent things that we see in the brain. But one of the biggest goals in neuroscientists to bridge the little microscopic rules we know about how cells work up all the way to these larger scale properties of how emotion and vision and decision making and actions and planning all work together. So you might be tantalized by the possibility of using similar ideas or trying to take a similar path that statistical mechanics used to understanding how like steam engines were to study something like the brain, although it's a much more complicated system. So this is just laying out why there might be a hope of doing something like this or why perhaps it might be an interesting thing to try to do. So then information theory. I find information to be difficult. It's one of those theories that have been not embedded in it. It's hard. So maybe steps with us here in terms of information thing to make you feel better. Like statistical mechanics is also considered like itself is considered one of the more enigmatic branches of physics, I would say. For example, even to talk from a practical perspective, if you're getting an undergraduate degree in physics or a PhD in physics, statistical mechanics is still one of these classes that varies perhaps more than any other branch of physics in terms of how differently it's taught at different places by different people. So there isn't like an agreed upon canonical path sort of a logical ladder of ideas like there are many other branches of physics in statistical mechanics. It's still very much this winding road that a lot of people still don't fully understand and we could get into these other details about why statistical mechanics is still even philosophically like a very confusing and interesting thing. I get to put it this way statistical mechanics, even though it's one of the oldest branches of physics is still an active area of research. Wow, I didn't know that. Like Einstein's theory of general relativity, let's say, or take special relativity is the simplest thing Einstein did, but I mean, it's still a massive leap. But it's fully understood. Like we understand there's no sort of outstanding questions that confuse us. Like if you take a special relativity class, it's understood basically how to line up the ideas and teach them. And by the end you understand basically all that's known about special relativity and similar for general relativity for the most part, which is Einstein's theory of gravity. But in statistical mechanics, it's still very much an active, very research. So there's still people devoting their entire careers to trying to understand what statistical mechanics is trying to tell us. And is that fundamentally that it's still really difficult to connect the individual interactions within the system with the emergent phenomena that comes at the end, whether it be temperature and pressure? Is that the core problem? Different people will probably say different things, but for me, I think it really stems from this fundamental quite philosophical question about what we mean when we talk about probabilities. When we say something's random, what exactly do we mean? So the word statistical, as we mentioned before, it basically means that that things were talking about our random, their statistical in nature, like flipping a coin or something. As we just talked about before, there's kind of two levels of description. You can think about temperature and pressure in these things and then all we kind of need to know is we don't need to know the microscopic details of each individual gas molecule. All we kind of need to know is like we actually only need to know the statistical properties. So like, what is the probability of some number of gas particles going at some velocity? So you really only need to know the probabilities of these things. At the fine grained scale, we like to think that if we really wanted to, we could really with infinite precision measure the position and velocity of every particle. So we like to think that there is some underlying truth that's deterministic. The only caveat being like quantum mechanics type things. But assuming Newtonian, we're thinking about like Newtonian mechanics, everything's deterministic yet. To talk about this larger scale picture, all you need to think about is probabilities of particles flowing left or right at some velocity. And the question is, are those probabilities real in some sense? Is there sort of inherent stochasticity in these systems? Or is it really just a statement about the details that we're including in our description of the system? Is it like a human choice about what we're deciding not to include in our description? So I think this rather philosophical question about probabilities is gets at the heart of why these things are very confusing. This is really interesting. I know where I was thinking, dear, a little bit of a segue here. But we know these statistics work. The question is, are they meaningful? Is it a bit like that sort of quantum mechanics question? We know it works, but whether that's actually what's happening. We get the right answer, but whether we're describing what actually happens or it's a human invention of what actually happens, is that the core philosophical difficulty here? Exactly. So it feels like at some point when you start talking about the larger scale description, in some funny ways, human choice comes in the matter. So you could describe it a very small scale, all the individual little gas particles. But at some point, if we want to talk at a larger scale, we have to make a choice of what we're calling the balloon. We have to have some notion of a balloon. And so you end up with this very self-refresherential problems, which it feels a bit like how in quantum mechanics, you still have this issue with observer paradox for you look at a system and it collapses. It feels like somehow the human was involved in that process, which confuses people in quantum mechanics in a similar way in statistical mechanics. There's a bit of a human choice involved at the larger scale of saying, what are we calling the system? And how do we define these probabilities? It feels a bit like there was human choice involved. And in physics, it's very uncomfortable for us when there's any human choice involved. Because it feels like the universe is what it is. Like if we weren't there to describe it the way we want to describe it, it should still act the same way. It shouldn't exist in reference to what we decide, how we decide to describe things. And so to be clear, these questions don't really come up much in the practice of doing statistical mechanics so much. But I think in terms of actually getting a full understanding of statistical mechanics, why I think that is still not fully solved in physics for lack of a better word is I think due to these questions about coarse graining and human choice and things like this. But anyways, all of this was a tangent just to say that if information theory scares you, I don't know if it's better to realize that maybe you should be more scared of statistical mechanics or it was it was just can we feel a bit better about being scared about information theory. But perhaps I just made you more afraid of statistical mechanics. No, it's brilliant. So information theory. What else is the key bit for us here? I've been using both of these sort of sets of ideas for a while now. For me, information theory actually is a bit more well defined and I can wrap my whole head around it for the most part. So hopefully I can convince you that and the listener that they should also be not so scared of information theory. But perhaps they should be scared of statistical mechanics. Information theory is one of these interesting subjects that was kind of founded in one fell swoop in one paper by one person in the mid-40s. So it's one of these moments kind of like what we were talking about Einstein before where he writes one paper and all of a sudden you have all of general relativity which is incredible. So there's this guy Claude Shannon who was working at Bell Labs in the 40s and he wrote this paper called a mathematical theory of communication. And in this paper and this one paper he basically lays out all of the fundamental ideas that we now call information theory. And information theory itself is still an act of very of research. So people have expanded upon these ideas. But the key fundamental pillars that you need to understand about information theory were all included in this very first paper which is quite incredible. And this also actually came from a like statistical mechanics was sort of spurred on by a very quite practical questions from an engineering perspective. So they were building wires to send information for example between parts of America or across the ocean over to Europe. And you would like to send information like through a phone call or through a telegram or Morse code things like this. You'd like to send information as quickly as possible. And if you have a cable that can carry a certain amount of information which we're going to talk about in a second you would like to be able to quantify how much information that cable can hold and how much you can really push through for example per second. But to be able to think quantitatively about these things you can already see in the words we're using that you need to be able to quantify information. Information is a quite sort of squishy word that we use colloquially to mean a lot of different things. I think for people that aren't that when they're not talking about information there if we just talk about information it's just something that tells us knowledge that we didn't have before. So what Shannon was basically trying to do is trying to put a number to that squishy feeling concept that doesn't feel like you could put a number to it. So he basically wrote down some axioms and said well we have this squishy idea of I'd like to quantify this notion that if you tell me if Sean tells me something I could quantify how much information that provides me and he wrote down some axioms for things that perhaps we could all agree on that we might want this quantity to have like properties we might want this quantity to have. So for example and I don't know if I'm going to I don't know if I'll be able to remember all the specific axioms but the first key insight is that probabilities are going to be important here. So that's the first key step and the reason probabilities are important are because if I already know something which means that I know this bit of knowledge with 100% accuracy there's no probability there like I know exactly what's going on. Then if you tell me something about that it's going to provide no information because I already knew it. Whereas if there's something that's sort of 50/50 in my mind and you tell me which one it is now that provides information so for example I haven't looked at my phone yet and I don't know what the weather is going to be like tomorrow. So in my mind there's maybe a 70% chance it's going to be sunny and a 30% chance it's going to be rainy and then you the weatherman tell me oh it's going to be rainy tomorrow that provides information to me because previously there was probabilities and now there's certainty and indeed the simplest way to think about these things is just by flipping a coin. So suppose you flip a coin and you don't show me yet what it is if you ask me right now what is the probability that the coin is headed tails I'll tell you 50/50 unless you're tricking me in some way. But then when you tell me when you reveal whether or not it's headed tails that provides information to me because now whereas before there's this 50/50 chance of it being headed tails now I know well it's heads and that provided information. So you can immediately see that think about quantifying information we need to be able to talk about probabilities like this. And so one thing that we might want this quantity of information to have is we might want information to be larger if the probability that I gave that event is smaller. So for example if you roll a die and then you tell me what number it was that provides more information to me than if you flip a coin and tell me whether or not it was heads or tails because when you roll a die there's six options and I have a one in six chance of guessing it correctly. So in some sense I really am a bit of a loss for guessing accurately what the die is. But then when you tell me it provides a lot of information so it took it from a one in six chance to a one in one chance of what number it was. Whereas with the coin there's a half chance I was going to get it right. So you actually give me less information when you tell me what the coin is. You give me more information when you tell me what's going on when you after you roll the die. So that's the first thing we might think information should have. Good. Just go quite practical on that because these are really core ideas and I think they are a little mind bending because as you say most of us think this is information as something someone can tell us and you know we're fine then. And I'm we sort of say that if I have a wire then I'm going to push down that wire the answer to a coin flip whether it's heads or tails or the answer to a die row whether it's one to six. If there's any static under that line or there are a little bit of data gets lost or whatever then you're much worse off if I'm giving you the information for the die because you've got one right answer and five wrong answers. Whereas you would a coin flip you've got one right answer one wrong answer. So you're much more statistically more likely to get the right answer is that it's harder to pass the die information. Exactly. So if we did this so you can imagine repeating this process many times and you have to send the answer down the wire so you can immediately see that if you need to send the message down the wire telling me whether or not a coin is heads or tails we could do that with just a one or a zero. So if it's a head send a one down the wire if it's a tail send a zero down the wire and then every time I get a one or a zero at the other end I know whether or not one of the coins was heads or tails. whereas for dice, if we do this a bunch of times, we need a way to send down the wire one of six options many, many times. And you can't just do that with a 1 or a 0. In fact, you can't even do it with two 1s and zeros, because then there's only four options. What you need is three 1s and zeros. With three 1s and zeros, you can write down eight different things. There's two times two times two different messages we can send with three 1s and zeros. So there's eight things. So if we wanted to continuously send down the wire with 1s and zeros, the outcomes of rolling dice, you can see that each time we roll a dice, we might have to send something like three 1s and zeros at a time, which is three times more than you needed to send just for flipping a coin. So you can already see how it's going to be more difficult. So there's more information we have to push through. The message has to be longer if we're rolling dice than if we're flipping coins. So this is one pillar or the one axiom or one assumption we can agree on that we might like some quantification of information to have. If things are less probable, but then you tell me the answer, then that contains more information. The second thing we might want some notion of information to have is actually a bit simpler to understand. So it's if we have two independent events, then their information should add. So if you flip two coins and they're not correlated in some way, you just flip two coins at random, then the amount of information you tell me about coin one and the amount of information you tell me about coin two, if you add that together, it's the same as if you tell me the information about both as some like four-sided die or something. If there's two independent things, their information's add, which makes sense. Like if you send one thing down the wire and then another thing down the wire, the amount of information should just be the first one plus the second one, which is quite easy to understand. And the third thing is even easier. Maybe we want information to be positive. We don't want negative information doesn't quite make sense. So maybe we want it to be positive. I might be missing one axiom, but basically with just those three ideas, the core one being the first one we talked about. So we ate our vegetables first and then the easier ones came second and third. With those three core ideas, what Shannon showed is that basically uniquely pins down a quantity that represents information. If you tell me the probabilities of things, I can plug it into this formula and tell you a number that represents how much information is contained in basically the process of you rolling dice or flipping coins or any other message that you might want to pass to me. And this was the key idea, basically. It is really mind-bending, isn't it? You go away from saying, I'm going to define the volume of information based on how many options are available for each one. And the more options there are for a particular piece of information, then the more information have to send to a separator, which option I mean. I mean, that's such a mind-bending, but brilliant, obviously. Yeah, it's a bit of a mind-bending, but you hear this or once you're told this, you can actually start to try to think about times in your life where, like, just moments, everyday moments where something provides you information and you can think, oh, it actually starts to make intuitive sense that the things that provide more information were the things that you thought were rare to begin with. So for example, if we just had a presidential election here at America, and if you were very convinced that one candidate was going to win, and then you close your eyes, let's say I put you in a coma and I wake you up after the election and I tell you that candidate won. That doesn't provide you much information because you were already very, very convinced they were going to win. But if I tell you the other candidate won, that provides a lot of information to you because you wouldn't think in a million years that that person would win. So you can already start to see that the less probable something is when it actually happens, it provides more information to you. And this realizing this is the key insight to having like an intuition for information theory. That one insight is the key thing to understand. So how do we bring these two ideas then together and apply them to the brain? What have you done there? Or what have people done there? I'll try to explain how we might combine statistical mechanics and information theory or how they interrelated. And then that might take a bit and then we'll see how perhaps we might hope to think about the brain. So interestingly, what Shannon showed is that this quantification of in the things we're about to say, there's words that are used that people still to this day like practitioners of information theory or statistical mechanics, they're still misunderstandings just because of nomenclature, which is a classic human foible. Just we often use words and loose ways and that causes confusion because the receiver of a message might think of one thing and the center of the message is actually trying to send a different message. So it's important to be very clear here. So when I was talking about this quantification of information, really the idea is that before you receive a message, you have some amount of uncertainty about what's going on. And at the end, I tell you the answer and now you have no uncertainty. So really that drop in uncertainty is the information. This will become very important. So right now, the amount of uncertainty at the beginning is the same as the information because at the end, we have no uncertainties. But imagine you have some amount of uncertainty and then I reduced that uncertainty all the way to zero by, for example, telling you whether it was a heads or a tails, that drop in uncertainty is the information. But you could imagine that it's not too far of a step for me to tell you, well, I could reduce your uncertainty not all the way to zero. I could reduce your uncertainty, let's say, by half. And that drop in uncertainty is what we would call information. So there's two key subtle difference here between uncertainty and information. Information is a reduction in uncertainty. And we can make this very clear. So if you roll a dice, in the scenario we were talking about before, you tell me exactly what the number was and that reduces my uncertainty to zero. I know exactly what number you rolled. But you could imagine you could tell me something a little more cryptic, which is, well, it's an even number. And that has reduced my uncertainty by half. So rather than a one in six chance of being right, I now have a one in three chance. So I still have some amount of uncertainty. It's either a two or four or six. It's kind of like a, in who wants to be a millionaire, where you, you'd use the 50 50. So the 50 50 provides information, but it doesn't reduce your uncertainty to zero. There's still a 50 50 chance that you're wrong, rather than a one in four chance that you're wrong. Okay, so that's a key step that even people that talk about information theory still don't, I think, communicate properly is, is that information is not uncertainty. It's a reduction in uncertainty. Okay, so now that we have that in mind, what Shannon showed in the quantity that we were talking about before, which I was calling loosely information, really we should think of it as uncertainty. It's the amount of uncertainty we have about some set of probabilities, like rolling a dice. And this quantity, this uncertainty is what we call entropy in information theory, lingo. If I say entropy, just think uncertainty. It's how random something is. So rolling a dice has more uncertainty. It has more entropy than flipping a coin because it's more random. There's more possibilities of what could happen. So now the second tricky bit is the fact that people use the word entropy. So Shannon used the word entropy to describe this quantity precisely because it looks a lot like another separate thing in statistical mechanics that we call entropy. Yep. And I must say I struggle with this. It's hard. Good. Yeah. So and it turns out that they're deeply related because mathematically, the reason Shannon called the information theory quantity entropy is because mathematically, in some circumstances, it is precisely the same thing as what you see in statistical mechanics. So in statistical mechanics, we talked about how you can describe things probabilistically. So if we have these gas particles bouncing around in our balloon, we don't need to know precisely the velocities in the positions of every gas particle. All we really need to know is the probabilities of some number of particles being in some state, like being in some position with some velocity, which we just call a state. And it turns out that perhaps the core quantity to understand statistical mechanics is what we call entropy in statistical mechanics, which if all of the different states are equally likely, the entropy is simply the log of the number of states. It's sort of this quite simple formula. So the entropy is just the log of the number of states. Now it turns out that in information theory, if we think of this sort of the system before the statistical mechanical system, the entropy of me telling you what state we're in. So you could imagine that we have all these gas particles, it's just like a giant dice. And there's all these gas particles flying around. We don't know what the specific state is that the system is in, but we can define the entropy in statistical mechanics as log of the number of states. It turns out that in information theory, if you told me precisely what state all of these gas particles were in, that is precisely the same thing as telling me like what number you rolled on the die. And the entropy from an information theoretic perspective is exactly the same as the entropy in the statistical mechanics perspective. And this is this deep connection. Well, it's not clear actually if there is a deep connection, it's clear that there's a surface level connection. And this was clear to shade. And that's in fact why he called the thing he wrote down entropy because he's like, oh, this looks like what we see in statistical mechanics. I'm going to call it entropy. And that has led to ongoing confusion among physicists and information theoretic people about what we mean when we say entropy. So sometimes I have to say I give talk sometimes and someone in the back raised their hand and they're saying, well, this doesn't have to do with anything with temperature or pressure or anything like that. I was like, okay, sorry, I meant information theoretic entropy. Whereas in a physical system, if you apply the same ideas in a physical system, then you can relate entropy to things like temperature and pressure. But in fact, you can define entropy in an information theoretic term in a much broader sense for anything with probabilities. That's the key idea here. Just sort of summarize that a little bit. In information theory, it's a measure of the uncertainty. So it's a measure of the number of possible answers that we could have six different answers. Exactly. So it turns out that the entropy of the dye is just log six. So it's the same as. Oh wow. So it's the same as it's literally that is the quantity. It's log six. And when we go to the gas then entropy is really the measure of the number of states. And what we mean by that is. And I will find this really difficult is it's the number of different versions of the underlying sort of composition that could produce the same macro state. Is that right? It just takes two molecules to make it really, really easy. Presumably we could have molecule one beside molecule two and that would give us a certain macro property of temperature and pressure. But you could also flip them and have them in the opposite order and that's another state. Exactly. Yeah, that's exactly right. Yeah, and you're even using the correct lingo which is micro state and macro state. So that's what people talk about in statistical mechanics. A macro state is what I've been loosely calling this like higher level picture. We have a balloon with some gas in it. And when I look at this balloon it has some temperature and some pressure and you could ask there's a ton of different states underlying states, micro states of the different gas particles that give me the same looking balloon. So the question is how many different micro states, how many different orientations of all these different gas molecules give me a balloon that looks the same. And there's many, many, many and that number log of that number is the entropy. That makes a really good sense. I'm going to ask a very dangerous question here that could, you know, give you up 20 minutes. Is it a relatively simple explanation for why it's the log of the number of states? Do we know where the log comes from? So you could answer this question from the information theoretic side or you could answer from the statistical mechanics side. From the information theoretic side, it actually comes from those axioms we were talking about before. You can show that just from wanting basically this quantity that what we call entropy now to increase with basically the number of possibilities or increase as the probability of each individual thing becomes lower. So that was the first thing. Then the fact that two independent things add, if you tell me what state the system is in, one system is in, and then you take a totally different system and tell me what state that's in those entropy should add. And then non-negative, it turns out that those are enough to derive that really what you need is a log. And the key step there's actually the second one, the additive. So it's basic probability theory, but if you have two independent probabilities, they multiply together. And so you can already see this because we know that if you flip one coin, it's heads or tails with probability one half, one half. And then I flip another coin, heads or tails with probability one half, one half. The probability of the first one being heads and the second one being heads is one quarter because you have to multiply the one half times the one half. And the simplest way to see this is that there's four possibilities for what these two coins could be. There's heads, heads, tails, heads, tails and tails, tails. And so you can see there's a one and four chance that it's going to be heads heads. And so the key idea is that probabilities multiply if they're independent. And it turns out from if you've taken high school math, you probably would have learned that log of two things that multiply log of a times b equals log of a plus log of b. So it turns out that the additive property is what basically tells us this has to be a log. So if we want independent entries to add, we need something like a log. On the mathematical side, that's where it comes from. And that's the clearest way that I think about it. So you're saying statistically, they need to multiply by each other to give the answer. But we need to add them when we start thinking of entropy. Yeah. So you've got to go from multiply into add and the log is the log is the log is the opening that does that. Yeah, you need to go from multiplying probabilities to a quantity that adds together. So you need to go from multiplying to adding. And that's exactly what a log does. And that's basically why there's a log always reminds me of the engineering scale rule that you get two log scales together. And you can now do multiplication, you can slide the rule of the relish feature. Anyone who's an engineer who's or anyone who's seen a polar 13, we'll see them using the scale rules. Exactly. Yeah. Okay, we'll hit pause right there in this episode. Give you a little time today, just for you just heard. And we'll see you again in part two. Thanks for listening to simplifying complexity. When we look at the key concepts of complexity science with expert minds from across the world, concepts like emergence, self-organization, adaptation, networks, scaling, tipping points and much more. This podcast was produced by Brady Hayward and Wadland Creative. To make sure you don't miss an episode, be sure to subscribe to or follow the show in your podcast app. I'm Sean Brady and I'll see you in our next episode. [Music]