How Is AI Changing the Science of Prediction?

Making predictions is a challenge woven into every part of our lives, often in ways we don't even think about. Will it rain this afternoon? How will the stock market respond to the latest news? What would mom like for her birthday? Typically, we build up a knowledge base and a theoretical understanding, at least in science, and apply what we know to predict future outcomes. But that approach faces sharp limitations, especially when the systems, to be analyzed, are profoundly complex and poorly understood. I'm Steve Strogatz and this is The Joy of Why, a podcast from Quantum Magazine where I take turns at the mic with my co-host, Jan 11, exploring the biggest unanswered questions in math and science today. For this episode, we're joined by mathematician and statistician Emanuel Kandaz to ask, "How are data science and machine learning helping us approach complex prediction problems like never before? And how confident or skeptical should we be in their predictions? Can we figure out ways to quantify that uncertainty?" Emanuel Kandaz is a chair and professor of mathematics and statistics at Stanford University. His work lies at the interface of math, statistics, information theory, signal processing, and scientific computing. He's a member of the U.S. National Academy of Sciences and has received a MacArthur Fellowship, a Colatz Prize, and a Lagrange Prize. Emanuel, welcome to The Joy of Why. Thank you very much for having me. And since you mentioned the National Academy, let me start by congratulating you on your election. This is truly awesome. Oh, you're too kind. Thank you. I'm honored to be joining you and all of our other esteemed colleagues. Well, so let us begin here by talking about something on the mind of just about everybody nowadays, machine learning models. We keep hearing so much about them. We know that they can pour through massive data sets and often pick up patterns that no human being could detect. But these models, people refer to them often as black boxes. And I'm just wondering, would you yourself use this phrase? And if so, what do we mean by a black box? As you said, a machine learning algorithm takes as input data collected in the past and given a set of features, tries to make a prediction about an unknown label. So I have to say that the predictive modeling culture is as old as a field of statistics itself. Statisticians starting with Galton and Pearson and Fisher have been very focused on making predictions from data. But they use relatively simple models, models that could be analyzed mathematically, models that we teach at college for which you can provide sometimes reliable inference. But I don't think I need to tell you that now we have passed this simple regression that we're using deep learning, gradient boosting, random forests, a lot of techniques that have become very popular sometimes in combination. And now this becomes so complicated, that it's very difficult. And we use a 10 black box to refer to algorithms that are so complex that they will resist analysis. There are of course a lot of theoreticians who try to understand what's happening in the black box. Thank you. Wonderful explanation. It's a whole new universe of statistics, seems like. Absolutely. But it doesn't mean that we have to trash what we've done so far. What my research group has been doing and what a lot of groups are doing worldwide at the moment is to try to get the output of these black boxes and treat them as statistical objects. And so we see a whole branch of statistics that is reasoning about the output of these black boxes without making any modeling assumption. So that the results of analysis can be trusted so that we can quantify uncertainty so we can make reliable decisions. And so all the stuff like the p-values and the confidence intervals, they are present in one way or the other. The concepts of p-values essentially a measure that quantifies how surprised should you be by a certain experimental outcome. And in the context of black boxes, if a black box makes a prediction, I can still ask how surprised I should be from this prediction. And so I need to be able to quantify the element of surprise. So I would like to be able to transform the prediction into what you refer to as a p-value so that I can actually calibrate what comes out of the black box. So surprisingly, we do not have to abandon what we've been doing. Mostly we're moving towards a world where there are less parametric models. But the concepts of having outcomes that are well calibrated of quantifying uncertainties is still there. It's so interesting. I really like the way you put that, that it's sort of like the black box can stay black. We don't have to look underneath or inside the model to make sense of what's going on analytically. So it's like we are taking the old methodology of the old desires of traditional statistics to quantify uncertainty and rebuilding the theory for this new world of these black box models. Absolutely. And actually, this rebuilding of the new world comes in many different flavors. But I'll give you an example. Let's imagine a world not too far in the future where people apply to colleges, for example. And because colleges say we see so many applications, we're going to outsource at least part of the decision process to a black box. And so let's say that now students apply to Cornell to your home university and that somehow you decide to predict how well they will do at Cornell using a black box, right? And so the question is, how calibrated are these predictions? But what you can do is you can say, well, I've trained my model and now I have reserved a set of students for which I know the outcome and I can actually see how the black box predicts these outcomes. And now I can try to understand for what kind of students is the error large? For what kind of student is it low? What kind of accuracy do I get from this black box? And from that, you can calibrate now when you're going to use a black box to perhaps screen a few candidates. Because you have observed the mismatch between the black box prediction on the test set, then you can understand a bit the accuracy of the black box and what you can actually conclude. By observing the outcome of the black box on a group of students for which you have the outcomes or labels, you're able to actually produce not a point prediction of how the students will do, but you can obtain a prediction interval that has a probability of containing the true performance, a prescribed fraction of the time. And when I say this, I say you do not have a model, you do not have a Gaussian distribution and you are in sight, you're only using the fact that you draw students at random, you look at what the black box does on the random subsets and you use these observations to actually generalize to unseen students. And it's very much a statistical spirit, which is you collect the data of features of students applying and what the black box says about these students. You're learning from this to be able to say things that are valid into the future. Good. I really want to unpack this example. It's so provocative. The language of features and labels I think is a little abstract maybe. So let me see if I get what you're saying. But so if I imagine a cohort of high school seniors applying to Cornell or Stanford, your institution either way, features might be things like their high school GPA, whether they played on a varsity sport, whether they are African-American or Latino or male or female, all kinds of things like this, you would call those features? Yes, these are features. These are essentially what's in your application file. So these are what you know about the applicants that can be sort of digitized. But I think we live in a modern world now. And so a feature might be your essay, because your essay will become a string of numbers. That's the revolution around large language models. And so that is also a numerical feature that you can use to predict how well do you write English? What is the richness of your vocabulary? There are lots of things you could use. Yes. But in terms of what the college might want to predict, just to make it simple, what if we said we want to predict the GPA upon graduation of the student? Or it could be even more simple, will the students graduate in four years? And so in this case, let's look at your first example, you want to predict the GPA after two years of undergraduate education. And I can say, what does a black box say about these students? And so by looking at sort of the distribution of errors, that is a difference between the true GPA of the students and the black box predictions, I might have a sense of the typical errors that the black box makes on a random student. And so when a new student comes in, I have a sense of the errors that I'm going to suffer. And I can sort of, instead of giving you just a point prediction, I might give you an interval that likely contains the true outcome of the student. And to our surprise, it might be that for some applications or for some students, this interval is short, we're fairly confident of how well they'll do. And for others, it might be wide. In this case, what would the interval be in this example? One interval might be, I'm predicting 2.9 to 3.9. So the center is around 3.4. And the other one, I'm predicting 3.3 to 3.5. And the interval is much shorter, they have the same center, the prediction is the same. But the range is very different. And so if I'm an admission officer, this is something I'd like to know about my prediction engine. How accurate are you? What level of uncertainty is associated with your point prediction if we're doing finance? And I have an investment strategy. And I say, Steve and I can promise you a 6% rate of return. There's a very different scenario between 6% plus or minus 1% and 6% plus or minus 10%. You might lose a lot of money and a lot of customers if you're in the second category. All right. Good. So this example that you've given either in the context of finance or GPA really does help underscore why we care, not just about means or what we might call point estimates, but also intervals within which we might have high confidence. I mean, anyone can see, I hope, how valuable it is to be able to make predictions of intervals, not just numbers. So if we could, I'd like to move now to another real-world example outside of the collegiate setting having to do with election forecasting. Just to be clear for our listeners, we are recording this podcast a few months before the 2024 US elections. But this episode we predict, if you'll pardon the pun, will air sometime right in the aftermath of the election. So I'm sure this is something very much on the minds of our listeners. I know you have worked in this area and your students also. The question is, what insights can you give us into some of the complex models that are being used to forecast our elections? So perhaps first I should be clear. I don't have really firsthand experience with forecasts of elections. I'm working with students at the Washington Post who is a data science desk and they actually do the work. And I'm just going to be a messenger for this part of the conversation if that's all right. I would like to give some credit to the young people who are involved in this. And I also feel like you may be a little bit modest as is a nice quality that you have. But isn't it true that Lenny Bronner and Stanford undergraduates who were working, I mean, at least in Lenny's case for the Washington Post, didn't they build on some of these techniques that you helped develop? That is true. But as you know, when you actually work in the trenches on something of consequence, such as predicting the outcome of an election, even though the general principles are in some of the papers we wrote, there's still an enormous amount of work that they've done to make it all work. Okay, good. Thank you. So what a news organization will try to do essentially is some poll closers and some precincts are reporting and some counties begin to report. In fact, this is a very cool problem because the ballots are already in the box, so to speak, and you have not opened the box yet and like, you know, what's in there. And a lot of the statistical work that is ongoing, for example, at the Washington Post, which is the organization I know best, is they're trying to predict unreported counties. And so instead of giving their viewership a point estimate of, well, Santa Clara will vote this way, you can tally up the forecast for unreported counties, aggregate them at the state level, and have a very nuanced picture of how California will vote. Now, how is this done? So obviously, we're going to need to predict how counties are going to vote. And this is going to be based on a lot of features. Is it a predominantly urban county? Is it a rural county? What's the level of education? What's the socioeconomic variables associated with the counties? And most importantly, how did the county vote last time? And so you're using all these features, you're trying to learn a model that can predict accurately how counties are going to vote. And that's your black box, if you will, except that they use models that are not too complicated from what I've seen that are fairly simple. But then the second part is a calibration, because you cannot just go on air, say, oh, California will vote this way, when in fact, it's just a point estimate, this has enormous consequences if you get it wrong. And so what they will do is they will report a range of possible outcomes for the state of California, that is dynamically updated as the election goes along, that reflects truly their knowledge about what they think will happen when the vote will have been completely tallyed. And so it's very cool what they're doing, because they're really projecting errors, they're projecting uncertainty. And you can see that their uncertainty ban, of course, narrows as more and more counties are becoming reported. And they're fairly faithful, they are back testing them as we say in the field that they are saying, okay, let's see how this model would work in 2020. And they want to make sure that the intervals that they project contain the true label, the true votes, the prescribed fraction of the time. And so it's all engineered very well. And I think kudos to the Washington Post to being so respectful of their readership in not just giving you point estimates, but a real sense of accuracy of their point estimates. Now, just to be dead clear about this, we're not talking about forecasting the election based on polls a year in advance or anything like that. This is election night forecasting based on the results that are coming in. Exactly. So the reader has to imagine that basically there are ballots in a box somewhere. And the only thing is that the box has not been open yet. But I've seen similar boxes open elsewhere in other counties, other precincts. And I'm going to use this knowledge to make a prediction about what's in this box. And it's going to be a very well calibrated prediction following the principles we laid out earlier. And you do have the right to use polls as features as predictive variables in your model. I suppose you could. I think a lot of people out there may be skeptical of polls. We've seen how difficult it is to do polling. But then again, the model may take that into account. Maybe it doesn't assign much weight. Exactly. The model will take this into account. Now, what's kind of tricky about polls is that polls might be different in different counties. Typically, when you fit a statistical model, you'd like the features to be the same for all units in your data set. So going back to the example we had about college admission earlier, right? Everybody has a high school GPA. Everybody has a yes/no answer to are you on a varsity team? And so what might be tricky regarding your poll to use it as a feature is that some counties might have its others may not. So you have to be a bit careful about this. Good. All right. Let us take a little break here and we will be right back. Welcome back. We've been speaking with Emmanuel Candes about statistics, prediction models, and the inherent uncertainties in them. So let's move along to another real-world example. I'm thinking here in the context of medical applications of prediction models, drug discovery, but of course very important with life and death consequences. So, for example, there's a move to generate artificial data using artificial intelligence to increase our sample size. That sounds kind of hard to imagine that that could work, but apparently it can be a helpful strategy. So what you're asking is very, very interesting. And I think you're touching again on the future of statistical science as a discipline. Statistics has always been an empirical science that tries to make sense of the world around itself. And so now we're dealing with GNI, for example, or extremely fancy machine learning algorithms. So to understand drugs, we started in vivo, like we would just inject people with stuff. Then we did this in in vitro. And now we're moving in in silico, as you point out, right, which is that now we want to use algorithms to make predictions about what drugs will do. And so, let's say you're a big pharma company, then you're sitting on a huge library of compounds. And it can be 400 million, 500 million. And you would like to know which of these compounds will actually bind to a target. So what do you do? Well, you should take your compound one by one and experiment whether they will bind to your target. But as you can imagine, this takes an enormous amount of time and money. So now people are using machine learning to guess whether they will bind. And in the past few years, we've seen things like Alpha Fold. We've seen a lot of models that try to predict the shape of a compound from just sequences of amino acids, for example. Now, that will not replace physical experiment. But what machine learning does in this instance, it's going to prioritize the compound that you should try first. One of the things we do in this area is to say, okay, we're going to train some extraordinary fancy models. And they're really black boxes. I mean, they're so complicated, I have no idea what they do really. But they produce an affinity score, an affinity of a compound for a target disease. And I say, can I trust this? And so without any statistical model, just looking at what the algorithm predicts on molecules on which it was entrained, we were able to select data adaptive threshold, if you will, that says that if you select all these molecules whose predicted affinity is above this threshold, you're guaranteed that 80% of what I'm giving you is actually of interest to you. Downstream, you will do some real experiments on some real things. But here, what's very exciting is that AI can really speed up the prioritization of drugs that should be passed on to a lab. No, it gives a whole new meaning to the concept of an educated guess. These are now brilliantly educated guesses that still have to be tested, as you say. They still have to be tested. Now, there's another thing, which is perhaps this time a bit more scary, which is that what if we use Gen AI to build what people might call digital twins, things that are not physical, but can be generated by generating AI. And so here, there's a new line of research. So for example, suppose I want to study statistical properties of some drugs, right? And the problem is I have too few samples. Let's say I want to kind of estimate which fraction of drugs have a certain property. And the problem is I have a lot of sequence of amino acids for which I have not measured the property. And as you can imagine, the tendency is to use a predictive model, a black box, and replace the real measurement with a prediction, and then pretend that it's real data. And then average now these predictions and say that's the overall fraction of drugs that have their property, and that's wrong, because this method introduced biases. We want to use this predictive model. We want to use Gen AI to filling missing data to possibly create new data set. But at the same time, we need to understand how we can remove the biases to draw conclusions that are scientifically valid. Let me give you an example. Let's say I want just to estimate the mean of a random variable. Let's call this Y. And I have some feature. Let's call them X. And so what I could try to learn is I could fit a model to predict Y from X. And now I can replace the true label, the true amount by the prediction when I don't have it. And I could average those, but they're going to be biased. But guess what? I can remove the bias because I have an estimate of bias from the labeled data you gave me. And so if I do this correctly, I can effectively augment the sample size enormously. If my prediction has reasonable accuracy, then it's as if I had a sample size which is much bigger. And so the level of accuracy of what I can tell you is much higher. Well, I can't resist asking you since it's such a rare treat for us. You're very well known for contributions to an area that people call compressed sensing. And I don't know if it exactly fits into our discussion today, but I feel like I want to ask you to tell us how does compressed sensing and maybe its applications to medical imaging, to MRI or other things, does that fit into the framework we're talking about? And even if it doesn't, could you tell us a little about it? It doesn't fit directly. I think compressed sensing is the fact that sparsity is an important phenomenon. So what we're seeing at the moment is people measure everything under the sun because we don't know ultimately what will matter, right? And so we need people like you and me to sift through what matters. What compressed sensing says is that if we measure the lot of things, but if only a few things matter, and if we use the right algorithm of the kind suggested by compressed sensing theory, then we should be able to build a very accurate predictive model. Like, we will understand that a lot of variables have no business in predicting the outcome. And it will quickly focus on variables that have something to say about the outcome and then build a good predictive model from then on. So you've been using the word sparsity. In this context, does it mean all those variables that don't matter, we can effectively set their contribution to zero? Exactly. So it's saying that in this case, just for our audience, it might say that even though I measured a million genetic variants, the distribution of the phenotype does not depend on this million thing. It may be depend on 20, on 30. That's sparsity. And so the question that compressed sensing asked is that when something depends on a few but unknown from a long list, how do you go about and find them? So the technique or the method will identify which are the key 20 or whatever small number it is? Exactly. Exactly. Let's think about this as almost a matrix problem. So I have a matrix. It has a million columns because these are all the genetic variations. And then I have a response y and these are the rows of this matrix. If I want to solve a system y equals ax, like which genetic variations matter to predict y? Well, classical theory will say that, well, I need as many people as I have unknowns. But compressed sensing theory says, no, that's not true. Because if you know ahead of time that only a few of these genetic variations matter, then you can deal with fewer people. And that's why we can develop predictive model for phenotypes that do not need a million patients. It's wonderful. One of the big issues that seems to be everywhere in science these days is a crisis of reproducibility. And I just wonder if you have statistical comments for us about that. Yeah, it's very interesting that you asked this. I think, first of all, I will make an observation about the reproducibility crisis. It occurs at a moment where people have enormous data sets at their disposal, usually prior to the formulation of scientific hypothesis, access to extremely fancy models that depends on billions of parameters. And so I would say to start with that it's not a coincidence that this crisis occurs at this time. Because I give you a data set, you believe it's gold, you're going to try a model, it doesn't pan out, and you're going to try something else. And so you're fine tuning parameters, you're fine tuning a lot of things until something clicks. And there's nothing wrong with that. But I think what we need to do as statistician, and there's a lot of us are working on things like this, is how can we build safeguards around the freedom you have in selecting models, or such that at the end of the day, the discoveries you claim have a chance of being reproduced by, let's say, an independent experiment. The statistical community is developing a lot of methods so that when you think you have something, you really do have something. And so this is a very exciting moment for the field, to develop methods that are not really quantifying the uncertainty in your prediction, but actually calibrating in such a way that when you report findings, we make sure that a good fraction of what you're reporting is correct. Well, I would like to sort of now back out to a broader, like even societal scale, to think about education just for a minute. Every learned or educated citizen should know something about the ideas of probability and statistics, including in their modern incarnation that we've been talking about. And I wonder if you have thoughts about this, what we could be doing as either educators or communicators to promote greater statistical savvy. That's a good question. I think what I see at lower levels of statistics teaching is a reliance on formulas, which formula should I apply when, and I think that's not helpful. As a student, I learned, of course, mathematical reasoning, and that was important. And then a bit through high school and college, I learned physical reasoning, and that's distinct from mathematical reasoning, and it's extremely powerful. But in grad school at Stanford, I learned about this new thing called inductive reasoning, which is neater of the first two. And I think we need to be doing a good job at teaching this at an early stage. What is inductive reasoning? It's the ability of making generalization out of particular observations. And how do we do this? Okay, so I would promote an approach which is not too mathematical in nature, which is trying to make kids understand how it's possible to generalize from a sample to a population to individuals we haven't seen yet, and what makes this possible? There's a bit of a tension between fields. Should we go towards more mathematics, or should we go more towards CS and where AI is mostly taking place? I think there's a danger of losing the ability to reason statistically if we go either too much towards math and too much towards CS. It might be a bit abstract what this is, but I find statistical reasoning extremely powerful, extremely beautiful, because I don't want to talk about it in general. I'll give you one problem. And it's a famous thing that happened in the '30s, I think. I think Corbett was studying butterflies, and he went to Malaysia for a year, and he was a very concentrous man, so every day he would try to observe species of butterflies, and he wrote in a notebook, "These species I've seen once, and these species I've seen twice, and these species I've seen three times," and so on and so forth. And he came back to England, and he approached one of the founding fathers of the field, R. Fischer, and he asked, "If I go back to Malaysia for six months, how many new species am I going to see?" This is a kind of question. It's different from math. The answer is not in the question. And I don't think deep learning can be very helpful, and that's what statisticians do. And this is a very modern question, which is that you have a lab and they're looking at cancer cells, and they're going to do exactly the same thing. This is how many cancer cells I've seen once. This is how many cancer cells I've seen twice, and they say, "How many cancer cells I have not seen yet?" And if I continue looking for cancer cells for six months, or a year or two years, how many new types am I going to expect it to see? So this is what you learn when you study statistics, and I find it fascinating. Oh, well, that's just great. It's really interesting to hear about the culture of statistics, how it's distinct from that of math or computer science, because nowadays with the rise of what people are calling data science, there's a kind of muddying of the waters. Who owns statistics? Why are we calling it data science? Why isn't it statistics? I'm sure you have an opinion about this. Of course, because there are lots of activities in data science that you would not find traditionally represented in the stats department. So I have a colleague, Gioj Laskowicz, who is a very recognized data scientist. Cornell PhD. Exactly, exactly. He's a brilliant person. And so when COVID hit, people were calculating this beta number, like the model where you're susceptible, exposed, infected, recovered, and you have these differential equations. And if the beta number is greater than one, we have a problem, stuff like that, right? And so this is a very macro model. And what Gioj Laskowicz did was created an enormous digital data set. He tracked 100 million Americans in all major U.S. cities, and so he would see where they would go during the day, where they come home at night. And so instead of fitting like the epidemiological model everybody knows at the global scale, which doesn't really make sense because the behavior in California and the behavior in Florida were very different, then you can fit it at kind of nodes on the graph. And so you're going to fit a model which is adapted to the mobility of the people where you are. And that is data science, because what Gioj did, which you will not see in a stats department, is he basically tracked 100 million people for a few weeks. I would like to claim that I have some colleagues in a stats department who would do something like this, but I cannot name any. And that is modern data science. This is not something I typically see in a stats department. So my position is quite clear on this. Data science is much bigger than the traditional field of statistics, but statistics is one of its intellectual pillar. Oh, I'm so glad I asked you about that. I hit a gold mine with that one. But all right, you've already expressed your fascination with statistical thinking. Is there something in your research that brings you particular joy? Yeah, I think so. My job at Stanford is unique in the sense that the students I get to work with are phenomenal. I feel that it keeps me young. It keeps me alert. I don't fall asleep because I just have to catch up with them all the time. And I feel that it's strange to say this on air, but I'm going to age better because of this, because like mentally, physically, they keep me fit. And it's a joy to see them develop, become great scientists last year. I had two former students who received a MacArthur Fellowship in the same year. So the students I've got to work with are tremendously accomplished. And so it's just a privilege. It's a privilege to feel so much energy, so much enthusiasm for the subject. And selfishly, I would say that it's good for my health. Well, thank you very much. It's been really fun to talk to you. We've been speaking with mathematician and statistician, Emanuel Candes. Thanks again for joining us on The Joy of Why. Thank you for your time. It's been a pleasure. Thanks for listening. If you're enjoying The Joy of Why and you're not already subscribed, hit the subscribe or follow button where you're listening. You can also leave a review for the show. It helps people find this podcast. The Joy of Why is a podcast from Quantum Magazine, an editorially independent publication supported by the Simons Foundation. Funding decisions by the Simons Foundation have no influence on the selection of topics, guests, or other editorial decisions in this podcast or in Quantum Magazine. The Joy of Why is produced by PRX Productions. The production team is Caitlin Falz, Livia Brock, Genevieve Sponsor, and Merritt Jacob. The executive producer of PRX Productions is Jocelyn Gonzalez. Morgan Church and Edwin Ochoa provided additional assistance. From Quantum Magazine, John Rennie and Thomas Lynn provided editorial guidance with support from Matt Carlstrom, Samuel Velasco, Arlene Santana, and Megan Wilcoxon. Samir Patel is Quanta's editor-in-chief. Our theme music is from APM Music. Julian Lynn came up with the podcast name. The episode "Art" is by Peter Greenwood and our logo is by Jackie King and Christina Armitage. Special thanks to the Columbia Journalism School and Bert Odom Reed at the Cornell Broadcast Studios. I'm your host, Steve Strogetz. If you have any questions or comments for us, please email us at [email protected]. Thanks for listening.