Machine learning and climate risk adaptation

[Music] Welcome to Science Sessions, the podcast of the proceedings of the National Academy of Sciences, where we connect you with Academy members, researchers, and policymakers. Join us as we explore the stories behind the science. I'm Paul Gabrielson. It's difficult to plan for future climate hazards, because climate projections themselves have uncertainties. If you're planning to protect a large city like, say, New York City, you have to be prepared for a range of scenarios. But mitigating every scenario is probably too costly. There's a balance between costs and benefits. In a recent PNAS study, NingLian of Princeton University and colleagues, employed a machine learning method called reinforcement learning, to explore ways for New York City to adapt to a range of climate hazards at minimal cost. The results show that reinforcement learning models may help New York and other coastal cities prepare for an inherently uncertain future. Ning, let's start with some background. How does uncertainty and climate projections affect risk management? So, risk management and specifically climate change adaptation strategies have to be based on climate projections of extreme events or other climate events in the future. And those climate projections have large uncertainties, because the complexity of the Earth system and also because of uncertainty in our future climate policy. And those uncertainties will be transferred into the uncertainty of our strategies and our risk management methods or applications. We know that still our world is rising, but to what extent is still quite uncertain due to the uncertainty of science in terms of the ice sheets, behavior, under global warming. So, we may be seeing a general distribution of sea level rise, but we also are concerned on some large increase of sea level rise, we are the normal distributions that we are dealing with. And if we determine our strategies based on a distribution that does not account for large disruptive levels of sea level rise in the future, we may be under protected or under a debt, which may induce large impact in the coastal regions in the future when the extreme event happens. So, we have to be aware of this uncertainties and incorporate this uncertainties in our decision making so that our decision making are robust to this uncertainties. How does reinforcement learning work? Reinforcement learning is an area of machine learning regarding specifically how agents or decision makers were made decision act in a changing environmental states in order to maximize their cumulative rewards. So, in this study, we investigate this potential and performance of reinforcement learning when applied to adaptive climate decision making. Specifically, we are applying reinforcement learning to design coastal flood protection strategies that involve over time according to future observations of sea level rise, with the goal of minimizing the total cost, including the investment and the damage over time. Why did you focus on coastal flood protection for Manhattan? Although it's located at a really high latitude and affected last by hurricanes as compared to the lower latitude regions like Florida or the Gulf Coast, New York City is actually highly vulnerable to storm surge due to its geophysical features and socioeconomic significance. And then in 2012, we personally experienced superstorm Sandy, which brought that stating flood damage to both New Jersey where we are and also New York City, exposing our high regional vulnerability to storm surge flooding. Since then, various proposals on coastal protection for the region, especially for New York City, have been discussed. For example, we had a big U. It's a protective sea war system around the lowling Manhattan. It was proposed and actually partially constructed. Recently, the US Army course of engineers proposed protection plans with multiple strategies including building sea walls and dikes, also relocating communities around the New York City area. But those plans, including the big U and recent proposals, are based on traditional design criteria, such as designing static sea walls for the 100-year flood level without fully considering the dynamics and uncertainty of climate projections. How did the reinforcement learning model do at risk management? So in our study, in order to assess the performance of reinforcement learning, we compelled reinforcement learning with a big U design and with previous optimization approaches in terms of designing the flood management strategies for lower Manhattan. So our results show that the reinforcement learning can minimize the net cost, life cycle cost, or climate adaptation decisions, specifically reinforcement learning method outperforms the big U design method, also cost-benefit-based static design, and other previous learning-based adaptive methods in achieving lower expected total cost over the life cycle of the project. What surprised you about the results? So there may surprises to me. One is that although the reinforcement learning method we applied aimed at achieving the minimum expected cost, it also effectively limits the tail risk. When we say tail risk, that means the extreme events with high consequences and low probabilities. So reinforcement learning could limit this kind of risk compared to other methods. The reason it is important is that the trade-off between risk and return is like a fundamental principle in financing investment theory. So it is noteworthy that reinforcement learning can achieve minimizing the economic cost and at the same time limiting the risk. So that's one surprise. Second, we found that reinforcement learning method can achieve lowest regret among the models. By regret, we mean the loss we have due to some uncertain climate conditions that we didn't know. So our decisions may induce so certain regret. And we found that reinforcement learning can achieve the lowest regret when we have deep uncertainties in our future climate conditions. This is particularly important given that the projections of future climate states are largely uncertain given that they depend on future climate policy and complex climate physics. So we are happy surprised to see these additional advantages, including lower expected cost, because those mean that the strategy is outperforming other methods in multiple measures. What are the takeaways for emergency planners? So this study demonstrates that advantage of reinforcement learning method, but essentially it demonstrates that advantage of observing and learning in flexible climate adaptation. The study provides a numerical tool that planners can use to design learning-based adaptation strategies. Some planners may think that this is a new idea of developing adaptive strategies and may not think it's easily implemented, but I want to say that it has certain advantages that actually can be applied better than traditional methods, because the initial investment from learning-based adaptive methods are lower than those by non-adaptive methods. Specifically, non-adaptive strategies need to be more conservative as they need to cover the large uncertainty and risk for their entire life cycle. What are adaptive strategies can be more optimistic at the start, and they can adjust themselves over time according to future observations. So in practice, both the large uncertainty and a high-inissue investment reduce stakeholders' willingness to actively implement protective strategies. So we can argue that the adaptive design can provide a promising approach to climate adaptation as stakeholders may be more willing to invest in policy or projects that respond to future scenarios and operate at lower initial cost. What are the caveats or limitations of the study? So our study focuses on economic optimization while real-world decisions must balance the completing inches of multiple stakeholders and political entities. Some of these factors may be accounted in this optimization, such as indirect economic costs or social costs, but some may not be pointifiable mathematically. So the planners need to consider other practical constraints in making decisions in addition to the economic optimization that we can provide here. Also, the current algorithm is rather complex and a computational intensive. So we need to simplify it, possibly through other immersion learning methods. Finally, some decision scenarios are complex and there are a lot of them. We have vast number of design scenarios. So we need to develop some better visualization to support the decennemakers to use these tools with understanding of these different scenarios and apply them more effectively. Thanks for tuning into Science Sessions. You can subscribe to Science Sessions on iTunes, Spotify, or wherever you get your podcasts. If you liked this episode, please consider leaving a review and helping us spread the word. [MUSIC]