James Boyle - AI & Personhood

Support for this podcast comes from Is Business Broken, a podcast from the Meirotra Institute at BU Questrum School of Business. A recent episode asks, "Are boardrooms ready for the new geopolitical reality?" Stick around until the end of this podcast to preview the episode. WBUR Podcasts. Boston. This is on point. I'm Megna Chakrabardi. When the US military launched its air attack on Iran this weekend, it used artificial intelligence tools to help with the strike. Now, this is not a surprise, but what is a surprise is that the DOD reportedly used AI services from Anthropic, the company that created the AI tool Clawed, and it did so just one day after the Pentagon declared Anthropic a supply chain risk. That is, a threat to national security. But the military went ahead and used Anthropic's AI in the most consequential military operation in the Middle East since the Iraq War. So, is Anthropic a threat to US national security? Or isn't it? Well, let's take a moment to recap the quite complicated backstory to this question. Anthropic is one of the premier contractors of AI for the US military. And many analysts say that Anthropic's tools are superior to others that the Pentagon has at its disposal. But last month, a major dispute broke out into the public when defense secretary Pete Hegseth started blasting Anthropic as, quote, "woke AI." And what fueled this defense secretary to deploy such a Trumpian smear? Anthropic has a very hard line on how its technology can be used by any military. It will not allow its AI tools to be used on mass surveillance of Americans. And it won't allow it to be used in weapon systems that do not need human oversight to launch attacks. Over a series of very public statements reporting and meetings and Anthropics CEO Dario Amadeh refused to water down those rules regardless of pressure that he received from Secretary of Defense Pete Hegseth. So finally, on Friday, Hegseth ordered the Pentagon to designate Anthropic a supply chain risk. And just hours after that decision, Anthropic CEO Dario Amadeh appeared on CBS to respond. And the very first thing he wanted to assure the American people is that he actually supports the use of AI by the military. I believe that we have to defend our country. I believe we have to defend our country from autocratic adversaries like China and like Russia. But he says defending the country also means upholding America's long-held tradition of democracy. I have always believed that, you know, as we defend ourselves against our autocratic adversaries, we have to do so in ways that defend our democratic values and preserve our democratic values. His unwillingness to let the military use an unfettered version of Claude AI also rests on a simple technological truth. The AI systems of today are nowhere near reliable enough to make fully autonomous weapons. You know, anyone who's worked with AI models understands that there's a basic unpredictability to them that in a purely technical way we have not solved. And there's an oversight question too. If you have a large army of drones or robots that can operate without any human oversight, whether there aren't human soldiers to make the decisions about who to target, who to shoot at, that that presents concerns. And we need to have a conversation about how that's overseen. No, of course, Secretary Hegseth and the military disagree and insist that AI used by the military should be completely unfettered. Now, the Pentagon V Anthropic Fight is about so many deeply and definitively important issues of our time. How should the military use AI? What are the limits if any that should be placed on that use? And what are the consequences seen and unforeseen when the federal government seeks to kneecap a company simply because that private business will not sell the exact product the government wants. So let's start with Stephen Levy. He is an editor at large at wire and he's recently written about this in a piece called AI Safety Meets, the war machine. Stephen Levy, welcome to one point. Thank you. Happy to be here. So where are we right now? Just bring us up to date with the latest. If there's been any change since Friday on this supply chain risk designation and the relationship between the Pentagon and Anthropic. Well, there have been changes for one thing as you mentioned. We're at war now and Anthropics AI, Claude is going to war for the United States. And this is as you point out, the technology which is sufficiently dangerous in the Pentagon's view, they want to not only stop using it themselves, but they don't want to do business with any other company that uses it. And depending on how that's interpreted, that could be a critical factor in whether Anthropic lives or not as a company. Okay. So doesn't this not only as an existential threat to Anthropic, but how many companies does it sort of leave in terms of the ecosystem of all the tech that the US military contract out for? I mean, it seems to be a huge, huge decision. Yeah, well, the president's social media message seems to imply that the whole government will enforce this ban. So basically every major corporation does business with the government. So if you interpret that broadly as the president seems to, and Pete Heck's death doesn't seem to disagree with that, that would be an unbelievable limit on who Anthropic and Celto. So it, especially since Anthropics business plan is the Celto corporations. They've gone to an extreme not only to say we don't want to include you in our military suite of armaments because you won't let us use it as we want, but we're going to punish you and basically try to end you as a company. From what I recall, back less summer, the Pentagon actually gave the same amount, roughly a $200 million contract to not just Anthropic, but Google, OpenAI and XAI. But the thing is, is that Anthropic was the first to be cleared for classified use because military officials considered it the most advanced and secure model for really sensitive military applications. So that's the Pentagon saying that Anthropic was delivering the best product, right? That's right. I think they still believe that. Now the other development that happened very recently is that OpenAI, which is Anthropics public key competitor, has jumped in and completed a contract with the Pentagon. They haven't done any implementation yet, but they've gotten the classified status and are willing to work with the Pentagon to take the place of clawed in the Pentagon's plans. And there's an amazing backstory to that in that Anthropic was founded by people who worked for OpenAI, who left because they felt that OpenAI wasn't developing as technology with safety provisions that they felt was necessary. So I don't know if you can give us any insight into this, Steven, but Anthropics has always been very public about these really hard lines that it follows regarding how its AI is going to be used. Now the Pentagon had to know this beforehand. So why even go forward with any kind of contract knowing that ultimately what it wants to use, Anthropics simply wouldn't let it do that. I mean, do you have any insight on that? Yeah, I do actually. So when they formed the contract last year, Anthropic felt that they had those safeguards. They wouldn't be using it for mass surveillance of Americans or for autonomous weapons as you described. So what happened was they got those assurances, but something happened that's under dispute a couple months ago. After the government displaced Maduro in Venezuela, Anthropic met with Palantir, which was accompanied, it uses their technology and is used by the government. And Palantir told Anthropic that by the way, your stuff was used in that raid in Venezuela. And now this is under dispute. Anthropic denies this, but apparently the people of Palantir heard that Anthropics say, "Well, we don't like that," or something that was misgiving about that. In any case, they apparently went to the Pentagon and said, "You know, Anthropic might not be on board for all you want to do." And it very well could be that incident led the Pentagon to say, "You know what, we have to have a total green light to use whatever technology we're buying. So we're going to push this and make those red lines go away. You know, no longer will we accept the red lines. And that might have been what forced this confrontation." Interesting. And Palantir, of course, was launched what? Some 20 years ago by Peter Teal. So this a very entangled web of tech and politics. Stephen Levy, Editor-at-large for Wired. Thank you so much for helping us get started. - Okay, no problem. - We'll be right back. This is on point. (upbeat music) - Support for this podcast comes from Is Business Broken, a podcast from the Meirotra Institute at BU Questrum School of Business. When a geopolitical crisis hits, how should a board respond? Perhaps by helping the company find its core values. - Who are we? What is our vision? What is our strategy? As an organization, what is our ethos? What are our morals and values? - Follow Is Business Broken, wherever you get your podcasts, and stick around until the end of this podcast to preview a recent episode. (upbeat music) - Right now we are living through some of the most tumultuous political times our country has ever known. I'm David Remnik and each week on The New Yorker Radio Hour, I'll try to make sense of what's happening. Alongside politicians and thinkers like Cory Booker, Nancy Pelosi, Liz Cheney, Tim Walts, Katangi Brown Jackson, Newt Gingrich, Robert F. Kennedy Jr., Charlemagne the God, and so many more. That's all The New Yorker Radio Hour, wherever you listen to podcasts. I'd like to turn now to Heather Roth. She's a senior research scientist at the Center for Naval Analysis, and back in 2020, she was the primary author of the Department of Defense's AI Ethics Principle and she joins us today from Loudoun County in Virginia. Heather Roth, welcome back to the show. - Hi, Magnagh, great to be here. Thank you for having me. - Can you give us a broad picture of the Pentagon's use of AI today? I know that that's like a 17 hour answer. (laughing) But just to give us a sense as to why it is so important in everything from big data analysis, we'd mentioned Palantir before we'll talk about that more later, too. Obviously, even as we solve the weekend, targeting and airstrikes in an active war in Iran. - Yeah, I mean, we can basically say, look, this is a suite of tools, a suite of techniques, right, where you can have very well-defined algorithms that are deterministic in nature and we've been using those in the Department of Defense for decades, right? Everything from missile defense to some, you know, rudimentary planning and things like that. And then you get into more of the more nuanced flavor of the day AI right now, right? With generative AI, LLMs, large language models, like Cod and ChatGPT and these other systems, you know, this could be everything from putting in different types of doctrine concept notes to figure out like what's the most important thing that I need to pull out of this to looking at data processing and large data analysis, patterns, predictions, things of that nature. But there are limitations to how far these models can go, especially when we're talking about high stakes operations. - Okay, so I don't know if you're willing to theorize, but I'm very fascinated if we can speculate how AI might have been used to say even over the weekend in those strikes against Iran. - I mean, speculating. (laughing) - Speculating, like look, what, you know, so Palantir has what's called the Mavinsmart system. These are, you know, very data intensive software platforms. There's data visualizations, you know, what you wanna see where ships are. You know, if you want to be able to look at satellite imagery for targeting without getting to into the weeds and on the high side, it's about data visualization and it's about planning and operations. And so I can see, right, if you're using Mavinsmart system as part of your intelligence surveillance and reconnaissance and your planning processes, then you're using, quote, unquote, AI. Right, there's a ton of information that goes into these types of operations. And so that's what these systems are trying to do, right? They're trying to get every single piece of data and then visualize it to whatever person they need to within the chain of command so that the command can make decisions. - Okay, can I just for a second put the Pentagon and what the Secretary of Defense has said and done recently against Anthropic? I wanna put that aside, because thus far in the conversation, we've been kind of operating as if everybody knows what the Anthropic is that people are familiar with, Dario Amade's sort of view on what AI should and shouldn't be used for. So can you just tell us a little bit about has Anthropic under Amade's leadership always had this kind of desire to put some guardrails around what AI can do versus other companies? - Yeah, no, I think he has. I met Dario for the first time back in 2014, 2015 and he and a group of then open AI folks as well as some other academics and myself. We wrote a big paper called the Malicious Uses of AI, where we thought about all the different ways AI could be misused coming in the future. It's kind of a warning shot like, "Hey, we should think about this." And so that was over 10 years ago and so Dario had that on his mind about how do I do safe development? He thinks and believes that AI can do good things but that we have to be measured and how we go about it. - Also, I'd say in the past year, he's been incredibly public, right? In sounding these along, writing op-eds in the New York Times saying like, "We need to be thinking more thoroughly about what AI can do before it gets too far ahead of us that we need regulatory regimes." For example, here Dario is in Davos earlier this year and he talks specifically about how certain types of governments could use AI technologies to bridge surveillance and attack capabilities. - I am concerned that AI may be uniquely well suited to autocracy and to deepening the repression that we see in autocracies. We already see it in the kind of surveillance state that is possible with today's technology. But if you think of the extent to which AI can make individualized propaganda, can break into any computer system in the world, can surveil everyone in a population, detect, dissent, everywhere, and suppress it, make a huge army of drones that could go after each individual person. It's really scary. It's really scary and we have to stop it. - So that makes me want to take us further, even further back in time, Heather, because we had you on in 2021, I believe. (laughing) That was before anyone, just sort of, we regular Joe's out there, even knew that we could use AI in our daily lives. It feels like a completely different world. But we had you on that back then to talk about the military and ethical concerns around AI, because you had written the Department of Defense AI Ethics Principles document. So what was the DOD thinking back then? Regarding ethical limitations or considerations that they should have when it comes to using AI? - Sure. Yeah, also, magnet, that just makes me feel even older. (laughing) Tell me about it. (laughing) - But yeah, no, I think, so that project started under President Trump's first term. - Right. - Yeah. - So the project started in 2018 when we started hosting a series of expert round tables and discussions within the department and then without externally, and then it culminated right in these five principles with this very long supporting document that explains all the principles. There was a consensus around, yes, we want to use these tools, but we want to make sure that we have sort of the right grounding for what is required. Why are these tools different than any other tools that the department uses? And why do we need a separate set of rules for them? - Five ethical principles. - Correct. - And I have them here in front of me. I just want to take a second to go through them because it does tell us a lot about how the military was thinking about this at the time, right? So I see here that the five ethical principles begin with responsible, that the DOD personnel will exercise appropriate judgment and remain responsible for AI development and use. Tell us a little bit about that. - Yeah, I mean, humans are the responsible agents. The tool is not responsible, right? It's like, if a rock fell down off of a mountain, you wouldn't blame the rock. It's so, there's a chain of command and so responsibility grounds all of the other principles, right? This is a tool for human use. We don't want to design tools in such a way that sort of obfuscate that responsibility. So responsible is that grounding principle, then it goes into the notion of equitable. And this is about really unintended bias. And the new AI strategy that was recently released talks about sort of indirectly the equitable principle as one of these woke principles. But that's not really what was meant when this was put together, right? It's about biasing the system to look for certain characteristics or patterns in the data. I want it to go in a particular direction. I want the system to push towards these types of cases. And that can be everything from, you know, gosh, all of a sudden, people's pay rise. we're going in a direction I didn't know, right? I mean, can be everything from business to logistics to operations. - Right, and then, so I'm sorry to jump in here, 'cause then you also, these all build on each other, right, you have traceability. - Yep. - So like being able to trace the systems, its decisions, audit them, et cetera. Reliable, that one's like self-evident, right? We need the system to be reliable. I have, incovernable, right? That to avoid those unintended consequences, that the system could be deactivated if necessary. And what was really interesting to me is like, the overall goal here, as far as I understand, Heather, and this is of course back in 2020, was to ensure that AI systems were, would be secure, reliable, and compliant with, - Yes. - the ethical frameworks, such as the laws of war. - Correct. - Make sense. Now, this is no longer the ethical principles that are active at the Pentagon, right? You're just saying that there's a whole new set here. - I mean, it's hard to, I don't know how you want to frame it. So once the principles came out, it became a lot of a mouthful to talk about the AI ethics principles. And so people just started short-handing that to responsible AI, or R-A-I. And so if it's responsible AI, it's meeting these principles, these requirements, right? We're doing the right testing. We've designed it in such a way that the human understands what's going on. You can figure out where the errors are. We can pull it back, right? There's, that was sort of what we're talking about, when we're talking about responsible AI. It's unclear to me if just the word equitable because become a hot button word. And so it was sort of skewed in that strategy document. But there's a pressure to move faster and to innovate more and to deploy prototypes and demos more quickly. - Got it. - That being said, it's hard to do that and go, are we doing all over the things we could to make sure that this is the best piece of equipment? Or are we just going fast? - Support for AI coverage in on point comes from MathWorks, creator of MATLAB and Simulink Software for technical computing and model-based design. MathWorks, accelerating the pace of discovery in engineering and science, learn more at mathworks.com. And from Olin College of Engineering, committed to introducing students to the ethical implications of artificial intelligence in engineering through classes like AI and society. Olin.edu. (upbeat music) - Heather, thank you for bearing with me and digging through previous versions of the AI ethics principles as the Pentagon saw them because at the time spent in the weeds, for me really helped sort of put today's debate in quite sharp relief. So I just want to listen to a couple of examples of different points of view on what AI can and should be used by the US military. So to remind folks at the top, anthropics sort of no-go line for the use of cloth and their other AI technologies by the Pentagon, what two things, one are weapon systems that could self-deploy without human oversight and two, the use of AI or their products for the mass surveillance of Americans specifically. So anthropics CEO, Dario Amade, talked about this on the New York Times, but one of the New York Times podcasts back in mid-February. And here's what he discussed regarding how AI could be used by authoritarian states or basically any government to surveil citizens. - Think about the Fourth Amendment. It is not illegal to put cameras around everywhere in public space and record every conversation. It's a public space. You don't have a right to privacy in a public space. But today, the government couldn't record that all and make sense of it. With AI, the ability to transcribe speech, to look through it, correlate it all, you could say, oh, there's this, this person is a member of the opposition. This person is expressing this view and make a map of all 100 million. And so, are you going to make a mockery of the Fourth Amendment by the technology finding kind of technical ways around it? - So here's another view. This is from Palantir CEO Alex Carp. And he thinks that American citizens have already accepted a certain level of surveillance. And he's downplayed concerns over what's called this pattern of life surveillance because it's enacted by corporations and not by the government. - The primary evidence for a surveillance state in the West is not government on consumer. It's company knowing every single action you have at all times. And we walk around with surveillance devices, called electronic devices, and every single thing we do is monitored, not I think primarily so people can eviscerate or have an understanding of, am I shagging with too many people on the side and lying to my partner or lying by a mission? Because they want to sell us like cornflakes. Okay Heather, so let's take this back to the defense department. The law says as far as I read it that the department of defense cannot engage in mass surveillance of Americans. So a lot of people say nothing to see here. There's no worry. They couldn't even use the AI to do that if they wanted. But other people, obviously including and Thropic CEO say, well then why do you want Claude to be able to do that for you? Like why is this a line that you say, we want Claude AI to be or Anthropics AI to be unleashed in order to be able to do that for the defense department. I mean, how do you read this Heather? I mean, I think there's a couple of things here that are, I don't know if they're strawmen, arguments or not. Like as you say, so Title 10, right of the US code, is about governing military behaviors, right, with the militaries allowed to do. Title 50 is about the intelligence community and what the intelligence community is allowed to do. When it comes to the surveillance of American citizens versus non-citizens and then people who are outside of America, right? So then it becomes a question of, well, are we, is the DOD using these to surveil on citizens inside the country? And you go, well, that's not really their remit. They're about fighting worse elsewhere. Now we can talk about whether or not, you know, using national guard troops within US cities. And if there's, that's a whole other above my pay grade decision that I am not comfortable getting into the weeds on that. But, you know, so you can say, well, here's the military's kit and it's designed to do what, right? So is it AI to suck up a bunch of data over here, here, here, and here and then correlate patterns of life for whatever you're observing? Well, then you have to figure out where am I using it? When is it appropriate? Right, who's using it? Now, it should also be said that Palantir sells it's kit to the Department of Homeland Security. Uh-huh. Right? So the Department of Homeland Security does surveil. Yeah. I was going to say that was Carp's misdirection there, right? Like, don't worry. It's only Kellogg's. It's watching what kind of cereal you eat. No, clearly not. No. And I think the other thing to know about Palantir's products, right, is that what they do is they take all of the different data points, right? Who's doing what at what time and what location? What are they doing? What are they doing? And they call that an object, right? So an object becomes a single point of convergence for all of the data around whatever is related to that thing. And that's how they're able to build these pictures of you, right? Oh, you spent so much time on wafer looking at deck chairs. And then you went to work today. You're in the car, right? There's all sorts of these devices that are gathering data. And it can create a very robust picture of your behavior. And that's sort of this semantic object model, right, that Palantir uses. And so we can use that model for anything, right? And so if we're using it in a military campaign or an operation and we're using it abroad, right, we're trying to get as much information from that's publicly available in the open sources, open source intelligence. You're trying to get all sorts of satellite data. You've got anything that you can that have a better information picture to undertake your operation. Now if it's in the United States, then Dario has to have a conversation with David Kerv about them selling the product to DHS. Well, so then let's turn to the other one, the other issue, because I think maybe that's more in a sense, more concrete, but at the same time, complex, more complex. And that is, Anthropic says said, no, it does not want its tools to be used to create weapons systems that could basically work entirely without human approval or human oversight. Now this one, I have heard from a number of people who are thoughtful military analysts saying, we need to build structures that would prevent accidental fire rings or the starting of a nuclear war or whatnot. But at the same time, we cannot say that the military should not have such a capability, because well, everyone else is going to do it. China is going to do it. So putting the shackles on the Pentagon here could actually make us weaker overall. Yeah, I mean, I've always thought that that's sort of a red herring for an argument. We want to be responsible, but not when it's hard. If it's hard, then we want to make sure that we have all the rules are there are no rules. And so I would say, look, when it comes to autonomous weapons, that in a few minutes. of itself as a definitional thing, right? I don't know any military that's going to be like, you know what I really want to create is a system that launches itself and I have no idea where it is and what it's doing, right? Like that sort of, I think outside of the bounds of what any reasonable military in the world would whatever want, right? Now, do they want to create systems that have autonomous behaviors that they can operate in denied environments so they can locate on their own to a particular location and then select and engage a target? Yeah, some militaries do want to do that, right? ours included and our military does not preclude that from happening, right? The DOD policy, which is DOD Directive 3000.09, states that for autonomous and semi-atonomous weapons they have to have certain levels of approval when they're being tested and validated and sign-offs for acquisition and things of that nature, but it does not preclude us from going and generating them and procuring them and using and deploying them. Can you give us a quick example? I don't know if there's sort of autonomous systems within things like submarines? Yeah, I mean like, well, I mean you, putting the definition between automated and autonomous sometimes it gets a little bit gray. Okay, and so you can some say something like a landmine, you could say, well, it's autonomous, it selects and engage at a target on its own, right? And the selection is just how much weight goes on that pressure plate. Some people would say, no, that's not autonomous, it's automated, right? It's automatic. We use automatic target recognition on a large portion of our missiles, our precision guided munitions, right? They do terrain mapping, they can maneuver themselves in time and space by themselves, right? A human is not guiding that missile. Once they find their target, you can look at anti-radar missiles, right? They can find a radar signature lock on that signature. So there's a large number of systems that we use that are pretty sophisticated, right? And you could say, well, that's autonomous, that's not autonomous based on your definition. For the DOD, the definition is that a weapon system that can select and engage a target without intervention by a human operator, right? So we can fire and forget weapons. Israel has them. There's a large class of the Shahad drone that you know, Iran's been using. There are, you know, FPV drones in Ukraine, right? There's all sorts of these different types of systems that you say, look, I'm going to launch it and it's ATR, it's automatic target recognition software is going to figure out when it's got to that place and it's going to say, yep, that's the thing that I was trained to do. And these are the sensor inputs that I have. And it says to this confidence level, that's the thing I'm looking at. And therefore, I engage, right? That's an autonomous behavior. And this already exists, you're saying? Correct. And this is something that the human oversight comes from what target selection to begin with? It comes down the whole chain, right? So, I mean, these weapons don't just design themselves de novo. So, right, you have to figure out like what's the delivery platform that the weapon system is on, right? Is it a missile? Is it a drone? Is it a boat? Is it a torpedo? And then what's the targeting software? And then what's the data that that software is using, right? We can talk about it as AI or we can talk about a software, right? But what is the data that it's trained on so that it knows what it's looking for, right? So, it sounds like you're saying, though, that, and I don't know if I'm hearing this incorrectly, that anthropics concerns then would be moot if there are, as you're describing it all, like, human. Well, I think they're, I think where they're going is there think fully autonomous weapon systems. And so they're, it's like, well, okay, then are you okay with semi-autonomous weapon systems? And where is the line between semi-autonomous and fully autonomous? And that's to me where it's really unclear. Like, if your definition of fully autonomous weapon system is a system that just launches itself and decides to go to war one day, I don't know any military system that has that, right? Like, when you engage in an operation, right? There's, there's operational planning. You are trying to figure out, right? Like, what is the military objective that I'm seeking? And then what is the force that I'm going to apply to get to that military objective? What is the likely probable collateral damage, right? Is there civilian casualties? Is that, is it objects and civilian objects that are going to be destroyed? What is the, so all of these considerations are happening, right? And so when you're, when you're saying, well, if it's fully autonomous, is it doing all of those things? And there's nobody? Well, that seems to mean a little bit not how militaries work. And so we can say that there are times when we might have autonomous systems that are out for extended periods of time. Now we have to think about policies and training and tactics and procedures about how long it's too long. You know, if this is a submersible system that's out for, is it a day, a week, a month, right? Like, what are the types of training and testing and validation that we make sure that that system is doing the behaviors we want it to and not learning while it's deployed and it's getting, you know, skewed. You're like, oh, it's learned some new tricks. Heather Roth, Senior Research Scientist at the Center for Naval Analysis. And back in 2020, she was the primary author of the Department of Defense's AI Ethics Principles Document. Thank you so much for joining us again, Heather. Thank you. Have a great day. I'm Agnachakrabarty. This is on point. The Trump administration says it's battling illegal immigration. But the real war it's waging is behind the scenes against all legal immigration. A normally operating Congress would be demanding oversight. That's not happening. How the White House and Department of Homeland Security are shutting down legal immigration pathways to the United States. That's on the next on point. Support for this podcast comes from Is Business Broken, a podcast from the Meirotra Institute at BU Question School of Business. Follow Is Business Broken wherever you get your podcasts and listen on for a preview of a recent episode that asks how boards should navigate geopolitical tensions. Kurt, I think that your intuition about both dynamics being stressed is exactly the point. I think that in 2025, the biggest problem is board bandwidth because we ask boards so much. We ask them to deal with those geopolitical tensions and we ask them to do their security, cybersecurity and the climate. And by the way, also the normal things of financial reporting and succession planning and higher the CEO and executive pay. And even if the directors are really, really great and they're independent and they expert and they have all the information, you need like the perfect dynamics, the perfect processes, the perfect pre-reading, pre-meeting materials. You need everything to be perfect just in order for you to fit all those huge issues into one agenda. Find the full episode by searching for Is Business Broken wherever you get your podcasts and learn more about the May Rotra Institute for Business, Markets and Society at ibms.bu.edu.