Speaker 1In the wake of Dario Amadei's note about pacing the frontier, published this weekend, and the corresponding agreement from the other lab leaders, there is now a shift in the conversation to figure out where we go next. On the one hand, some folks are getting more nuanced, trying to break apart questions of whether we should be having a debate about AI risks and the best way to place guardrails around them and to involve society in the decision-making of the AI labs, from questions of the specific proposals that the labs are bringing to that effect. And then there is the partisan wrecking ball, the thundering cataclysm of congressional calls for regulation, and of course, an absolute torrent of truth social posts. The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. All right, friends, quick announcements before we dive in. First of all, thank you to today's sponsors, KPMG, Blitzy, Robots and Pencils, and HyperAgent. To get an ad-free version of the show, go to patreon.com slash AI Daily Brief, or go to www.patreon.com slash AI Daily Brief. Or you can subscribe on Apple Podcasts. To learn more about sponsoring the show, send us a note at sponsors at AIDailyBrief.ai. AIDailyBrief.ai is also where you're going to be able to find everything that's going on in the community. For example, right now, we have the multiplayer AI sprint for Teams Live. It's all about finding the agents that are going to live at the intersections of your work, which I think is going to be a key trend for the months to come. Again, you can find a link to that at the top of AIDailyBrief.ai. Prominent mathematician Terence Tao has led 25 field medalists in an open letter objecting to AI companies engaging with academic mathematics. It seems like so long ago, but prior to the middle of last week, the biggest conversation in AI was about OpenAI solving a Millennium Prize problem. NYU professor Tristan Buckmaster, who had been working on the problem over the past year using AI tools, questioned whether OpenAI could have accessed his logs to scoop the prize. OpenAI denied accessing the logs and also claimed to have made significant progress on a second Millennium Prize problem, with rumors swirling that Anthropic had solved yet a second Millennium Prize problem. Also, there is a meaningful prospect that three of the six outstanding problems could be solved this year. These problems have stood for decades as the pinnacle achievement in the field. So it is unsurprising that an AI solving them is raising big questions about research mathematics more broadly. On Friday, field medalist Terence Tao led 25 of his peers in signing an open letter called A Severe Misalignment of AI in Mathematics. The misalignment they pointed to was not between AI and humanity, but between the AI companies and the scientific fields they're entering. The mathematicians wrote: "Over the last few months, the mathematical capabilities of LLMs have improved dramatically, to the point that they can solve major outstanding problems in many fields of mathematics. However, the push by AI companies to solve mathematical problems as a benchmark is detrimental to the science of mathematics and to the mathematical community. The goals of the AI companies and the goals of the mathematical community are severely misaligned. We see these as part of broader alignment issues impacting other scientific and creative professions, as well as the whole of society." Now, some viewed this statement as lamenting a loss of status among mathematicians. Solving a Millennium Prize problem makes one a shoo-in for the Fields Medal, which is the equivalent of the Nobel Prize in Mathematics. Yet, when you dig in, the letter was more about how automating scientific progress undermines the entire endeavor. The letter stated, "Famous problems have often served as landmarks and lighthouses against which one can measure an improved understanding of this landscape. Solving one of these problems has been a certain sign of new insights and interesting methods, which would then be studied by a community of mathematicians through a long, and arduous process of talks, discussions, simplifications. Some of the mathematical ideas pursue their journey even further to become, decades or centuries after, tools that are understood and used by the whole population." The letter expressed a concern about students being trained to think deeply about unsolved problems, about AI companies rushing to publish a result in order to beat humans working on the same problem, and even the field of mathematics itself. The mathematicians wrote, "Solving problems is only a tool and proxy for achieving the primary goal of conceptual understanding and insight. Forgetting this in the world of AI may turn the tool against the primary goal. Indeed, the mass production at faster and faster pace of true/false statements could destroy fertile ground instead of breathing life into new ideas." Ultimately, the mathematicians who signed the letter framed their small corner of academia as a microcosm for broader human society and one that is being disrupted quite early compared to most fields. They wrote, "We are witnessing a general threat to intellectual work, with misalignment between the outcome of the use of AI and its initial purpose. In many fields and activities, years of training have traditionally served not only to produce a final answer or product, but also to develop understanding and the ability to formulate new questions and ideas. However, building on a vast body of previous human work, AI systems are becoming increasingly capable of producing the results of such work directly and these goals cease to align. The issues the mathematical community faces now are similar to issues that other scientific and creative professions are facing and indicate issues that all of humanity might face. How to make sure that, as AI changes the way work is done, we do not lose sight of what the work was meant to achieve in the first place." Ultimately, the letter acknowledged that the AI is here to stay and is, in fact, extraordinarily beneficial for the difficult work of research mathematicians. Terence Tao himself has been a slow convert to AI-assisted mathematics. He saw relatively little promise in the technology until recently, but has now adapted his process to make better use of the tools. Now, some rejected this out of hand. Eric Weinstein wrote, "I'll start: Who is part of this so-called mathematical community? Am I? Because anyone who follows me knows I've been warning about this kind of unwanted priesthood holding back the mathematics community by pretending to speak for all of us. And I'll be inviting the AI savages in." Author Nicholas Taleb wrote, "Something bad for the existing class of mathematicians is they lose their privilege, but not necessarily bad for mathematics and our body of knowledge?" Mathematician Steven Strogatz responded to Taleb and tried to articulate his own personal sense of loss. In fact, he admitted that while talking to Wired for an article, he actually got choked up. He writes, "So why did I get choked up? It wasn't about AI taking my job. I'm 67 and nearing retirement. I also love playing around with AI and find it exhilarating . I use it in research, brainstorming, and learning, and, as you can see, to put this thread together." So what was it then? I think it was when I started telling the reporter about the magnificent 4,000-year history of mathematics. It's an honor to be part of that long conversation, all the beautiful and important work, and the benefits that have accrued to humanity. Then, suddenly, the tidal wave of AI hits math in 2026. Something about the grandeur of the tradition and the suddenness of the change conspired to get me worked up. And basically, he goes on to argue that change always involves some amount of loss, even when it involves good things as well, and that even if one views change as inevitable, that doesn't mean you still don't mourn what is lost in the transition. Speaking of the transition, we continue to keep an eye out for if and where there is evidence of AI disrupting specific industries or specific categories of people, and a new study from the Census Bureau has certainly found that the market that college graduates are entering into, one that coincides with the rise of AI, is an extremely difficult one. The study looked at hiring and salary data for college graduates between 2016 and 2020. Based on a dataset that represented around 29% of bachelor's degrees earned in that period, researchers separated graduates into cohorts based on how exposed their profession was to AI disruption. The study found a notable impact on job prospects for graduates, beginning from the release of ChatGPT in November 2022. There was a differential effect based on exposure to AI disruption, with those entering highly exposed fields seeing a 5 percentage point drop in employment and a 13% reduction in earnings. This earnings reduction was in part due to graduates taking lower paid roles outside of their area of work. The study also found that job prospects were more likely to be impacted by the decline in earnings than by the decline in salary. The study noted, "This earnings decline is comparable in magnitude to the earnings losses associated with graduating into large recession." Now, if you are looking for evidence of AI-related job disruption, this type of area, early career employees, is the most commonly pointed to. Eric Brynjolfsson and his crew at Stanford called it the "canary in the coal mine." The challenge is that causation is a lot less clear than correlation. Many pointed out that companies have been fairly slow to adopt AI, certainly advanced uses of AI, and so the idea that as soon as ChatGPT was launched, they were all of a sudden changing their hiring practices is kind of suspect. Many will also point out that there have been a lot of other factors happening in the same period. The tech industry has seen significant layoffs that go back to 2022, but many point to other causes outside of AI like a correction from pandemic-era over-hiring. There have also recently been studies that have pointed out that a lot of what some has attributed to AI is equally correlated at least to work from home, which some speculate could be negatively impacting the junior and senior jobs in the future. If this is not the case, then there is no guarantee that the company will be able to continue to work from home, and it is likely that the company will continue to work from home. employees who haven't had a chance to build real deep relationships with their colleagues and bosses. Whatever the reason, it is clear that college graduates are entering an extremely tough job market right now, and this is absolutely worthy of continued study. Moving over to fundraising, Chinese open-source model company ZAI plans $5 billion to fund expanded compute needs. The fundraising was announced on Sunday and will be split between new stock issuance and convertible bonds. ZAI said that they would spend 60% of the proceeds on training their next-generation models, which includes building a "fully self-training life." This is the first time a Chinese lab has openly declared they will build recursive self-improvement, but it wasn't the only announcement along those lines. On Monday, a group of 33 researchers with backing from ByteDance published a research paper called "The Last AI Built by Humans: Towards Genuine Recursive Self-Improvement." The paper outlines a theoretical roadmap for RSI, but concludes the method is not quite there yet. Specifically, AI can automate certain parts of the training process, but at this stage there is no evidence that it can improve on the process itself. Still, the news reinforces that if recursive self-improvement is the major concern for AI safety folks, then any slowdown is indeed going to require coordination with the Chinese. And that, I think, is probably the perfect place to jump from the headlines into our main episode. A new study from KPMG and the University of Texas at Austin found that when people work with AI, similar skills don't guarantee similar outcomes. Researchers studied more than 500 early career professionals and found that the best performers consistently amplified the value of AI by guiding, evaluating, and refining its outputs. These top performers, called AI amplifiers, weren't defined by what they knew alone, but by how they worked with AI. Learn more about what separates AI amplifiers from everyone else at kpmg.com slash US slash AI amplifiers. Blitzy deeply understands your codebase before it writes code. Here's the first place that pays off. Security in the age of AI. Vulnerabilities don't live in isolation. They live buried inside millions of lines of interconnected code, where patching one thing quietly breaks three others. That's why surface level scans fail. Blitzy starts from its knowledge graph of your entire application, identifies and surfaces CVEs across the full estate, proactively recommends patches, and can execute the PR. Each fix is grounded in how your systems connect and validates, or nothing new breaks. And the knowledge graph dynamically updates, keeping you ahead of an ever-accelerating threat landscape. One Blitzy customer resolved 21 active CVEs across six core microservices in four days. Zero compile errors, every validation scan clean, tons of planned work fixed in less than a week. Security remediation grounded in real architectural context at the speed of compute. Harden your codebase at blitzy.com. That's B-L-I-T-Z-Y dot com. The best teams don't have a single star carrying everyone else. They know their own strengths and each other's weaknesses and play to both. That's the team Robots and Pencils has built on purpose. Nobody there is grinding through busy work to pad a headcount number. People come for the hard problems and they stay because everyone around them is leveling up at the same time. In a market full of companies that are just trying to hire fast... That's worth a look. Check out robotsandpencils.com/careers. This episode of the AI Daily Brief is brought to you by HyperAgent, where you run fleets of agents your team can manage together. Forget local agents and chat workflows waiting on your laptop to be prompted. HyperAgent deploys always-on agents in the cloud, doing real work across the tools your team already uses. Marketing agents turn competitor moves into landing pages. Sales agents enrich leads, draft emails, and updates the CRM. Ops agent chases the paperwork and tracks the budget. Every agent has a story to tell, and every agent has a story to tell. Every agent has a story to tell, and every agent has a story to tell. Every agent has a story to tell, and every agent has a story to tell. Every agent has a story to tell, and every agent has a story to tell, and every agent has a story to tell. Every agent has access to shared context and follows your rules about scope and approvals. It's time you had agents that feel like teammates. Hire yours at HyperAgent. Get $100 in credits at hyperagent.com/AIDailyBrief. Welcome back to the AI Daily Brief. Yesterday, I argued that the combination of Dario Amede's Pacing the Frontier proposal, replete with more specifics than we'd had in the past, plus the other lab leaders stepping in to agree specifically, would make a huge difference to our team's team's success. With Dario, even going so far as to repost his article, represented a meaningful inflection point in the discourse. A recognition that the next phase of AI development would be one in which the labs were not the sole arbiters of the speed at which new AI comes to market if it comes out at all. And so, I contended, I was viewing Dario's letter as the opening salvo of a negotiation. A negotiation to figure out how the power would be distributed in that new period. And some evidence yesterday after recording, certainly supported that view. Take, for example, the updated thoughts from A16Z's Martin Casado, who had been fairly hostile to the letter initially. Yesterday, he posted on X, "The non-cynical view here is that the labs are trying to navigate impending, unsophisticated federal regulation in a high-strung, well-financed Doomer complex, and it's just a tough, tough spot to be in. It's entirely consistent to applaud their attempts to find a governance approach to questions around safety and security. To express skepticism, around the proposal, and also express concerns about the larger safetyist complex. It's just important not to conflate the three." I'll put aside rhetoric on near-term species extinction. Like any major epic of our industry, new technologies have security implications. And I do think we're in a much better spot with open dialogue between Anthropic, open AI, and SpaceX on how best to handle, whether it's pacing or not, an independent oversight body or not. I'm in the not camp. I joke in exasperation about heavy-handed federal regulation which would be disastrous for us all. Unfortunately, I do think it's a likely outcome if we don't collectively get our together. I have no special knowledge, but from what I've gathered over the last few days, the labs really are trying to do the right thing here. It's a very complex and shifting landscape, and the broader safety complex still sucks and continues to be a national security risk. So from someone who was fairly aligned, at least initially, with the "they're just doing this for publicity" and/or "because effective altruists are insane" camp, this is a much more nuanced take that tries, as Martin says, to separate and deal with individually the difference between applauding the attempts to find the right governance approach, having skepticism about the approach that they're proposing, and to have even greater concerns about the implications of some of the larger safety agreements. And yet, that type of nuance would not, in fact, be the discourse that dominated yesterday. Because on Monday morning, President Trump weighed in on the AI safety debate, and his thoughts were not even remotely in the realm of subtle. In fact, when all was said and done, he posted seven times to Truth Social. The first and most repeated was this: "The only control or guardrails that AI needs is a strong and smart high IQ president, and the USA has that in spades. The Trump administration has stopped AI people from doing bad or potentially bad things like Dario, Anthropic, who is now pretending to be a perfect little angel, and we will continue to do so. We already have tremendous criminal and regulatory power over these companies. There is a 'sick' conspiracy going on against AI and data centers, and the only one that is happy about it is China. Whoever wins AI wins. We are leading China and all others, and we will continue to do so. Conspiracy theorists, treasonists, traitors, and leakers, beware. Thank you for your attention to this matter, President Donald J. Trump." About an hour and a half later, "The only reason the AI data center outburst is happening is because the United States is leading by a lot every other country. Don't kill the golden goose." Two hours after that, "Concerning AI, when, in the history of business, did anyone see the leaders of an industry call for regulation that, if strongly implemented, will drive them into human bankruptcy?" AI taking over the world, destroying humanity, and all other things bad, is a hoax. No different from Russia, Russia, Russia, Ukraine, Ukraine, Ukraine, impeachment hoax number one, impeachment hoax number two, and all of the other hoaxes and scams that America was forced to endure through the destructionist and deviant's foul play in illegal conduct. President Xi of China just announced that China will be doing absolutely nothing to stand in the way of AI or its future. Google has recently stated that they want to build a massive plant in Finland because they are finding permitting too difficult in the United States. I am not happy about this, and want them to change their thinking. AI and data centers will be the greatest economic development engine in history, bigger than oil, gold, diamonds, or even the internet. It will not be stopped by brilliantly run destructive forces during the term of President Donald J. Trump. And by the way, I know that I am going into exacting detail reading all of these, but this is the primary source material, and to get seven messages about this shows you just how top of mind it is right now. Three hours after that last message, Trump again says, "The people that say AI is going to destroy the world and that data centers are bad for your neighborhood, are the same people that said just a short time ago that the world would be extinguished by climate change. That hoax never worked out for them, and now they're on to the next one. These people are revolutionaries, but revolutionaries for a bad and evil cause. Soon you'll find out they're working for people that do not have the best interests of the United States in mind." Just four minutes after that, "I am the hoax buster, and I am now breaking another hoax, that AI is going to take over, consume, and destroy the world, and that robots will be marching into our cities and getting rid of us all. This is even wilder than the Russia, Russia, Russia hoax or the global warming scam. Thank you for your attention to this matter." Nineteen minutes later, the claim that AI is going to take over the world is a hoax just like global warming, quote unquote, where everyone was going to be dead by now. Russia, Russia, Russia, Ukraine, Ukraine, Ukraine, or impeachment hoax number one or impeachment hoax number two. These are all radical left Democrats who never give up but seldom win. Hopefully the voters have seen the light. Finally, three and a half hours after that, the AI hoax being perpetrated by the radical left Democrats is reminiscent of their global warming scam of not so long ago, where everyone was going to die from extreme heat. What happened? Make America great again. Now, as is frequently the case when Trump goes off on Truth Social, there were follow-up comments from JD Vance that tried to reinforce the message without so many exclamation points, air quotes, and strange capitalizations. Speaking at Andrews Air Force Base, Vance told reporters, "Personally, I feel a little bit weird about the fact that you have so many frontier AI tech companies kind of coming to the government and begging the government to regulate them. It feels a little bit to me like a bit of a Trojan horse." The Department of War CTO account on X also got into the fray, sharing a graphic that said AI First War Department, with the post copy, "Americanism, not effective altruism. The United States will continue to be AI dominant." Now, there were, as you might imagine, a few different categories of response to this. Some were pleased. Computer science professor Pedro Domingos wrote, "Thank you, President Trump, for seeing through the AI fearmongering." Psych professor and AI safety advocate Jeffrey Miller, on the other hand, pointed out the potential political consequences of this position. Pro tip, he wrote, "If Republicans want to win the upcoming midterms, it's not prudent for Trump to call everyone concerned about AI safety a conspiracy theorist, treasonist, or traitor." He then goes on to list a bunch of polls that show that Americans, by and large, are not super keen on AI, and concludes, "Human survival and AI safety are bipartisan issues, and it's bizarre that certain Republicans are trying to make it into a partisan issue, especially given that there is nothing conservative, traditionalist, or Christian about the AI industry trying to, quote, 'summon AI demons' to radically overturn human civilization and to ultimately replace humanity." Now, the idea that these are entirely bipartisan issues is one that is looking a little more suspect these days, but we'll come back to that in just a bit. Mostly people got the memo that if this is indeed a negotiation, the lines have officially been drawn. Ryan Orhan wrote, "There is absolutely zero ambiguity left. Trump thinks the AI extinction argument is a hoax and wants America building as fast as possible. China isn't slowing down, and as far as he's concerned, America would be insane to handicap itself." Now, one person who broadly agrees, if who might use more diplomatic language about it, is NVIDIA CEO Jensen Huang. an interview at the All In Summit, which of course had the events of the past weekend as the core focus, Huang pushed back on proclamations of AI doom, calling them irresponsible. Early in the interview, Chamath Palihapitiya asked Jensen how we should explain the notion that there is a 10% chance AI could exterminate humanity to people who aren't as familiar with the industry. Jensen responded, we shouldn't because it's made up. The prediction, he said, is troubling and alarming and it shouldn't be done because it's irresponsible. Jensen then went through the track record of predictions from the AI doomers. The idea that AI would replace radiologists, eliminate white collar work, and that 90% of code would be generated by AI. All of these predictions, he said, have been wrong. We have to take accountability for all of the stupid predictions that were made. We have to keep track of all that and remind us that those predictions are inconsistent with America winning the AI race. Then later in the interview, in a twist that would have been rejected from Hollywood writing rooms for being too unrealistic, Jensen received a surprise call from the Oval Office, live on stage. Jensen told the president, good job, sir. The fact that you saw through all of this, that there's a lot of complexity, but the fact that you saw through all of it, we're really grateful. After Jensen figured out how to put the president on speakerphone, Trump said, it's almost this conspiracy. The happiest group is China, but a lot of states are very happy that weren't going to get anything. I'm telling you, it's all a hoax. The data centers are great. They make people wealthy. They make states wealthy. It's the oil of the next 20 to 25 years. They're just playing right into the hands of a lot of people who don't want to see it happen. That could be political people. That could be China, but it's a hoax. The robots are not going to be taking over the world. Jensen responded, we're not going to let it happen, sir. Now, while those were the two sections that gathered the most attention, Jensen's entire framing of the current moment is worth unpacking. While he dismissed existential risks, he did not dismiss the need for AI safety. In fact, Jensen was broadly supportive of the idea of independent auditors embedded in the AI labs, as Dario Amadei called for over the weekend. He did caveat his support with the idea that auditors need to be truly independent from the community of AI doomers, but he was still supportive of the idea. Jensen framed the safety issue as largely an engineering problem, that the frontier AI labs are struggling as they transition from research to engineering. Events like the Hugging Face incident demonstrated to Jensen that the labs need much better security frameworks and need to increase their focus on safety. He also noted that the safety incidents we've seen so far have all been driven by giving agents access to near-infinite compute. He said, it's sensible that the frontier labs will be where the most danger comes from. It's unlikely that a high school student did something because they won't have enough compute. In fact, you can look across the planet and everybody won't have enough compute with the exception of the frontier labs. Jensen argued that the solution needs to be rooted in reality and engineering, commenting, the first thing you have to do is root cause the problem from an engineering perspective. What happened? What could we have done differently? And what are we going to implement or institutionalize, whether it's technology or methods or processes, to make sure that we don't let it happen again? I would bet you money that in every one of those cases, that it's within their control in the future to prevent it. In other words, even if the call from the president was the dramatic showstopper moment, the overall conversation had a lot more of exactly that sort of nuance that many of us have been hoping for. Now back to the question of just how bipartisan this issue is. While that may be true in the polls for right now, politics is doing what politics does. As I mentioned on yesterday's show, Trump isn't the only president weighing in on the AI debate. Last weekend, Politico reported that President Obama had urged Democrats to focus more closely on AI policy as a campaign platform. At a fundraiser last Thursday, he reportedly said that AI would be, quote, one of my central agendas if he were running for president in 2028. On Monday, Obama took his views public, which may not seem like much. But in general, Obama has been pretty disinclined to wade deep into policy discourse. In a post on X, he wrote, I was encouraged this week to see the leaders of the frontier labs agree on the need for them to slow down the pace of AI development. Given the stakes, it's a good and necessary first step. But I'm even more encouraged by the growing recognition that how this powerful new technology develops should be at the center of our public debate. I've been watching the progress on AI for over a decade now, and one thing that's clear to me is that the potential impact of this technology is not overhyped. It's also moving at lightning speed, and even faster than those who are engineering it can keep up with. I'm not an AI accelerationist who believes it will lead to some techno-utopia, and I'm not a doomer who thinks it will inevitably lead to humanity's destruction. But whether this technology results in amazing breakthroughs in medicine, energy, and education, or unleashes huge economic disruptions, greater inequality, and potential catastrophe, will depend on the choices that we make now. Choices that should be not just by the companies involved, but by all of us. Obama then announced Project Blueprint, an initiative from Ron Conway's venture capital firm, with the stated goal to, quote, rebuild bonds between leaders in the artificial intelligence sector and Democrats who are demanding new rules to regulate the technology. Pirate Wire's Mike Solana wrote, After Kamala Harris chimed in to lend her weight to the calls for slowdown, Rohit Krishnan wrote, Ryan Orhan again chimed in, This is becoming exactly the nightmare scenario for AI safety. The second AI safety becomes attached to politicians and political tribes, half the country is going to oppose it simply because the other half supports it. Democrats start lining up behind slowing down, Trump declares the hoax and promises America won't stop. I absolutely hate watching this become partisan. Meanwhile, over in markets, the first blush was to be a little bit nervous about the slowdown rhetoric. The Nasdaq index was relatively flat, but the semiconductor index lost 5.9% on Monday. Financial media is discussing a pivot to SaaS, or even the much-maligned Indian IT stocks as somewhere to hide out while the AI sector takes a tumble. So that is where we are right now. The next thing to watch will be to see whether this jumps from the realm of politics to the realm of that's going to do it for today's AI Daily Brief. Appreciate you listening or watching as always, and until next time, peace.
PodcastTranscript AI