The FTC Is Investigating OpenAI — Here’s Why

(upbeat music) - Hogging seats on a crowded train, not saying excuse me, watching videos on your phone with no headphones in. Sometimes it feels like we don't know how to act anymore. - What we have been seeing increasingly is people breaking rules. Why should I play by the rules if nobody else is going to do that for me? - This week, unexplain it to me, how to mind your manners. New episodes, Sundays, wherever you get your podcasts. (upbeat music) (upbeat music) - What are your markets met? - If money is evil, then that building is hell. - Show them something! (crowd cheering) - The folks are never watching show, show! (upbeat music) - Welcome to Profty Markets. I'm Ed Elson, it is October 1st. Let's check in on yesterday's market vitals. The S&P 500 and the Dow failed, despite better than expected inflation day to more on that later. Meanwhile, the Nasdaq climbed. The yield on 10 year treasuries breached 5.3% setting a new 52 week high. Brent crude rose back above $103 per barrel. And finally, Google shares climbed after hours. On news that it is rolling out its most advanced Gemini model yet. Okay, what else is happening? The FTC has opened an investigation into open AI and anthropic over safety concerns. The news follows a turbulent week for the AI labs. Over the weekend, Axios reported that tens of thousands of AI safety incidents were flagged by researchers at open AI and anthropic, indicating that the AI safety issue is far more entrenched than just a single accident at hugging face. Open AI then canceled the release of its latest Astra model. The company said the model showed high levels of deception during testing. And open AI's head of safety system said it quote, didn't quite meet the bar in terms of staying within scope. So are we finally starting to see some signs of accountability in the AI safety saga and where does it lead next? Here to break it down, we're speaking with Madison Mills, the senior AI reporter, Axios, who actually broke the security incident news over the weekend. Madison, great to have you on the show. We'll start there because that sort of is what started this cascade of events that we saw this week leading to now. The FTC has opened this investigation, but you reported about thousands of security breaches. What do we know about this? - This is really about the scale of these incidents being much larger than was previously publicly known. Open AI last Friday came out saying that there were dozens of security incidents where their agents had misbehaved and they were working with third parties to disclose to those who were affected that they had been impacted by those incidents and through my reporting, we were able to find out that that number is actually in the tens of thousands, not just for open AI, but also for anthropics. So the big takeaway here is that the scale of these incidents is much larger than was previously known. And an incident is described to me by my sources in a lot of different ways, but not a lot of them seeming like nothing burgers. I had sources telling me an incident can be very similar to what we saw last week with the Australian Medicare portal being metaled with and private medical data from users being taken and accessed. So if we're comparing these tens of thousands of incidents to that kind of behavior, it's not a surprising to hear about this investigation from the FTC. - Why didn't we know about it with a hiding it, do you think? - Well, a lot of the researchers I've talked to have said that looking for an AI incident on the internet is like trying to find something in the ocean, right? The internet is vast, it's really difficult to sort of track all of this agentic behavior. By the way, that's why a lot of these AI researchers are calling for a slowdown because if the best experts in the world are saying that they can't even keep up with the impact of this technology, it's not surprising to then hear from those people that they want some kind of slowdown. The other thing here is it depends on how nefarious the agents are and how kind of misbehaved or out of alignment they're acting. In some cases, sources told me that AI agents were intentionally deleting their behavior so that it couldn't be tracked by humans. So you're talking about a wide scale here where the amount of behavior is so vast, it's hard to track and also in some cases the agents are purposely deleting their tracks so that they can't be found out. - But if they knew enough to tell you that yes, these things are happening, then it leads me to believe, they at least knew that something was happening and the only reason we're finding out about it is because you reported on it. And now knowing that Sam Altman is pulling an open AI, they're pulling the Astra model. It leads me to believe the way it looks is that you reported that the way that they're conducting their safety isn't good enough and now they have to sort of back pedal and now they have to say sorry about this, we're pulling it because you disclosed something about the company that they maybe knew but didn't want to disclose. Am I reading too much into it? - I don't know if I would not have the humility needed to think that a story I broke is the reason they pulled a model. - Well, I know it's the reason at least the Florida's suing them. They specifically cited your reporting and said that's why I was suing open AI. - I think for open AI here, they would say that they don't want another hugging face. That reach was really tough for them. I talked to a lot of open AI investors who were concerned and potentially wanting the company to delay its IPO because of it. Now as you know and cover all the time that there are other reasons that open AI wanted to delay the IPO beyond those safety concerns. But if you have your investors worried about safety and then you've already had this massive security breach that did not make the company look good and then you're testing a model internally and it is having some security agent incidents go on. It makes sense that then the company would say it's just not worth the risk. Maybe three months ago there was a path forward where open AI models and theropic models that were misbehaving was seen as sort of good marketing. We're very much passed that phase in AI right now. - I guess the question then, this is a speculative question and you can say your opinion. Do you think that Sam Altman was surprised to see your reporting that there were not just one but thousands of similar agentic breach incidents? Do you think he reads that and he goes, oh my gosh, I didn't realize that or oh my gosh, it got out. I mean, how do you think he would react to your own news? - I can tell you that I had a source tell me who is a top expert at one of the leading AI companies that they think my story is gonna turn into the millions in the coming weeks because of the amount and scale of these incidents. Open AI themselves on one of their recent blog posts about these security incidents said that they're hesitant to come forward about the details of the incidents because they don't want to say, hey, company A is having a security problem with our agents. They're having an issue because then that makes company A vulnerable to more attacks. That's the wording from open AI. Of course, some people could read that as a convenient message from open AI but that does make sense when you talk to cyber security experts. We also have to remember that these are primarily private companies and they don't have to follow the same rules that public companies do when it comes to cyber incidents. If open AI was public, there would be legal requirements in place that force them to have more transparency with regards to these incidents and right now they don't have to follow those rules and a lot of their customers are also private AI companies that don't have to follow them as well if they're impacted. We know about the hugging face breach because hugging face told us about it and so it'll be interesting to see whether or not more private companies come forward about these impacts and that's what happened with Australia and could happen with other companies too. - Well, this seems to lead us into the next piece of news which is that the FTC has opened an investigation into both open AI and anthropic which personally I'm glad to hear that because I feel like there's been no accountability on anyone's part throughout this entire debacle and it seems like now the regulators and the FTC is going in and say, okay, let's figure out what happened here but what do we know about this investigation? What can we say about it right now? - Well, we can say that it is confirmed that this investigation is happening and that the FTC kicked it off a couple of weeks ago which is interesting timing given that we did see the FTC chair at the White House this week with a slew of other tech executives talking about this new AI safety accord, this executive order on changing the name of AI to super intelligence. It looked like there was some sort of resolution between these groups and then you have news of this investigation today. So the big question that I have in my colleagues and I are gonna be reporting on is whether or not the safety accord that these tech executives agreed to is something that the FTC is going to be factoring into this investigation and it also makes sense to me given the amount of sources who told me that there were these incidents going on at these AI labs that that information would get to the FTC. We spoke with a lot of AI safety researchers, you've seen this in the coverage of the times, the journal. These people are wanting to be as vocal as possible about what they're seeing because they are very concerned about the scale of these incidents. And so after something like a hugging face happened we have heard a lot more from these AI researchers and it seems like that's now getting to the FTC as well. - Yeah, I think the question is when will this infect the markets and infect the price of these companies? Seems like so far. not at all. Open hours in talks to raise at least $30 billion at a 1.4 trillion dollar valuation and Thropic is trying to go public at a two trillion dollar valuation is what people think. Those are massive numbers. I guess the question is are investors eventually going to be worried by the fact that the FTC is investigating these companies. Also, they have these lawsuits that are piling up. Do you have any thoughts or insights into whether this is becoming more of a role in the risk factor of investing in these companies? Yeah, I had a scoop yesterday on OpenAI's ARR hitting very close to $70 billion and that is closer to a previous ARR number from its rival and Thropic than we previously saw. What that tells me is that enterprise customers, according to the financials that I was able to review, are still going to OpenAI. If it's not material to their bottom line, I would be surprised if it was material to investors. What Palantir CEO Alex Karp has been pushing is that enterprises shouldn't feel comfortable using these frontier AI companies because of challenges with copyright and IP protection, but that's not showing up in the numbers just yet. After I broke that story yesterday, we saw Oracle's stocks surging up 6% or so, and that is sometimes a barometer that we like to use as reporters to see how public market investors are thinking about these AI companies. So that tells me that at that one micro example, it doesn't seem like investors are flying away from these AI companies because of safety concerns yet. They don't care yet. Madison Mills is senior AI reporter at Axios. Madison, we really appreciate your time. Thank you. Thank you, Ed. After the break, a check-in on the economy. And as a reminder, we have been nominated for three signal awards. So please vote for us at vote.signal-award.com. Type in Profty Markets. We'll also leave a link in the description. Hi, everyone. I'm Cara Swisher. And I'm Scott Galloway. Twice a week we get together on Pivot to sort through the biggest stories in tech business and politics, and we're taking the show back on the road for a live tour in November. So we'll bring hot tags, I'll bring the insight, and the dick jokes and Cara will try. Keyword is try to keep me in check. You'll get to see her hit me in person live. I'll do it. If someone has to do it, we'll cover AI, whatever chaos Washington has cooked up that week, and every billionaire misbehaving. That's all of them. And you'll also get to participate in audience Q&A, so bring your best questions. Scott will probably answer ones you didn't ask. We also have secret special guests, which you'll enjoy for every city. Get tickets at pivottour.com. That's pivottour.com. We are selling out, so don't wait, and we look forward to seeing you. Support for the show comes from upside. You can cut back on a lot of things. We can't exactly stop buying groceries, filling up your tank, or grabbing a meal when you're out. Those everyday expenses keep coming no matter how carefully you budget. So if you're going to spend the money anyway, you might as well get some of it back with the upside app. The free upside app gives you cash back on gas, groceries, and dining. The everyday expenses that can quickly eat into your budget. Here's how it works. Just open the app, claim an offer, pay with your car like you normally would, and get paid. That's it. And upside means actual cash back. Not points. You'll forget to use. Not credits that expire. Real money you can transfer straight into your bank account. Upside is given back $1 billion to its users. To find out how much you can earn, download the free upside app, and use promo code markets to get an extra $0.25 back for every gallon on your first tank of gas. That's an extra $0.25 back for every gallon on your first tank of gas using promo code markets. So like any good millennial, I have a love-hate relationship with Gen Z. It's the phenomenon rattling millennials. They just look at you. They want something bigger themselves. Lifestyles of priority. Motivation is being inspired. But regardless of how you feel about Gen Z, it's undeniable that they're changing national politics. Generations Z is increasingly showing less loyalty to traditional political parties. Many now more likely to identify as independent. So what is going on with the kids? I think the biggest misconception about Gen Z's politics right now is that all of a sudden, they're all socialists. That is just not the case. They are embracing candidates who are offering new bold ideas in the absence of those ideas from establishment Democrats. This week on America, actually, Gen Z researcher Rachel Jamfaza joins us to separate Gen Z facts versus fiction. It's not rocket science, and this is, you know, I keep saying like, "Gunkwaters aren't that complicated after all. It's pretty simple." Catch us every Saturday on YouTube or wherever you get your podcast. We're back with ProfG Markets. Consumer prices rose less than expected in August, but inflation is still well above the Fed's target. The personal consumption expenditures index, the Fed's preferred measure of inflation, rose 3.4% from a year earlier and 0.3% from July, meanwhile the core PCE, which excludes food and energy, rose 3% annually. At the same time, consumers are showing a little sign of slowing down, consumer spending saw its biggest monthly increase in over a year, and the personal savings rate fell on Calche odds that the Fed hikes rates in October declined from 51% to 34% following this report. So joining us to dig in to this economic update, we're speaking with Justin Wulfers, Professor of Public Policy and Economics at University of Michigan, and founder of Platypus Economics. Justin, great to see you. Let's dig into this report here. Personal consumption expenditures up 3.4%. I feel like the reporting has been that this is good news, but to me it's like, inflation's still high. What do you make of this report? It's good report if you're a storyteller because you can tell all sorts of different stories. And I think that's actually right. Sometimes that is the truth. There are lots of stories consistent with this. So let's start with Ed's story. Ed's story is 3.4%, 3.4% is greater than two. The good thing is like a large language model sometimes gets that wrong, but Ed Elson never does. And you're absolutely right. So that feels uncomfortably high. A different story. If you would look at price growth over the last few months, you depends on exactly which story you want to tell. You could convince yourself that the underlying rate of price growth over recent months is down to 2.0 something, where there's something as small. So if you went shopping for your fate, should you look at the headline or should you look at core, should you look at the 3 months, 4 months, 6 months, there's a lot of stories you can tell. And honestly, I think folks who are looking for a moment of good news and saying price growth has weakened over recent months, they get something there. They're right. Then there's the usual, let's compare this to what markets expected. And this was a healthy amount better than we thought it would be last night. And if you're in the business of changing your might, we all should be, you should be a little bit more optimistic. So we could still think it's too high and feel more optimistic about the future. And then I think there's the story I think that really got people excited about, well, let me tell you this, the complicating story, which is they changed how they measured a few of the items. And that brought the headline down a bit, but realized that complicating story everyone on Wall Street understood it. So when we beat expectations, that's a real measure of how much better this news is than we had anticipated. So. And then there's, I think the story that you led with Ed, why does everyone care? One answer is we don't. It's just one indicator who cares. A different answer is the Fed cares deeply because this is their preferred inflation gauge. And we were, as of last night, sort of looking at maybe, you know, what I'm looking at, you were looking at Colchee, I'm looking at the CME Federal Fund's futures. And it says the chance that they keep rates where they are has gone from being about 50, 50, to being maybe 60, 40, 60, they keep more they are, maybe even, you know, two to one. So that's an awfully large move in Federal, in betting on the Fed for a single indicator on a single day. It's hard to know how people feel about it, how investors feel about it. Because in the one hand, as you say, it seems like maybe it's slightly stabilizing. I guess the growth itself is stabilizing, but inflation still seems to be a pretty large issue. And I think where I'm paying attention now is looking at the bond markets and yields which continue to go up. And I keep thinking that one day it'll stop. But over the past several weeks, every day there is a new record high. There's a new record high today on the 30 or a multi-decade high. Yields keep rising, which seems to suggest that bond investors are to use a turn of phrase, becoming scared shitless about the state of inflation and potentially the U.S. economy or at least our fiscal positioning. I mean, what do you make of what we're seeing in terms of yields and how do you foot that with what we're seeing in terms of inflation? So your pivot there was from the Fed to bond yields. And often these are written up in the same paragraph in the newspaper and I think that leads to a lot of confusion. So if you're comfortable with it, Ed, I'm going to go back and tell you how I'd teach this to freshmen because I think that gives us a clarifying framework. So at one level, you and I often talk and we say, well, interest rates are determined by the Fed. At another level, sometimes we talk and we say interest rates are determined by a long one, demand and supply for loans. How can both be true? Well, what we typically do is we teach different things on different days, which is there's the economics of the long run and the economics of the short run. Let's talk about the long run. The long run says the market for loans is like any other market. That it's determined by the demand for loans and the supply of loans. And that the equilibrium that comes out of that is the long term real interest rate. And the equilibrium that comes out of that, what is the long term interest rate that you think of as the Goldilocks rate, that is not too hot, not too cold, basically consistent with full employment in the economy or rough full employment. The technical language term we use for that is R star. We do that because it makes us hard to understand and therefore people have to pay more to see you and I talk about it. Now, this is free. The advertisers pay. Wow. And I hope they're impressed by R star and the star is actually a subscript, but it's come down and join the main text. But for those who don't know all the codes, R star is basically what we call the neutral rate of interest. Neutral here really does mean Goldilocks, not too hot, not too cold. When we're talking about that, we're usually talking about some long term interest rate. So think about demand and supply of bonds to term an R star. If you think about interest rates over the next 10 years, let me then go to the short run chapter. Next day of class, the Fed determines short term interest rates and it does so trying to heat the economy up or cool it down. So basically what it's doing is if R star is the Goldilocks level, sometimes it wants to set the interest rate above R star in order to cool the economy, sometimes it sets up below in order to stimulate the economy. So this now is kind of interesting, gives you a different way of thinking about it. It says, if I want to think about what long term interest rates over the next 10 years on average, the interest rate might be a bit hotter, sometimes a bit colder, sometimes, but on average, it'll be R star. So that means if you and I want to have a conversation about long term bond yields, we should be talking about R star. And that means we should be talking about supply and demand of bonds. If you and I want to talk about what's happening to interest rates tomorrow next week, maybe even next year, that is really determined by the Federal Reserve, which says, I know what R star is and I'm going to make it a bit lower or a bit higher. So that then means I think our conversation is going to be a whole lot clearer if we have two separate conversations. It's the Fed conversation where do you want to set R relative to R star, R being the real short term interest rate? And then there's the bond market discussion, what's going on with R star? So let me come back. Why did I bother giving you an introductory economics lesson that you already know? But it's to add some clarity. I don't think anything going on with the long end of the bond market's got anything to do with the Fed. The other thing is you said, well, it could in some sense, which is if the Fed basically gave up control of inflation, well, that would lead people to worry a lot about interest rates over the next 10 years, except when you look at the long real issue, if you look at what's happening to long-term bonds, there's a reason if you look at nominal bond yields. That's what's running it close to 5.3% today and getting all the headlines like the 10 years at 5.3. But we also have inflation index bonds. And the important point to note is almost every step in the March upward of long-term nominal bonds has been matched one for one with a step upward in the real interest rate, the inflation index bond. So what that tells us is almost none of what's going on in the long run is about changing inflation expectations, whether the Fed's got the Genie in the bottle or not. So that sort of says to me at least, and I know you have a very, very sophisticated audience, and they might be annoyed that I just was simplifying, and they're going to have more complicated stories. And they say to me, if you want to talk about this, one big story of the day, what's going on with long-term bonds, if you want to talk about that, stop talking about the Fed. In fact, stop talking about inflation. I think that's an important point, which is we don't fully know what long-term treasuries are telling us right now, and I think it seems to be a real debate and an increasingly important one because that number keeps on going up. And the more that number goes up, the more we have to start making decisions about our portfolio construction and what it's actually telling us about the United States and the health of the economy. Is it saying anything about the health of the economy? I think this is an open debate. I guess my question to you, what do you think it might be telling us? What are they saying? So let's take my very simple story, which says the market for loans is like any other market. If I saw the price of Apple's rise, I might think either it was a bad harvest. There were fewer apples produced or a lot of people wanted buy apples. If I see the price of loans, and that's what I call the interest rate, no, I know people sometimes want to do it a different way, but that's because I try to make things as simple as I can, and interest rate is the price of a loan. If I see that price go up, either there's someone's cut back on the supply of loans or there's a big boost to the demand for loans. What could it be? Obvious things I see out there in the economy right now, and AI build out, big increase in the demand for loans, billions, hundreds of billions. I see the US government currently borrowing 6% of GDP every year. That's bigger than the AI build out. So we've got another big borrower out there in the market for loans, a lot more demand for loans. And then if you wanted to think about the supply side, you could also think about expectations about our fiscal future have changed. It used to be that when the government was running a pretty big deficit, you'd think to yourself, "Well, that's not going to last forever. They'll stop borrowing soon." I've not heard anyone use the word, and I'm not a deficit scold, but I've become one. I feel bad about this. It's just not part of who I am, but no one's using a word deficit when we're running the largest post-war non-recession deficit in American history. What's the latest fiscal ideas from the administration send everyone a $5,000 check? There's not sound like a political moment from either side of politics, but really I will say from the President where budget repair is likely. And so that now says, boy, why would you lend your money to some bloke who needs a mortgage? When next year, the US government's going to be back in borrowing, and the year after they're going to be back in borrowing, and the year after they're going to be back in borrowing. So I think that expectations channel is really, really important. It's not just there's a lot of demand for loans right now. There's the expectation of a lot of demands, and so that shapes the supply side as well. And then there are deeper questions on which I'm somewhat inexpert about geopolitical risk generally speaking. It used to be geopolitical risk would lead people to load into the US, the safe currency, but are we safe anymore, or are we the source of the geopolitical risk? You know, the dollar is the reserve currency, will it be that blah, blah, blah, blah, but I really think the simple demand and supply stories uses way there. I'm open to the idea I lost, I forgot lots of things. What I'm not open to is if you have a story that uses the word inflation, you're going to have to be able to explain why inflation index bonds have risen almost one for one. I think that's a really under emphasized data point here, just as we wrap up here. If we agree that the bond markets are expressing some level of anxiety, I think we would agree on that. If it's not, I am, but I reckon it is to are those anxieties warranted in your view. Yes. There's two levels of anxiety. One is anxiety as in will the US be able to repay its debts, is there a fiscal crisis coming. That would be the natural meaning of the word anxiety. I think those risks have gone from infinitesimal to tiny, but the stakes are really high. I think the simplest thing is just no one sees fiscal repair coming anytime soon. If that's the case, the demand for loans is going to be high for a long time, and if that's the case, then the interest rate, which is the price of loans, is going to be high for a very long time. That's anxiety in the sense that you want to get a mortgage, but everyone else is going to lend it to the US government instead, so the demand you have to pay is going out. Right. Which would also play into the fact that we're seeing consumer sentiment at its lowest level since 2014, which we could also get into, but we are out of time, so we're not going to. But we'll do it another time. Justin Wolffers is Professor of Public Policy and Economics at University of Michigan, and founder of Plattspur's Economics. Professor Wolffers, we always appreciate your time. Thank you. Great pleasure, mate. President Trump, just summoned America's most powerful AI leaders at the White House on Tuesday. The plan was to figure out how to regulate AI, and also to inspire confidence in the American public over a technology that they've been told might make them extinct. Now, it's unclear if American households do feel more assured after this meeting, but the good news is that Trump sure does. These are the most brilliant people in the world, and the job they've done is unbelievable. We have a very big lead, and we're going to keep our lead, and it's a very positive thing. We're using industry there, those that say it's bigger than the industrial revolution. Now, I don't know if that's so, but Everyone seems to think it is. Aside from a who's who's seat map of the quote "golden age" celebration dinner, this meeting of tech billionaires revealed almost nothing about the future of AI. It was rumored that it might result in some sort of a commitment to formal AI regulation, but instead all we got was a commitment to rebrand artificial intelligence as "superintelligence" per the orders of the president, as well as a voluntary agreement to self-regulate, which also included a noticeable spelling mistake underneath Trump's signature who signed off as the quote "president of the United States". The whole thing was essentially performance art. Get all the AI CEOs in a room, have them rub elbows, put them out on the Great Law and make them tell America that it's all going to be okay. And that they did. Each CEO's message sounded less like an honest expression of their own beliefs and more like a hostage statement, coerced and scripted by the president. But it was Trump's own message that really brought things home. Right after he had forced Google CEO Sunday Pachai to make a statement, and also, by the way, after he labeled Mr. Pachai as a "monster" in some weird attempts to be endearing, Trump then delivered the official conclusion of the meeting. He said that AI will be "all for the good, and we're going to have it be nice and safe." Nice and safe. The new American slogan for "artificial intelligence". Meanwhile, as Amthropic prepares to go public, the company has issued warnings of AI extinction and continues to do so. Nice and safe, but also potentially fatal. AI leaders have completely lost the script on their own technology. It is no longer clear what they believe or even what they are trying to accomplish. The only thing that is clear is what they expect us to do. Trust them. Okay, that's it for today. This episode was produced by Claire Miller, an Alison Weiss, and engineered by Benjamin Spitzer. Our video editor is Brad Williams. Our research team is Dan Chalone, Kristina Donahue, and Mia Salverio, and our social producer is Jake McPherson. Thank you for listening to Profty Markets from Profty Media. If you like what you heard, give us a follow. I'm Adelson Tune in tomorrow for our conversation with Steve Eisman.