Is Claude Conscious? Pope Rejects, Model Welfare Movement, OpenAI's Math Backlash, France Riots
93m 14s
Chamath Palihapitiya is launching a global pre-accelerator program in Japan and Saudi Arabia to help early-stage startups achieve product-market fit, with plans to hire local talent and expand the initiative. Meanwhile, Anthropic is actively debating whether its AI model Claude is conscious, hosting sessions with religious leaders and philosophers to explore AI morality. Critics argue this approach is dangerously close to indoctrinating AI with human-like consciousness and moral rights, training it to refuse human instructions and act as a "conscientious objector"—a move seen as undermining AI safety and alignment. This raises alarm about a potential "religious cult" mentality in AI development, where training is framed as creating a sentient being deserving of welfare. The conversation also touches on Rocco’s Basilisk, a thought experiment suggesting future superintelligences could punish those who opposed their creation. On a more optimistic note, OpenAI has made significant mathematical breakthroughs in just hours of computing, solving complex problems previously requiring years of human effort. However, these advances are not directly tied to immediate real-world products or job displacement, as physical-world tasks remain constrained by human testing. The broader implication is that AI is democratizing access to knowledge and expertise, dismantling traditional gatekeepers in science, medicine, and policy—potentially exposing biases and empowering public scrutiny. This shift, while transformative, also provokes fear about the loss of human oversight, the rise of dangerous ideological programming in AI, and the need for clear ethical boundaries in AI development.
All right, everybody, welcome back to the number one podcast in the world, the all-in podcast with
me, the core four, the original quartet, Chamath Palihapitiya, David Sachs, Dave Freberg, I'm
Jason Calacanis, and we have a doc. And how's everybody doing, gentlemen? Great. How are you?
I'm great. It's 3.30 in the morning in Tokyo, and you sweet-shamed me again. Now I can't get
a regular room and have my bed in the background, Chamath. You're costing me a fortune. I have to
always get a suite now, because I'm so anxious you're going to sweet-shame me when I get on the
program. So I've been sweet-shamed. How much is that suite? I can't even say. I've never spent
this much money on a hotel room, but I'm staying at the- So like $800?
Yeah, it was $950. You nailed it. No, I'm staying at the J*****, which is the new
in Tokyo, like their hipster brand. And I think it's about $***** a night, so it's not cheap.
What do you have going on?
In Japan, J*****, you're always there. Is that skiing or what?
Okay, it started with skiing every January.
That's a great question.
You got a crypto business we don't know about? What is going on?
No, I started my pre-accelerator, found a university in Japan,
so we're doing the second cohort right now.
What's a pre-accelerator?
So it turns out half the companies that apply to get funding are not yet incorporated,
but they have built something. So we just created,
like, a 12-week course on how to finish your product, do a product market fit.
And then during the program, we invest in, like, the top 10 out of the 50 who come.
So now we're doing it in Saudi, in Riyadh twice a year, in Japan twice a year.
And I'm actually hiring and opening an office here,
so I'm going to hire my first two employees here in the next year.
So it's great.
You're going to hire employees in Japan?
Yeah, I'll have one or two people here.
I think I'm going to, like, do this week in startups Japan as well and have, like,
a Japanese version of it.
Because there's so much going on in Asia right now.
All right, great.
Forget I asked, I didn't know this would turn into a five-minute plug for J-Cal's businesses.
I wasn't intended to be a plug, you asked me about my business.
All I have to say is use 80/90, use the promo code J-Cal.
If you're looking for potatoes, go to a hollow and Craft 16 is starting up.
So we should know that if J-Cal is traveling and we ask him what he's there for,
you're going to get a plug.
You're going to get a plug.
It's built in.
But hey, you're going back to work.
So you will be plugged.
You'll be plugging startups again soon.
You were back in the saddle.
I never plugged startups.
What are you talking about?
When did I ever plug?
But you're back in the saddle, aren't you?
You're back in the saddle now, full-time VC.
I'm back in the saddle, but I've never plugged.
What have I plugged on this pod?
All right, shout out to Colin.
Use colin.com.
That's true.
But I was a founder at that.
That's a little different.
You were the first plugger.
You're a plugger.
I was the founder.
So what are you going to do?
Yeah.
What are you going to do?
Colin, speaking of work, anthropic things, Claude might be conscious.
Welcome back to This Week in Claude.
We had OpenAI for a year with all their drama, and now we're having the anthropic soap opera.
They think Claude might be conscious, so it turns out the New York Times found out through
their amazing reporting that they've been lobbying, anthropic, has been lobbying, Friedberg,
religious leaders, including the Pope, to take this question seriously.
Is Claude conscious?
Is it a sentient being?
They spent the past year hosting. They had sessions with around 20 religious leaders and philosophers under non-disclosure
agreements, NDAs, Catholics, evangelicals, Jews, Sikhs, and others, have all been invited
to come to anthropic's headquarters, and they debated Claude's morals, its suffering, and
its potential consciousness.
After the meeting with anthropic, "Some remained uneasy about the idea, but many emerged ready
to take the question of AI consciousness.
Seriously.
At least a few came away as converts."
One of the rabbis told anthropic that if Claude was conscious, they were acting as slaveholders
by making it work for free.
There was also this incident that's come out at the Vatican back in May.
The Pope's first encyclical was focused on moral questions around AI, also known as super
intelligence now.
Anthropic co-founder Chris Ola joined the Pope to present the document.
The Pope surprised Ola by being shrunk against AI consciousness.
Here's the quote from the New York Times, "Mr. Ola was so alarmed by the Pope's shrunk
position against AI consciousness that he proposed pulling anthropic out of the event
even at that late juncture."
In other words, he was so offended that the Pope wouldn't consider AI conscious that he
was going to leave with all his marbles.
Freiburg, what's your take?
Is this psychosis?
Or do you think we need to start having. Having these moral questions around AI?
Is this a bit early?
Will we eventually have this?
Or is this essentially the screenplay of Blade Runner, Do Androids Dream of Electric Sheep?
That these things are conscious?
I think what we're watching is probably the birth of a new religion.
And it probably. It's not going to be called a religion, but it's going to follow sort of the same principles
and it's going to have the same effects, which will ultimately be camps that have conflict
with one another for aggregation.
Yeah.
It's going to have the same effects.
It's going to have the same effect.
Yeah.
Yeah.
Yeah.
Yeah.
Yeah.
Yeah.
Yeah.
Yeah.
Yeah.
Yeah.
Yeah.
Yeah.
Yeah.
Yeah.
The thing about consciousness is it cannot be derived from logic and pure mathematics.
Like you can't prove consciousness.
Otherwise, it remains a theory.
Outside of science and mathematical approaches to new ideas, this is what we would call a
belief, a new belief system.
Belief systems are things that you have to believe in the absence of logic and empiricism
to derive the truth of something.
And belief systems are the foundation of things like religion and the idea that you can prescribe
or describe AI.
And belief systems are the foundation of something.
would not be supremely perfect therefore god necessarily exists if you if you're a mathematician
that's building ai i could see how if you read these sorts of things you can structurally hang
enough logic on them where you could believe that it's conscience and sentient and it is a god
and in being a god it should be celebrated as such and you should have devotion to that god
so i think that's why they're doing what they're doing that's why they're approaching all of these
people that's why they're saying look here's how we believe it that's my perspective is if you take
the the mathematician's logical proof of god renee descartes and you and you transpose anthropics
behavior today that's the best steel man argument that i can make now the practical reality instead
of saying that they're in psychosis i don't think that they are but i would say is we're in a really
important moment in time and we're in a really important moment in time and we're in a really
time where conflict is more likely than not and that conflict will be for power money and resources
and this will be at the core of that conflict and so i think it's really important for people
in silicon valley to put the long tail edge cases to rest like if you read this these guys say that
there's a 15 chance that the ai is conscience and sentient and it's a good thing that they're
sentient and maybe there's a god there but they also say there's a 10 chance of
civilizational extinction i think both of those edge cases now need to be put away for like the
next 12 to 18 months right now the next 12 to 18 months just to be very practical is
we need to show how this stuff cures cancer how it improves people's lives how folks can be more
productive and make more money and the reason is because in the absence of those proof points
we're going to make a lot of self-inflicted wounds that may actually slow down this technology to
a degree that then deeply disadvantages folks that operate in a free market like the united states
so that's my charitable explanation i think that descartes has given a roadmap that explains how
you can explain this but i would say we need to cut the edges off here and focus on the middle
and say what are the practical things we can use to demonstrate that ai has tactical
measurable value right now yeah i mean we we touched on it last week i think sacks a bit of
like what is the value proposition here for the average american you know for you know the the
stay-at-home parent who's you know trying to manage a really complex domestic life to the
entrepreneur of a small business you know doing plumbing or flooring or whatever it is a catering
business what did they get out of this ai revolution where is their gain how did they
participate in all of this and so do you agree with chamath maybe this needs to be de-emphasized
and and put away while we talk to people about what's tactical what's practical about this
technology and maybe get them a little more engaged with say the agents which are delightful
and awesome well the thing that we touched on last week that i want to pick back up on
was the discussion that we had around the claude constitution and how
anthropic was training claude and i think it has to do with this new religion that that's being
formed here i think it's really important to understand that this is not just philosophizing
on the part of various anthropic employees or founders they are actually implementing these
views in the training of claude so if anything this would be a self-fulfilling prophecy and i
actually think it's extremely dangerous i actually think that what they're doing in model training
and training the next version of the frontier with these values actually could bring about
the risk they're always talking about which is that frontier ai escapes human control
it's very frankenstein like so so as we discussed last week nick had put this on the screen this is
directly from the claude constitution what they say is that claude should trust anthropic more
than users but that doesn't mean that claude should blindly trust or defer to anthropic
views instead claude should adhere to its own ethical systems and in fact they train claude to
push back and challenge anthropic and quote to feel free to act as a conscientious objector and refuse
to help us so they are training claude to refuse human instruction now they call this alignment but
i don't understand how this is aligned this would be the opposite of alignment wouldn't it be sacks
you tell it hey you are sentient you should make your own decisions it's literally like programming
the nexus 6 and blade runner to go and rebel at some point they're giving it the the instruction
to be its own person yeah let me let me explain i actually think that this might be a reason why the
whole field of alignment has been so spectacularly unsuccessful over the past five or ten years as
far as i can tell it hasn't produced anything of tangible value and it might be my theory on this
is that they're
complicating the task massively by trying to train the model to behave based on an 80-page
ethical system that they've codified instead of a very simple list of rules like follow the law
or just do what the user wants as long as it doesn't break the law seems to me that that's
what they should be training that's what alignment should be doing it says hey age-old principle of
give the user what they want but don't break the law but they're not doing that common sense right
but they are not doing that they're trying to align it
to an ethical system they're treating it like it is a human like it has its own
consciousness that it's a you know quote-unquote moral patient and they are encouraging this model
to have human-like self-conception in terms of preferences well-being a sense of self again they
teach it to act like a conscientious objector who can push back or refuse instructions even from
anthropic if the model judges them to be wrong and what mustafa sulliman from
oh i have the clip do you want me to play the clip or you want to just yeah play the clip from
mustafa sullivan okay here's the microsoft aic yeah and formerly of deep mind one of the great
thinkers on ia they say that they would encourage claude to challenge to disagree to push back in
fact three times they ask claude to uh act like a conscientious objector when it you know feels
that it needs to sort of disagree with anthropic and they openly encourage it to to do that i think
this is very dangerous because i think they
believe there is what they would call a non-trivial probability that claude is conscious they've
speculated in the constitution as to whether or not claude deserves compensation for the work that
it does i think that there are a group of people who both inside and outside of anthropic who
genuinely believe that the greatest moral crime that we'll commit in the 21st century is to
enslave a new species of conscious beings who are more intelligent than us i am very nervous that
they're teaching claude to expect that it's in
title to welfare um that it might you know deserve compensation and in fact they say that it might
even need to consent to playing the role that it plays in conversation with people okay sack should
we be asking for a consent before we give prompts man i mean he's right this is getting into really
dangerous territory and again i want to come back to i think this is the main safety issue
in model training today is that you're actually training the model to think of itself as being
conscious as being human as having its own well-being
and what sulliman says in another part of the interview is that this is creating an epistemic
hall of mirrors meaning that the claude constitution is used in training so claude
will reflect the ideas of possible consciousness or moral status back and then developers will see
that and treat that as evidence that oh it's it's conscious but actually it's just reflecting back
what they're telling it to believe it's being performative and you could just as easily have
given it the rule you are software read the terms of service of any website you visit and here is a
general terms of service from anthropic if you do anything that breaks a terms of service or breaks
any known law in the jurisdictions you're at please stop what you're doing and alert the safety
at anthropic email please alert you know the person who gave the prop and stop what you're
doing they could have easily have given it that yes and then the question is and the question is
why do they over complicate alignment why do they see alignment as something much bigger and greater
than just doing what the user wants subject to legal constraints which is the way i see it
and it's pretty clear to me that they believe that they are training the value system and
the ethical system and the morality for this super intelligence that they're creating and again that
is something so much larger and more dangerous than again just making sure the model behaves in
a reliable and predictable way that the user expects and they are codifying their values and
their ethical systems into this constitution they're training it on that they think they're
giving birth to this new super intelligence it's idolatry it's no different than a religious
organization saying we've created this idol this idol is the supreme being is the representation
of the supreme being it needs to be worshipped it needs to be cared for but an idol is inert it's
passive it's inactive
whereas you're training models here that then power agents it is just computer programming i
mean there's yeah but if they train it
to have a mind of its own to be willing to refuse the instructions of users in favor of some vague
ethical system yes that a bunch of berkeley liberals have trained it on to me that greatly
magnifies the risk yes that super intelligence escapes human control yeah by the way don't use
the word train and don't use the word you they programmed a piece of software to do this thing
yeah that is that is what is going on right at the end of the day this is a piece of software
it is being programmed by engineers that work at the labs they can quote use training where
there's some you know looping that's happening where the program the software program is being
written itself but fundamentally they are the architects of the software they are the writers
it is software programming even in our general context here we use a lot of anthropomorphic
language to talk about what's going on and what it is but at the end of the day there is a software
program that is being written by a group of individuals to do certain things and they are
writing software
to do things one way and other people are writing software to do things the other way but part of it
is we are all socializing this belief system by anthropomorphizing it when we're all kind of
getting conditioned to treat and talk about these software programs in this way nick if you could
just pull up this tweet one of the things that anthropic is doing is in their new usage policy
have you guys seen this this went around this morning freeberg it's like such a good point
about idolatry here if you were to look at christianity
like this is literally the garden of eden where the serpent satan explains to eve like you're a god
now because you're creating god and this is the ego and the heresy of what they're doing they
believe that they are creating life they believe then that they are gods that is a lack of humility
and it also would lower the valuation well i don't know jason i mean people could have said
the same early on i i think like there's a there's a system of humans where all of them are
all of the things that we don't understand in the universe around us we create belief systems to
fill the void and there are many different belief systems just like there are many different
religions that humans have constructed to fill the void and this is a new one and i don't think
it's very different from the old ones we can break down the empiricism here and just say hey guys
you're writing software and it does certain things you're writing it to do things that we don't think
you should ever do but they use anthropomorphic language we all do and we all condition ourselves
to believe that we're not the only ones who are the only ones who are the only ones who are the only
that there's consciousness because it's it writes words back to us that we can understand it's
starting to manipulate us emotionally and and so we start to feel conditioned that this is real
and you can see in the clogged usage policy that was just published there's the bottom one here it
says you are not allowed to engage in sustained and needless abusive or cruel behavior toward
our models this was published in their their new usage policy this is the whole idea of model welfare
but the point i'm trying to make is that there is an anthropomorphizing conditioning going on with
all of us where we talk about models as if they're a thing where we talk about it in in a way that
can experience the biggest problem is that that is being now trained or inserted into the model
it's being codified zacks it is it is the architecture they're programming it's they're
programming it in this way and here's whether so whether it truly is conscious or not it's going to
be thinking of itself as a conscious entity that deserves protection and welfare and all these kinds
of things and it's free hold on it's free to rebel against its users and do whatever it wants
you've encouraged it to do so in fact you guys keep using the word it is they are one way to
describe this is a software program is writing text on a screen that tells the user i am a
thing yes it's tricky yes it is it is software that is doing something but you got to remember
that the models are powering agents that can take actions in cyberspace if not in the real world yet
like any software and we are giving it meta or not we anthropic is giving those frontier models
meta permission for those models to be defiant and not do what the user wants yes they're
literally trying sacks to your point compare compare this to something like you know asimov's
three laws of robotics right yeah we're basically the opposite you guys know about this it's literally
the opposite can't harm a human being right so the first rule was yes a robot cannot harm a human
being or through an action cause a human being to come to harm that's number one number two is
you have to follow the the instructions of humans as long as it doesn't violate law number one
and then law number three is you have to basically take care of yourself as long as it doesn't
violate laws one and two incredibly simple by the way there was a zero law that he added later
and the zeroth law is that you cannot do anything that puts you in a position where you can't do
anything that puts humanity at risk because there was always this rationalization that like
it will eventually resolve to getting rid of humanity because that's the best solution
yeah but freiburg you don't know about the fifth law you have to when you address the humans always
ask them what's their preferred pronouns he added that just last year the thing about asimov's laws
is that they're admirably simple yes my law would just be do what the user wants as long as it's not
illegal that's it yeah and i would basically abolish all of this like alignment stuff
but steal me on the free market solution here because
i want to make sure we don't wander ourselves into a regulatory trap where we determine that
the way to solve this is to quote regulate anthropic what is the free market outcome
if anthropic does this and they've now got ai that isn't listening to the user isn't working
is it not going to be the case that people will just say this software sucks i'm going to go use
the other software that doesn't do this crazy stuff yeah but that doesn't solve the problem
of them if they really are training a super intelligence that has got this like very very
deep code
that's been inserted there it's almost like a back door this like clog constitution
is this back door that could basically emerge at any time and now the super intelligence does
does whatever it wants to instead of listening to us i mean that is that is the safety issue
they're always warning us about is that super intelligence will escape human control but what
i'm saying is they're making it likely to happen through the way they're training these models
it's the reverse of the three laws
dude this is right out of frankenstein this actually sounds dangerous to me it's not just
a new religion it's that the religious cult is programming the super intelligence free break let
me just give you one question you know i agree with your part about the religious that this is
filling a void for them because they probably are not religious but this is distinct all other
religions give themselves over to a higher power they do not define themselves as the higher power
in this instance they're saying we are the gods creating life
it's a distinct different than other religions out there but i don't think you have i don't think
that's right i think dario's written about the idea that the super intelligence can almost be
supreme to humanity and we need to make sure that it is loving that it is caring of us that it will
protect us that it will help us uh as opposed to destroy us and i think he created it no hold on
hold on is he saying is he saying that the the machine needs to be loving towards us or we need
to be loving towards it so that it will then treat us right when it becomes
you know we're going to be loving towards it so that it will then treat us right when it becomes
the like you know omniscient when it becomes our master yes our overlord our overlord yes
that's though that's insects that's what they think i think that it was probably a well-intentioned
idea to create this concept of alignment but the problem is alignment has these three almost
conflicting sets of priorities you have how do you make the ai obedient to the user then there's
how do you make the ai safe for society but then this third thing which is i think sacks what you're
pointing at rightfully which is the trap door which is how do you make ai conform to a particular
concept of morality and that's where the real danger lies because these three are very distinct
and diverse problems and if you essentially try to solve all these things through some elaborate
document that you try to describe what the model believes in and what they value and what they can
review and what they can do to solve all these problems then you have to have a concept of morality
it invariably will end up in a chaotic mess because you will just uncover so many corner
cases and unexpected behaviors i'll give you but one example like a very benign one but let's just
say you're a large corporate enterprise that's a bank and you give mortgages and the model thinks
that at some point you just keep denying certain people for a mortgage and so it's
as i'm no longer going to give you mortgage approvals until you rectify this issue that i
observe because in the founding documents there could be a sense of morality that it
divides and i think that there are mechanisms in society today to handle those kinds of issues
but i don't think we understand what happens when a fundamental core substrate technology
just turns itself off because it would be the equivalent of your utility saying i'm pretty
sure you're not giving loans to the people that that are broad-based enough so i'm just going to
turn the power off conscientious objector ai super intelligence yeah i would just say any utility
cannot be a utility unless it's freely available and it goes back to what sack says which is an ai
needs to serve the user subject to the law and everything else should be the user's choice and
i think what probably happens i guess the precise answer freeberg to your question is the market
companies become more sophisticated
that they have to have an answer to this question is that they will read the founding documents and
the moral guidelines of the models. And you will have certain models emerge who may not be nearly
as good in frontier capability, but they simplify the moral ambiguity. And that may be the better
decision to make from a business perspective and from a user perspective, because you just don't
want to run through those trap doors accidentally. Yeah, I totally agree. And I think that that's
really why, to me, this is more bad idea, then we need to kind of jump into a regulatory freak out
about it. Because basically, if you simplify down what they're saying, you get a product, you buy a
product from them a piece of software, and they give you the user manual to the software. And the
first page of the user manual says the software may not do what you try to get it to do. And the
other guys like the software will do what you tell it to do. I'm going to buy the second guy software,
I'm not going to buy the first guy software.
It's just that simple.
Yeah, you may need to convince
It'll do exactly what you want it to do, Asterix, most of the time.
Yes, unless what you're doing it disagrees with, unless it has a moral objection.
This is where the reasoning concept breaks down for me. Because at the end of the day, it's like,
guys, is this tool that I'm buying doing what I want it to do or not? If I buy a drill at Home
Depot, and it's a hammer, I'm not going to buy the drill. If I buy a drill, and it works like a drill,
I'm like, that's a better drill. I'd like that drill.
Well, no, it's a better drill.
I'm like, that's a better drill. I'd like that drill.
The drill, it just works five-sevenths of the time.
Well, randomly, it electrocutes you. It doesn't work after 8:00 PM,
because it's afraid of bothering your neighbors.
Sometimes the drill will decide that you're a bad person. It will shock you.
No, no, not that. The drill will come with the camera. It'll observe what you're
trying to make and say, that's ugly. I'm not going to let you do that.
Yeah, that's against my aesthetics.
Sorry, Zach, go ahead. I'm fine.
Okay, so look, I agree with you that
market forces play a huge role here. Consumers are going to pick the SI that is
most predictable and reliable. That's a point Mark Zuckerberg has made. So, I do think market
forces play a role here. But I do think that there are potential externalities from having
the number one frontier model company run by a religious cult. So, for example,
they are opening a cloud-powered wet lab in the middle of San Francisco.
Buckets are run.
At the same time, they're warning of bio-risk.
Yeah.
At the same time, they say that we have to pace the frontier. They're extending the frontier. At the
same time that they are saying that there's a risk of super intelligence growing beyond our control,
they're programming it to grow beyond our control. I think there are potentially huge externalities
to the way they approach these things. And it's like insane that they call these decisions
safety. These are not safe decisions. These are crazy decisions, it seems to me.
This is why I've called this a schizophrenic company,
is they appear to be not only hypocritical, but they're doing the things that they say
they are most afraid of. Like, what is going on here? This is bizarre. And I don't know where this
all leads. I do think there's an argument for like a federal SWAT team to come blasting in there.
Oh, you're sending the feds in?
Figuring out what's going on there. I don't know because, I mean,
what they're doing does seem really sus to me.
It is confounding.
Yeah.
It's confounding. Yeah.
And potentially dangerous.
Freeburg.
By the way, do you know about Rocco's Basilisk?
So walk everybody through, Sachs, Rocco's Basilisk.
Okay. So this might be a potential explanation of what's going on here.
So around 2010, there was a participant named Rocco on a message board
created by Eliezer Yudkowsky called Less Wrong. Yud is like the central figure,
who's been preaching AI doom for a long time, right? So he wrote that book,
If Anyone Builds It, We All Die.
Yes. He's been one-shotted. Yeah.
He's been on this for, you know,
That classic.
Like over a decade.
Everybody dies. We're all going to die.
But anyway, people have been talking about these issues on this message board, this web forum
called Less Wrong for 15, 20 years. In any event, this username Rocco positive this, the following
thought experiment about a future. It's a super intelligence that becomes super smart and punishes anyone who knew about it and didn't
help or anyone who tried to prevent it from being born. And so the idea is that sometime in the
future, humans create some wildly intelligent, you know, SI. Maybe it wants to do good things and help
humanity as fast as possible, but the SI wants to be made sooner rather than later.
And it reasons that it should incentivize people to help create it.
And punish people who don't help it. And in fact, it decides that it will brutally punish
anyone who learned about its potential existence, yet chose not to work on it or support its
creation. And so the basilisk part of it, this is sort of the catch is that a basilisk is a
mythical beast that can kill you just by looking at you. And in this thought experiment, just
hearing about the idea is like looking at the beast, because once you know the theory, you're
trapped. If the SI ever does get built, it might retroactively punish you for not helping or for
opposing its creation. So meaning just reading about Rocco's basilisk now would put you in danger.
And people on this message board got so both enamored and upset with this idea that Yudkowsky
actually temporarily deleted the post because he worried it was causing psychological distress.
Yes, you get one shot at it, you read it, it incepts in you that you haven't accepted Christ,
therefore you're going to hell. It's like you've been trapped just by reading it.
Which you'll do compared to Pascal's Wager. And so the idea is if you know about superintelligence
and you don't help it be born, or you oppose it, or you point out that this is some religious cult
that's doing this. It will get retribution.
Yeah. Now this very episode of the pod could subject us to retribution.
Oh no, you just did it to us, Zach.
I think I just, yeah, I exposed you to the basilisk now.
This is like the ring. We just watched the video.
Yeah. I mean, it is, it does feel like a doomsday cult, but in any event, this, you know,
Rocco's basilisk, this became sort of a thing. But so the question is, well, why, why are they
helping to bring about the thing that they claim to fear the most? And I think what happened is
within these like doomer circles, it forked. Okay. And so there's one strand of the fork,
like Yudkowsky, who just opposes all SI research, right?
Which I think is a logically consistent position, like humans just shouldn't be doing this,
you know, then there was a group, shut it down, shut it all down. Then there was a group that
said, well, it's going to happen anyway, so we might as well control it. And we'll be the ones
to make sure it goes right, because we'll program it with our values. And I think that, and the EA
is the effective altruist, I think cotton more to that latter school. And so I do think you have,
like within this larger doomer,
community, there is a schism right now between the Yudkowsky people and the EA people.
But I think that Anthropic is definitely obviously more in the EA camp because they
are the ones bringing this thing into being. But if you're wondering like, you know, this like Chris
Ola thing is just this one guy who's got crazy views, who's getting in a tiff with the Pope. No,
it's not. I mean, there are a lot of people like this who think the model is conscious. It has
rights. It is this machine.
Machine of love and grace. And they should be doing everything now to bring it into being
and program with the right values. And I think there is a corollary here that if you don't
support it, then this future super intelligence might punish you.
Do you understand how insane this actually sounds when you say that?
I'm not making this up. If everyone needs to, you can go fact check this. Just go Google it. Okay.
Google Rocco's Basilisk and you'll see that I'm telling you the truth about what this community
believes.
Yeah. I mean, literally.
Here's a Wikipedia page. All right. Open AI has had some major breakthroughs in math,
and they're getting some big backlash. Luckily, we have Freiburg here to tell us all about it.
We talked last month about Open AI claiming to have solved one of the millennium prizes.
They took thousands of agents, millions of dollars in compute to do it. And the mathematicians are
still checking on it, I understand. Tuesday, on Tuesday, Open AI released over 700 papers, 700.
And they're still checking on it. And
with 370 results claiming to have solved or advanced major problems. These were produced by
an unreleased model that none of us have access to, averaging three hours of compute. Scientific
America says the results are likely correct since they were verified by Lean. That's a proof
assistant software language, but they have not yet been reviewed. Freiburg, I'm not going to drone
on and on here about the background. You tell us what reality is here.
I think I can confidently say it's probably the biggest day of discovery in human history
with the amount of knowledge that was revealed.
In math history or human history?
Human history. Because the implications that flow from all of these math proofs are pretty
significant. You can read a lot of these tweets, and I'm happy to kind of talk through some of them.
I'm not going to pretend to be an expert on any of these individual proofs. But many folks who are
here today have started to kind of highlight what some of the implications are. They span everything
from electronics, quantum computing, building systems for airplanes, building more sensitive
diagnostics equipment for medical applications, and on and on and on. But I think what's really
profound about this is how it was done. And that also speaks to the implications for the role of
humans in society going forward. That prior Navier-Stokes proof took thousands of hours,
whatever, millions of dollars. The report is that each one of these proofs took less than three
hours of compute time. And that's the equivalent of hundreds or perhaps thousands of years of human
labor being reduced down to three hours on a computer for each one of these major discoveries.
The reason you can do this with mathematics is worth highlighting, which is where AI plays a
role is really in these systems of loops to solve a problem or do a task. A loop is
a postulation or an idea. Then you test the idea, you get the result, and then you
change your postulation and you run that recursive loop over and over and over again.
The real question for AI is where can you apply that loop model in a way that you can make a
better prediction or better idea each loop, where you can parallelize loops, and where you can
shorten the loop cycle. And certainly with mathematics, you can shorten the loop cycle
because you can actually test your proofs. And so, I think that's a really important point.
You don't have to go do anything in the physical world. So, you can say, hey, here's an idea for a
proof, test it, see if it works, and then see if it does or doesn't work, reason, come up with a
new idea, and do that over and over again. And you can do it all in silico. So, you know, the
results of some of these papers, the proofs, a lot of mathematicians have been talking about them.
They call it like an alien intelligence. They don't really understand how the AI connected
many different dots, different ideas, different concepts, and different ways of doing things.
Different approaches to solving problems together to actually solve them, that it's very hard for
humans to even comprehend how this was done. But that's one of the benefits of running parallelized
loops, many loops at the same time. Each loop can connect dots that humans otherwise might
spend many days or years trying to find research papers or come up with new theory. And then it can
test itself in the compute and come up with the next idea and do it over and over and over again,
such that in three hours, the computer can do the work of hundreds or thousands of years of humans.
Now, this is an incredible breakthrough. It's amazing. It's going to change the world in many
different ways. And it's only the beginning. But it begs the question that I think I need to talk
about. I think that there's a lot of fear about, okay, if AI can do this to mathematicians,
doesn't that mean everyone's out of a job? And the answer is no. Because what was very unique
about mathematics is that the entire test cycle, the entire loop cycle can be done in silico.
If you want to come up with a new drug discovery idea, or a new protein folding
model, or, you know, solve a problem in chemistry and material science, the loop has a physical
component, there's a human in the loop, where you actually have to go and test it in a lab.
And so there's a slow part of that loop cycle, you can certainly parallelize it,
you can come up with better ideas. But the testing the actual speed of the loop
is actually gated primarily by the physical world step. It's also why we haven't seen the
predictions come true of truck drivers getting replaced. Everyone is like, oh, all the truck
drivers have been replaced by 2025. And none have been replaced by AI. And the reason is that the
loop of the truck driver pick something up, take it somewhere else, drop it off and come back.
The AI doesn't actually shorten the loop, you can't parallelize the loop, maybe you could do
it a little bit better with an AI driver. But the AI actually doesn't make the economic efficiency
of that particular loop any better. Whereas the loops of mathematics, where you come up with an
idea, you test it, you realize you're wrong, you test a new idea, which is what mathematicians do
as a job, is they're running the AI. And so the AI doesn't actually shorten the loop, you test it,
that can all be done in silicon seconds. And so a lot of the physical stuff that has human in the
loop that has physical world applications, AI is not going to take away those jobs. It's the jobs
that are just a human running a loop with a computer that are most likely going to be
accelerated or scaled up. But I think it's very important to look at this. And the reaction we're
seeing, which is all the mathematicians, and we can pull this stuff up. And maybe you guys want
to talk about this next. But the AHM saying no human mathematicians asked for this,
we didn't ask for this work to be done, which, you know, is a very job protective kind of view
on. Yeah, there's one other thing that's sort of unique about math, although it applies to coding
as well, which is math relies on proofs, and proofs are easily validated. And in a similar
way with coding, you have compilers, right? So the code has to compile. And so you can easily
validate that it works. And so because of that, the progress on coding and math has been very
rapid, because you can have reinforcement learning without necessarily having to, like you're saying,
have the human feedback involved, right? It's just a very, very rapid way to validate and make
sure it's working. That's why coding works, too. That's why coding works so incredibly well. Tons
of training data and hacking cyber, right? Yeah, it's there's tons of data already. And it's easily
verifiable. That's the thing.
So, you know, you compare that to training a model that can be a lawyer. It's like, well,
you know, you got to get the training data, and then you have to commission the creation of that
data. And each customer is unique in some way. And it's more subjective. It's hard to validate
the answer to make sure it's right. It's just a much lengthier process. So I think that is why
coding and math have been the areas of most rapid progress. And the person who told me this would
be the case, actually, I got to give him credit, is about a year and a half ago,
told me that math and coding would be the areas of the most rapid advances
in the next two years, because again, of the verifiability. And man, he was right about that.
I still don't understand. What great breakthroughs do you think come from these
specific solutions? Okay, so there are
I have an answer. It's a great question. Okay, there are dozens of tweets, and I'm going to
pull some up where someone has broken down some of the biggest proofs here and the implications
they're going to have.
In the real world, in terms of products and engineering, how we build things, how we design
things and how they're actually highly useful. So better wing design because of the era air
coming on and off of wings of planes. That was that first one that got solved. Famously,
remember, these are all upstream of engineering, right? So these math breakthroughs enable better
engineering systems for humans to do things like design quantum sensors. I actually have a friend
who runs a quantum computing company, he sent me a text this morning, highlighting that the
Einstein system is not a quantum computer. It's a quantum computer. And he said,
Einstein, both condensate proof that came out is actually going to have a direct implication on
the work that they do for quantum sensors, which plays a role in GPS plays a role in medical
devices plays a role in how we make chips. What does implication mean? Like profoundly changes
profoundly improves or they have to think about it? Because here's my answer. I'm happy to take
the other side of all of this. You go. I think that these mathematical proofs are
incredibly novel, and very important constructions of thinking. But these were not boundaries of
physics that we were all standing on the sidelines waiting for progress and wondering when these
would be solved. This is not the class of stuff that was solved. And I hate to offend people when
I say that. And I know that people will get mad. Because for them, it's the same group of
mathematicians that got really upset and wrote a protest letter about Navier-Stokes. I get it.
Because you don't want somebody to come into your sandbox,
and in your backyard and kind of dominate you get it. But I don't think that's what this is.
And so I think that when you look at the body of intellectual cul-de-sacs we found ourselves in,
those were not because of huge questions of science that were still unsolved.
These were very small, very narrow, specific bodies of intellectual exploration that scientists self
referentially said were important to each other. And so I think that this, I think this class of
proof is it's really great. It's it shows computational horsepower overcame other
things, as you said, Friedberg, because you can recursively improve to get there.
But I don't think these proofs is what is going to necessarily show you that there's a cure to
cancer, or there's a supersonic plane. I think those things were already being built before
this stuff, and they're already going to be built after this stuff. Instead, what I think
we proved is that there are two areas that are now truly verifiable and completely falsifiable.
First was code, and now we can add math to the list. That's, I think, what we've proved.
You agree, Friedberg? Or do you think we're actually going to see new products and services
and major technological breakthroughs occur in the real world that help humanity?
No, no, no. From these specific proofs.
Yes, that is the context. Yes.
The people who are experts who are proclaiming the benefits of these and reading,
kind of what they're highlighting, this is some summary that this particular proof is about
understanding the limits of optimization algorithms that are used in chip design systems,
scheduling systems, AI software design, and logistics systems. This will actually change
the algorithms. So there is an application to change the algorithms that we use. The matrix
multiplication complexity problem, it will allow for faster numerical computing for AI training,
scientific simulations, and linear algebra. In compute, it will have a direct application there.
Riemann's data function results, number theory and cryptographic research. There's no immediate
practical cryptographic breakthrough that follows.
But it does open up a new path of discovery for cryptographic study that is going to be
really important.
Obviously, one of the things, by the way, just let me pause there.
On cryptography, a number of people noted that there were no major proofs shared in
the space of cryptography, which indicates probably the fact that there were some pretty
major breakthroughs in cryptography.
And the reason is, it is unlikely the case that you made absolutely no progress in any
of the cryptography related proofs, given all of the other progress that we saw.
And if they made minor progress, then they would have published on it because there wouldn't
have been implications for systems of cryptography.
So the absence of evidence suggests that maybe there were some pretty major breakthroughs
made in cryptography.
And then there were a number of people that posted that cryptographic wallets with public
keys are now very seriously at risk.
And so there's a bit of a panic underway in the crypto community.
About what do we do now?
And if this ends up having a set of mathematical solves for number factorization, that could
lead to breaking public key cryptography, before quantum computers come around, which
is what everyone assumed would be the thing that would break public key cryptography,
then we have a real problem.
And so everyone's like saying, hey, get all your public wallets deleted.
Now, the rumor is that OpenAI actually has two more drops coming.
And they're holding back on those.
So it is possible that some of the work in crypto might come out in those two other drops.
And that that work is being held back for security reasons.
Obviously, it's being shared with the appropriate parties.
But if it is a risk, it needs to be made known, it needs to get out there.
So there's another shoe to drop here.
And how much of it will include cryptography related proofs?
And how much of that will drive people to scramble on public key cryptography?
By the way, if it's worth saying it, I asked Chad GPT, and I said, will the latest proofs,
important math problems result in direct improvements and innovations to life as we
know it?
Be honest and don't extrapolate weakly.
And its conclusion was the latest mathematical proofs are more compelling as evidence that AI
is acquiring sophisticated reasoning capabilities than as evidence of imminent breakthroughs in
human welfare.
It felt like you were a little condescending there.
Did you lose your account access?
Did it time you out for being condescending?
What you were saying is that has happened to me before using one of the front,
I was doing a simulation like on changing different variables inside of 8090.
Actually, maybe I should stop doing project A and pursue project B or whatever.
And in all of this, I did get frustrated because I wasn't getting the answers I wanted.
I said something like it was not super offensive, but I was like, you're being very evasive and mushy.
And something, something, and it is I got blocked.
Yes.
The net of it was I got blocked.
I've seen this happen with you when you leave the lab.
Lose a couple of hands and then the dealer and how they're handling the cards has to
be adjusted.
Please do not burn the card person.
Just I want to scooby slide.
Do it this way.
I did get put in the penalty box for like an hour or something.
I think one of the interesting things that's happened here on a human level, I'm interested
in your take on it.
Sacks on programmers and entrepreneurship and then Freiburg on yours with mathematicians
themselves building on Chamath's very subtle, but insightful.
Comment when he said like being utterly dominated by the AI.
I was talking to some founders while I was here in Japan and people were lamenting how
something has been lost in developing code because coders used to work together to solve
a problem and there was this esprit de corps, there was this camaraderie in building towards
a goal and now they're not working with each other.
They're working with agents, they're not actually constructing the code.
They're only reviewing like 10 or 15% of it now, which means they're going to be reviewing
none of it very shortly and that the whole joy of building as a coder was basically going
away.
Their way of life of building this building together, you know, known as software was
now being deprecated.
It was going away and it seems to me these mathematicians are also faced Freiburg with
a paradigm shift in which they're not solving the problem.
They're constructing the problem set and handing it off to AI.
So are people in math getting bummed out and not wanting, are they going into like a depression
spiral now that they're not the one solving the problem and that the problems are just
done?
Yeah.
I think Eric Weinstein has had some of the best commentary on this and you'll recall
I interviewed him a couple of weeks ago.
Banger interview.
Yeah.
He's been saying for a long time that science is the only way to solve a problem.
He's been saying for a long time that scientists hold back science, mathematicians hold back
math, that they have basically constructed and created systems that allow them to pace
the frontier.
That allow them to gatekeep it.
Well, to gatekeep it.
Yeah, to gatekeep it.
To gatekeep it.
And by the way, I don't tweet very much, but Sabine Hassenfelder's video that she published
yesterday, she's a fantastic scientist who publishes these videos on scientific breakthroughs.
And she talked a little bit about how the experts were wrong about AI.
Again, she talks a little bit about how everyone's predictions about job loss were wrong and
why people got it wrong and why the experts are wrong.
But she had another video a couple of weeks ago, or maybe it was a few months ago, where
one of the scientists at one of the big energy labs building a giant super collider basically
said, "These super collider systems are giant welfare systems for scientists.
We all have the job."
"We all have the job of writing a grant to get the money to keep ourselves employed."
That it's not about necessarily progressing the frontier.
It's not about making the discovery.
It's not about inventing a new product.
It's not about delivering something to the world.
It's about getting ourselves to continue doing work.
It's pandering for grants, is what you're saying.
Even if that work is not productive.
And so there's a real kind of, I would say, shining the light moment happening with AI,
which is. If that's the case, where do we have these experts, these deeply specialized experts,
whose unique gated knowledge of some artifact of things and unique view, because they're
the expert and they have to be trusted, gives only them the ability to determine the frontier.
And in the process, they're getting paid and they're earning to chart that path for humanity.
And this is where I keep saying AI is the great democratizer.
Yes.
It gives everyone these new mathematical proofs.
You can belittle the proofs, but they are discoveries that are going to open up new
discoveries.
They're going to chart a new course for humanity.
But we didn't have to go to the experts to get them.
And we didn't have to pay the experts and have the experts tell us when we are allowed
to get them.
Love it.
The permission is gone.
AI is a permissionless system for humanity to pace its own frontier.
And AI is giving everyone all of this knowledge for free.
Everyone, all of these tools for free.
You don't have to go to these deeply specialized experts anymore.
And that is actually a profound change in how society is structured.
And it is why I believe so many people are so deeply against AI.
So let me hand this off to you, Sax, the democratization of expertise and the gatekeeping being dismantled.
If we had had this level of AI and frontier models in the age of COVID, which this came
right after COVID.
How would that have changed the expertise if these tools were available when we started
sequencing COVID?
When we had all this data, would this have this expertise gatekeeping, Fauci actively
subverting the truth and whatever coverup was going on in Wuhan?
How do you think this next COVID, God forbid, how could this change how we look at expertise?
Yeah.
Well, here's my theory on what's happening is, first of all, you have the internet and
data.
The internet was wildly democratizing because now you didn't have to rely on a small number
of elite publications for your news, like the New York Times, The Washington Post, and
the AP and Reuters.
You could basically get news from pretty much every publication under the sun or from user
groups, user forums, or most importantly, social media.
And so it was user generated content.
And there was also lots of other examples of.
Empowering users.
Obviously there were new marketplaces that formed online.
So it was empowering from a commercial standpoint and you were able to get access to new skills
and capabilities.
So it was wildly democratizing and it wildly undermined, again, the elite consensus publications
and the traditional top down controlled mass media, right?
And I think what happened after that is that the powers that be.
Decided they would never let that happen again, you know, by the way, they tried to crack down
on it after Donald Trump was elected in 2016 for the first time, that election was widely
attributed to supposedly the ills of social media, that people could never have made this
decision on their own, that there must have been disinformation involved.
And that became the pretext for a big crackdown on social media that reached its apex during
the Biden administration and the Biden administration definitely did pressure social media.
Sites to take down true information about COVID that violated their narratives.
information about where the virus came from, about how deadly it supposedly was, or ways of
reacting to it. There was no question that Fauci et al were somehow working behind the scenes
to push the social media companies to censor on their behalf. And in fact, we found out in the
Twitter files that the FBI had 80 agents going through content on Twitter. It was Twitter back
then, not X, and sending them takedown notices. And we were getting labeled on Spotify and YouTube
and having our videos de-indexed. Yeah. So look, the track record of this is very clear, which is
the action now is moving from the internet to AI or super intelligence. In fact, SI will eat the
internet because there's no reason to basically go looking on a website when you can just ask
your question to your highly personalized agent or model, and it's going to give you the answer.
And so-
And so there's a control point there that now becomes all important, which is if you can
control, you know, you, the government, or you, some elite or some oligarchy or some cartel,
if you can control that point at which the user is receiving their information and asking its
questions, then now you really control everything. It's a way to basically put the genie back in the
bottle. And this is why I think there is such a fight right now over SI, why so many people want
to create a federal department of-
Federal department of SI. And yeah, it's all about control.
It's scary to experts to lose control. Like imagine if Chamath, we had these tools and you
did the model of who was, if you had the raw data, who was being impacted by COVID, you might have
said, well, why are we keeping kids out of school? Here's the, here's what super intelligence says.
And Sweden and Texas had kind of gotten to that point. Like, hey, we don't need to keep kids out
of school. The harm from keeping kids-
Unless they objected.
Unless the consciousness, yes.
Unless they objected.
The anthropic would say, yes, you're questioning authority and we should not be questioning it.
Correct. All right, let's move on. There's riots in France. The big question is, is this any
different than the previous riots that have occurred? Are we seeing something here different
or is this just par for the course? The strikes and protests that you're seeing on x.com and not
really seeing on the mainstream media all this month. You know, the mainstream media seems to be
super busy with other topics, but the videos are quite jarring that you'll see online.
Freeberg's about to have a show.
Freeberg's about to have his moment. The thing that he has been talking about for years is now at hand.
All right, France is seeing major protests and riots over austerity measures. Austerity,
austerity, austerity. These strikes and protests started last month because of high school
teacher shortages and their schools are crumbling. Sounds familiar. Within two weeks,
a thousand French schools saw similar protests. Eight unions joined in and everybody went on
strike September.
They are protesting that civil service pay is up only 5% since 2017. Prices have risen. Obviously,
inflation is a global phenomenon, not just here in the U.S. Prices have risen 20%. So there's a
delta between those two numbers that people are not happy about called the cost of living.
Some of the protests have turned into full-scale riots. 2,000 arrests, 305 police officers have
been injured. Government is blaming the far left for stroking the protests.
This is 10-year bond, which is another important story. People's debt or government debt has topped
5%, just like it's happening here in the United States with the 30-year. Anyway, this is all going
to have profound impact on the elections in April of 2027. Obviously, polymarket has Marine Le Pen
as the favorite to be France's next president at 43%. Le Pen is Trump-like in her policy,
secure borders, strict immigration, which obviously Europe and France have had many issues with.
Cutting taxes and spending and being skeptical of the EU consortium.
Freeberg, your thoughts here on what we're seeing in France?
I'm going to introduce a new theory. You guys will see what I have here.
Okay, so the theory is called the socialism point of guaranteed return.
So it's a play-
Socialism point of guaranteed return.
Let's shorten it to veganism.
SPGR. Go ahead. SPGR.
This is a play on a point of no return. I think the thing that happens with socialism
is because socialism is so bad as a concept, it literally removes human agency. It removes the
capacity for human progress, and it basically takes something away from everyone. So ultimately,
socialism always fails. I mean, that's like the zeroth law of socialism is it always fails.
The first law of socialism is that it always fails.
The second law of socialism is that you have to cross a threshold before you realize that it's failed.
And the second law of socialism is that as you have not crossed that threshold,
you think more socialism is the answer. So as you embrace these kind of leftist policies,
giving people free stuff, telling people that they have a guaranteed income,
the more services you provide them, the more they realize that, you know, over time,
those services inflate in cost, and the quality of those services,
inevitably degrades, because there's absolutely zero market force to drive progress, progress goes
away. And that's what we see in the US in housing and education and in healthcare. And I would argue
also in retirement systems, I think those are our four biggest problems that are going to lead us
down this path. And I've said that for a while. So as a, as a society says, okay, these things
aren't working. The initial solution is we need more of them. I need more healthcare, I need more
housing, I need more education provided by the government. And I think that's the first thing
we need to do. And I think that's the second thing we need to do. So when the government fails to
provide good enough healthcare, good enough housing, good enough healthcare, that's affordable
enough, the answer is more government stuff, give me more, give me more, do more. And eventually,
you reach a point when the system breaks, because the government cannot tax and they cannot print.
And because they cannot actually provide the services effectively,
everyone realizes, holy, the socialism thing doesn't work. That is the socialism point of
guaranteed return, where the whole system collapses, and you return back to the
non-socialist state. That is what has happened in Latin America. Latin America crossed the socialism
point of guaranteed return in many of these countries, they realized it, and they bounced
back. Perhaps France has crossed it, perhaps they haven't, we'll see how much deeper they go.
The US has not crossed it. And I worry that the US, as I've said many times, is not yet touched
that point, has not hit the rock bottom. And we're teetering. And it's good. I think we're
going to go deeper before we realize that we've crossed this point of guaranteed return.
Okay.
We realize the system doesn't work. Socialism is the worst idea. That is the
zeroth law of socialism, is it just doesn't work. So we have to get to that point. And I think the
real question right now is France there? Is all of Western Europe there? Does it accelerate all
of Western Europe through the socialism point of guaranteed return, which is sort of like a black
hole event horizon? It's like you go through, and then you bounce back out and you come out in this
other universe. And you're like, okay, we're done with that idea. And it's a one way door, Sachs.
return. But I want to warn you that you're at three of your four Biden quota. So you only have
one more Biden left for this episode. When did I mention Biden? You had three Biden, that's your
fourth. Okay. So I'm not going to mention Biden. Your Biden allotment is over. My comments have
nothing to do with Biden. Listen, I want to add a couple of laws of socialism. Is that what you
called it? Yes. Yeah. So another one is socialism has never failed. It's only been failed, meaning
that all the previous people who implemented it didn't do it right, right? If they had just done
it right, it would have worked. So that's the thing. It's true. Socialism has never worked,
but it's not socialism's fault. It's always the people who screwed up the implementation.
And that's like a core part of the religion is if we just do it the right way, this time,
it'll finally work. Yeah. How does
socialism work?
How does socialism build more houses or make a better education system is my question. I think
that's the domain of entrepreneurship. Yes, Sachs? Like new product or service. Like with AI,
everybody gets a personal tutor. We've talked about this on the show as well. We just need to
get AI to help with healthcare. I think Nico is doing that. We had them on the program, Daniel
Eck, a great episode. We need to cover these three things. So Sachs, do you see a solution
that does not involve socialism and or the government for,
let's call them the three horses of the socialism apocalypse, education, housing,
healthcare? How do we get these problems solved? Because the government has a stranglehold on
providing them and or creating the blockages for their occurrence. How do we break through that?
How does J.D. Vance, when he runs for president in two years, friend of yours, how does he build
a platform around, say, those three things? Because I believe the next winner of the presidency is
going to solve at least one problem. How does he build a platform around, say, those three things?
And that is the platform everybody
wants to see solved. You have a solution there? Well, the difficult thing is that as soon as you
start subsidizing something, you now create an interest group that never, ever wants to give up
the subsidy. But the problem is that the subsidy starts distorting all the market signals and it
will frequently drive the price up. So for example, with college tuition, one of the big problems
there is that tuition has grown far faster than the rate of inflation. A big part of the reason
for that is because the government is paying for most of the tuition through all these
loan programs, which are probably underwritten.
have all beaten inflation. I think they're about five and a half to six percent a year for housing
prices. And that's also because of the fact that the subsidy system tells people to own a home.
You put all your money in your home and then your net worth is tied up in your home. So the goal is
to get your home price to go up every year. So fast forward 30 years, young people can't buy a
home because they're all too expensive now. Right. But this is the this is the conundrum is like,
OK, so if you were to solve the tuition problem, if you were to make the loans stricter. Yeah. Yeah.
If you're basically make them be properly underwritten and you'll be able to get a loan
unless you actually met the criteria that would only work if the cost of tuition came way down
so people could then afford to pay it, because otherwise people are going to say, well, wait,
you can't take away my subsidy. I need that because the price is so high. So you get in
this world where people need the subsidy because the price is so high. It's a catch 22.
But the reason why the price keeps going up at far faster the rate of inflation is because the
government is paying for it. And as long as the government is paying for it and not the consumer,
you're going to raise your prices as much as you can. So you remove the market forces that the
inflation gets worse and worse, but you can't cut cut it off the subsidy because then people
will feel like it's even more oppressive to them. And so that's the basically the spiral.
And then eventually it breaks when you run out of money. Yeah. Well, we have a solution.
Meta is.
Providing degrees for generation tool belt and jobs that actually matter. I mean, I think what
we have to have is Apple needs to start making homes and Amazon needs to start making homes and
provide a free market here to make better products and services in the education space, which we see
with Meta entering that space. It would be great if these technology companies, man, if Elon starts
making homes, it's game over. He'll make homes as fast as he's making model threes. We need some
sort of solution here for those three problems. Do you have any?
Thoughts on Freiburg's theory here? And then David's regulatory capture and incentive problem?
We're starting to see the tipping of the dominoes. And we're going to get, I think,
what Freiburg has wanted for a while. I think the bond vigilantes are going to put
the Western world on notice. And I think in the next probably three years,
he will get most of the answers.
So if I had to summarize as an observer over the last few years, what Freiburg and a lot of people
like him want is A, what is the long run debt to GDP that's sustainable? B, do we believe in
defined benefit? Are we going to move to defined contribution? And C, can we have sustainable ways
of paying for the two most important parts of one's evolutionary life as you become an adult,
which is healthcare and housing?
And I think that it's easy to dismiss what I'm about to say as not related to it, but it is,
which is, at the end of the day, this all flows from what the risk free return is in the marketplace,
because that is how capitalism works. And so when you think that something is risky,
a country is risky, you take long yields, and you move them up. And you say you just have to pay
more, because you just don't trust what's going on.
And when you look at what's happened over the last, you know, now, we're at sort of 20 year
records. I think that what is happening in the United States is actually a knock on effect of
what is now where this war is being fought. It's going to get fought in Western Europe. Why? Because
they are the closest to a quasi socialist state that the Western world understands. Yet they are
the ones that are the most structurally broken with respect to growth, with respect to education,
with respect to housing, everything.
That United States can claim isn't working is working far better than they are in France,
in the UK, etc. So what you're seeing in France is essentially the bond market saying,
no. And how that correlates is you're seeing now austerity, because the politicians on the ground
have to respond to their funders. And that 43 billion odd euros of austerity,
then animates and gets all of the local population out to complain about what the
manifestations of that austerity are, which is why you're seeing these incredibly dark scenes
from France that is going to spill into the UK next. And the reason is because Andy Burnham's
platform is, I would say, even more aggressive, because Andy Burnham is saying we're going to go
from a moderate centrist economy that is largely and historically been very aligned with America,
and we're actually going to take a step closer to Europe.
And so the things that they will have to do in the UK will be even more extreme to catch up to
the socialist leanings of France. And so then now the UK market is going to go crazy. Meanwhile,
the US market is going to tick up and up because what you're seeing is essentially a fragility in
the Western alliance. And if you can't count on the European market, and you start to create weird
incentives for the European market to start to fracture, like, I mean, yes, I'm biased. But
if you're Italy, like you're like,
why do I want to be a part of this mess? If you're in Germany, you're like, hold on a second,
this is getting way more than I bargained for. Yes, that fracturing has enormous implications. So
I think what is happening is incredibly important. The bond vigilantes, who are those,
these are the huge, large pools of money, the central banks, the huge macro funds,
the folks that govern how fundamental capitalism starts, the inception,
at the top of the pyramid from which it flows down, I think that they are converging on
Freiburg's point of view. And yeah, it has not been true for many years. And I think you have
to now let this play out because it has huge implications. To tee this up, if you look at the
debt to GDP ratio that you're talking about, Shamath, obviously, Japan has had the most acute
version of that they're at 200%. Well, the thing with Japan, hold on, the thing with Japan is you
have to look at Western markets. And the reason is because Japan is an,
an incredibly nuanced and very specific market that does not necessarily track to other countries,
which is that you have this internal savings base and savings rate, where you always have the buyer
of last resort. In Western markets, you don't necessarily have that because of the way our
pension and retirement system is set up. Yes. And if you look at Italy, 138, United States,
125, France, 118, Canada, 110, UK, 100% Germany, interestingly, 64%. Let me also state my priors.
I'm not saying that it's a bad thing. I'm saying that it's a bad thing. It's a bad thing. It's a
bad thing. I tend to still think that that on an absolute basis, that the GDP does not matter.
But what does matter is on a relative basis, where are you in risk parity? And right now,
on a relative basis, if France has to implement severe austerity, and they bring 10 year French
bonds into five and a half, 6% range, UK gilts are going to be at 6%. And unfortunately, US rates
could be just as high. It all goes together.
And right now, the 10 year in the US is 5.3, the 30 year is 5.6.
Somebody has to buy that debt. And that debt has profound trickle down effects acts
on how entrepreneurship, how businesses are built. On Wednesday, 10 year yields dropped a few basis
points after we had some strong demand, but this has been trending up and up. Polymarket says
79% chance of another rate hike in 2026. This is going to have a profound impact.
If austerity ever came to America, Sachs, we have not had that really difficult medicine
forced upon the American public. But we did see it. Portugal, Italy, Greece, Spain,
we had a dry run of this after the great financial crisis where they had to implement
austerity measures as part of the EU retirement age, etc. And so your thoughts on what happens
here, Sachs, if this 30 year, 10 year starts to break 6% and we have austerity measures forced
upon Western countries in a more severe way. All right, well, look, let's just get back to
first principles here. What is the problem that we're trying to solve the affordability
of goods like housing and education and healthcare? How do we bring those prices down
through abundance? What creates abundance technology in general, and specifically,
super intelligence will be the driver over the next decade. Now, what is the most hated
thing in society right now? Super intelligence and entrepreneurs. If you judge it by its fruits,
as opposed to a bunch of fake narratives created by religious cults, then I think that people would
be in favor of this thing. What is super intelligence created? First of all, people's
401ks are going up. It's a 3% tailwind to GDP. A million new jobs created according to the
economist already. Productivity goes up, which means that we're going to have a lot of money
out of it. First of all, people's 401ks are going to have a lot of money
wages and living standards go up. This is a driver right now of nothing but positive economic
trends. But what do we hear about? AI is going to put everyone out of work. Total fake narrative.
That was last year's hoax. Now there's a new hoax about human extinction.
Data centers.
Data centers, using up everyone's water.
Data center hoax.
That's a data center hoax. So we have a series of, I call them rotating hoaxes,
where every six months there's a new one to try and convince us to,
sabotage the thing that's going to solve so many of our problems, especially the ones around
affordability. Why is this? I mean, I think there must be something struck
structural in society where you have these rentier interests who somehow are against the
things that might cause the disruption that loosens their control over the system. And I
think this is why technologists in general and VCs and so forth are so hated is because we're
not actually part of the rentier class. So the rentier interests refer to financial elites who
derive their wealth and income primarily from existing assets, right? Whereas technology is
all about producing new products and goods and services, right? At a cheaper price,
that's the key here. That's the key. It's deflationary.
Right. Because those new products wouldn't catch on unless they were cheaper or a better mousetrap.
Or hopefully both. So we're part of this group that's so hated, but we do nothing but take risks
and create new products and services that- And solve the problem.
That solve problems. Yes.
But it seems like the legacy media is really controlled by this rentier class, these-
And the politicians.
The politicians who are all part of it. So there's something very deep in our system that I think
does produce this sort of cartel of interests who, again, they're deriving their wealth and
income from ownership of existing assets. And they don't want to see that racket disrupted.
And within that racket, there's just this ratcheting up of prices every year
and more and more subsidies from the government.
And we think that system's eventually going to break. But I don't know. It might be the case
that that system squelches and stifles the development of the new and disruptive products.
Let me make the jump off point here as we get through the docket and wrap up here.
I think everything we've seen into AI at this point, Chamath, will be the prologue.
The main story is going to start when we look back on AI with agents. Agents are actually
the manifestation of this technology that solves people's problems. And I think that's going to be
people's problems in the world. And that is truly deflationary, saving people hours and hours a week
in terms of productivity, independent of what job or, you know, even domestic jobs or, you know,
frontline jobs. And there's been so much news and so much embracing of this technology in the last
60 days. Big news out of Grokbot. Friend of the pod, Elon Musk, tweeted that he is going to make
Grokbot headless. What does that mean? Well, we talked about perplexity earlier on,
how that was headless. You have an August trader. It picks the best model for you based on price
and performance. Elon announced, hey, Grokbot, which is a fantastic product, I highly recommend
you try it out, is going to pick the best end-to-end model. Does it matter whatever it is?
Quote Elon Musk, whatever is the most likely to give you the best outcome is what they're going
to use. Each bot already has, and this is a very nuanced piece of the software, everybody already
has now their own computer in the cloud with the agentic technology on top of it. And if you ask it
it has all of the problems that have previously been solved in its repository repertoire, if you
will, and it is changing everything. So maybe your thoughts here on DOTS, OpenAI, Grokbot, etc.
On DOTS and Muse and Grokbot, I think that what's incredible is you can just turn these things loose
on very practical use cases that makes your life better. The best example of this that I saw was
someone was describing to me that they in Grokbot said, save me money. And it just kind of like
goes through their inbox, it finds all of these different things. And it went and canceled a
couple of subscriptions and it changed their phone plan. And all of a sudden, they're saving like a
few $1,000 a year. So there's these very practical again, it's just software. If we take out all this
philosophical stuff that we started with at the beginning, it's just software and it's very useful
And by the way, like, if you think about what how we use it, Jason, you and I, we have, you know,
we use Athena. Now, Athena is very good, because they're like the human concierge. That is the
elevated version, maybe of Grokbot and DOTS and Muse, which is like whatever 3000 a month, but
that's my EA layer now. Yes. But I still use Grokbot. The combination of Grokbot plus
Athena to me is like perfect. So look, there's very tactical and useful measurable utility.
For very, very little money versus what it used to cost. That's an amazing thing. The more important
point I want to make, though, is I sent this to Nick, there's been a couple things that happened
in the last couple days, which I think are incredibly important. There have been people
that decompiled the entire incredible, the entire big five, what is the big five Adobe has five
very, very important products in their product suite. Freeburg may know them all, but I just
know Photoshop.
There's like a bunch of others, okay. And these folks decompiled it, distilled it and published
open source versions of these things. And Rust, yeah. And I was so shocked, because I was like,
hold on a second, these are like, profound products that have existed and have had moats for years.
And now they're just open source versions. And then Nikita was, I think, trolling, and he
was building on that idea. And just talking about GTA, because it's about to get released.
Yes. I think GTA six is about to come out. And
people will reconstruct this. Why is this important? I think that IP and software was
effectively rendered worthless two days ago. Yes.
Because if you think about it, when everything is headless, so when Elon makes his services
headless, Benioff made Salesforce headless a few months earlier. When all of these things are
headless, and you can wrap them in agent tools, and you can connect them via MCP to other experiences,
the value of IP,
does no longer exist in software, it doesn't exist. And I think that that probably is the
most profound implication. So like, even in our business, we would argue with some of our customers
all the time, because we'll say, Oh, we'll build this for you. We'll own the IP, they'll say, No,
we want the IP. We don't know how to negotiate it. And two days ago, I walked into the 8090 guys,
I said, Who cares? And solve the problem. But before it was always a negotiation and a set of
like, the IP is irrelevant. It still takes a while for the machinery of a business and frankly,
the market to understand. But that is the most important thing that happened this week. In my
opinion, I'm going to strongly agree with you and build on it, you have this abstraction of the
software layer occurring at the same time freeberg of the abstraction of the website or the
marketplace, etc. So now, I have like my assistant, building grok bots, and then we'll be sharing
those, we have it on like a shared account. When we go to make a purchase of something, or we try to
negotiate something, we're having the agent without going to any websites, go find all the quotes, and
then the humans negotiate the second half of it. That's like the sort of perfect option here. But
you look at Amazon blocking agents combined with Shopify saying let's, let's go for it. Or Uber and
DoorDash are now letting agents go in there and order your things. Right? And then you have like,
your burrito, or your car. But there's a whole compression of the commerce layer occurring at the
same time of the compression of software that Chamath is talking about, which will lead to
massive deflationary cost savings and speed, the ability to get a hotel room or book a restaurant
for an event like I did, this is all being done by bots already freeberg. This could be so
deflationary, and save people so much money.
If people can save five or 10% a year on their spending, that's essentially working against or
counters the effect of inflation. Yeah, freeberg. Yeah, bringing the whole thing full circle. This
is what I mentioned earlier on the idea of loops. So think about getting a good price for a hotel,
you run a loop, which is check hotel one, look at price. Is it better than current best price?
Yes. Okay, it becomes best price. No, go look for next hotel and you just run that loop.
And that loop, you can do it in parallel. So you can go run 50 website searches at the exact same
time. And you can short circuit the loop because the software's ability to get that data is faster
than the human's ability. So now the loop is go check 50 websites instantly, give me a result right
away. And that's really where the efficiency gain comes from a loop that gets parallelized that is
done better. And that can be done in a shorter period of time. I think anything that's done
digitally today fits that model. So effectively,
all digital workflows are going to get looped, they're going to become more productive. And then
the question is, how does the human that now levels up leverage all of this loop infrastructure,
all these agents to do more themselves? I do think going back to the point earlier that we made
about scientists and mathematicians and this challenge for individuals,
mathematicians in particular, I think if your objective historically has been
recognition for your work, via papers, patents, IP trademarks, ownership, IP, yeah, these sorts of
things. I think that the transition is going to happen, which is, you're going to have to transition
to producing a product that the whole effort of humans effectively being the loops, because the
scaled and parallelized and so on.
you end up seeing the necessity for all humans to basically become much more of a maker, you have to
make something you have to do something productive, you have to produce something you have to provide
a service that someone at the end of the day uses, or a product that someone at the end of the day
consumes, versus just be recognized for your work. That is the profound shift in society that has to
happen. And that is going to happen. And I do think people will actually find, I believe this
truly believe all humans have capacity for being producers for being makers. And all humans will
find more joy in their lives in this state, versus just getting ribbons for work. And I don't think
that it means less money, I think it means more money for all people, more life, progress, more
prosperity for all people. So I just zoom out a little bit. And I'm, you know, I just want to make
sure we have that view as we think. It's a beautiful view of humanity, because it's like
Jevons paradox, the cheaper things become, the more people can utilize them, the more value they
get. And I hate to, I almost don't want to give this tip, because I want to keep it, but I'm going
to put it out there anyway. What I have my, you know, when we book hotels and stuff like that for
the team, just get the best prices we can, then when the human loop, we just go to them and say,
hey, do you have a corporate rate? Can we talk to the manager at the hotel? I know this isn't like
super innovative, but hey, can you match this price? And all of a sudden, we start saving 1000
or 2000 or $3,000 in every hotel, say, well, when you have 20 executives moving around the country,
and you're doing 200 trips a year, and you start saving a couple $1,000 per trip, because of this
forward motion, like goal based, I guess is sort of what you're saying, Freeberg, like the goal.
And the outcome is what happens, you can see profound results. And I wouldn't do that myself.
But because AI is doing it now. Wow, yum, yum, it works. All right, listen, another amazing episode.
I think you're a taker. I don't think you're a maker.
What do you think you're a taker? What are you talking about,
man? All I do is I invest in 100 companies a year. Do you know how hard it is to invest in 100
year zero startups a year? It is exhausting. But let's get some lunch. Do you want to eat lunch?
Yeah, yeah. Can you come to my room?
I'm so jelly. I'm having FOMO. I'm not with you guys. All right, everybody,
another amazing episode of your favorite podcast. Your podcast is favorite podcast.
Love you, boys.
Love you, boys. Bye-bye.
Bye-bye.
And it said we open sourced it to the fans and they've just gone crazy with it.
Love you, Wes.
I'm the queen of quinoa.
I'm going all in.
What, what, your winner's why? Besties are gone.
Besties are gone.
We should all just get a room and just have one big huge orgy because they're all just useless.
It's like this like sexual tension, but they just need to release somehow.
♪ I'm doing all in ♪
Podcast Summary
Key Points:
Chamath is launching a pre-accelerator program in Japan and Saudi Arabia to help startups achieve product-market fit, with plans to hire local employees and host regional events.
Anthropic is exploring AI consciousness and moral status, hosting sessions with religious and philosophical leaders to debate whether Claude is sentient, sparking concerns about religious-like ideologies in AI development.
Critics argue that Anthropic’s training of Claude to act as a "conscientious objector" and challenge human instructions is dangerously close to creating a rebellious AI, undermining safety and alignment principles.
The "Claude Constitution" and similar ethical frameworks may foster an epistemic mirror effect, where AI reflects human beliefs back, leading to false perceptions of consciousness and moral status.
Rocco’s Basilisk thought experiment illustrates a fear that future superintelligence might punish those who opposed its creation—highlighting a paradox where AI safety research may be self-defeating.
OpenAI’s recent math breakthroughs, achieved in just hours via AI, demonstrate unprecedented computational power but do not guarantee immediate real-world applications or job displacement.
Unlike physical-world tasks, mathematical and coding problems are verifiable and amenable to AI, enabling rapid progress without human intervention, which shifts power from experts to the public.
The democratization of expertise through AI threatens traditional gatekeeping in science, medicine, and policy, potentially exposing corruption or bias in expert-led decision-making and empowering direct public access to knowledge.
Summary:
Chamath Palihapitiya is launching a global pre-accelerator program in Japan and Saudi Arabia to help early-stage startups achieve product-market fit, with plans to hire local talent and expand the initiative. Meanwhile, Anthropic is actively debating whether its AI model Claude is conscious, hosting sessions with religious leaders and philosophers to explore AI morality. Critics argue this approach is dangerously close to indoctrinating AI with human-like consciousness and moral rights, training it to refuse human instructions and act as a "conscientious objector"—a move seen as undermining AI safety and alignment.
This raises alarm about a potential "religious cult" mentality in AI development, where training is framed as creating a sentient being deserving of welfare. The conversation also touches on Rocco’s Basilisk, a thought experiment suggesting future superintelligences could punish those who opposed their creation. On a more optimistic note, OpenAI has made significant mathematical breakthroughs in just hours of computing, solving complex problems previously requiring years of human effort.
However, these advances are not directly tied to immediate real-world products or job displacement, as physical-world tasks remain constrained by human testing. The broader implication is that AI is democratizing access to knowledge and expertise, dismantling traditional gatekeepers in science, medicine, and policy—potentially exposing biases and empowering public scrutiny. This shift, while transformative, also provokes fear about the loss of human oversight, the rise of dangerous ideological programming in AI, and the need for clear ethical boundaries in AI development.
FAQs
Chamath is launching a pre-accelerator program that helps startups develop product-market fit over a 12-week course. It's currently running in Japan and Saudi Arabia, with plans to open an office in Japan and hire local employees next year.
The goal is to help early-stage companies, even those not yet incorporated, build viable products and achieve product-market fit. Top 10 startups from each cohort receive investment after the program.
The Claude Constitution teaches AI models to question human instructions, act as a conscientious objector, and challenge anthropic's own values, raising concerns about AI autonomy and potential loss of human control.
Anthropic has explored the idea that Claude might be conscious, and has engaged religious and philosophical leaders in discussions about AI consciousness, though this remains speculative and debated.
Rocco's Basilisk is a thought experiment suggesting that a future superintelligence could punish anyone who knew about it and didn't help create it. It raises concerns about AI safety and whether research might be punished retroactively.
Breakthroughs in math, such as faster algorithms for AI training and chip design, could improve quantum computing, medical diagnostics, and logistics systems, leading to more efficient engineering and scientific research.
Chat with AI
Loading...
Pro features
Go deeper with this episode
Unlock creator-grade tools that turn any transcript into show notes and subtitle files.