Welcome to the Sound of Economics, the podcast series by Google, the Brussels-based economic
campaign.
I am Yu-Yun-Jian, host of our monthly podcast, Zhong Han-Lindus, that brings you recent updates
about China's economy.
As usual, I am with our Senior Fellow, Alicia Garcia Herrerao.
Hi, Alicia, how are you?
Hi, Yu-Yun-Jian.
We're also glad to have with us today, Kristi Loki, a maths research fellow focusing on China's
AI innovation and governance strategies, and their implications for global AI governance.
Nice to meet you, Kristi, and welcome to the Sound of Economics.
Thank you so much.
Great to be here.
Thank you for joining us.
Artificial intelligence is rapidly becoming a strategic issue, not only for technology companies,
but also for governments and the global economy.
China has been moving quickly to develop its AI capabilities, while also building its
own approach to AI regulation and governance.
So what does China's approach look like?
How does it differ from the US and Europe?
And what could it mean for the future of global AI governance?
Today, we will explore all these questions and what they mean for China and the global economy.
Kristi, when we talk about China's AI governance, what does that actually mean in practice?
And how does China position itself in the current AI race?
So at the most basic level, AI governance is really about risk management.
It's just that with regards to AI, it becomes much trickier.
Because with deep learning in the form of AI that we see nowadays, it's usually really
hard to predict what it's going to look like when it's implemented or deployed at scale,
and sometimes it's even hard to predict what the kind of development process will look
like in terms of risks.
And so that's tricky.
And in this sense, AI governance is very much a global challenge.
And then in terms of a thing of note between, let's say, USV China in terms of perspective
on AI governance, I think rhetorically and philosophically is also quite different.
So in the US, we often talk about AI governance as something that has slow innovation down,
whereas in China, since really the chat GPC moment, somehow the government has been consistently
emphasizing how governance and development are actually mutually reinforcing.
And so one way to think about this is that by essentially introducing topical and timely
governance, you can move fast without breaking things, which is very much the Chinese ethos
when it comes to AI.
And then in terms of China's positioning in the AI race, I think it is very much one of
economic and productivity transformation being the heart of the equation.
And because of this, it's very much a marathon and not a sprint towards a theoretical, short-term
AGI finish line.
Alicia, you recently in an op-ed describe AI as a comprehensive strategy for China.
What makes China's approach to AI different from simply, say, having an industrial policy
for a new technology?
So, you know, Chris is right that the economics behind AI are important for China and rightly
so, because China's structural deceleration is all about AI to mitigate that structural
deceleration.
I think that is not taken as a silo, but rather as a comprehensive strategy.
So it's also a social strategy, it's also a political strategy, it's also a foreign policy
strategy, you think about recent developments, 29th of July, I've got correctly, in Shanghai
there's basically new global governance in AI, beyond China's own governance.
So, you know, the whole idea of using AI for every purpose, in a way, to upgrade China
if you want.
I think that is something that if first, quite substantially from what we are doing in Europe,
where, you know, our view of AI is much narrower.
So yes, we won't compete in this, but we don't think of AI for foreign policy or AI
for political purposes.
On the contrary, we worry about that, and that's why we kind of regulate AI in a way to
avoid exactly what AI can tell in a way, is pursuing, while regulating other matters that
are very different from that overall objective, and we can go back to that later.
Yeah, you can really see China is plugging AI into every aspect of the society and in its
policymaking.
So, what are China's AI organizing priorities?
You talked about four fronts, could you explain an unpack of it for us?
Yeah, I also want to really strongly agree with what Elisio was saying about China's
seeing AI as not just, you know, in a one-dimensional way, but very much in a multi-dimensional way.
And so, if we put our focus on the domestic side of things, the way that the state organizes
priorities or deprioritize certain things around AI really is organized by four priorities,
or four set of organizing concepts, is how they call it.
And so, that framework, or that organizing concept, is actually called four orientations,
and it's actually a term that's personally associated and introduced by sitting by himself.
I find this framework really useful because it helps us understand how China's been behaving
and what it will probably act like, how probably act going into the future as we go into
different technological junctures and all that.
So the first dimension here really is to do with almost competition, staying competitive,
but they also frame it in a more cosmopolitan sense, right?
So, get close enough to the technological frontier because this is something exciting for
tech practitioners, but it's also something that is, as Elisio was saying, important for
a composition and economic development at home.
And so, the framing of getting close enough to the frontier is really interesting because
it indicates that China's willing in the short medium term to not always chase the frontier,
but reserve some of the energy for other important things.
And then the second problem, or the second dimension, has to do with the economy again.
So China's fundamental problem at home is slowing growth, resulting from an economic playbook
that probably doesn't sustain itself in the long run, right?
Because manufacturing and various things has a lot of environmental damage, and the state
notes that.
And so, effectively, what they're trying to do with technology in AI in particular is to
boost their productivity.
And so, the way that they frame it is that if you have really good technology, you have
productivity gains with productivity gains, you get even better technology.
And so, it's a very positive loop that they're after.
And so, when we look at Chinese policies, pros 2022, we actually see a lot of emphasis
on AI diffusion, which has to do with exactly this problem.
And the third dimension has to do with technological sovereignty and security needs.
And this really has to do with the fact that this concept emerged within the years after
US containment of 2018.
And so, China has to also organize itself to build out some of the technological supply
chain or chip supply chain that is now missing.
Of course, that has to do with development, but also security in the long run.
And the fourth component, which is really interesting for someone who is in AI governance
nerd like me, it's also interesting because this is a new addition since 2018.
So, this concept is from 2020, and the fourth dimension isn't new addition since then.
And the reason for adding this, if I can kind of speculate here, is because between 2018
and 2020, there were actually a lot of AI governance debates across the world around
algorithmic bias, and then within China, there was this backlash against powerful AI-powered
delivery apps that were basically making impossible targets for drivers to meet and therefore
really threatening their lives and safety.
And so, that created a lot of discussions around AI governance, and then in the fourth dimension,
the government actually talks about how technology seizes all purpose to exist if it's actually
not making people's lives better, and that people's livelihood or people's well-being
and not protected.
So, this prong continued to be very relevant today as we talk about pacing AI and various
AI governance strategies.
Indeed, and also like you said, China has been quite early in regulating generative AI as
well.
Do you see a tension between controlling the risks of AI and giving companies enough room
to innovate?
I think they too themselves solve that tension.
I think the tension is there, but it's not impossible to say overcome in some sense.
And China's example is a really good way for us to think a little bit deeper about whether
the two are in complete tension.
So, the way that China went about AI governance posed Chatchy PT in late 2022 is very much
to almost forcefully and very definitively from the top level of the government, coming
out and saying, "You know what, we're pro-innovation.
We're pro-AI development, but as you do that, please also do risk prevention."
So, that was really interesting because at that point in time, which is, you know, let's
say early 2023 when they came out with this, China was still two to three years behind
the US AI frontier.
And so, the fact that they decided to focus on the two, given that, was interesting, what
was also particularly challenging, I think, in the Chinese case is that at that moment in
time, it came out of, of course, COVID, but also years of kind of complications, relations
between big tech and the Chinese government, right?
There is the famous tech crackdown, which some would say is because of political control.
I'm kind of in the camp that thinks that it has to do with alignment.
strategic incentives, which is about innovation instead of just social media
big tech that the state was trying to do when it comes to readjusting its
relationship with these tech companies. But anyways, so there's a lot of
potential chilling effect to of the Chinese government come down really strongly
and say, you know, we need to act very cautiously. We need you to do a lot of
content security constraints and all that. What they ultimately did, if I can
use another 4-prong way of summarizing what they did in the subsequent three
years, I would say they're the forming components to how they overcome this
innovation, the governance challenge. The first prong really is that they stay
very narrow when it comes to red lines. Instead of saying most AI is dangerous,
we need to do XYZ before you can do whatever. Effectively in 2023, they just
said, we need you to act cautiously and act in good faith. So be as safe as you
can procedurally. But at the same time, we're super innovation pills, right? So we
want these companies, you companies to move fast. But please make sure that you
observe certain content security rules. The way that they do this is that they
set up this AIGC interim measures and then the subsequent regulatory regimes so
that they can look at these public facing models, public facing apps before
their release. And so have that line of accountability for political control and
all those other reasons. What is interesting here is that it stays narrow, right,
in terms of red line drawing. And as a governance nerd again, I'm very interested
to see if they're going to expand this into frontier AI risk governance, which
has more to do with like pre-release a pre-deployment testing as an additional prong of
this kind of red line. And some folks in China that are very influential are
actually calling for that. The second prong to this kind of foreprong approach to
governance in China is staying extremely iterative. China just came out with
something called the AI Safety Governance Framework 3.0. So it's the third
iteration of this framework, which then sets some example and guidelines for
business to follow before the law hardens into hard law. And also for
national standards body to have a basis, like a consensus book for them to build
upon. And so that's really interesting because with something like AI, where
things are moving so quickly, kind of, or edit a disadvantage if you just come
up with a very definitive horizontal law. Kind of in the sense of the EU AI act
rated, it becomes very tricky to have it stay relevant and be implementable
over time. And then the third component is very surprising for a lot of China
watchers, if they also read AI governance stuff, is that China is very much
bottom up in some of its approaches to AI governance here. Because in many
writings that I've read, the government set verbatim that really the brain of
innovation rests within the private sector. A lot of the times the scholars and
the experts are not within the government. And so how they went about this is
that they said they want to do an AI law. They said they want to make sure
governance keep up with development. And it is really about consulting the
industry and getting these experts to draft their own law proposals, which they
then slowly incorporate into actual law. And then the fourth component is
extremely important in the current debate that they were in. Because the fourth
component is actually about China absorbing the lessons from the rest of the
world. So if you look at Xi Jinping's speech in the early in July at the
World of AI conference, the first few sentences was about Darmuth meeting in the
US, which kick started this deep learning pathway that we're on. And so it's
very much about acknowledging that China didn't invent AI, but China's here to
observe like what the frontier debates are and then taking those lessons and
adapting them but also adapting them at home. And so some example is that we
try to following this pacing argument. It's also following some of the it was
following some of the compute thresholds and capability linked thresholds when
it comes to governance in the Biden administration, for example, and considering
implementing it at home.
Alicia, you already mentioned this a bit earlier. Europe has taken a rather
different approach toward AI. We care more about risking. And then you know, we're
trying very hard to regulate it. So how does the European model compare with China's
and the US approach? And where do you see potential areas of convergence or
divergence? Well, I should start by saying that China does regulate AI differently
in areas where we probably avoid regulation and goes with that saying that
that relates to political freedom. I think that needs to be said. So, you know, as
much as bottom-up innovation is certainly foster in China is a clear top-down
regulation of content. Bandar to alignment with special ideology, culture,
training data, filters, freelance reviews, you know. So, you know, I do see when
it say this because it sounds like, you know, somehow that Europe is hyper
regulation and China is leaving, you know, a free flow AI within its borders. I
think something, however, important because it brings us to a clear difference in
my view is that China has a double objective when it comes to exporting AI. I mean,
China hasn't yet explored hardware or software layers simply because they
can't produce enough for their needs. But China does export obviously of the
wake models. And interestingly, you know, the amount of this top-down regulation,
we want to call it regulation for the lack of perhaps harsher words. It is
very different within China than half that time. That's totally different from
Europe, yeah. So, I think Europe's approach to regulation is totally on the
risk management front. Which China also has on big face and, you know, it's not
that China does have that. But I would say that the risk Europe is ready to take
in terms of actually training model. So, you know, advancing on AI innovation is
much smaller. So, I would say on the top-down and kind of ideology that goes
in, I think China is much harsher for all these reasons. And on the bottom of
innovation, pure innovation, training, and perhaps even the speed at which, you
know, the cyber security agency even approves those those models. It's
probably much more acting as Christy was saying to allow the right sector to
excel. So, you know, it's kind of a very mixed reality because people can't
understand within that political system how can China do what China is doing?
And I think it's because China's segments, the regulation related to
innovation as opposed to that related to ideology or political alignment. So,
it's a very divided reality. So, that's hard perhaps for us Europeans to
understand. But it is their reality of China's ecosystem and important for us to
understand that you can actually do both. And that's China's, I would say,
success because otherwise, it would have been very hard for China to export
these models. So, the sectors, I would say, is beyond what I just said, is the
fact that on the top-down, it isn't applied as strongly outside of China. That's
another thing that we kind of forget because otherwise, the sectors wouldn't
be there, trips are overweight mode. So, it's a very, very smart, you have me
way to approach comprehensive regulation. You regulate when you need, but you let
it be when you don't really need, when you want to itself. So, this is the trick.
I think let's see a touch on a very interesting point here, which is that China has
much tougher content security requirements, which it doesn't have when its
models and products go abroad. And in the case of open-weight models, for example,
like some people, because they're not very familiar with the technology, we'll
say, well, these are CCP kind of infuse flair and values within these models. You
can say that, but then again, for those who know how to use open models, you can
easily remove some of those kind of pre-baked biases or kind of positions and
still use the model and open the model, which is a part of some of the
discussions that pad with Lithia. And so, it is indeed quite smart. And of course,
I'll go back to the emphasis that China's keeping the red lines relatively thin so
that the innovation can still go on. And something that's pretty clear from
looking at the documents and some of the framings up close is that they also
emphasize a lot on tolerance, right? As long as you have shown good faith that
you're doing the right thing, we're going to tolerate your mistakes. But this
only extends to things that are not super sensitive and content security related.
Thank you very much for that. We are recording this episode at the Trump
She meeting in Washington. Christy, what do we know about the AI
dimension of the meeting and what coordination or lack there are and competition
are we seeing in this area? Yeah, so I think what we saw, I think it's still ongoing.
in terms of the summit in between sea and Trump in Washington, is very much an extension
of the respect or goodwill building that started last October. So I think a lot of analysts rightly
diagnosed that last October at the Busan summit, it was very much a watershed moment in some sense
because after years of trying to contain China technologically, let's say, by the US under Trump
and under Biden, the US side is realizing that it's not really yielding the same effect as it
was before in terms of the strategy working well. So for example, China used the soybean card
and then the railroad card and that subsequently resulted in some relaxation on the chip containment
or chip control side and it almost formed a bit of a tenuous ceiling in terms of US China attention
relationship technology but tenuous because there's a lot of competitive elements to this
relationship of course. What I thought was interesting in sea speech yesterday is that he basically
gave a gentle subtle but as obvious of a nod, you would expect a Chinese leader to give to the
pacing of the frontier argument that is happening in the US right now, heavily debated. Trump set
no and then backtrack and said, "Guard Rails is still necessary," you know, things like that. And so
the nod that he gave essentially is how he framed the US and China's responsibility and what can
be achieved. So just to quote what he said, I believe it was something along the line of US and China
as the major AI powers can and have the responsibility to manage AI and make sure that it goes well
and also make sure that the risk are matched. So I would call this very much a can-do attitude
to AI governance instead of saying, "Oh, these existential risks are going to overpower us. China's
response is very much the US and China are going to work together to make sure that doesn't happen."
Alicia, what's your key takeaway from the summit when it comes to the AI component?
I think Xi Jinping's speech on back to Chris this point is in a way a double-led sword in my view.
So on the one hand, because of China's nature of AI governance, which is very much, you know,
human centric. In a way, by nature, it aligns with that kind of slowed down of AI in terms of
the contextual reason why, you know, at his own paper, US hyperscalers want to do this.
For me, the reason might be actually quite different, but we could talk about that later.
But in essence, and yeah, I need to explain the reason. Otherwise, I wouldn't be able to talk
about the other side of this world is that I don't think China buys into this nice humanity of
US hyperscalers at this point in time. From me, they realize that they have so many things
against them now, whether it's, you know, the political will or pushed by the social
Afro or about, you know, electricity or other constraints related to
data centers and probably the simple fear of losing your job, you know, because of all
of these things happening in the US, probably US hyperscalers are thinking, okay, China probably
doesn't have these problems for a number of reasons. Your electricity is not as limited.
They can control much better, you know, the social sentiment about AI blah blah blah.
So probably that is why they're asking for this to delay the competition. And I think
China is not binding to this. So while keeping the rhetoric that we want to human AI,
China is however pushing and saying, sorry, if because of your fears, I go first, you take it.
You see, that's the nature of competition. So it's a very, very
again, double meaning for me in Xi Jinping's speech. Yes, I hear you. We told you so, you know,
like AI has to be human. However, I'm running my way and you're not going to stop me. So,
which is again, very smart. I mean, nobody's saying this is not smart, but, but when it comes to
the consequences for us Europeans or others just saying, again, this is a N minus 2 problem.
As I like to call it is it's people there at the UN watching the show of these two Ketsun and
basically deciding on the future of AI, both not looking at the rest, the N, not even the US
hyperscalers with their own worries, which might not be as human as they claim there. But overall,
I think the race is there. That's the end game of these long discussions, sorry for that, but it's
because yes, sure, but yes, sure, but and in the case of Trump, it's not even yes, sure, it's
but yeah, it's like we go for it. So for me, if anybody is worried about the speed at which we're
deploying this technology, you have no respite from this summit. I mean, there we go. There's not
going to be at least as far as the US administration is concerned. Of course, the US has the problem
of its own population, but barring that nothing's going to come from the US administration,
nor is it come from China. That is my thing on this summit and it is pretty worrisome for the rest
of the world. So I leave it there. I have a somewhat short response. So I see it a little bit
differently. The way that I see the Trump presidency and the incentives or constraints that
Xi Jinping faces at home, because as we were talking about China needs AI and other fast-moving
technologies to go well and diffuse it so they can solve its economic problems and move along this
kind of like economic industrial upgrading rate. So that's clearly something of interest.
And then from President Trump's side, he also needs AI to go well because at this point in time,
the AI industry is just taking up most of the US stock market and there's strong incentives
for a high-profile incident or the next high-profile incident to not happen. And so with this in mind,
they are getting various warning shots on both sides. So for example, the hugging face,
cyber-incident was a problem and we recently learned that the Australian government is also hacked
because of this. And from China's side, they do not want chaos. They do not want this fast-moving
technology to kind of ruin their orderliness. And so they have very clear common sticks
in preventing the most frontier capabilities, especially when it's critical infrastructure
elephant, like cyber-relevant, nuclear-relevant, biologically-relevant, they do not want it
to be misused by malicious actors. At a certain level, China doesn't want, you know,
Amnitho's model to be targeting its own critical infrastructure. And so there are a lot of dynamics
that play some of them with Syrosum, a lot of them are common in terms of shared concerns,
which is why Besson also focused on the shared aspect to this relationship. And so because of that,
it gives me some level of moderated optimism that they will keep talking to each other and working
on this and they have agreed to keep talking to each other, which is great. Something of note is
the incident sharing or incident communication mechanism that they're interested in because I was
looking at so much of the domestic governance systems of the two countries. And what is politically
feasible, technologically feasible for them to work on in terms of coordination, I feel like that's
probably one of the first thing that they should be working on. And I was arguing that before the
summit as well. And keeping our costume dialogue sharing some best practices, moving up a coordination
ladder is probably the best we can hope for. So I think they're moving in their right direction,
but maybe not as fast as some people hope. Yeah, the thing I like the most about our podcast is
that I am always the most negative. So, you know, I leave like relief by listening to people and,
you know, kind of calming my fears about it in every single episode. So, you know, I hear you
and I wish I could agree. And in a way, I do because, you know, it would be too doomed to be
true. You see what I mean, but there is one thing that I want to say is that as opposed to,
because this is the red phone of the nuclear race, you know, this is what we're talking about. I mean,
if something big happens, hold me up. I mean, that's that's the type of cooperation we're discussing.
It's not cooperation in, you know, joint models or, you know, we're really talking about
call me a private series. And it's already very good. But what worries me about this is that
the use of AI is different from nuclear. In other words, you know, what nuclear was really a
military threat, a security threat. AI has so many positive deployments that if you fall behind
a lot of such things, you know, it's like the way if you want or calling the high is larger. And
therefore, what you will give up by moderating your development might be less. For me, this is the
ultimate why I'm very worried about this because I can sense that the race is just too important.
And then this is Trump himself, you know, making this so how they can really moderate that
race is very uncertain beyond, you know, major events. But major events might still happen because
you don't realize they're coming. This is like because you're not causing, you're not even
slowing down the development. You're only there to communicate in case something happens. It
might not be enough to avoid additional risk. So for me, this is too much gain compared to
nuclear development. Also because nuclear, indeed, the safety may happen now. The society just
didn't take it. This time around, the society in the Western world may not take it. And then you
would have this very unbalanced development. And in which case, probably the degree of interaction
will be very, very small because you have this very different levels of speed. So, you know,
that's why I'm a bit worried because again, this is fundamentally more transformational than
even nuclear energy. That's for me the key point is to worry a little bit. But and they have
about your positeness. And I'm going to sleep well tonight. So that's it. That's why we need to,
you know, invite different guests to the podcast at least to give us, you know, some positivity.
So let's go back to what we discussed earlier, picking up what Christy has mentioned about AI
safety. Does China care about AI safety? Yeah, I think that's that's a really good question. And
a lot of people, a lot of the discourses in the West, so you see nowadays, started asking this
because there's a really high profile, actually a multiple exits or resonations from the top AI
laps in the US. So one particular one is Jacob Coxon who left Anthropic and warned that there
might be soon existential risk for humanity. I think 10% was the number. And so that prompted a
lot of people to ask, is this slow down proposed by all these AI leaders in the US possible? And if so,
even if it's possible in the US, can we coordinate with China and get it done? And so a lot of people
started asking, does China care about AI safety? I would say that a lot of the, where the debate
is at is still a little bit unhelpful. A lot of like top-level officials in the US and various,
like, opets have been written that China doesn't care about AI safety. And the evidence-based for
them is that during the Biden administration, there was this very short lift US-China AI safety
dialogue in 24. And it didn't go well. It was the only dialogue that took place. Nothing happened
because China was asking for relaxation of export controls. US was asking for maybe more frontier
ex-risk safety related stuff from China. The way that I would put it at this point in time is that
2026 September is very different from May 2024. And as we all know, in late 2024 and early 2025, deep sea
happened. And if we actually track the AI governance discourses within China after, they became
even more governance-pilled. So the first study session on AI, which is led by seeing pain,
and also a way for the top leaders to talk amounts to the elites, you know, the top of the top,
you know, the polyburel members, to figure out what to prioritize around AI. And there, he again
placed the emphasis on development and governance. So very much their recognition after deep sea,
to the surprise of many people in China that I've spoken to is, whoa, we're able to develop a
frontier model despite compute shortages. So that means that we have a way forward to be at the
frontier or near frontier. And because of that, we saw a lot of, like, enthusiasm and confidence that
I grew in the open model space and the rest is history, right? And so essentially at this point
in time, China is firmly within tier one or tier one point five of frontier model building.
Because of that, the government is deeply concerned about the security ramifications of these models.
And as a result, they deeply care about AI safety. It's just that they don't buy into existential
risk the way that some individuals within these AI companies do. But that doesn't stop them
from considering this as a serious risk. And in fact, they're actively thinking of ways to
extend the current regulatory regime to account for these very quirky, semi-unpredictable risks
into the future. Let's say through pre-deployment testing and testing for certain risks like
cyber capabilities. So yes, they do care. And I would also kind of implore these AI CEOs in the US
to understand that the Chinese state is actually reading a lot of what they're writing.
And so when the framing is too much about AI safety is actually the same thing as containing China,
the China is going to, you know, turn sour at the concept of AI safety very soon. So it's important
to talk about AI safety in as scientific and as neutral of a sense as possible so that we can
protect those and and keep those research going and keep the Chinese engagement going as well.
Alicia, do you want to respond? Yeah, I would say that I agree with and this was basically
basically what it in much tights or words or not. So okay, they're something I'm not professional
that mine, but it was exactly the same point. But I think by using AI safety as a throw to China
to slow down, China could actually ironically believe less in AI safety, which is not what we
want China to do. So in a way, it's irresponsible. I quite agree with that. But I want to add something
it's like again, this double edge sword when it comes to safety is that AI is also a tool
for military purpose in other words for the all consider of safety. In other words, I mean, we can
argue that China, but this is the same for the US wants AI safety domestically, but what if push
comes to show and again, that doesn't mean that China needs to start this conversation. China
will want to develop AI in the same way as the US does for cyber not security for cyber use.
And in that regard, going back to why I think the cooperation or is very difficult is that it's
not only about safety is about action and both sides somehow avoid the discussion on this
double use on understanding that the safety of one could be the again, when push comes to show
in certain circumstances, the success of the other or the unsafety of one would be. So that is why
this conversation gets tougher once we move beyond commercial use. And I think that that is
again, why I think there is a very limited scope for cooperation because of the other side of
the story. Yeah, I mean, hard when needed rather than safety and as a kind of domestic objective.
So the domestic versus the cross-border objective when push comes to country for matrix,
I think that may and again is that the other side of the same coin we're not talking even about
different technologies. So yeah, that makes the again, the cooperation very, very difficult in my view.
So again, Alicia, the most negative person in the podcast?
Always. Because my ending question would be the US-China AI coordination and then you just answer
that question by saying there's very little possibility in there. But okay, let's hear from
Christie. So maybe she will give us, you know, like something more positive.
I think that's room for cautious optimism as long as we are a little bit more,
we say pragmatic, right? So I work with a lot of people who like feel very strongly about AI safety.
And so some folks will go all in and say, we need a grand bargain with China today or yesterday,
right? And then there are other folks who will say, well, you know, people who are closest
to the policy side will say, why do we have the US and China both work on hardening their safety
at home, safety and parallel, if you will? And they can still be engagement, but, you know, we can
move cautiously and more domestically. I'm firmly in the camp, the middle cap, which is to say that,
I think there are certain events that will push the two sides towards each other very quickly.
The fact that suddenly almost out of nowhere for like someone who doesn't look at AI safety or
governance, the US and China are talking about an instant hotline or an incident communication
mechanism. What prompted that? Well, events from April all the way to, let's say now or July,
prompted that because Mika's happened and it showed that frontier models with cyber capabilities
can be a real problem. And another component to this is the hugging phase incident, right? And
that prompted a lot of discussions around operational security of these models, not just model
capability. And so I think the two sides are pushed with each other to talk and to have
means of clarifying, did you mean to harm our critical infrastructure in that way or was it just
a bad AI accident, right? So having that hotline is actually really, really crucial. One reason
that I believe that's crucial and some other people do is recently there is this high profile
ship in the Middle East that the US site with the help of Claude or other AI systems mispredict,
misstagnosed to contain nuclear. And
That could have created a really big, negative, horrible incident, but somehow they managed to clarify or somehow they managed to know that it's not actually indeed China's.
Or that simply didn't exist.
And so having that clarification mechanism is actually extremely important going into stepping into the kind of AI era that we're already in, unfortunately.
And then the other point of kind of how do we look at AI coordination?
I think a more helpful way to look at it beyond the two extremes of immediate coordination versus no coordination is to think about coordination like a ladder, right?
So the more the more good will experience of working with each other and understanding of the other system and concerns you have, the easier it is to step up this ladder.
So the first step can be something as simple as dialogue. So when I proposed this, the dialogue didn't was in there. So I was proposing this earlier this year and really glad that the dialogue is in fact in place and continuing going forward.
So dialogue is really important for them to get a more similar degree of understanding of what is your perspective on how the risks are going to look like in the next few months?
What should be really concerned about and what should be prioritizing working on? You know, some of those concerns might be serious. Some a lot of those are not.
And then as we step up this ladder, the next step can be, is it possible for some of the leading lats people at deep seek and traffic, whatever, maybe an ontropic.
It's a little bit difficult, but it's possible to get the brightest safety talent within these companies to talk to their own governments or talk to each other about potential best practices for AI safety at the frontier, right?
Because these are the two countries that have a share stake in figuring this out soon.
In terms of containing and aligning AI, which are two important risks that we focus a lot on in my work.
To step up this ladder, we can think a little bit more about things that might impose asymmetric, hopefully by then it would impose asymmetric cost on either side when they go into an agreement.
For example, is it okay to talk about setting common standards around open models, but also closed models.
And then as we go a little bit further up, we can then talk about pacing. We can then talk about can we pause AI? Can we stop AI?
But we need to move up that ladder and to jump from one to the other, unless an event has triggered that is just unequal.
And so that's how it's see it. And because of that, I think we're actually moving slowly, but shortly in the right direction.
Would you agree that we are at the very bottom of the ladder meaning that we don't even have a technical dialogue. Would you agree with that? I mean that what we have so far is still very political and very, you know.
Or do you think that that technical dialogue is is is is the dialogue that.
My reading of the summit is, I mean, I hear you you say there will be dialogue actually for me is just like again an incident communication mechanism.
I may have not fully understood what what is going to be proposed, but for me, the dialogue remains, you know, a pre-summit dialogue at the political level.
And if something really serious happens, pick up the phone, but that is if that's what we are building that ladder will take very low time.
I like the ladder is lovely, but the point is we're really, you know, like, like literally in the basement of, you know, not.
So how do you see people actually moving out that ladder because yeah, yeah, it's like that dialogue said pre-summit dialogue no more. I mean, are you thinking there will be more?
I don't know if this is sufficiently helpful, but one way to look at this is think about Trump in March.
So the the US government put out a document called nationally, I frameworks there's no mention of safety, no mention of like pre-deployment testing, which is really something that companies do themselves, but have no regulations impose on them.
And then within April, the conversation entirely shifted after mythos happened globally, and within the Trump administration by June, early June, there's an executive order asking for us actually essentially something more of a mandatory kind of oversight regime that the US government is kind of seeking.
And still finalizing, but the power of the state is effectively starting to be executed by the Trump administration because they are indeed very concerned.
There's also a lot of cybersecurity policy documents coming out of the administration after that.
So I think the way to characterize it is they may not want governance, but governance just finds them every step of the way and increasingly so.
And so things move very quickly in AI and potentially in AI governance as well.
And then in terms of whether there's a technical working group to my knowledge, there's no formalized technical working group between us and trying to on front to AI safety.
But then again, because Scott Bassett and the Treasury from the US has been very involved in hands-on in terms of talking to the frontier AI labs in the US about their concerns about what the government expects, I think he's pretty well informed on whether the most dire risks front.
And from the Chinese side, not even a question, I'm pretty sure Holy Fong is very well briefed because I've seen the density of Chinese AI governance debates, discourses and proposals just coming from all different academic research and various channels.
And so it's almost like a technical working group has been formed in that sense, but of course it's not sufficient.
Hopefully we'll see more development there.
Thank you very much for this conversation. Alisa, do you want to talk about Europe's role in here?
I just wrote and this will probably be the news editor for us, which ends on Europe and the idea of how much Europe has tried to build governance or regulation and governance beyond its borders.
And how little it has achieved because we are in that in minus two world, basically, in essence.
It's very, very hard for any player outside of that too, that quarreling couple, I would call them, but still a couple to listen to somebody else around the corner. And I think that is Europe's main problem.
They may have a lot of the things that anthropologists announce as part of the plan to slow down AI, it are already a reality in Europe.
But the point is nobody's day, or that is being borrowed from European regulation.
So for Europe, it's kind of, I told you so, but nobody's listening to it, I told you so, so it doesn't really matter.
So I think that should prompt Europe to realize that Europe cannot shape the direction of AI, even on regulation in these circumstances.
So it should basically stop doing so and do something more useful because it's just not feasible.
And I feel this is kind of the moment of truth for Europe where they will realize that no matter, you know, the meetings at the UN or whatever, the action was elsewhere.
And nobody even realized that they could have been, that the discussions would have been useful for that solid.
Not even that. So it's a moment of truth if you ask me for Europe in that regard.
I look at this again, I love it differently. Yeah, so again, maybe just like a little bit of optimism injection again.
So I actually think that at this current moment in time, in mixed sense of the US and China of focusing on each other first, because it is where the next risks are going to come from, and how soon they're going to be, what form they're going to be, these are things that even they themselves are not clear about.
And the lapse of worries, the government's worried, and so they should probably figure out, you know, what to prioritize and really start managing things.
So, sitting thing was talking about this yesterday, he said, with the US and China working together, they cannot solve everything in the world.
But without them working together, nothing can be solved. And it's not that much of a hyperbole, I think.
And so it's good that the two are at least engaging with each other and it's a good first step.
In terms of what Europe can do, and in terms of the bargaining power when it comes to making rules or contributing to governance, we use each other earlier this month.
And I think, Alicia and I, and I think something that I think is really important is for Europe to figure out its own edge, right?
If the leading powers of focusing on frontier AI building and frontier AI risk management, of which, which are bringing them plenty of headaches, maybe Europe can figure out how to do reform at home so that it can move faster.
Safely and faster in terms of applications and diffusing these technologies, because at the end of the day, if you don't diffuse the technology, not much good is coming out of it.
Besides electricity bill and ginormous expenses when it comes to frontier tail and hiring.
So, maybe, maybe there could be a focus there and using all the advantages that Europe has would be a great starting point.
Thank you so much, always for your optimism, even when you explain the ship incident, you know, for many people will be like, oh my god, this happened because of AI, you know, like it was false identified, there's a nuclear component.
before you it's like oh bad
managed to solve it in the end. So something good is coming out of it. So I really love that
explanation as well. So thank you very much both for this conversation and like every other podcast
we've been talking a lot about AI this year that this episode really focused on the governance
and the philosophy behind it. So it's something that will be different from our usual content.
I am Yuyun Jan and I'm with Alicia Garcia Herrero of Brugel and also Matt Stalo Kristi Loki.
For listeners you can find our research on
[email protected] and sign up for
Zhonghua Mindes, our monthly China podcast and newsletter with thank you in the show notes.
Until next time, bye bye.
you
PodcastTranscript AI