National Commission into the Regulation of AI in Healthcare: Recommendations. With Prof Alastair Denniston, Chair of the commission
from Digital Health Podcast- Royal Society of Medicine
47m 24s
The UK has established a comprehensive national AI commission to address the urgent need for a modern, adaptive regulatory framework in healthcare. As generative AI evolves rapidly and outpaces existing regulations, the commission’s 44 recommendations aim to strike a balance between innovation and safety. It proposes a shift toward proportionate, lifecycle-based regulation—allowing lighter-touch oversight for low-risk tools and stronger, continuous monitoring for high-risk applications. Central to the framework is a shared responsibility model across manufacturers, clinicians, and health systems, ensuring that safety is embedded from design to deployment. Patient transparency and choice are prioritized, with clear rights to know when AI is used and to opt out where possible. The commission also highlights the importance of system readiness—where clinical needs, technological capability, and systemic preparedness intersect—as the true source of value in AI healthcare. While direct-to-consumer devices face unique challenges, they are regulated as medical devices, offering potential for real-time feedback. Critical to implementation are investments in digital infrastructure, workforce skills, and post-market monitoring systems. These recommendations signal a strategic pivot from rigid, pre-market rules to a dynamic, responsive model. However, turning theory into practice will require coordinated efforts to resolve ownership, funding, accountability, and human oversight challenges—ensuring that AI integration in healthcare remains safe, equitable, and truly patient-focused.
your listening to the Royal Society of Medicine Digital Health Council podcast, where we explore
health-tech innovations that are transforming healthcare, with me your host Dr. Annabelle Painter.
We are at a unique moment in AI and healthcare.
Generative AI has arrived in health and it's evolving at extraordinary speed, but it's
being regulated by frameworks that were largely designed before we had this kind of technology
that's non-deterministic, there's general purpose and that's constantly being updated.
And for the UK there's another layer to this, AI arrived just as we were leaving the EU,
putting us in an interesting position.
As a relatively small but globally important regulatory player, what do we do?
Should we follow the EU, follow the US or try to forge our own regulatory path?
How do we make the UK an attractive market for AI and a home for innovators?
And how do we make the most of the NHS and its scale, its data, while still protecting
patients?
These questions led to the launch of an unprecedented national commission on the regulation of AI
and healthcare convened by the MHRA.
For the course of a year, the commission brought together industry, clinicians, regulators
and members of the public to advise government on what a future regulatory framework in AI
should look like.
I was fortunate to be part of that conversation, sitting on the technology working group,
and it's been fascinating to see how the different perspectives have come together.
On the 10th of September, the commission published its recommendations to government.
44 recommendations covering how we should approach regulation of AI and healthcare.
The commission was chaired by Professor Alistair Deniston, an ophthalmologist, academic and
professor of regulatory science, and I'm delighted to have him with me on the podcast today.
We're going to unpack what the commission found, what its recommendations mean, and perhaps
most importantly, what they could mean for the future of AI regulation and innovation in the UK.
Alistair, welcome to the podcast.
Hi Annabelle, thanks, it's great to be here.
It's such an honour to have you here Alistair, I'm really excited to talk to you about everything
to do with the commission and the recommendations about the regulation of AI and healthcare.
But before we jump into that, it would be great to hear a little bit more about you and
to understand how you came to be sitting in the position as chair of the National AI Commission.
So tell us about your background and what's brought you to this point.
Thanks Ava, so I'm an NHS doctor, I'm an ophthalmologist at Happens based at University
of Hospitals Birmingham, and I'm a professor leading a research group in the University
of Birmingham, focused on the evaluation regulation and implementation of AI health technologies.
Path to getting there was kind of always just interested in science and so on through
school, wants to be an engineer, sort of slightly accidentally explored medicine and really
enjoyed clinical medicine, but always continuing to sort of explore the research elements
and their kind of innovation and I guess always kind of interested in the technology and how
we use technology to improve the delivery of healthcare.
And then specialised in ophthalmology, did a PhD in ocular immunology, so super lap-based
kind of looking at cells and immune staining and so on.
But it was really during that that I discovered that actually what I really cared about was not
so much that kind of very discovery and lap-based stuff, but it was actually how do we get innovation
through to patients, how do we improve what we're doing today so that tomorrow be better.
So as a clinician thinking okay, these diagnostic tests we're using are better than we had ten
years ago, but couldn't we do so much better than this and similarly with treatments,
they're kind of okay but they still have lots of side effects and they don't work for everyone
and I'm sitting here in clinic and maybe I can rapidly diagnose 80% of the people coming
to us and maybe we've got good treatments for 70%, but what about the other 20% and what
about the other 30%. So there was just a sense of okay what we're doing is good but surely we
can do better than this. So I guess that's fed my desire to always push us to do better through
research and innovation. I think if you look at a theme through my research both when I was
working in the sort of vertical depth of ophthalmology and now as it worked more in the horizontal
of AI, I think a common theme has been around just how can we do this better? So it's almost like
you know at the meta level, how can we do just this whole thing better that kind of moving
from discovery of in this case a new technology and all the amazing things that are happening
in the kind of discovery of AI and the opportunity there but how can we flow that through to actually
being able to be have something that's useful and safe, inclusive, equitable in our day-to-day
clinical practice and that's that's not just a technology question that's not just a research
question you know that's a kind of people question as well isn't it so no it's a policy question
and a practice question. So now as Chair of the AI Commission how did that come about? How did you
get into that role? I guess what most people won't know is that's actually the second report
I've done to government on how to regulate AI. I sit on something called the Regulatory Horizons
Council which is not specifically a medical thing it's set up by the UK Government to advise them
on where there is an opportunity for regulatory reform to better support innovation and it's
really interesting role because partly because it's not purely medical so I sit with other experts
in this space and we look at things like nuclear fusion or regulation of space or looking at hydrogen
power and so on but we've also within that looked at regulation of medical devices and specifically
AI health technologies and one of the values of it is that you can learn so much from from other
sectors and actually many of the principles are very similar and of course something like AI cuts
across multiple sectors so as part of the Regulatory Horizons Council I led a report to governments
published in November 22 and actually if you look at that report now you still see that many of
the recommendations are highly relevant I was you know a very pleased I thought we had met the
challenge very well but something else happened within a week or two of us publishing which was
Chatchy PT went public so that was that for me just highlighted one of the challenges here is that
you can you can create a framework that is good for now but it's almost immediately good for
yesterday and this space is moving so fast and so I think if you did read it now you would go well
yes this feels like it belongs from a previous era because it really hardly touches on
the challenges of generative AI at all so I have been working in this space for a while
the team that I have the privilege to lead based out of Birmingham have been doing some really
good work and working with regulators to try and help both evaluate the existing regulatory system
but also think about how we can optimize the regulatory system so very much sitting in an
academic space rather than in a policy space but I guess those were the reasons probably that
the MHRA came to me and said we would we are looking on behalf of the Department of Health and
Social Care to establish this National Commission would you be prepared to lead it and about you
know me well enough to know there was definitely a moment of going oh my goodness am I the right person
for this and really took time to reflect whether actually leading something that was so important
you know you want that the right person to be there you don't want to just jump at it because
it sounds interesting but I did say yes and I think the value of having an academic potentially
leading this is that you're bringing a particular set of skills of impartiality kind of
independent scrutiny of the opportunity here the risks here the evidence the ability to convene
people together recognition that no one person or no one sector has all the wisdom we need we're all
like jigsaw pieces coming together and I guess that's what I saw my role within the commission is to
just ring together this wealth of expertise and knowledge the diverse viewpoints around
how this should be done both in expert groups in the wider public and practitioners on the front
line and health leaders in kind of the regulatory space across government international UK-based
foreign edge based just bring that all together and work systematically through to say okay where
are we now where do we need to get to and how do we get that and I think from your answer it's
It's very clear that you're happy to find out.
And certainly from my experience, I think everyone would agree that you have done a brilliant job in leaving the commission.
Why now?
Why was the use of mission convened at this particular moment?
Because it is quite an unprecedented commission.
So I would really like you to sort of give us that bigger picture view on what is it about this moment,
technologically, politically, strategically that has made this feel like it was needed right now in the UK.
Yeah, so I think it's really interesting.
And to highlight the uniqueness of this is that we have had a lot of interest from around the world, saying this is really interesting what you're doing.
Can you tell us more about it?
How have you set up this commission?
What's the scope that also tell us about how you're bringing different voices in?
And what they said was look, every country and every kind of organisation within the country is bringing out AI policies and bringing out really thoughtful papers, et cetera, about this.
But they're all based just on expert groups and they're all kind of a single part of the system.
What we think is really unusual about what you're doing is that you, the UK, are actually taking a moment to reflect on what you want your regulation to look like in order to build and shape that future healthcare system that we want to live in.
That is understands that it's going to be AI enabled.
And I think what this reflects is a real clarity that we had right from the beginning that before we got into the weeds of the regulations and, you know, what turned out to be 44 recommendations in a report.
There was a need to set direction and affect agree our compass points.
And I think that's what spoke so powerfully both within the UK and across the world.
So those compass points were effectively we wanted to ensure that any new regulatory framework was achieved was safe.
It was fast recognising it needed to be agile for a rapidly moving technology space, but it also needs to be fast in the sense of being able to get technologies through and either weed them out or through to benefit patients as, you know, faster than we do now.
It needed to be trusted. It needed to both earn and maintain trust when that's always true of regulation, but even more in this space.
And fourthly, and in fact, most importantly, we need to be person centred.
So what we're looking at is a future healthcare system that will be increasingly technology enabled.
But part of our role is to make sure it was always person centred.
And that of course is about being patient centred first, but it is actually also around enabling professionals.
I think we took a view though, to be honest, if the sort of introduction of these technologies didn't improve the working lives of health professionals, then it wasn't really very good.
You know, surely with all the technology capabilities, we can achieve both. We can really improve patient experience, improve patient outcomes whilst also empowering health professionals and enabling them to actually have better working lives, tackling burnout at the same time as we're introducing technology that improves the lives of patients.
So I think that those those compass points of safe, fast, trusted, and person centred have defined how we then set up the commission to achieve those aims, but that was where we wanted to get to.
That's a great vision of what we want regulation to be going forward.
I would like to dig into some specifics about why that vision was needed and why it was felt that there's something needed to change.
And I'm specifically thinking about what the advances in A.I. that are pushing the boundaries of current regulation and where current regulation is struggling and where that change was felt it was needed.
So that from a technological perspective, but also from like a geopolitical perspective about the UK's position compared to other regulators.
Thanks, Annabelle. And yes, you're quite right. I didn't answer your previous question. I think there were a few things coming together at this time. And in a sense, I don't think these discussions about how we regulate A.I. going to go away because, you know, I think you and I could be sitting here in 20th time and the exact detail may have changed, but this is an ongoing, both opportunity and challenge as these technologies continue to evolve.
But I think the reason it happened, particularly now, was as mentioned earlier, a need to respond to the opportunities and challenges of generative A.I.
I think it's also that we have moved from, say, four years ago, where the introduction of A.I. was much talked about, but actually there were probably far fewer examples.
It was in a sense much easier to contain and constrain to actually being very widespread, including through the use of A.V.T.s and so on.
So it's just like the challenges of A.I. and ensuring safety is now widespread across the whole health system, rather than just being in a few higher resource settings.
And we just dig into that one thing, because we mentioned that generative A.I. is the problem, but can we just go into why generative A.I. is a problem with current regulation?
Because to my mind, it's things like the fact that it's non-deterministic, that it might be more rapidly updated. It might be more generalist, rather than having, for example, a static deterministic model that can be evaluated pre-market.
Like, what is it about the generative A.I. that particularly makes that a challenge?
That it's broad purpose and the kind of fact that you can potentially have infinite inputs going in and potentially infinite outputs in non-constrained generative A.I.P.T.s model, the fact that at least kind of the base models have often not been designed for use and health care.
There's a whole host of problems, but I guess what you're trying not to do is you're not trying to say, "Oh, well, I'm going to put this in the two difficult box, and therefore we're going to just ban it outright."
It's clear that there is an opportunity here, so we need to move regulation to the opportunity of the innovation, not hold back innovation to where regulation happens to be from 20 years ago.
I think one of the things I haven't said yet was that the commission was actually partly set up in response to the NHS 10-year plan.
It's a describing analog to digital shift, and they talk about A.I. as one of their big bets, and they mention A.I. more than a hundred times through that report.
But it's just a recognition that if we are going to use A.I. as an enabler for what we want our future health care system to be like, then we need to have a regulatory system that keeps the safe, is trusted, but also is efficient in the ability to get new technologies that could be genuinely helpful out the front line through into routine use.
And as I say, getting beyond the, oh, it's another pilot, and it's another proof of concept, and it's generated this academic paper, but to a kind of, you know, business as usual, where actually don't really talk about the fact that it's A.I. not.
It's just part of, it's just another tool that we use, and that's a bitch, you know, I'd love to get to the point where we're just, we're just talking about health care, and it happens to be A.I. enabled just as it happens to use a bunch of other tools, do you know what I mean?
The focus is on the quality of health care, not on the tool we use.
Let's get into the kind of strategic political reasons as well, positioning the UK, because obviously we've got, you know, big global regulatory jurisdictions in the US and Europe.
What is it about this moment that made us think what do we need to do as a country?
Well, I think there is an opportunity here. One of the things is that I, as Chair of the Commission, I, it's been a privilege to take this big overview picture, including all the political aspects and the kind of opportunities.
But I'm, I always come back to the kind of, what does it mean for patients, what does it mean for health practitioners in the frontline, which probably reflects, you know, where I sit in the system.
I'm unapologetically pro innovation, and I'm unapologetically pro patient safety.
So, and I don't see these as operating in opposition. I see these as part of just getting the system right.
I see this as making regulations smarter so that we get better innovation through, through faster, whilst still weeding out stuff that it just isn't ready or simply isn't useful or safe at all.
And the reason I come to this first is that I think actually every health system in the world, every government in the world is struggling with that balance.
And I think the UK is really well set up to get that right or achieve that in a way that most countries will really struggle.
And partly that is that we do have a national health system. We know that it operates and quite fragmented, disconnected way at times, but still overall there is an ability.
And it's exemplified actually by the work of the commission to bring you know which bits of the system do what and therefore who you need round the table to try and work this out in a connected way.
I think that there is an opportunity for
the UK to work out how it positions itself in terms of enabling good technologies that are
developed elsewhere to come through faster. So in regulatory terms we talk about recognition
and reliance routes. So to what extent can we recognise a regulatory approval from outside
the UK? And so yep this is this meets the standards we we require and yes we can safely bring that
in to be used in the UK. There is also we have as a country we actually have a really strong
medical device sector who've also been engaged through the commission it's been really great to
have have their input but we can build an environment where they can flourish and that's good for
them as companies it's good for the economy it's good for our employment but it's also good for
patients and health practitioners because again that's a kind of engine room of great technologies
coming through faster. I guess when I look at regulation I'm looking at two things how do we make
it easier for people to do the right thing and how do we when you do have people deliberately or
accidentally do the wrong thing how do we detect that early and actually bring down corrective
action and stop happening again for ideally preventing it before before it happens and in terms
of being able to do the right thing again I think countries around the world struggle to provide
the clarity that developers need but one of the things you'll see throughout the report is a
real emphasis on both improving predictability for innovators so that they know what the pathway
looks like they know what they need to do when and clarity so that both innovators themselves but
also all of us are much clearer about for example when is a medical device a medical device as
opposed to a non medical device health technology if it is medical device what is the risk
class it sits at what are the exact requirements in terms of the kind of evidence that needs to
be provided in order to show that it is it is safe and and who needs to do what once it's out in
the system who has you know what related to the roles and responsibilities to control the risks
and so on. Yeah and it was great to see that that recommendation really running center there early
on about clarity in terms of risk clarification because that's you know something that obviously gets
talked about a lot in the industry and let's go go into what exactly the commission was what did it
comprise and what was the process. So yeah it was a big initiative formally announced in September
2025 and ran to earlier this month so September 2026 and the final report landed just a few days ago
the commission had a core group of commissioners chaired by myself and Henrietta Hughes
14 people representing kind of senior leadership and senior voices from across the wider sort
of health and the government ecosystem and then critically sort of feeding into that
or commission group were four working groups of more than 100 experts and they represented there
was a technology working group which represented for example small companies large tech also
clinical entrepreneurs in the NHS there was a health systems working group which was chaired by
Henrietta Hughes which was frontline staff senior health leaders people working across the health
system then there was a cross-white hold across government group which included things like professional
regulators different government departments because again recognising if you want to make changes
the regulation of AI and healthcare you may need to get sign off from other parts of the system
and then really importantly we had four nations working group where you had senior digital leaders
from across Scotland Wales Northern Ireland and England meeting together and essentially and
you know because you were part of one of these working groups what we did was we brought
really quite detailed papers to look at hot topics there might be around post-market surveillance it
might be around professional responsibility and agentic systems and all sorts of interesting
challenging areas which the working groups would work through they could feedback on
and this would then continue to be worked on and iterated throughout the course of the year
so that was if you like the kind of main route for expert inputs in a kind of formal really deep
dive way then there was a public call for evidence which ran December to January which we had 761
detailed responses big chunk of those were from members of interest in members of the public but
also some sort of formal responses from many of the sort of professional bodies and other institutions
etc fantastic material came through that great description of the problems that we are in the
challenge we see but also very thoughtful suggestions of what needs to change and we then had
big surveys so we surveyed more than 10,000 people all together 8,000 members of the public more than
2000 health professionals that was run by health foundation and has been published and looking at
people's views around AI etc and then we had like international contributions and so on
but perhaps the thing I'd want to most highlight because maybe people go yeah yeah we'd expect
people to you know the commission to do things like that I think one of the things that I was most
struck by was their sort of public detailed public dialogue we did so this was a bit like citizens
juries which people would be very familiar with the concept and so we had one set of these public
dialogues led by health foundation and one by national voices and we took members of the public
away and spent time over several days just looking in detail about the opportunity of AI
the challenge and we took three case examples of specific technologies we took an ABT we took a
sort of digital mental health chat box we took a skin cancer triage-diagnostic tool and we explored
with each of those those example technologies the potential how it could be used taking really
drawing from our experience of real life but also some of the challenges the decision points for
example what happens with that skin diagnostic where the evidence is strong for using pale skin
but is much less we don't have such good evidence in darker skin etc exploring that's
meant for public we then also talked through different ways of regulating how the regulatory system
works now and different suggestions in terms of how it could be regulating the future including
self-regulation which is obviously quite in the news at the moment in terms of wider kind of AI
frontier models we looked at things like trusted developer status where effectively people could
earn the right to self-regulate we looked at the current system where it's very heavily pre-market
it's all on a pre-market stage of manufacturer to MHRA or to an approved body conversation versus
something that is more ongoing assurance across the life cycle and so on and and we got their
feedback and what was just really interesting for me is that it was possible to have this very
nuanced conversations and detailed conversation about the regulatory system just removing some of the
technical more technical jargon but still absolutely not dumbing it down in terms of what was
actually happening and it's very similar to the kind of conversations that you and about we've
been part of in the working groups the kind of themes and the recommendations came through
were very similar so for example in terms of what the members of the public in those dialogues
wanted from regulatory system they wanted they were really clear they wanted a risk proportion
of that system they were happy for very light touch for vulnerable risk products but they definitely
wanted much stronger controls on their kind of high risk products and they definitely wanted
continuous monitoring they were very uncomfortable with an approach that put all the way right up
front but they were also comfortable then with the fact that we might have less evidence at the
point of it coming to market and it would be okay to come on to into healthcare what people
commentated called the learner plates on so that there was an appropriate risk controls in that
initial phase with a view to it then getting it full market assurance, market approval slightly
later on so it was a big process it ran for a full year involved more than 12,000 people
gave me a lot sleepness nights and but I'm really really grateful to everybody who committed their
time through that process the level of engagement these are all very busy people I think we've landed
somewhere very significant where it is a sort of foundation that we can now build from it
sets direction it is as you know it these are recommendations independent recommendations to
government this is not government policy we have to wait now hopefully it's in the order of a
few weeks in terms of government response but that's outside of my control and we will see what
changes are actually going to happen in response. So as you can hear a huge piece of work with very
widespread involvement and what this has resulted in ultimately is the publication that's gone out
last week which contains 44 recommendations and you know each of those is there for a reason
but I want to hear from you about which of those recommendations do you feel will really make people
sit up and take notice, which are the recommendations that are most likely to make headlines or raise eyebrows?
There's a few different questions in that. Well, I've been asked before as, you know,
what's your favourite recommendation? So well, I don't have any favourite children. They're all
important. But I think you're right. There are a few themes, I think, I would first want to
call that. So if you look, they're ordered, the fruitful recommendations have a coherence to them
because they fall under three key themes. And those themes are a shift to more proportionate
life cycle regulations. So that real recognition that it may be, you know, that we can be light
to touch in some areas, essentially, that the regulatory system we've inherited is not optimized for
these technologies. So it may be over heavy in some areas and overlight in others. So something
around risk proportionality, but also this life cycle approach. Now, the life cycle bit is the bit
that I think is going to, it's going to be most challenging. People, you know, those universal
acceptance, I would say, that that's where we need to go. But there is also very legitimate,
yeah, we need to, we know, we know we need to go in this direction about how. And I guess one of the
things that I would caution against, it's kind of magical thinking in the space. So you could
be in a position where they're, ah, we're going to manage everything through post market surveillance.
But if you don't put the infrastructure in place to, to ensure we have robust post market surveillance
systems, post market monitoring systems, then you can say we're doing something, but actually,
you haven't built the, the safe system that enables that to happen. I think the second major
theme is around system wide responsibility and safe management. And I think, I think that might
also challenge some people in the sense that what it does is it says, look, in order to ensure
these technologies are used safely, we all have a role in responsibility. And that includes very
much, including to manufacturer. So the manufacturer has responsibilities in terms of the design of
the product and the kind of development and ensuring that safety is sort of baked in the whole
way through and identifying what are the risk controls that need to be put in place at the point
that is kind of leaves the factory in quotes. And so the responsibility doesn't stop at the end
when it's packaged out. But the health provider or a hospital or a GP practice or wherever
also has responsibilities to ensure that their system in which their technology is being deployed
enables that to be used safely. And then you or I and about as doctors and wider health professional
community also have responsibilities in terms of how we use this tool in the same way that we do
for using any other tool that is part of our practice. I think most people again have really
received this and said, yes, we recognize that we all have responsibilities. We just are asking for
greater clarity in what those responsibilities are. And that touches on the liability conversation.
And you had a brilliant discussion on your last podcast with Sarah from NPS on the exactly subject.
And they asked for clarity. But I think most people accept that we do have responsibility. But
I think some people will go, oh, no, the manufacturer should do everything. This should be
perfectly safe out of the box. But for a complex tool, I don't think that is reasonable. I think
there is conditions of use effectively. And then the third principle which cuts right through this
is around trust transparency and productivity. And in fact, when we surveyed, it's part of the
call for evidence, you know, 83% on average across when you put everything together, ask for more
transparency around these tools, kind of when they're being used, what the safety profiles are.
So there are again, there's some quite challenging recommendations into the MHRA and the wider
system about how we can improve the transparency of these tools. There is a kind of right to know
for patients that they should know absolutely when technology is being used, where possible they
should have the opportunity to opt out. I personally don't think it is always possible, you know, if we
take the analogy of electronic health systems, we can't opt out and say, well, actually I'd like a
paper system, please. And so once there are many examples where you probably can, it may be very
reasonable to be able to explore whether you can have an opt out option. I think that won't always
be possible. But I can see of no circumstances where it would not be appropriate to be informed.
I think that's something we would all expect. So yeah, there's a few in there. I've not picked out
one for you, but I've tried to hide it. Some of the things that are important, but may challenge
the state's group. Thank you. And I think I would agree with you. I think the ones that stand out for
me are certainly this question or this concept of the life cycle and the so-called learner plates or
L plates for technology. And I think that feels like, you know, a very sensible step, but also one that raises
quite a lot of questions as well about, you know, how we actually make that happen. I think some other
ones that stand out for me as well is the concept that the powers of the MHRA might increase in terms
of fines and things as a consequence for organisations who aren't compliant. And then yeah, this
question about accountability, system-wide and what that means for organisations, what that means for
clinicians, and also the point about redress and kind of how, whether the manufacturers of organisations
might be held to account. I want to just get on to this question about the ongoing monitoring,
because I think whilst most people would agree that we definitely need ongoing monitoring,
I don't think there's agreement on is how we really shouldn't make that happen. You know, you've
mentioned infrastructure, and there's a big question there about who owns that infrastructure,
who's responsible for it, who pays for it ultimately. And as part of that discussion, one thing that
you haven't spoken about yet, but I think is an important thing to inject here is quite a lot of
medical technology, medical devices are actually D to C, so direct to consumer and aren't implemented
within a healthcare system. So whilst it may be responsible for organisations and clinicians,
etc, to take responsibility for ongoing monitoring when it is deployed in that kind of setting,
a lot of these are increasingly so more and more commercial products are sold D to C. So
who funds this? How do we make it happen? What are your thoughts on how we make that ongoing
monitoring a feasible reality? So I think one of the things is, some of this goes beyond the scope
of the commission, so I'm just trying to be a bit clear that much of this is me taking my
commission chair hat and saying where, you know, just as a commentator like you in terms of
somebody who has the privilege of seeing, you know, into different parts of the system.
From a commission point of view, there is clarity about where we need to get to, sometimes there
are multiple ways for getting the same route and the, you know, mapping that route and what
I think we should look for the government response to see some of the details to kind of how some
that infrastructure is going to be put in place to respond to that challenge from the commission.
I think there's a lot of good work already happening. If you lurk across groups across the UK,
organizations always kind of bottom up. There's been a lot of innovation in terms of, I don't know,
hospitals or GP working out how to kind of put the monitoring in place. There are tools being
developed to help institutions set up systems, so a concept that we've talked about a lot actually,
this is me wearing kind of my researcher hat on is AI redness. There's a team that have been
working on AI redness framework and a checklist that goes with that as part of the centre of
excellence that I lead, so I'm called Cersei AI. So if you look up there and look up AI redness,
you can see some tools that we've developed that kind of help organizations that want to
deploy AI technologies, essentially look at how they ensure that they've got safe systems in
place, including thinking about monitoring. And that's just to say they're kind of, it's worth
looking broader. Those are just the ones I've been involved with. There's increasing interest in
preparing, deploying organizations in terms of, in terms of these technologies. I think
I'm particularly interested in where we can reduce the burden on humans for doing the monitoring
and they can use technological solutions. And I think if I was an innovator and manufacturer,
I would be very much looking at what can I embed within my design and my product to actually make
it easier to collect both for us as manufacturers but also for the system to collect that ongoing
performance and safety data. I think there is, there is an opportunity, so it's a for the NHS
to bring its digital infrastructure up a level. The Health Foundation has done a report estimating
the cost of that, which is significant, but they also highlighted that without doing that kind of
digital renovation, if you like, we simply won't be able to unlock all the opportunities of things
like AI. I think that the point around direct to consumer, it's interesting. I think you can look
at that two way. Well, first of all, from a commission point of view, everything we said about
medical devices applies, whether it's going to be deployed in a hospital or a GP practice
or it's direct to consumer. If it qualifies to medical device, it's a medical device,
whether it's direct to consumer or not. So all those things still apply
But again, from a scientific point of view, I think when the interesting things about
Direct to Consumer is you can look at that and say, "Oh, well, it's introducing new risks
because there's not a health professional in the mix." But you could also make the argument,
it has less variables. If you're going Direct to Consumer, you actually have fewer variables
to think about. You as a manufacturer potentially have a direct line of sight into the device
in which the user is using. You've got much more opportunity for direct feedback in terms of
safety monitoring, improvement, etc. So I think the Direct to Consumer space is really interesting.
I think there's an opportunity to improve our monitoring systems and a real-time feedback
in a way that actually, it may be that the innovation that happens in Direct to Consumer
space is something that we then learn from back into the more complex healthcare system. So
I would actually see that as a really important area, but that actually is an opportunity to learn
how to do this one. I was to this so many more questions I want to ask you at a really
conscious time, and I know that we're having to the end of our time together. So I want to close by
asking you a final question, which again involves stepping back and looking at the bigger picture here,
and saying out with the commission if we're thinking about regulation and success of AI and
healthcare in general. What are the things that you feel like we need to change most to make that
a reality? Is that practical? Is that infrastructure? Is that policy? Is that how we think about AI?
Yeah, so that's probably another whole podcast, but maybe just to sort of
so a few sort of concluding thoughts around that. I have the privilege of being asked quite
regularly, you know, where do we, from an NHS perspective, where do we find the value of AI,
where do we unlock the value of AI in healthcare? And I think there are lots of complicated
questions that we've been talking through just now, but I think in my head this one's quite simple,
and that is that the opportunity or the value of AI in healthcare lies in intersect, and that
intersect is defined by effectively three circles. It's a defined priority need met by a technological
capability in the context of a system that is ready to use that technology or system readiness.
So those three circles have an intersect of need, technology capability and system readiness.
And I would argue that that intersect is relatively modest now. It's probably a lot smaller than
many people outside your listenership think because I think if you listen just to the headlines,
etc., you imagine that's like a massive intersect that kind of like the AI could be used absolutely
everywhere in healthcare, which I really don't think is true. But it does have a value, and that
value will grow over the time, and it will grow for two reasons. The needs will stay the same,
but the two things that are going to change are the technology capability is going to increase.
Now, but step change from more narrow AI to generative AI and general purpose AI is an example
where that technology capability has suddenly expanded to cover more of the things that we would
find useful where it really defines needs in the health system. But the other thing where we can
really make a difference, and I think many of the people who listen to your broadcast sit in
this space and could really make an impact is that system readiness. And what I mean by that is
if you imagine building a bridge, we've been so focused on building the one side of the bridge,
which is like that technology capability, you know, that half the bridge is looking really shiny,
it's so much investment's gone into that space. But the other side of the bridge is potentially
looking a bit rupee. It's that ability of the system to use that technology. And to get that
side of the bridge in a state of readiness to be able to meet in the middle and start actually
creating something that's useful and that addresses our needs. We need to invest and that's kind of
financially invested. It's also thoughtfully to share that it's about people,
skilling up people, it's about building digital infrastructure, it's about getting the systems
right, which includes a regulatory system. And I would see the commission as being one of those
contributions to that level of system readiness to meet in the middle. So I think what I would hope
to see over this next period is that effectively those of us are working the health system and
who are interested in this space, working coherently and together to just to ensure that we are making
sure that we are equipped to, you know, we have systems in place, we have the people that are
both competent, but also confident to use these technologies. And we can actually start to use
those safely and moving from it being kind of pilots and research studies to something where it's
just business as usual. And we're, we're starting to see a very, a step change in the kind of
quality of care we can deliver because it benefits people. You know, there's a really interesting line
in the tenure plan that talks about making the NHS the most AI in the healthcare system in the world.
I have a different vision, which is about making the NHS the best AI in a healthcare system in the world.
What a great note to end on, Alistair. Thank you so much. It's been such a pleasure having
you with me on the podcast. And look, we will share the recommendations in the show notes
for everyone to have a look at. And I do encourage listeners to do that. And it will certainly be
keeping an eye out for all of the ramifications of these recommendations as they're put into action
over the next months and years. So thank you, Alistair, for all you've done.
Thanks very much, Annabelle.
So what should we take away from all this? The recommendations highlight how the UK is
answering a key question in AI healthcare regulation. How do we construct a regulatory framework
that isn't just fit for now, but will be adaptable and robust enough to regulate technology
that's accelerating capability at an unprecedented rate. These recommendations are an interesting
signal into what might be ahead, but they really are only the beginning. Whilst they answer some
important questions about approach and strategy, they raise many others that will require a coordinated
system level perspective. Questions like where responsibility sits, what meaningful human oversight
actually looks like, who owns and funds the infrastructure and workforce capabilities needed
to make this a reality, and how different organisations will be held to account if regulation is not
followed or harm occurs. Answering these questions responsibly and aligning levers and incentives
strategically will be crucial to determining the impact of these recommendations. As with technology
itself, the most challenging part comes when we turn theory into practice, when recommendations
meet the reality of clinical deployment, navigating human factors, commercial incentives,
and increasingly capable systems.
Podcast Summary
Key Points:
The UK launched a national AI commission to develop a future-ready, adaptive regulatory framework for AI in healthcare, responding to rapid technological advances and geopolitical positioning.
The commission emphasized four core principles
Key recommendations include shifting to risk-proportionate, lifecycle-based regulation with "learner plates" for early deployment and ongoing monitoring.
System-wide responsibility is assigned to manufacturers, health providers, and clinicians, with clear expectations for design, deployment, and safety oversight.
The framework promotes transparency, including patient rights to know when AI is used and to opt out where feasible, while recognizing limitations in direct opt-out scenarios.
Direct-to-consumer medical devices are treated as medical devices under regulation, offering new opportunities for real-time safety feedback and innovation.
Ongoing post-market surveillance requires infrastructure investment, shared responsibility, and embedding monitoring into AI product design.
True value in AI healthcare lies at the intersection of unmet clinical needs, available technology, and system readiness—highlighting that investment in systems and skills is as critical as technology development.
Summary:
The UK has established a comprehensive national AI commission to address the urgent need for a modern, adaptive regulatory framework in healthcare. As generative AI evolves rapidly and outpaces existing regulations, the commission’s 44 recommendations aim to strike a balance between innovation and safety. It proposes a shift toward proportionate, lifecycle-based regulation—allowing lighter-touch oversight for low-risk tools and stronger, continuous monitoring for high-risk applications.
Central to the framework is a shared responsibility model across manufacturers, clinicians, and health systems, ensuring that safety is embedded from design to deployment. Patient transparency and choice are prioritized, with clear rights to know when AI is used and to opt out where possible. The commission also highlights the importance of system readiness—where clinical needs, technological capability, and systemic preparedness intersect—as the true source of value in AI healthcare.
While direct-to-consumer devices face unique challenges, they are regulated as medical devices, offering potential for real-time feedback. Critical to implementation are investments in digital infrastructure, workforce skills, and post-market monitoring systems. These recommendations signal a strategic pivot from rigid, pre-market rules to a dynamic, responsive model.
However, turning theory into practice will require coordinated efforts to resolve ownership, funding, accountability, and human oversight challenges—ensuring that AI integration in healthcare remains safe, equitable, and truly patient-focused.
FAQs
The National AI Commission for Healthcare was established to develop a future-proof regulatory framework for AI in healthcare. It was created in response to rapid advances in generative AI, the need for a person-centered approach, and the UK’s unique position between EU and US regulatory systems.
The core principles are safety, speed (agility in regulation), trust, and being person-centered—ensuring patient and health professional needs are at the heart of AI deployment and regulation.
The commission advocates for a life-cycle regulatory approach that is risk-proportionate, moving away from strict pre-market evaluations. It recognizes that generative AI is non-deterministic, fast-evolving, and broad-purpose, requiring ongoing monitoring rather than one-time approval.
Health professionals, patients, and the public were actively involved through expert working groups, public dialogues (including citizens’ juries), and surveys involving over 12,000 participants to ensure diverse perspectives shaped the recommendations.
L-plates refer to a phased regulatory approach where high-risk AI tools are introduced with lighter, more flexible oversight initially, allowing for learning and adaptation before full market approval and stronger controls are applied.
The commission emphasizes system-wide responsibility and ongoing monitoring throughout a product’s lifecycle, requiring manufacturers, health providers, and professionals to maintain safety and performance, with infrastructure and funding being key challenges.
Chat with AI
Loading...
Pro features
Go deeper with this episode
Unlock creator-grade tools that turn any transcript into show notes and subtitle files.