True skill mastery in AI requires continual learning through accumulated real-world experience, not just text-based knowledge transfer. Just as a student learning to play the saxophone must physically practice and internalize experience, AI systems need to absorb and integrate actual performance data over time. The current regulatory framework, which assumes AI models are static after initial training, is ill-suited for future AI systems that improve daily through ongoing interaction and learning. This dynamic evolution means safety assessments must shift from one-time pre-deployment checks to regular, ongoing risk evaluations—such as monthly or quarterly inspections. Locking in rigid safety rules now could result in outdated, counterproductive policies that fail to address the real risks of evolving AI systems. Without such adaptive oversight, regulatory frameworks may become obsolete or ineffective as AI capabilities grow more complex and distributed across diverse work environments. Continued learning is not just beneficial—it is fundamental to achieving human-level competence in AI.
I've explained elsewhere why I think actual continual learning is needed.
I don't think you can have AIs that perform whole jobs as competently as humans,
if they are forced to just write markdown files from session to session.
Just to give an illustrative example,
imagine if this is the way that students learn to play the saxophone.
So you have one student, he's never played the saxophone before.
He goes into the music hall,
he tries to play it, of course,
this is his first time, so he fails.
He writes down a bunch of notes about what went wrong,
and there's the next student who's waiting outside the music hall.
He comes in, he reads all these notes.
He's also never played, so of course, he messes up,
and he continues to add on to these notes.
You have an infinity of students who are outside
the music hall who keep writing notes to the next person.
I don't think there's any sequence of text they could write to each other.
They will allow the subsequent student to just nail the saxophone from the first try.
At some point, you actually have to accumulate the relevant experience into your brain.
I think the same thing will be true for a lot of skills that we want AIs to
actually accumulate from all the different workplaces in which they're deployed.
Okay.
So what changes once we have actual continual learning?
One, I think that a lot of proposals that have been put forward about regulating AI,
assume that you train a model and then you deploy it.
Therefore, if you run a bunch of checks on the model before it is deployed,
we can make sure that it's not going to aid in cyber attacks or do something crazy.
I don't think this assumption necessarily makes sense in the future.
This is one of the many reasons I'm actually worried about locking in
some safety regulatory regime right now,
because we don't know what kind of technology we're going to
be dealing with even within a year,
let alone within five years or 10 years.
What if the model is improving every single day
based on the millions of sessions of work it does in that day?
If that happens, we could potentially be locking in
an archaic and potentially counterproductive approach
to dealing with the threats from AI.
To the extent the government wants some way to do
some kind of safety evaluation on model providers,
I think it would make more sense to do monthly or quarterly risk inspections,
rather than trying to single out some special moment that occurs after training is done but,
Podcast Summary
Key Points:
Continual learning is essential for AIs to develop true competence in complex skills, just as humans must accumulate real-world experience.
Relying solely on text-based knowledge sharing between AIs—like students learning from notes—cannot replicate the deep, experiential learning needed to master skills such as playing an instrument.
Current AI safety regulations assume models are static after training, but continual learning implies models evolve daily from real-world interactions, making such fixed safety checks outdated and potentially ineffective.
Summary:
True skill mastery in AI requires continual learning through accumulated real-world experience, not just text-based knowledge transfer. Just as a student learning to play the saxophone must physically practice and internalize experience, AI systems need to absorb and integrate actual performance data over time. The current regulatory framework, which assumes AI models are static after initial training, is ill-suited for future AI systems that improve daily through ongoing interaction and learning.
This dynamic evolution means safety assessments must shift from one-time pre-deployment checks to regular, ongoing risk evaluations—such as monthly or quarterly inspections. Locking in rigid safety rules now could result in outdated, counterproductive policies that fail to address the real risks of evolving AI systems. Without such adaptive oversight, regulatory frameworks may become obsolete or ineffective as AI capabilities grow more complex and distributed across diverse work environments.
Continued learning is not just beneficial—it is fundamental to achieving human-level competence in AI.
FAQs
Continual learning allows AI systems to accumulate experience over time, similar to how humans learn through repeated practice and feedback, enabling them to improve progressively rather than relying on static, one-time training.
The analogy shows that students learning an instrument must build on each other's mistakes and experiences; without accumulating real-world practice and feedback, no single learner can master the skill from the start.
Current regulations assume AI models are static after training, failing to account for models that continuously improve through daily interactions and real-world use, making those rules outdated and potentially ineffective.
Such models cannot adapt to new experiences or improve over time, limiting their ability to perform complex, evolving tasks like human-level job performance or skill mastery.
Since AI systems evolve daily through real use, static pre-deployment safety checks become insufficient, and more frequent, dynamic risk assessments are needed to ensure safety over time.
It allows AI to learn from diverse work environments and tasks, building domain-specific expertise that improves performance and adaptability over time.
Chat with AI
Loading...
Pro features
Go deeper with this episode
Unlock creator-grade tools that turn any transcript into show notes and subtitle files.
PodcastTranscript AI