AI på ett av världens ledande barnsjukhus: Om skugg-AI, ”tysta införanden”, och riskerna med att inte använda AI
44m 3s
The transcript discusses the challenge of shadow AI, where employees use unapproved AI tools, and strategies to manage it. Greg Kennedy from Sick Kids Hospital in Toronto explains their approach: providing secure tools, raising AI literacy, and implementing policies. Sick Kids has launched an enterprise AI program called Sick Kids AI, which streamlines AI from idea to implementation through oversight and co-development. The program prioritizes projects using a matrix that balances clinical care, operations, boundary-pushing initiatives, and quick wins. Generative AI and agentic AI have democratized development, enabling non-technical staff to build solutions rapidly, but this requires adaptive governance. A multidisciplinary management group meets every two weeks to oversee AI, using risk-proportionate assessments. For example, with agentic AI, they rolled out simple knowledge retrieval agents to 2,000 staff, resulting in 256 agents in two months, and used a quick risk assessment tool to escalate high-risk ones. The key is to balance innovation with safety, ensuring AI is used responsibly while avoiding the risk of not using it at all.
This shadow AI or gray AI is very prevalent and we think there's a few ways that we have to get a handle around it. A is that we have to put the best tools in the hands of our people in secure ways for the organization. We educate AI education and literacy so we need to raise our organizational IQ for everybody. Hey, you're welcome until AI Sweden podcast. It also comes from a forward or artificial intelligence system, which is my Greg Kennedy from Huyke, who is just six kids in Toronto. Greg, welcome to AI Sweden podcast. Thank you and thank you for having me and thank you for accommodating me in English as well. We have to do this in English, it makes sense for both you and our listeners. You are chief strategy officer and director for AI adoption at sick kids in Toronto. To help our listeners to understand a little bit about who you are and what sick kids are, could you tell us a bit about the hospital and your roller? Sure, I'd be happy to. So sick kids or the hospital for sick children in Toronto is a standalone tertiary, quadenary pediatric hospital. We are an academic health sciences center, so we have a three part mission to advance clinical care, research and education. We just this past year released a new five year strategy for the organization. This at the beating heart of that is a movement to what we call precision child health for far too long kids and probably other areas of the health system of what as well have treated towards a population average. And we know that many kids have unique differences that we need to better account for in the clinical care they receive. So the tagline we use is that we want to integrate many forms of data from the genetic code to the postal code and use that to individualize care for kids. And that's really what's at the heart of the strategy at sick kids and topical for today. I think AI will be a really powerful tool to drive that change. And my role in strategy and AI is at the confluence of the health sciences and the business side of what we do at sick kids. And it seems to be a nice fit because I think our organizational strategy and AI are going to be very intertwined in the years ahead. But let's start that because what you are saying is that you want to move from treating the average to three, treating the individual kids. What we change for kid that comes to your hospital if you can make that transition. And what we know is faster, treat smarter, predict better is kind of what we put at the top of our strategy. And we think all those things are going to be necessary by better accounting for individual data and characteristics. We think we'll get faster to diagnoses and avoid some of the negative ripple effects that can come with delayed diagnoses. We think that we'll have much more targeted and tailored treatments. And on the prediction side, we think we'll just be able to see what's coming often before it happens and get upstream of a lot of the illnesses that the population community we serve can face. Hopefully all that together translates to better outcomes for kids. And you said that you just formulated a strategy that stretches towards 2030, where AI is then becoming or is a big part of achieving what you want to go. So from like leadership perspective, having this goal of being a great healthcare provider for kids and using AI as a tool for becoming that or strengthening that position because you are ranked as one of the best or even the best Schilderne hospital in the world. How are you organizing your work with AI in in way that makes that possible? So we launched a new program in March of 2025 called Sick Kids AI. And it was built off the foundation of years of work in the AI space led by some leading computer scientists at Sick Kids, where they solved a lot of the data and engineering challenges and even some of the policy challenges around how to take AI from an idea to an implemented solution. But what they found was a ton of AI was getting developed and we maybe make the publications about it, which is great. But those AI solutions would never translate to impact at the bedside or clinically. And the reason for that was there was a chasm of questions where people will get pinballed around between ethics questions and legal questions and access to data questions and technology questions and clinical adoption, etc. And people would spend sometimes years trying to take a solution to implementation, banging their head against a wall. And so we built Sick Kids AI, which is our enterprise AI program and I can dive more into the details of it. But it's really meant to be that pathway where it is now streamlined to take something that is even just the seed of an idea through to an implemented solution with almost a concierge navigational service and development supports along the way, whether that's something that we're procuring as an organization from an outside vendor, whether we're jointly developing it or whether it's an AI innovator at Sick Kids. We've now built the yellow brick road, if you will, to implementation. So that is like putting the blueprint in place for going from this idea to actually providing value for a sick kid. That's right. Can you talk a bit about how does this yellow brick road works at Sick Kids? Sure. So there's three components of our AI program. One's focused on discovery, so that's how we use AI to generate new knowledge in our research space. It probably a bit out of scope for the conversation today. I think a feature component is what we call our Sick Kids AI service. And this is a new service that's there to advance AI in the enterprise. And it does a couple of different things. So first, it has the AI oversight role for the organization. Any AI solution at Sick Kids that we want to deploy has to be assessed through the AI service before we turn it on, whether that's inbound from industry or internally developed. We take a look at that solution against an established set of responsible AI principles to make sure that it's safe before we ever turn it on. Safe in what sense then? Because when I'm doing other interviews in this regard, you suddenly realize that you can talk about risk, but it gets even more interesting when you start to talk about risk, like economic risk, legal risk, risk for the patient, risk for the staff, it's like, when you are talking about safe, what does that mean? So it's very multifaceted. And I'd add one risk to your list and it's one that we try not to forget and it's the risk of not doing anything in the AI field. When sometimes in the health sector, we can be risk-averse and we've really tried to lean into AI and be risk-managing. So when we look at a solution, it's across all facets of it. It's the financial components. It's legal. It's ethics. It's data governance. It's human or clinical adoption. It's patient and family safety outcomes. It's quite a multifaceted look. Model itself and the performance of that, how that performs in an AI augmented workflow. It's quite a comprehensive look at the solution so that we're confident it is aligned to responsible AI principles and practices. And we do that through a very multidisciplinary group who comes together regularly that I can say a little bit more about. They've really become the brain trust or the engine for AI at Sick Kids. So that's the oversight function of our Sick Kids AI service. And then there's a development or co-development function. We realize that Sick Kids that almost everyone in any corner of the organization in time will have a problem or opportunity where AI might be the right tool to solve it, but they don't necessarily have the right AI expertise on their team to advance that solution. So the AI service is a central set of AI talent knowledge and resources where people can submit their project ideas, those go through a prioritization process. And then we deploy our resources to co-develop those solutions and deploy and scale them across the organization. Could you talk about a bit about that as well?
because the way I started to realize, and you just confirmed that is that, hey, I could really touch upon everything that's happening at a hospital and I could help with administrative tasks or treating or diagnosis, et cetera. What processes do you have in place to actually prioritize between different use cases? Because they're all vast. I think this is super important. Over time, this is really evolved at Sick Kids. When in the early years of AI, I think we had a bit more of a let the flowers bloom approach and projects bubble up from various AI innovators. You know, we place a heavy weight on academic freedom. So AI projects would come out from different labs. And in a time of financial and fiscal sustainability, and ability challenges, we know we need to be a lot more strategic. So we've started to sketch out what we think the right balance portfolio of AI enablement projects can look like. And we're starting to pick projects that really align to that matrix of priority problems to solve. And then within that, we have prioritization criteria that we run potential projects through in order to make decisions around where we invest. Is that matrix only about what kind of use cases you have? Or are you also adding the technology to be part of that? What I'm getting here is that perhaps you have a low impact project that depends a lot of development around large language models for instance. So if you just look at that use case, there might not be that great value coming out of that. But building that solution would have you learn a lot about large language model in a new way that you could then leverage into a big plurour of new opportunities that you couldn't do otherwise. So it's learning like organizational learning about new ways to use AI part of that matrix as well. It is. And that's a nuanced pick up. What that matrix looks like at sick kids, I'll just give you the big buckets. So, Toronto projects will focus in clinical care and clinical operations. And we're going to pick a few target areas where we'll do a lot of work because there's the economy of scale in working in a single area where you understand the work clothes, the data types, you structure the data in the right way. So that's one category. One is back office operations in a place like Sick Kids, which is 12,000 people. These back office operations probably don't look that different in a hospital than they do in any place in the industry. We are looking at some that are patient and family facing. So putting the tools in the hands of the community that we serve, some that will really enhance quality and safety, some with commercialization potential. And then the last category to come back to your question is we call them boundary pushers. So they're the projects that we select that are still solving a business need for the organization, but we're going to test our design in our engineering and learning and push our teams forward. Historically, we might have picked too many in that category. And that can inhibit pace. So we're trying to have a better balance of the quick wins and the low-hanging fruit and yet still be pushing our engineering as well. Do you see any trends or changes in what kind of use cases you take on now? Like you did like five or 10 years ago as a result of the whole organization getting more familiar with AI and more mature and realizing what kind of challenges that I have as a doctor or nurse could AI actually help with? Is there an evolution within the organization's-- Oh, happening. I would almost call it quite seismic shifts. OK. Yeah. The historically are looking back several years. A lot of what was being bought or built were really focused, targeted, deterministic AI solutions. And in very recent years, generative AI exploded onto the scene. And there's so many emerging solutions coming out of generative AI that for sometimes hard to understand reasons have even much quicker adoption. And then the more recent one is agents, agentec AI systems. And what's been quite kind of blew our hair back in this past year was how democratized the builder function became. So years back, if you weren't a data scientist or a computer scientist or a highly technical, those were the only folks developing AI solutions. And now, just with plain language and some of the tools that are available on mass, people can spin up an agent in 20 minutes. And so we're sort of one foot in all of those areas where we have the historical deterministic models, a presence in Gen AI now, we're really starting down the agent path. And we're trying to be adaptive in our leadership and governance to span all of those areas. One thing building this understanding is of course, colleagues of yours experimenting on their own with two new tools, like Cloud Code or OpenAI Codex or whatnot. And if you go back a decade or two, the IT departments in big organizations will concern about bringing your own devices. Now we have bringing your own AI of sorts starting to happen here. And you are in a very heavily regulated business and where you also treat humans. How do you-- at one hand, I would guess that you want to stimulate this experimentation happening because that opens up new solution spaces because your doctor sometimes has thought you think about this. But you perhaps don't want them to go all the way to actually build tools that they deployed themselves. How do you think about this challenge opportunity? Yeah, I would very much agree with you that our belief is we need to put these tools in the hands of our people and enable them to use them responsibly. Adoption of AI and health has been probably a little slower than some other sectors. And I think the only way we're going to scale it is to democratize that builder function with the right guardrails in place. It was another big surprise for us. And we're not unique in this. It's probably any health care organization or really any organization. So we have a certain set of institutionally approved tools that we've bought or built. And we did a scrape of our network to see just over a month how many AI tools websites had been accessed through the network. So you can see the number of users and the amount of data that flows. So checking the logs from the gateways to see what outside service are getting called from our local area network. Yeah. And there were 106 tools in the last one month snapshot that we took. And I think of the top nine tools, eight of them, were not institutionally approved tools. And probably 85%, 90% of what was on that list was not institutionally approved tools. So we know that this shadow AI or gray AI is very prevalent. And we think there's a few ways that we have to get a handle around it. B, educate AI education and literacy. So we need to raise our organizational IQ for everybody with respect to how to use these tools responsibly. We have some policy back stops around it. The things thou shalt not do, which creates a line. And we're thinking of what we call the-- we call it sick kids AI service. We call it sky. And we've been talking recently about creating the Sky Force. And the Sky Force is almost a group of champions who have heightened level of skills and be able to identify, especially, a genetic, so maybe less sophisticated builds and sit in, I don't know, tens, maybe hundreds of teams across the organization and identify opportunities where AI is the right tool for a problem opportunity and help locally build that. And then from our more centralized sky program, we can see the tools that really work, that have good adoption, that solve a problem that 50 people at sick kids might have or 50 teams might have, and we can centrally scale those up and build them out in even stronger ways. Because I think this is interesting because you said, what one of the risks with AI that I didn't mention is the risk of not using AI. And here we have--
many people at Sikki who actually are using AI, but not the approved tools. But they are exploring and understanding. This is quickly from a leadership perspective, I think, becoming really, really complex and hard to navigate or how wrong. It's easy. No, it's uncomfortable. I mean, in the example I gave, they're using the approved tools. They're just also using a whole other set of tools. And I've never seen a technology move this fast and evolve this quickly. So in a publicly funded environment, it's almost impossible to keep the latest greatest tools in the hands of all your people. So it's about putting the right tools where we can and then putting in the right safeguards for how to use things we know people are going to be using in the safest ways for the organization. But democratizing that builder function in the use, I do think is the path forward for any organization that wants to scale it. It is uncomfortable how you govern that isn't as clear as it is for a focused deterministic AI model of past years. And then these things will start to combine into a gen to workflows that are deterministic and probabilistic and these agents will start executing tasks. So I think adaptive governance here is going to be really important that we have the right controls and guardrails and accesses set up and keep building the muscle of how you have oversight and ensure what's safe and what isn't in practice. What does that look like? What kind of roles or colleagues of yours are included in decisions that put these guardrails in place. So we have a management group that we stood up and that governs the activities of our AI program. It's about 20 people and it's from it's very multidisciplinary and it's not lost. I think we joke sometimes there was CIA field manual after the Second World War put out and one of the tactics to disrupt the enemy if you will was just create committees. We think we've created a fit for purpose, nimble one to oversee AI and it's a group. All our technology leaders are there. There's clinicians, there's strategy folks like me, there's clinical informatics and then a few of things that are truly important as well. We have an AI legal lead, we have an AI ethics lead. So this group has come together every two weeks for two hours to design. I want to record a venue like that. Yeah, and it was rife with creative tensions. We say we made AI a team sport and at times it felt like a blood sport. There's a lot of aspirations and we work through these creative tensions and it's been amazing to see how our organizational decision making capabilities, we sort of raised the bar by through dialogue at that group. So to come back to your question, what does it look like for governance in this evolving space? I can use the example of agents. So we have put the tools in the hands of about 2,000 of our staff to be able to build agents. They're quite simple. They're knowledge retrieval agents only, so they're not multi-agent workflows executing tasks plugged into our systems of record. And I think in the first couple of months that they were out there and we took put this in context, we have probably in the neighborhood of about 40 AI solutions that are running at sick kids today. And then we rolled out the agents and in the first couple of months we went from zero agents to 256 agents in the span of like two months. And it's not to say that these are all high quality agents or even having great utilization, but there's a lot of them. And you need to have some understanding of what they are and what they're doing and you know, because they have cost considerations as well. So we identified like a very quick risk assessment tool where we can bucket those that are highest risk out of those agents and just those ones escalate into a more rigorous review. This whole AI service I described when we assess something prior to turn it on, that assessment is really proportionate to risk. So these agents are just because they're knowledge retrieval agents, we have a sense they're on mass, pretty low risk. But some of the ones that have a flavor that might be impacting clinical decision making, for example, will kick them up into a higher level of review. Can that scale? That's I think an open question for us, but that's one example of how the emergence of agents as a newer fast of the technology stressed our existing governance and we had to evolve and I'm sure we'll have to evolve many times over. I read an interview, this ties into an interview that I read with a colleague of yours that was published like a year ago where he expressed that out of the pilots and prototypes that you start building in with machine learning and et cetera, you have to select a few or actually getting implemented in the end in the clinical work. And you are saying something similar around the agents as well that you have a fatura of agents and with a very degree of actually value creation at the hospital. Notice that you can and have to explore so many different ideas before finding the ones that really get implemented and create adds value to the patient or to the kids' family et cetera. Is that a feature that is new with AI or healthcare always had to do this kind of expansive search for the right solutions? Right, I would say that the more deterministic machine learning models, more where we started are nuanced and different, but a bit more similar to the digital solutions of old, you know, those digital solutions are largely static, AI is much more dynamic. So there is differences there, but experimentation is required to get things right. I think where it's a bit different is that we try and fail fast in developing those types of AI solutions, consider the last mile engineering right from the ground floor, but if a model is falling apart at the development stage, you know, don't hesitate to abandon it. Okay. I think where the pace of innovation is interesting with the agents because you can build them so quickly. And I think as you go to multi agent systems, that's a longer, more sophisticated build. But for the ones that you can build more quickly, it does allow for more rapid innovation, experimentation, iteration. So I think we have to think about the modalities within AI technology a bit differently by category. So getting an understanding that AI is a blanket term for a lot of different technologies, which where each have its own characteristics. That's right. Yeah. Yeah. One set of solutions at Sikis, good through what's called a silent trial face. Yes. What is that and why is that important in a hospital setting? So in a healthcare environment and just reminding listeners that we're a children's hospital, these are often very young kids and the margin for error can be very low. So when an AI solution is used in clinical care, we have to be very confident in its performance, the human adoption, how it fits a workflow, etc. Before we ever turn on that AI model. So we would never have it influence clinical decision making before we had that confidence. The silent trial does once a model is retrospectively developed and we have confidence in its performance, we deploy it into what's called a prospective silent validation. So it might be running next to a clinician and making prediction about something that's being seen with the patient, but that prediction or alert does not surface to the clinician. So it does not influence the clinician's decision making. And then we can see how the prediction of that model performs against what actually happens with that patient, what decisions were made. And then again at the end of that trial period, we can compare those two things and it gives us a better sense of how the tool is performing without impacting a workflow. And if all things look good at that phase, it would go into the next phase of prospective validation where a prediction or alert does surface and does impact decision making. So you kind of implement the backend part of the solution, but never present in the UX towards the clinician, what kind of result that's come out of the calculations that the AI does. That's right. And there's different names for it. I've heard it called silent trial. I've heard it called dark trial, but that's as you described. Then I'm curious because I didn't interview with. Sorgrenska universiteten lär två och tre episodes ago, där vi är diskussingar en similar ting, och som i en av det här är att det är det som är en similar som är silent. Implementatius, men också det är ju att du säger att det finns en tjugo opportunity att värrema att det modelen gör vad det är intresserat och inte inte ha bra or wrong recommendations to definition, pornography halpsज्द milieu konfieldans i de installationen faktiskt är försvåret små för att du har dags om det gjort en potential simplen och skulle larsa i tekniskeidan eller så kommer det ju понад ett konst i nord enning i enning om förtro bars inför alla grund manifestations och hela min mind dr Larsen Rö~! De har sjuksen där karier, för de vill hjälpa på sig, och de vill ta mycket tröjd i en tröjd, och vi vill ha en bra resan. Här kommer vi ut med ett teknologiet, att ta en helptom, men också som är en black box, hur ska du tänka om det är en challenge för alla videon, implementationsen, där du vill ha en AI på en kritisk decision, eller en diagnosis, eller en kreatment, för att ta en konfieldans i relan om vad det AI presentar du. Ja, det är det trust, och ett jobb, en en workflow integration, jag tror att det är ju en porten. Det är ju nu en dag, det är ju en hårdrappart, då är det en teknologietjön, en bränning, en solution. Vad vi gör, och det är ju, i en halvdär vi är developing AI-projekts, speciellt, men även när det är förkörd, är att ha en annan kliniklärsämpjens, på en tabel från en area, där det modeler är att vara öppnade, och där är det ju en impact. Men det är ju inte en full set av enda utgörs, så de är ju en person, en utgörsämpjens, vi är ur att man är ur att göra vad det inte gör, och det spännande av en stor tid från en idé till en implementat solution, för att det är förstålla att göra ett problem, för att det är ur att integrera i en workflow, för att det är en fördagsdär en alert, att det är ur att det är ur att det är ett rätt sätt, där det inte är en annan. Och då är det ju att man har gjort det, på alla sättet, att börja med dem att sätta förhållet, att understålla i en performant, att kontextuala det kliniklärsämpjens, och jag tror att det är en tända effort, är hur vi har buildat en trust, på en av de en stor performant av en solution, i en workflow, att builda en trust att få en adaption och ha ett team, på en dag som vi har börjat gå, att få en confident, capable och prepared, att få en adaption. När man är säkert att ha kompetens som har varit med på 2030, om vi har en strat till det. Har du en ny roll som är en plötslig, och det är en brugd av teknologi och kliniklärsämpjens, etc., som förstår att det är två särskilt, som kan vara proxis av translators, som är med de andra kompetens som är i de smultadisplanerar, så hur är det att ta och att det är ett annat. Vi måste vara av de andra som du sker. Kliniklärsämpjens, kliniklärsämpjens, kompetens, och brugd av teknologi, och de andra som har en strat till det, är väldigt röra. Vi är priviljad att ha en av de andra som är en av de andra som är en AI-team. Jag tror att vi måste builda en mussel och få en morg talent, som är i organisationen. Men i det här är att vi kommer till en multidisciplinary grupp som vi kräver. Vi har en komponat som har arketypsat i table, som kan dra på en grupp och på en år och en halv två år, som har en riktigt kommerisatio, som är att ha en komplexa problem med AI. Vi har en stor grupp som har en mer stort kommerisatio och en stor utönt. Men jag tror att de andra som kan ha en fattat, en teknologi och en fattat, en fattat, en fattat, en fattat, en fattat och en subject-matter-expertis, som är i en av de kliniklärsämpare som är i andra industrie som är något annat. De andra som är väldigt important för ett avdagsstånd. Det är ett educational system som är adoptat för de nya röra och kompetens. Om det de sprinnator är den som berhumande, så must vi göra앵커 i тебя. Det är ett ene ena jobbade frambildar. Det kan vara rätt, jag vill tänka en använt Barniprom i innovation i tema. När jag tror om en diagram av teknik och komputer-scientist har en cirka av det andra graf av leders och exekitives, är det därför jag har sett på de där med ett ögon. Jag tror att det är en av de andra grupper där det kanske är en av de translationala brorgen som är att de två som är en portengård tillbaka. Jag ser ju nästan AI-educationer och literacy-work-tärgning av det här grupper. Vi har en AI-ready, en organisational AI-educationer och literacy-program, och vad våra lägeninstitut var att sätta oss av vad de har intäts. Det var en sån som är svårt att få en av de som är som att beskriva en av de här dagar av informationen och av det och det är det att det är ett intressant ekonomi. Och jag tror att det var 300 personer som är som en av de första två hårdare som är öppnade. De hade ett 1000 signal om vad de som är att det var. De kreated lärna personerna. Det är av de här förändringarna och de har väntat vad de hade för att fära de i 5-6 lärna personerna. Och de har kraftat en kärkulam-talor till de här. Och jag tror att det är något som är att det är så hävligt att det är så öppnade. Jag tror att organisationsen ska tänka om vad de har en diversitet av stäffneds och vad en AI-team eller en AI-enabilitet support-service i den här dagar. Och att det är för de som är specifikt i dagar. Och jag tror att det är en sjukssäkrapp som är nu och det är det som är med att det är en nyhållande industrie, men det är att vi är tänkt om det. Men jag är kurser för att nu är vi ju ett av divergen av det av stäffneds. Men AI-solutionser är ju en av det som är oftast som som är som som är som som är som som är generalist. Vad du är med i en term som på en rekognition för inställning i en klinkning i en del av hospitalen- -mattar det väldigt simulat om vad mannärktering i industrie är med i en del. Jag är kärs att hur mycket inspirerar eller röntor eller interaktierar du har med en del av aktierar- -och eller eller eller i kanal- -att att lära sig att du är att man är att göra det som är att vi har inspirerat- -att göra det som det är från en del av aktierar som är av stäffneds och en helskare. Jag har personalit med det alldeles av det. Efter en annan år av min kuerna är det att ha en helskare konferens. De har börjat ha samma eller små konversations. Jag började att ha en annan sektors och industrie för lägen för lägen att vi kan pårera. Och det är industrie där det är bättre än att de kustorna är service-poster- -att den är en helskare organisation. Jag ser andra organisationer av det här och i en helskare i att säkert. Jag tror att industrie som vi är mest kluslig part är det en teknisk industri. Det är probably less än att lära sig från ett öppning av AI och det är mer om hur vi kan- -jämna att utveckla. Jag tycker att det är en väldigt intressant area av evolution. En publiksektor, en av oss och en 3-årige company. Det är det nya delar av våra öppningarna och en öppning av våra öppningarna. Jag tror att med så många korta huvudet är kring den här ägningen. Vi måste ha en ny menu av businessmodell-options, där de kunde ha en av våra öppningarna- -och att det är ju en av våra öppningarna och att det är det nya teknologisor och solutions- -om den är en organisator som som är data och en utveckling- -och det är kanske förra en förra öppning. Det är en teknisk och liten om man kan sommer och sommer. Jag tror att vi måste ha en av våra öppningarna och en av våra öppningarna-
get to a more sustainable place. And I'm really excited about the opportunity that a lot of tech companies are bringing for a new look at partnerships. Because in the academic research, there is a lot of opportunities found, but within the healthcare sector, there isn't the tradition or the processes or everything in place to actually build the implementable solutions based on these academic results. Is that what you are saying? I'm saying that, you know, we will, an organization like us is going to buy probably on balance most of our AI technologies, but in an organization like Sick Kids where we have the capabilities, we're going to build a good chunk of them too. And I just don't think without strong partnerships with technology companies, either of that's going to scale. I think that they have things that they can learn from us and coming into the health space. And I mentioned like validation with data, workflow integration, just like decades of expertise in the health space that they can soak in that knowledge and working with us and build it into their products and services. And on our side, we just can't afford to buy and build all of these technologies. They have a high cost where we have financial sustainability challenges. So unique partnerships that aren't just buyer vendor relationships, I think are an exciting path forward for healthcare organizations. And there's never been a more exciting time because you see all of the big hyper scalers pushing into healthcare. You see all of the AI companies, the new or age ones pushing into healthcare and releasing health products and where it meets organizations like us. I think there's a lot of opportunity to start wrapping up here Greg. I'm curious with more personal questions about just excitement. What is special about in being in this field with AI and healthcare in 2026 looking into 2030? What what excites you in in this area right now? Yeah, I'm originally from the north in Canada. So I don't know how familiar you'd be with the great lakes and what they look like on a map. But there's three great lakes in Canada that look like a banana peel and I'm from right where they all meet and it's the land of lakes of forest and rivers. So I'm not naturally a technophile. I'm more of a tree hugger and rather be cast in a fishing line into a river. But I see immense upside in this technology if we use it responsibly. I don't think there's going to be anything that we do in a healthcare organization like sick kids that isn't touched by AI in a few years. So I feel it's imperative upon us to figure out how to do that in a responsible way and to evolve, you know, as quickly as we can to keep pace with how fast AI is a technology outside our walls is evolving. But we have massive sustainability challenges, aging populations, strained workforce, layering on barnacles or patching over solutions, incremental change just won't work anymore. We need sort of step function evolution and I think AI is a technology that presents opportunity to do that. Fingers crossed that we're going to happen that engraved. Thanks a lot for coming in on AI Sweden podcast. Thank you very much for having me. I loved my time in your beautiful country. Thank you.
Podcast Summary
Key Points:
Shadow AI (gray AI) is prevalent in organizations, with many employees using unapproved AI tools.
Solutions include providing secure, approved tools, raising AI literacy across the organization, and implementing policy safeguards.
Sick Kids Hospital in Toronto has launched an enterprise AI program (Sick Kids AI) to streamline AI development from idea to implementation.
The program includes oversight, co-development, and a prioritization matrix balancing clinical care, operations, boundary-pushing projects, and quick wins.
Generative AI and agentic AI have democratized builder functions, allowing non-technical staff to create AI solutions, but requiring adaptive governance.
A multidisciplinary management group (e.g., with AI legal and ethics leads) meets regularly to govern AI, using risk-proportionate assessments.
Summary:
The transcript discusses the challenge of shadow AI, where employees use unapproved AI tools, and strategies to manage it. Greg Kennedy from Sick Kids Hospital in Toronto explains their approach: providing secure tools, raising AI literacy, and implementing policies. Sick Kids has launched an enterprise AI program called Sick Kids AI, which streamlines AI from idea to implementation through oversight and co-development.
The program prioritizes projects using a matrix that balances clinical care, operations, boundary-pushing initiatives, and quick wins. Generative AI and agentic AI have democratized development, enabling non-technical staff to build solutions rapidly, but this requires adaptive governance. A multidisciplinary management group meets every two weeks to oversee AI, using risk-proportionate assessments.
For example, with agentic AI, they rolled out simple knowledge retrieval agents to 2,000 staff, resulting in 256 agents in two months, and used a quick risk assessment tool to escalate high-risk ones. The key is to balance innovation with safety, ensuring AI is used responsibly while avoiding the risk of not using it at all.
FAQs
Shadow AI refers to the use of AI tools by employees that are not officially approved by the organization. It's prevalent in many workplaces, including hospitals, where staff access unapproved tools via the network.
Sick Kids aims to put the best tools in secure hands, educate staff on AI literacy, enforce policy boundaries, and create a 'Sky Force' of champions to identify and build local AI solutions while centrally scaling effective ones.
Launched in March 2025, Sick Kids AI is an enterprise program that streamlines the path from an AI idea to implementation. It includes oversight, development support, and a concierge navigation service to help projects reach the bedside.
The three components are: discovery (using AI for research), the Sick Kids AI service (for oversight and enterprise AI advancement), and co-development (central resources to build and scale AI solutions across the organization).
Sick Kids uses a strategic portfolio matrix with categories like clinical care, back office operations, patient-facing tools, quality and safety, and 'boundary pushers' for learning. Projects are prioritized based on alignment with these areas and other criteria.
Risks include financial, legal, ethical, data governance, clinical adoption, patient safety, and the risk of not using AI at all. The evaluation is multifaceted and aligned with responsible AI principles.
Chat with AI
Loading...
Pro features
Go deeper with this episode
Unlock creator-grade tools that turn any transcript into show notes and subtitle files.