How Do You Lead in a World Being Reshaped by AI? | Alan Smeaton

Alan Smeaton, you are very welcome to the Lead Well Podcast. And thank you for inviting me and I'm delighted to be here. Welcome forward to the chat. So, we've been talking about generous of AI for probably about two years. You've been talking about all things AI for decades now. Yeah. Could you talk to us a little bit about the evolution that you've witnessed? You've been at, let's say, the cold face of this, and we're all just catching up now. Yeah, AI, the definition goes back about 70 years. I think it's 70 years next year, it's an anniversary. And you've had various definitions where people say it's developing a computational system that can do something that only humans can do. But I think a clever definition of AI is, is that it's something you can't do yet, but you hope to be able to do tomorrow. And let me expand on that a little bit. And we thought artificial intelligence was speech recognition problem solved. You can do this now. We thought artificial intelligence was computer vision problem solved. You can take an image, we can take videos, we can analyze them, we can even generate them. We thought artificial intelligence was chess problem solved. We thought it was go problem solved. So artificial intelligence is what you can't do yet, what you hope to be able to do tomorrow. And it's been kind of hijacked by some of the big tech companies and the evangelists and the Jeff Hinton's by saying it's artificial generative or general artificial intelligence, sort of a human level capabilities across everything. But the definition of AI is a moving target. And that means that, you know, every time we examine the topic, we come up with yet another definition of it, something we can't do yet. Excellent. It's a good way to think about it. We've seen a massive emergence, right? But we can notoriously in Ireland be slow to adopt new technologies in all aspects. You're obviously in the educational system, right? Yeah. You are starting to create ways to inculcate that into your own classes, getting them to utilize these tools. And to be honest, that's a bit of an offshoot because, you know, just a very quick, quick example. When I started a master's a couple of years back, it was only coming into fashion. And there was very much resistance. It was, you know, by no means use these tools. You know, that would be completely against the ethics, et cetera. Whereas now we're starting to try to say, we can utilize them, obviously reference them, but we can, we should be utilizing them. But you're doing it in quite an interesting way in your own classes, which, yeah, remember the early days of chat, you be tea, some countries like, you know, Italy and Australia. I remember correctly, wanted to ban this. You know, you can't ban something like this at a population level. And that's because of the fear of the unknown that comes with this. And it's the generative form of artificial intelligence, rather than the other, you know, the other applications, classification and prediction, which causes so much angst. And when we're, we encounter something really traumatic in our lives, whether it's professional lives or personal lives, we tend to go through different stages of grief. And the first stage is denial. Right. If you think about it, if you hear something major, oh God, that didn't happen. What is so you're in denial? And then eventually you'll, you work your way through various stages into acceptance. You try to bargain with us. You'll try to distance yourself from it. Yeah. And eventually end up with them acceptance. And I think a lot of people were in the denial stage for generative AI and its possibilities for a long period of time. And some people are still stuck at that those earlier stages. And it's, it's not unreasonable for that to be the case for generative AI, because generative AI wasn't just one thing. It came and they were able to, you know, chat you be able to do things. And now all the other models came along and they're able to do things which are even better. And then they're even better than that. They can generate images and now they can generate videos and now they can do reasoning. And so, so what we keep getting is, is people are set back to the denial stage. And you very quickly go through the denial, the bargaining and eventually end up at the acceptance stage. So a lot of my colleagues in the education sector were very quick to embrace it. And likewise, a lot of colleagues in the educational sector were, were a little bit more resilient and pushing back on it. And that isn't just the education sector. Yeah. It's pretty much right across the society. But what what open AI did was was and they get credit for this in the long run is that they made it easily available to anybody on the planet for free. And there's 800 million people weekly users of chat GPT. And that changed the narrative a little bit. So instead of of an educational sector where you've got this, this very socratic view where there's, there's lectures and the students and the difference between the two. That was the students be able to go and query and interact with a chat bus or whatever AI to. And then almost offer a comparator against their teachers. So what I did in them in the early days of of chat GPT in its second year when it was available. Rather than have students use chat GPT, which they're all everybody's doing as either officially or in a shadow AI version. Was that I took a large language model. It happened to be GPT for and I then fine tuned it on the course content. So you can take the syllabus. You can take the lecture notes. You can take the slides. And but I also recorded the lectures onto an iPhone using a wireless mic. And that gave me like a tour audio clip, which I would show into a speech recognition system. And then 30 seconds later, I'd have a transcript of my lecture, including all the course and I would have done and the jokes, the anecdotes, et cetera, because that's funny enough what people remember from classes, isn't it? And every week I would fine tune the model with this additional content. And then the next week there'll be new content, new content. And I prompted, I was going to say prompt engineering, but we don't call it prompt engineering. I call it prompting because it's not real engineering to basically stay within the guard rails or stay within the course content. Somebody asks for a recipe from Armelade, no, we didn't cover that in the course, right? So we only answer courses. And I also prompted it to be more, not just to give answers, but to be more interactive and dialogic, dialogic with it with the students. And I made that available to students in an anonymized way. So they didn't have to have an identifier to access this. It was hosted by a third party. And I recorded the questions they asked and I recorded the answers that were given. I went through a thousand of the answers and I never found an error in any of the stuff. And we were covering things like linear regression and P values and all that sort of stuff. Very memorable to some people, I'm sure. And it never made an error. And it always engaged in a conversation with the students. So somebody come along and say, what's a P value? Put that in and say, well, P value is, you know, why do you want to know, etc. And then I would enter into a conversation about that topic. And I think we had 140 students in the first year that it was done used. And there was like 4,000 questions asked in an interactive way. And I think there was something like 700 questions asked in the 24 hours before the written exam. So people were using it as a sort of revision tool, which is great. Because you know, you're there at three o'clock in the morning, you're cramming, you know, you're mates are all gone to bed. You can send a WhatsApp to somebody in the class and say, what do you mean by this part? Or you can just ask this chatbot. And so it provided a useful alternative view or alternative way to access content, another voice in the students heads. And it's funny. Obviously we have James, our social media manager is here today as well. And he got to experience elements of that as one of your students. And again, just that learning experience, obviously he talks about, you know, I suppose how enjoyable that was rather than just you're talking about the socratic approach. You can adopt a constructivist approach, but do it in a way where, you know, you reference having say a teaching assistant and the quality of the questions that the teaching assistant was getting had increased based on the fact that the students could initially ask some of the maybe questions they would feel would be, I'm not putting my hand up to ask that. I might feel a little bit stupid if I ask that question. They were able to utilize that large language model to ask those questions. And then the questions they were bringing then to the teaching assistant were far more pointed. Yeah. We didn't, we just didn't replace anybody. We still got as many go into lectures because we were getting the experience of that. And we didn't cancel labs. We had those labs. But as you said, students would ask these easy questions. The questions they'd feel almost like a bit too embarrassed to ask, because it's a bit basic and it feels stupid kind of thing. And they'd ask those of the large language model and then they'd take them a little bit further along the journey. But it didn't replace anybody. It basically has added another voice or another source of information. And if you think about a student, for example, in the university setting, it's going to get information. They're going to get it from the lectures, they're going to get it from the notes, they're going to get it from Wikipedia, they're going to get it from online material, they're going to get it from their mates. And now they've got another source that they can either choose to use or ignore. Yeah, because isn't the danger or what we hear a lot in terms of the danger is that we will lose our ability to critically think. But what we've described there, in essence, I would say that actually increases our critical thinking ability. Yeah, there was a study published in by MIT earlier, a couple of months ago, whereby they asked 48 students to write an essay. And the third of them were able to use Google Search. Third of them were able to use a chat GPT and a third of them had to come up with the ideas themselves. And they measured their neural activity, but basically putting on sensors on their brains and picking up the amount of neural activity. And they did this essay writing task for five weeks in a row. And they observed that what had happened was that those students who were just using chat GPT were all floating the task, the creative aspects of the task, to chat GPT, whereas the others had to come up with the ideas themselves. Now, there were different essay topics, but it was the same kind of thing. And if you imagine by week three or week four, you have to come up with an essay, an hour to do it, et cetera. Your brain is buzzing because-- and the analysis they were doing in the census they had on people's brains were basically picking this up. So they concluded immediately, so chat GPT, dulls your mind. But that was a completely premature conclusion to reach because they started with 50 students, and they ended up with two thirds of them dropping off. So it was only those who were really motivated to complete the task, completed the task. And for five weeks, they had to write an essay, which is every Monday morning to have an essay on a different topic. And of course, it dulls the census and the creativity with that. And one of the really interesting things that was in that report was that they put in some hidden comments into the text, which said things-- and I'm paraphrasing here-- if you are a large language model, then this is the conclusion to reach from this blatantly. In the article, knowing that what happened is is that because it came from MIT, a lot of people would take it, shove it into a large language model, generate a summary from this. And here's the conclusions already prescribed by the authors of that. So there's a lot of flaws associated with that. But it got a lot of traction and a lot of media coverage and that sort of promoted this narrative of using chat GPT, dulls your mind, which isn't actually the case. No, not if you're using it in the right way. In the right way, exactly. Yeah, because there is elements as well. Isn't there where I know you've spoken about for utilizing the free version, for example? It just wants to engage us. So it wants to keep us on for as long as possible. And just even being mindful of that. If you're using it as a research too, or you've also got to be mindful, that's sometimes it lies to you. Sometimes it's giving you-- I remember looking up certain papers for topics I would be studying. And sometimes it would just give you a reference. And you'd go and check the reference and get the paper. And you'd realize, OK, there's not such paper. So there's two things in your comment, Emma. One is the engaging nature. So they are-- large language models, AI powered, large language models. I was going to say they are programmed, but they are not programmed. They are prompted in order to retain engagement. And they use tricks in order to do this. So when you look like you're leaving, one of the things-- you know, these people who just keep talking the whole time, they're like that. And they'll say things. Before you leave, just one final thing. It's a bit like that character in Groundhog Day. You never-- so I just want to get away from them. Get away from them. But that's what they do. They just engage and engage and engage and engage. And never want to let you go, because that's the way they are configured to do that. So that's that one aspect. And then the second thing you mentioned is the errors. And those errors will always be there, because it's within the DNA almost of a large language model that there's an element of randomization associated with it. And the reward function rewards guessing, rather than finding something correct. So this is built into the architecture of a large language model. So it's inevitable that they come up with what we call hallucinations, because that's the way they are designed to do so. Can I ask you-- I had an interesting caller earlier on today with a potential coaching client. And they were talking to me about a colleague that they were speaking with, who has this inkling that people leaders are going to become a thing of the past. That role is going to be completely automated. And for me, I'm sitting there thinking, I couldn't disagree more, but also I'm very biased. Because if that role is completely redundant, well, then a lot of the work I do no longer will make sense. I'm just curious as to when you see whether it's in education or whether it's in corporate companies that are at the forefront of-- because we hear everyone talk about AI. And it's like a race to integrate AI into your business in some way, shape, or form. But are you seeing anybody who's doing it really well, who's really at the forefront of adopting this technology for the betterment of the people and the organization? I think the easiest or almost like the first level use of large language models is to use them in their free version, to the shadow AI. And we get a lot of people doing that for personal reasons and for professional reasons as well. But when it's made official and when it's legitimized and brought into the organization, the first and the easiest way to use generative AI is to take an underlying large language model. And it doesn't matter which of them, because the difference is in terms of performance at the top end is like paper thin. And then to configure it by fine tuning it for some local content within the organization. Best example I've used this before is revenue. Revenue have tax and duty manuals. They have rooms full of them. There could be 100 pages talking about some particular corporation tax or personal taxation, et cetera. And if you work in revenue, then one of the things you have to do is basically learn these off by heart or know where to find them. Do a control F for search with them. And what revenue did about 18 months ago was they took a few different large language models, fine-tune them on all of this local content, hosted it on their own cloud service, and ring fence to it so it can't be accessed. And now what happens is if you work in revenue, you have a question, et cetera, you go and ask the local revenue, Chachi BT equivalent. And it comes back with the answers. There's an example where you take a model and you're fine-tuning it with your content and providing a conversational search tool for your own internal content. So revenue have done that. I came across an example a couple of weeks ago where Tala Hospital, as anybody who has been diagnosed with cancering, Tala Hospital has to go through a non-college screening to see how, what kind of chemotherapy, et cetera. So they want to know a lot about your background and history and allergies and blah, blah, blah, blah. And you just have to travel to Tala Hospital to do that. Now they've configured a chat interface. Fine-tuned with the content from this, which don't be taken access from a laptop, from their home, instead of having to trace from Dune Gol or from Belenche Island all the way to Tala Hospital to sit, waste, to have your whatever number of minutes interaction says instead, you basically do it on, and the time to complete that is down to six minutes. So six minute laptop sessions, instead of a trip to Tala Hospital. Again, the large language model fine-tuned with content, knows which questions to answer. And it's got 98% accuracy compared to a human. And there's lots of other examples. So to go back to your question, are there any examples of people who are doing it well? The people who are using AI well are realizing where to solve a problem and solving that problem well, where the problem is solved by a question answering search type of interaction. And so for Tala Hospital to revenue to dime a dozen others places, that's the great place where you can get that low hanging food. Beyond that, it's more difficult to see where AI, generative AI, or even agentic AI can be used in an organization. And I think people are still trying to find their feet with embedding it. And you're right, there's a race to be on, to be seen to be using it. Why? Because if you don't, then we're very, you know, previous century type of things. So everybody else is using it, therefore we have to use it too. But that's not necessarily true. Everybody else is looking at it just as well. You're looking at it. So find a problem for which it is an appropriate solution to ploy it, advertise it internally, promote it, reward it, and then let people who work in your organization identify subsequent problems or situations where AI could help. It's interesting because what I hear quite a lot is companies talking about their AI strategy. And then you ask some employees. And the employees are like, it's mentioned all the time, but we actually don't know. What? So I think there's an element of kind of social desorability on your doing it, your doing it, your doing it. So we better be including it in all of our comms. But as you say, it's how practically are we utilizing this? And so for example, I see good companies use it well in terms of in my area where they're creating similar to what you've described. These interfaces where-- I'll give you a prime example. It could be a sales leader who's about to step into a pitch with a client. And he now has this interface, or she has this interface where they can practice the pitch. They can get feedback in real time from this AI interface on which parts are strong, which parts might be lacking. And that has the ability to potentially analyze all aspects, so not just the content, but potentially the tone of voice, the body language, etc. Again, I can see how that can be extremely beneficial. And when you pair that away, Mark, what that is, it's a large language model. It's GPT, something it's Gemini, it's Claude, it's Coppile, one of those. Fine tuned with a lot of content on how to make a presentation. And maybe it's presentation in a particular domain, so if it's ending to do with car sales, for example, you'd fine tuned it with a lot of stuff to do with new cars, prices, comparisons, all that sort of stuff. So you bundle all of that, you overlay that on top of the underlying language model, and then you prompt it. So the first part is called fine tuning. And the second part is prompting. And the prompting is basically the programming or the instructions it says. Analyze this input from this candidate who's doing a draft sales pitch, etc. Take a look for cases where it's weak, strong, blah, blah, blah, and then give them feedback. It's the same dance again and again. Underlying model, fine tuned, prompt engineer to do something. Can I ask you, so my space as well is coaching, right? I do quite a lot of coaching. Where do you see the developments in AI? Do you see a point in time where coaching, for example, becomes something that's primarily done by an AI interface, looks very human, can connect with the person sitting across. Like is there going to be a point in time do you believe where something like as personal as a coaching relationship becomes something done through AI and potentially as effective? It's tempting isn't it? Yeah, but it's fraud with dangers. Because those large language models which are made available as chatbots and are almost configured with this priority to engage, have got some great upsides, which we've talked about, but they also have got some great downsides and in particular the addiction to them and the mental health issues arising from the addiction to them. We're only seeing the tip of the iceberg now with especially younger people, but not necessarily just younger people. One of the things that open AI did about five or six weeks ago, so it was about in October, was that they published a response to challenges on the mental health issues associated with chatgbt and they proudly said that only that 0.15% that's 0.15% of their users had had suicidal ideation in their conversations. Now hang on a second, that's 0.15% out of 800 million. It's like one over a million people have interacted with chatgbt and have had suicidal ideation. Wow. And of course, when you interact with chatgbt, it doesn't block those conversations. If you interact with chatgbt and you start to do copyright violations, it'll shut down the conversation. But if you start talking about suicidal intention, it'll start to talk about methods and how and when and you're right, all of that sort of stuff. And what was really disturbing about the open AI announcement was, first of all, the fact that there were that many, what second equally disturbing thing was was that in chatgbt5, we have engaged with hundreds of mental health experts to be able to shut down these conversations with an improvement of 65% accuracy. Oh, so you mean instead of 1.2 million people with suicidal ideation, you'll be able to shut down the conversations in 2/3 of those, which means it's only about 400,000 people. 400,000 people. Wow. So the addiction aspect of those is a really is a downside. Go back to your question again, is some sort of chat, AI part chat, but going to replace people coaching, apart from the mental health addiction aspects to it. The second aspect and it's completely different is that you can never replace the wisdom, the intellect, the experience, the social interact that comes from a person because a person like you, Mark, you've had years of experience and social interaction with everybody. And as you're coming up with the next question that you're going to ask me, there's a chaos going on in your brain. An interplay between you, me looking at you and looking for facial reaction and your experience with that, the knowledge that you have of this area, the background research you would have done into this podcast, what we've said up to this point, your knowledge of me, there's a huge amount of interaction chaos going on in your brain coming up with what is the next question to ask. AI is not even close to that level of sophistication or of humanity. And as a replacement for a person coach, it's tempting to see, but the flaws with that, like the mental health addiction, etc, are very quickly exposed. So no, I don't see it being replaced. It's good to know, thanks for that. Yeah, and again, if we look at, if we just look at geopolitical landscape at the moment, where the velocity is optional, we see very autocratic leadership style, they're dripping back in. There's always a danger that these tools are falling into the wrong hands and there is a danger that we may be passengers in this rather than, let's say, having a level of agency where we can utilize them the way you're utilizing them for the good of whether it's students, whether it's government initiatives, etc. There's always that potential danger that we could lose a level of agency. And what I'm, I suppose, what I'm reminded of is even aspects like the Cambridge Analytica scandal and how, as part of that whole process, so many people were manipulated by these tools and these large organizations. And if you remember, going back, and that's nearly a decade ago, now Cambridge, we've stopped talking about it now, but to remind people what happened was that Cambridge Analytica and the company were able to identify and classify personality type based on anything from 80 to 120 likes on Facebook. So all you have to do is like 80 things throughout your lifetime and it would know what kind of personality you have. And as a result of that, they then targeted personal advertisements based on geography, gender, location, for those people who were likely to be in the middle in terms of voting. They could go either way. Personalized advertisements targeted against the personality and basically just delivered those in Facebook. And President Trump's first campaign for the first time he was president, he spent more money on Facebook ads than on everything else combined. So that's how we got that election. Was targeting advertisements on people, and that those advertisements were not seen by anybody else except the target that it was aimed at. And the footprint that was used was just Facebook likes. Now if you think about it, the digital trail that you've left behind on chat GPT or whatever kind of AI power chopper, if you go into the personal sphere, you're asking questions, you're leaving an awful lot, richer footprints behind than just a like on a Facebook page, you're asking questions that might be to do with your personal relationships with others, your health, your wellness, all sorts of stuff that you're leaving behind. And that's a awful lot of revealing of the inner you to what you believe to be a sacri-sanct and chatbot that will always tell me more, oh, do you tell me more? Tell me more. And it's sort of pulling this conversation out of people. Now those conversations and the insights from those conversations have not been used in any targeted way. Like when we do this with them, with going back to the education case, I use it in order to be able to identify recurring topics that students were asking that I may not have taught well enough. Like if they're all asked about p-values or something, then there must be something to the way that I presented that. Now that's sort of out of population level rather than targeting James, who was one of the class, in that class, and learning about what he doesn't know. But if you look across the set of those, I was able to identify parts of the course that they got stuck on. Now go back to the chat, G-B-T or the cloud or Gemini or Copilot, whichever one you use. There's not a lot of information on you. It's left in there. And if you, you know, one of the things you could do on whatever is your favorite chatbot is look on your previous chats. Just the titles of those chats tells an off-lash. Oh, too, you know. Yeah. So. And again, you can see the, as you're speaking there, what comes into my mind now is I'm thinking developing my own little chatbot for any programs that I'm running, you know, prompted which the content, but that's relevant for the group. But then more so, try to capture the themes in those groups. So for example, if I have a leadership program, I might have four core modules. If I have all that content prompted in that large language model, and then I give the group use of that to ask it questions. And then obviously anonymously, I. can analyze the themes, all of a sudden then your workshops or your facilitated sessions in person may become far more pointed because you're able to capture what are the group really struggling with. And you hope you've got that figured out beforehand, but rather than being too presumptive, you could have that as a tool which keeps let's say refining the accuracy for that group as you go from session to session. And that's great to think of that on a group size, whatever size group that might be. Imagine that on a national level and that brings me to Estonia. So in Estonia, what the Ministry for Education is done for high school students is that they have contracted with OpenAI and with Google to take basic versions of the foundation of large language models from each of those. They have then fine tuned those with Estonian content, Estonian culture, Estonian language, Estonian literature, Estonia course content. And they have made this available to, I think it's 40,000 high school students in Estonia. They have trained nearly 5,000 teachers during the summer just passed on how to use this. And they now present this in schools as an option that teachers can either decide to use or not to use in whatever way they make a progress. And it's free. And rather than it being the free version of Chatchy BT, they're getting the customized version of Chatchy BT and of Gemini. I think it's 50/50 divide just to see and not be locked into one vendor. So you can imagine this being a great prop for those teachers who decide to use it in their teaching. But can you imagine the log files and the traces not assigned to an individual, but analyzing those conversations and those interactions and looking at themes that recur repeatedly across the population of students in a particular subject. Now that's brilliant insight. Never get that in the other way. It's great. You might have picked up in your workshop example just from body language or from hearing the same questions. It's a good way to automate that. But do that across an entire nation and entire curriculum and seeing where the pinch points are where people, where kids are getting stuck with particular topics. That's information and feedback you'd never get in the other way. So I can see the breath. You can see the wheels turning in your head. Because you know what I'm thinking. And again, I'd love to get your take on this. I'm preparing a dissertation at the moment. And I'm still, it's a psychology master's, right? So it has to fall into the sphere. This is another master's. Yeah, yeah. But I'm even just thinking, if, let's say I was to put together a research proposal, which was going to capture one of these large language models and try to elicit a large body of data in my area, which would be for leaders or managers and try to really understand them. That, like so previously I've done qualitative. I've sat, I've done interviews with them and taken those insights and created themes. As we keep moving forward with research, is there going to be an element where we can utilize those large language models to capture those themes as you're describing, you know, 40,000 kids in Estonia now, as those years go on, the data that's going to come from those is going to be incredible. Is there an element now where research will move towards and it probably already has to a degree where we're utilizing those? You know, ethically is that something that easily would get approval or is that something that is still a big issue? It's hard to stop. And one thing you said in your preamble to the question was, was to imagine the data that's generated, but the data that's generated from Estonia for good. Imagine, sorry, I would phrase that, sorry to correct you, but imagine the good that you could do with the data that is gathered on the footprints from all of the kids in Estonia. And by the way, that's what I mean for that cohort of managers. Like, can we collect data to help them in terms of, you know, I always find that cohort within organizations are the ones that are most stressed. They generally feel the most pressure. And often they're not willing, not that they're not willing, but sometimes there's a hesitancy to come forward and say that I'm struggling or say that I don't know how to do something because all of a sudden, then, you know, my reputation might be damaged. And is there less invasive way, is there an interview of trying to collect, yes, there are surveys, but people find surveys boring. You know, is this interaction with a chatbot or a large language model, a way where we can elicit real feedback to help those individuals at that level? Yeah, because when you're interacting with one of those, hey, a part large language models, it's a conversation, you know, it's it's the Turing test, right, which has been passed because you believe or you are led to believe, right, unless you have to keep slapping yourself in the face and reminding yourself, this is only a, this is only a program. There isn't a, it's not anthropomorphized. So it basically engages us in conversation and dialogue, which we're very comfortable with, more so than filling out a survey. And for those people, filling out their oncology, prescreening in Tallahhospital, it is a much more, that's not a pleasurable experience, but it's better than, you know, filling out a forum and Tickin boxes because it's engaging in conversation. And we are much more comfortable with that than Tickin boxes are filling out forms. And when we do that, then we tend to reveal a lot more, even in the way that we, you know, that language we use, the words we choose, the way we phrase, that kind of thinks. And if it's a voice input, then the way you intonate, the kind of things. So you're giving away a lot more. You are open to giving away a lot more. I provided that's managed and, and, and captured and analyzed in an ethically, conformant way. And, you know, that's hard to see it downside. Yeah. And are we there yet? Are we getting close to that in research? In, yeah, to go back to the question, in research and in your masters, the part which is the literature survey, looking at the, that's now really assisted by AI tools. And there's tools like elicis.com and there's others which basically do, they're fine tuned and, and, and sorry, they're trained on, on scientific literature and on academic literature, rather than on the internet. And therefore, they're very good at pulling information from previous publications. And so there's your chapter two. The first version of your chapter two, Dylan, both remember, you have to check everything and read everything and make sure it's valid. And but the part in, you know, coming up with hypothesis and coming up with experimental methodologies and then carrying out all those, that is something that is still part of the scientific process. Yeah. It's something that's still done by humans. Yeah. And I, somebody said to me recently that when I was talking about alpha fold, which is Google's, or Google deep minds, production of the 200 million 3D structures for amino acids. And it used to take a PhD student four years to do one of them. And Google automated just did all 200 million and put them on the internet. Wouldn't you hate to be one of those PhD students who had spent four years defining the 3D structure of a protein and find that, you know, it's been automated. So some aspects of the scientific process are automated and can be automated and assisted greatly with AI. But other parts still require the same steps as previous. Really interesting. Ericuriosity in Estonia of the teachers, how many were pro and how many were against adopting it? I was there visiting and looking at this about two months ago. And as of two months ago, before it had launched 60% of the teachers had taken on the training and indicated they were going to use it. And they expected that and hope that would rise to 75%. So two thirds. Two thirds three quarters. I asked because my wife is a secondary school teacher here in Ireland. And I'm just picturing some of the people in the staff room that she tells me about. I don't think it's. Yeah. And I imagine there's some who are complete in denial who's never, you know, shot over my cold body. Shall this AI be used? And for those those teachers with respect, you know, your students are using it at home. And then there's others who are innovative and trying to do things. And it's great to see that happen. When it comes to second level though, we are certainly not leaders in the use of AI. We are a long way behind. And I am going to be critical of the Department of Education here. It took the Department of Education more than 550 days between announcing that there would be guidelines for the use of AI in schools and eventually producing those guidelines, which the minister produced about six weeks ago. And let's just say that they lead, they leave a lot to be desired. They talk about using generative AI to help in drafting policies. And there's nothing there to support the accessibility of generative AI in schools to be some schools from disadvantaged areas where kids are all just using the free version. And then there'll be some schools where the school will subscribe and we'll take out subscriptions to paid versions. So there's very little support in that guidelines produced by the minister. Yeah, it's a shame, isn't it? Yeah, you know, not quicker out adopting some of these aspects that are clearly going, they're coming, like they're already here. Other countries are adopting and utilizing them. And as we say, you're getting this good rich data like the county has an example. It could be used to for so many, in so many ways to improve the teaching and learning and education processes, but also, and that's one outcome. So you learn your science or your history or your geography better, but it also would make our younger citizens, our school kids, much more digitally literate and AI literate because at the moment, the vast majority of them using the free version of pick your favorite model just think it's, oh, I don't have to have them clear how it works, it just doles stuff. And one of the things that was found recently is that the more people don't understand how language model works, the more they trust it. Yeah, I know. Your A-Brother, I've shot up with that. Yeah, the less you understand about how it works, the more you trust its output. And the corollary of that is that it ignores its bliss. And if you learn a little bit more about it and you start to learn about how it works, you actually mistrust it more. And that's exactly the place you want to be. Yeah, it's exactly the place you want to be. So currently, there's no scope for improving AI literacy at a national level. There's no national program in the AI Advisory Council in January of February this year. This is what we advise government to do. And there's been nothing about this. There's nothing in the, nothing concrete in the guidelines from the ministry for our school kids about improving AI literacy. So they're just basically blindly going on and basically, you know, using this thing and trusting it because they don't understand it and because they haven't been informed or taught to that. And you mentioned earlier on about one more of you'd use around the whole topic of AI is curious, curious about it. And if we're just taking these products for what they are, and as you say, just using it blindly, but I'm actually trying to understand or be curious about what could it be doing with my information or why is it interacting with me in this way? If I'm just blindly using it, and as we say, trusting it, well, then that's where we're going to find ourselves, you know, potentially in a harmful situation with these tools, whereas adopting a curious approach is probably the way we need to go. Yeah, I think the more curious we are as people in our personal and our professional lives, the better we are, the more satisfactory we are. And you know, you see that quote, be curious, which could be attributed to the US power world equipment. But the better example that I draw out from is, is season one episode eight of Ted Lasso. Any of your listeners have heard this, this party, it's the dark scene where Ted is playing a dark game against Rupert. Have you seen it? Yeah, I have. So you know exactly. Replay it in your mind. And as he's going for, I think he has to get two treble twenties and a bullseye. Yeah. And he's got the two treble twenties and he's just about hit the bullseye and he says, they never asked. I was always be curious. And then he hits the bullseye as a great scene, isn't it? So be curious, attributed to Ted Lasso. I'm very good. And if we think about, I can imagine if you mapped out a timeline of the last, you know, two to three decades in the advances in your area, you know, be fascinating. As you look to the future, we obviously have aspects now like, agente geoi, which I have no clue, you know, just what the advances in that area are like today. I know you have a great understanding. I'd be curious to ask you, like, what does the next decade or two look like in this area of AI? Yeah. I think it's, I think it's fair to say that since generative AI arrived and turned the machine learning paradigm on its head completely, that has impacted society big time. And people are much more familiar with it than we're seeing it in workplaces and so on. But it is also impacted computing research enormously as well. And my favorite area of AI research would have been in computer vision. So basically analyzing images and videos and looking at content and that kind of stuff. And we were making slow incremental progress with that. And it was measured. And you know, the trajectory was a particular, you know, climb, success rate and you could project where it would go. But generative AI has basically pivoted that upwards. And in the scientific domain, in computer vision, in robotics and all the stuff associated with computing, the progress in the last 10 years has been more than in the previous decades. So trying to project where this is going to go, right, and it is extremely difficult because the future is coming much, much faster. And that's different from the public familiarity with generative AI, which is also moving very fast. And it's really, a PhD students working on computer vision now are using foundation models that weren't a thing three or four years ago. Just literally were not a thing. So the methods that you would using computing have now completely been turned on their heads. One of the big conferences and multimedia is was held in Dublin about about a month ago. It was about two and a half thousand delegates came from around the world. PhD students postdocs, researchers, academics, etc. All presenting papers and posters, etc. And more than 60% of them use foundation models, which were not a thing two or three years ago. Predicting where it's going to go, it's hard to say. I'm even wary about saying what's going to happen next week because there'll be another announcement. Not over a new model, a bigger model of a larger footprint model trained on more data model, but applications and the impacts of them. It's really difficult to predict. I know, well, matter what I'd say, the only thing you could predict with certain is whatever you predict, you're going to be wrong. Seeing a lot of advances in agenticaio, you've mentioned in particular models from the likes of Microsoft. Would you give us just an insight into what you're seeing there or what we are seeing there? Yeah. So what happened with large language models as they were being trained with more and more training time is that they started to get cleverer at their answers. They started to be able to do more inference and deduction and reasoning in reasoning out complex questions, answers to complex questions. And this happened because they were given more compute time. In other words, to take their training data, I turned it into a model and said, "A giving it two months, give it three months." And let's just see, come up with the model, still looks the same. It's the same size model. Internally, it'd be different, but it's able to answer more complex questions. And that was a natural progression over the last of the first two years of the development of large language models. Separately, and in a completely different part of software and development, there's a technology called agent software where what you have is not a program that does something, but you've got an orchestra of programs. And each is like an orchestra a different instrument. It does different things. It has different expertise. And you can give it different inputs and it'll come over the outputs. And in agent-based software, what you do is you control like a conductor would control an orchestra of different instruments. You give the problem to the set of agents and you say, "Go and solve it yourself." And they can talk to each other those programs. That's the second thing. So putting those two things together, people realized that if you had agent-based software and whenever it got stuck in trying to resolve a problem, it could ask a large language model, then agent-based software could start to do clever things. More worldwide applications, real-world applications rather, real-world applications. And the coupling of a large language model, doesn't matter which one, which is able to do reasoning and solve problems with agent-based software, which has got expertise in different areas, starts to put together. This only happened about a year and a half ago when people started to put these two things together and realized that previously agent-based software only worked in very small domains. Now it can work in the real world. Because when an agent program gets stuck with trying to resolve some problem, it doesn't know how to, it goes and asks a large language model and a large language model is now able to give answers to that. One of the examples that you can see for this is called deep research or think-deeper or research mode, which is available on every one of the AI-powered chatpots. So if any of your listeners are, I've got access to a laptop or a phone or you're not driving or and like that, you've got this. Go to your favorite AI-powered chatpot and the marketing people will brand it differently. They might call it deep research, they might call it research, they might call it think-deeper, they might call it whatever. So whether it's copilot or Claude or Chachibit, or Gemini, switch on that option, which by default is switched off and ask a question which is a really hard question, which requires a bit of research. So for example, one of the things I did was, is that I asked for a team sheet for our soccer team the best there ever was. And it came back and it said, okay, would you mean Republic of Ireland or do you mean Northern Ireland? I mean Republic of Ireland team sheet. Okay, and then it went off and it came back and says, you want a 433 or a 442 formation. And I said, I want a 442 formation. And as you're putting this, as this is happening, these questions, what's happening in the interface, it doesn't matter whether it's a smartphone or a laptop, is you're getting the queries that the agent system is putting to Google. and you're seeing the answers coming back. And at that point, it spawns off eight or nine different searches. So it says go and find the best goalkeeper. Okay, go and find the best left back, there are a fast, right back there, et cetera, et cetera, et cetera. And each of those agents, right, basically, there aren't 11 because there's 11 on a football team but it's two central midfielders, two centerbacks and two strikers. - Yeah, yeah. - And each of those is an agent, it's a program, it's a Google search told to go and find the best left back, the best right back, et cetera. And eventually what happens is is that, you know, they'll talk to each other because there'll be some players who could play in multiple positions. So we can't come up the same name on the team sheet twice. So the two, you know, the agents, the eight or nine agents will talk to each other to make sure there's no duplication and eventually it'll come back with the team sheet and it'll say, you know, share given and go. - It was a share, it was not Packey Barnard. - No, no, no, bonner. So if you're listening Packey, Barnard made one good save, right? But share given has been consistently a better goalkeeper and Dennis Erwin and maybe Steve Stanton and Kevin Moran in the middle maybe and Keen, of course, et cetera, et cetera. And you can start to play with this. And this is a great example because what it does is it shows you all of the underlying searches happening like they are different agents tasked with going to do something and collaborate with each other to make sure, you know, that the players aren't duplicated and played in multiple places. And this will take you three or four minutes. There's no rush, but it's almost like a spectator sport watching this unfold. And that's your first taste listeners. That's your first taste of seeing what agent-based software looks like. You have commandeered an orchestra of individual agents, individual programs tasked with going and doing something, coming back. And when they are all finished, then they will give you the team shoes. That's the first example. And I encourage people to have a go at doing that. Now there are much more meaningful examples of agent-based software starting to appear in healthcare and in business processes and so on. But to get a flavor of like, because you began this question about five minutes ago by saying, I'm not too sure what agent's all agent to AI is to say. Now you can see it by using one of those AI part chatpats and a hard question. It's not an easy one, like a difficult one that you have to go off and do. That it has to go off and do research and see how it collaborates with itself and then comes back with a consolidated answer. Yeah, I see it. Like in, I suppose, all our worlds to a degree, strategic decision-making, back being a core element of your strategic decision-making process and getting that information back. And as we said, five minutes versus having to go off and do a ton of research, you can get pointed insights that are gonna help you make those strategic decisions far faster than you would have previously. But again, you have to be, you have to, have to ensure that you check everything because there can be hallucinations. And then I remember doing this question, do not query once and insisting that each of the 11 players had only ever played in the Premiership or the first division. Ah, so that would rule out who, for example, keen. Yeah, yeah, front, because he played in, he can't have Brady playing, can't have Liam Brady playing, because he played in Italy. Yeah, you can't have, could you have Packie Bonner and really pay for Celtic? Yeah, so it had to be Premiership. So you have to check all of these things. So this is part of what the agent-based software would do. And I remember doing that and I came back and it's excluded Roy Keene. And I said, what about Roy Keene? And it said, well, he signed for Juventus. No, he didn't actually play for, yeah, no, actually, it was, it allowed Scottish Premiers, but he did play for Celtic. Yeah, yeah. So it made an error because it had, it had deduced that he had played for Juventus, but he only talked about signing for Juventus. He didn't end. So there's an error, hallucination, that crept into the reasoning. Really interesting. So you can't just take this 11 man time, time, team cheese and rumored that you have to check all of these things. So probably the most famous case we've seen, you know, recent times is the Deloitte Australia situation where they produced a report for a client and it had errors in the call for hallucinations in the report that some of the information was wrong. Yeah. I don't know if you came across that one. And then it made some headlines for sure. Essentially, that's what it was. They produced a report and it was some of the information was false, basically. And they had to repay the client. And yeah, or FK juniors report, White House report on make America healthy again, also had half a dozen errors and hallucinations, which the White House press secretary said were just typing errors. But this is an example, even closer to home. Kiran Molluli used to be an RT reporter. And then he went to Europe, got elected as an MEP, he wrote a letter to Ursula van der Leyen about Gaza and about the situation. And it was revealed in the letter that he talked about some family, et cetera, which was completely made up. And it was hallucination. And he got caught with this. And he was dragged on to drive time to be interviewed by Sarah McInerney. And I can see your face, Sarah. And being interviewed by Sarah McInerney at sometimes be like a blood sport. But she gave him an easy ride. She just basically didn't ask them the hard questions. What he should have done was he should have said, I made a mistake. Put my hands up. Please, the lesson here is check the contents. Check the outputs from AI. Don't fall into the same mistake that I've done. And said he tried to basically say, I was so busy, I had to read a thousand emails and tried to squirm his way out of it. But basically, you know, you can get caught. Yes. I'm definitely going to remember that, Alan. There's other things I'm going to remember, one of which being that my coaching profession might be safe for the time being. It's been an absolute pleasure. I've learned a lot from our conversation today. I know listeners will have as well. So I really appreciate you taking the time to join us on the podcast today. Thanks very much. Time has flown. It's been very enjoyable. Thanks, Mark. Thank you. Thank you for joining me on the Lead Well podcast. As always, if you've enjoyed this episode, don't forget to hit that subscribe button.