Season 6, Episode 2 - Artificial Intelligence (AI) In The Room

This is for the love of community engagement, a podcast to inspire better public participation. We bring you the latest insights, innovation, information and interviews from across the world. Visit fortheloveofcommunityengagement.com for show notes and more. And here's your host, Becky Hurst. Welcome to season 6 episode 2. I am indeed your host, Becky Hurst. I haven't updated that little intro blurb thing because that's too hard. What you get instead is me introducing every episode. My co-host for this season, Dan Ferguson. How are you, Dan Ferguson? Amazing, fantastic. Well, if not slightly deprived, I'm certainly really stoked to be here. This is going to be an awesome conversation today. Dan and I have just come on to record this podcast and I've said, how are you, Dan? And he said something along the lines of there's been a bug. I've been up 36 hours and me and Mumland thought that meant like children promising, you know, people everywhere, just bodies kind of across the, you know, but no, it's a bug in tech world. You've got the Red Bull there, Dan. I've got the Red Bull and I do have the V. I'm fortunately not sponsored. If for some reason they're looking at a community engagement podcast, do reach out. Monster, my Red Bull, we should be. We should be. Community labs, converting cafe to code. Oh, you're smart. I'm on the caffeine as well. I'm on the straight coffee though. Okay, before we get into this episode, I want to acknowledge that the land I am recording this podcast on is the land of the Garner people and I want to pay my respects to elders past and present. I've learnt this week that we don't acknowledge emerging leaders anymore. Did you know this? I didn't know. No, I didn't actually. Why not? Yeah. And I think it's the thing of being a consultant on the outside of government. You don't always keep up with what the things are. So I always say past, present and emerging and I was told this week, oh, we try not to say emerging now because it's putting that pressure on the younger generations to kind of step up and lead. I thought that was really interesting. But I might be wrong. Who's country are you on, damn? I'm on the wild around people. You are indeed. Yeah. Okay. Let's recap before we get into this being non-stop, hasn't it? Goodness, this has been a. I really couldn't. look, we work with people, people and we work with engagement practitioners. What were we thinking? What was going to happen? But wow, what feedback we've had since just episode one. I honestly feel like we're on six, just with the number of conversations I've been having. Yeah. And I'm trying to sort of screenshot everything, capture everything. So for those of you just joining at episode two, Dan and I are working on this season together, six episodes to talk about the future of community engagement and because we don't do things by halves, as we're doing it, we've decided to write a white paper that is about the future of community engagement. And again, because we don't do things just by halves, we add even more on, we add cherries on top, we are involving the sector in writing that white paper. And so before we get into today's conversation, which I know is going to be massive, it's going to be fantastic. Yeah, it's kind of already started this week without people knowing we even had this record, this episode planned. I want to give a quick update on where we're up to with the white paper, because it's already growing way beyond what we thought it would. So we've started basically with. there were currently nine themes and we're only. this is only episode two, so this is going to change a lot. We've got trust, we've got representation. We've got depth, we've got the survey obsession, we've got professionalisation, we've got ethics, we've got AI, of course, we've got accessibility and what it all means for the future of this work. And what's coming through really, really strong is about being genuine humans. We're only one episode in, as I say, and the sector is just adding to this almost like on an hourly basis in the last week. It's incredible. One of the things, it's funny because I've heard that we would notice the AI data guys, but I'm so happy and thrilled that we're. I guess as a sector, we're kind of starting to move on. I don't think move on is the right term. Broadening the horizons beyond artificial intelligence. If we're going to talk about the future of our sector, AI just absolutely have a good dimension, I hope, but it has a place to play, but just with all of the other things you mentioned, accessibility ethics, we're really starting to have quite a deep conversations in comments sections about what tomorrow should look like in the conversations that we should have today in order to form the tomorrow that we want. It's really awesome to see because I just feel like it has this, this, this, this times an overtones of real ownership and passion and care from everyone. Everyone is engaging. And again, I shouldn't be surprised by that we are engaging practitioners, but it's just invigorating to season. It really is. And it's what's really struck me this week and has hit me quite hard with where the conversations have gone. And we'll no doubt touch on this in this episode as well as we record it. Is about, do we want that it's coming down to that pure value question? Do decision makers want communities involved because the tools are getting faster, they're getting quicker as a practice, we're getting more sophisticated. We've got so many cool ways for people to get involved in decision making, but it still comes down to that fundamental thing. Did the decision makers care? Do they want to hear from the community? Yeah, I wrote a piece. I mean, Becky, you've read it internally for community labs and the phrase that I was just talking with Finn, I'll see us in today. And something that really stood out to him was the term engagement as insurance. And it's not a new term or concept. Like I know everyone that I spoke to has felt that check box exercise, like get in there, get the yes and then get out of the way. But in a sense, like I feel like I'm a being, I feel a little bit like chicken little and I think a lot of other people feel that sense as well, which is we know the repercussions of doing engagement poorly. And I don't just mean not doing engagement or not meeting your people, but even just executives and leadership thrown people in at the last minute where they have no ability to inform and it's like we've made the decisions, go make it feel like it's the community's decision. And then practitioners are sent out the front to kind of cap the cop the backlash. And we know, you know, how poorly this goes down, we know the trust that erodes and we're seeing a plan out in live time. And we're I think we're all as an industry kind of sitting here going like the sky is falling. Everyone goes, it'd be fine. Like it's crazy. It'd be fine. And it stays with the same question. Yeah. Oh, is anyone really listening? No. And until we crack that one, I think I want to say almost shit is about to get real in the engagement because we as a sector are so much more sophisticated than we've ever been. Yeah. Yeah. Okay. And with this episode, because that's why people are here. I'm going to pretend that we're in a podcast studio, let's say in, where was I last time New York? Let's pretend we are in Hong Kong in a in a podcast studio and Lisa has just arrived in the waiting room. So I'm just going to go and fetch Lisa. I'm just playing. Yeah. And I'm now just going down set and I'm going to admit Lisa. Let's get her in. Awesome. Here she comes. We're waiting. We're like, we're waiting. Lisa. Hi, guys. We're so excited. Hello, I'm Becky. Hi, Becky. Hey, Dan. Very good to see you again, Lisa. Lisa, your background actually looks like what people have as their fake background. Like it's very nice. That's a real background. I'm so impressed. That's beautiful. I like you. I don't know how to get to share it because we have the standard corporate background, but not all. Oh, yes. Yeah. I'm so excited. We're actually recording already because Dan and I have just been recording the introduction. And I may chop this bit out because usually when the guests arrive, we chop out this little bit of discussion in the middle, but I may not. I may just leave it in because I was just pretending that you were actually in the waiting room and I was going downstairs and get you. Excellent. Let's just start. I reckon. We haven't actually introduced you there. So let's, Lisa, we're just going to have a chat about Drumroll, please, AI because AI has entered the room. I haven't met you before, Lisa. And I must say I've had a quick look at you online on Dan's recommendation and I was just so excited to see all of the work that you're doing around AI and clearly leading massively in this space. And I, we often talk in community engagement about how Australia is leading the world in engagement and I can see, I think, quite clearly you are leading the world in Australia in AI. So it's so good to have you here. Before I let you actually speak, I'm going to ask Dan to tell us who is Lisa Dan because why have you said, let's get Lisa on the podcast. Oh, okay. Why don't Lisa, I'd love you to introduce yourself because I'm sure I'm going to get, I'm going to miss some of the most critical things. But when we were discussing people to come on, especially for this topic of AI in the room, you're not going to be able to do that. Your name is the first to come up over the last three years of jail, community labs and kind of really pushing artificial intelligence into local government, state government. I think from day one, almost like your name has kind of been there as well, but obviously from. in sub-ideave local government. And so it's just been fantastic to see the things that you've been working on with your co-conspirators like Justin Daly. And what is it now you've got your leading the LG Pro, our enablement special interest group? Like it's as much as it's really awesome to be able to be on the outside as a vendor. It's really awesome to see the internal kind of pushes that there are in the benchmarks and the training as well of just what is good AI, what is good AI practice, how are we using AI and keeping the thumb on the pulse there? It's incredibly valuable information for like us to be able to know where is AI and literacy as a whole because that's been a massive part of our journey as well. It's actually just the education piece as much. So yeah, look, your name was the first to come up when we were talking about this topic, but please, if I know the special interest group is one of the amazing things that you're doing, but I'll hand it over. - Lisa, who are you? Did you get it? Did you get it well? - Yeah, I think so. Well, thank you so much for having me on today's pod. I'm excited to be here for a lot of different reasons. Community engagement feels like it is part of my role also because the organization that I work with in my mind is a community in and of itself. So even just to speak to our staff and our workforce and our executive team about AI, the implications of this technology, the seesaw of the risks versus the opportunities, how we actually navigate that, not only as an organization, but as a sector, has been the focus for me for over the last 24 months. I've been fortunate enough to step into an AI enablement lead position at our White Horse. And also I get this amazing opportunity to work with everyone across the sector because I hold the role of a co-convener at the AI enablement Sieg with LG Pro, also as a co-lead of the AI community of practice with Algon. And I've also had the opportunity of being on the Mav task force for AI. So across those different forums, I really get this opportunity to not only share what we're doing, but also hear about what others are doing and the challenges that they're facing and the opportunities that they're leveraging in their organizations. And it's the, I guess the most engaged I've ever been with the sector on any given topic. And we speak all the time. We have this amazing opportunity to really work with each other because we're not in an industry where we can pitch. So there's been an incredible community that's come together. I think our AI enablement Sieg has reached over 500 members. And there's just virtually not a meeting that we run where we don't get over 100 people attending. And it's just an incredible space at the moment and the willingness and the generosity of people to come together and share what they're doing. Yeah, I couldn't do it without them. - One of the things that you sent us a report, and I think we'll get onto it a bit later in the podcast about I think it's a report for a paper that you've just launched this week, I think. And one of the things that really jumped out at me in that, I think it was that document was said that local government are jumping on this, the staff, because they want to use it. It's like a natural curiosity. It's not that it's being enforced on them in some way, like top down or like you really should do this. It's just this natural people are playing around using it and going, I love this. This is amazing. Are you finding that? - Yeah, so I might just take a moment to show you guys that this is just come out yesterday. So yeah, I'm excited about this. So White Horse authored this, but also basically came together with LG Pro to work with 22 councils across the sector to survey their staff. And so the outcome of that process was that we had 2,500 respondents to the survey across 22 councils. And essentially what we were able to do was establish the first baseline AI adoption report for the nation, particularly in the local government sector. So what that's given us is a really clear view of our workforce. And this report specifically focuses on AI capability across the offices in our organisations. The risks, the opportunities, but also it really looks at the barriers to AI adoption as well as the enablers. So I think the biggest takeaways for me in terms of if I had to address the barriers versus enablers question of the cuff would be that our, the drivers of AI adoption, those enablers are extremely consistent. So we had over 1,500 respondents saying that personal curiosity is what drives them to use these tools. So that means that there are staff coming into work every day and they go, I want to check out what co-pilot can do. I want to go over to chat GVT, which happened to be the two biggest tools that are being used across the sector. Second driver is our time savings. So once you have that experience of AI for the first or second time, then the momentum starts to get going because you realise that there's actually a benefit. There's a what's in it for me. And then the third leading indicator of what's enabling staff is visible support from leadership. That nod you get from your leader, the permission to experiment from the executive team and from that top down is really important for staff because it creates that safe space for experimentation and for them to have conversations, important conversations about what's OK and what's not OK, what are our responsible usage guidelines, where do we stand ethically on this technology? How has it been grounded in the AI ethics principles or our values as an organisation? So they're the drivers of adoption. I just want to speak briefly to the barriers, which is the interesting stat that also came out of the data. The first barrier, second and third, were dramatically lower than the drivers. And what we saw was a huge disparity of what the barriers actually are across the data set. So whilst they were meaningful, they actually changed depending on who you're talking to. So that brings me to our user profiles. What we were able to do through this engagement activity was understand how our workforce is broken up. So we've got four user profiles. Firstly, it's our non-adoptors. So our non-adoptors are roughly-- I'm going to call it about a quarter of our workforce. These are people that are not using the tech. Then we have our novices who are exploring the technology. They've dipped their tail in. It's like maybe one to three times a month. They're going over to an AI platform of some kind. Then we have our explorers who are using the tech one to say three times a week. And then we have our power users, which are using the tech every single day. So if we take the full categories that I've just explained there, what's really interesting is that these user profiles essentially being driven equally by the same things, but they'll be held back by different reasons. So our non-adoptors, they're not what we would assume as being tech laggards or needing to be fixed in some way or dragged along. These people-- and what we saw in the qualitative feedback-- was that for them, it's really around professional integrity. It's often a principled choice not to use the technology. They've got ethical concerns. They're worried about AI thoughts scraping, copyrighted data, IP-- ethically, it doesn't sit right with them. And often, it's a deliberate choice. And so when you dive into that cohort of the workforce, sending them a mandatory webinar on how to prompt better is probably going to land as an insult as opposed to a genuine learning opportunity. So that was a really interesting kind of insight that we were able to gather and some myth-busting that we're able to put around that. I'll interrupt there just to say that cohort includes my 14-year-old daughter. She refuses. Yeah. Yeah, I know. I know. Yeah. Because of exactly what you've been saying, Lisa, it's an ethical thing. She's a bit arty in the way her brain, what she's a creative. She's worried that if we all start using it, there'll be no creativity in the world. So she's really doing it. I like what you've said from that ethical background. Sorry, carry on. No. And that's being reflected at almost all generations. So this is a jester. It's a young people, an old person. Yeah. It is across the workforce. And I get the privilege of having those conversations in a hallway, in meeting rooms, in training sessions. And it's often an opportunity for me to personally share why I got into AI. It's because I was terrified of the technology. People think I'm an enthusiast. But it was actually because when I saw what it could do, I was like, the implications of this, really, really shook me to my core. It set me down many rabbit holes. And what I tend to do as a person is lean into the things that scare me. And I do that through education. So that's kind of how I've made the pivot in my career to do this work. So I love speaking to my non-adoptors. And I have family members that are non-adoptors. And we have fantastic debates. And it's for me an enjoyable part of what I do, to address that segment. I'll move to the novices briefly. These people are not against the tech. They've got a very different barrier. For these people, it's about readiness. They don't have the time. Don't have the skills. They don't have necessarily the confidence. So for them, it's all, I don't have what I need to get going. Our explorers are really struggling with a mix of confidence and time. For them, it's about-- clarity, it's about understanding, am I allowed to do this? I often get a lot of questions from our workforce going, but is this okay? What is our rules and policies around this check? How do I do this without breaking any of those rules or breaching any of our policies? So for them it's about clarity. And when you move to our power users, the power users in the group, which are the daily users, these people feel that there's structural issues for them. They feel held back by a restricted by T settings. They want they want to build automations and workflows and they just want to dive in and they don't feel as though they have the environment to do that with the current setup. So for them, the organization is lagging and is it responding to their needs quick enough? Do we assume all three of us are power users? Is there a level higher than power users? Like, you know, people like me that use it like hourly? Yeah, I mean, I'm sure and I'm looking forward to seeing the data next year, which is the purpose of this report. It's about annual year on year comparative reporting. So we can see how our workforce is moving. How the barriers and the drivers are changing over time. I get the best insights by running these sessions weekly because I get to see trends, situations that people felt uncomfortable with a year ago don't seem to be what's giving them that odd feeling in their stomach when it comes to this technology. So I also get to see in here attitudes shift over time. And it's lovely. So have you seen any massive converts so far? Like from going from the non-user that group all the way to the power user quite quickly or vice versa? Does anybody ever go from being a power user to go in there? Don't like that. I don't have a lot of experience with people to understand if they've gone backwards in any way. I'm definitely sure even I personally have maybe dabbled and then gone now that doesn't feel right. I don't like to say I for this thing. Yeah. All right, dabbled in it and I've gone, oh, we could potentially use a eye for that. And then I've quickly realized that ethically that's not going to stack up. There's too much bias. Yeah, like for example, I really welcomed you case in in local government when I when I've spoken with staff around what would you appreciate? What would you welcome the most if we would give you some AI assistive tools? Recruitment assistance often is in the top five. Right. They want the ability to sort through the hundreds of applications that we're getting as a result of AI I might add. Right. So the barrier to entry of creating your CV in a tailored job application has gone completely down to zero. So we're we're seeing a doubling of applications for any given role. So yeah, yeah. So because I saw something on Instagram that was saying that I think it was Instagram my source of all knowledge. It was saying something like everybody's applying for jobs using AI and then everybody's selecting for jobs using AI. But you're seeing a volume increase because people are finding it easier to apply. Absolutely. And likely the staff on the other end, they're in undating it and they're saying, do we have a tool that we can use? And at the moment we don't have one that I can put at hand on heart and say, I mean these these tools go into decision-making. Once you start to venture into that space. So for us, we've got we've got principles around how we use this technology. And when decision-making becomes a part of that, you're now in a higher risk category and we're not able to do that without having a human in the loop. So for us to introduce a tool that impacts somebody's life from a premium perspective, ethically at this point in time we cannot venture there because we don't have a tool that doesn't that gives us enough comfort to say that there's no bias in this process that we can actually trust the tool. It's interesting because I think like that conversation was really had a few years ago, like pre-chatchy BT when we've started to see a lot of autonomous vehicles, like the massive inclined autonomous vehicles. And the question is, if this hits someone who's responsible, but we're seeing this on the micro level, not just the macro level of like, if this makes a decision who's responsible. Yeah, it's really interesting. I guess like how would the power users tackling that? Is it because they in order to have a power user cohort that must exist in a environment that has really well-specced out policies that's communicated well, that has the right tooling to give them the freedom of flexibility and without that ecosystem, you can't really have that. Or like, that's a really big assumption, but like, yeah, but you know, how do we address that? I guess. Going back to the data, we did ask the question, what is separating the capability score, the highest capability scoring councils from the least capability scoring councils? So just so you guys know, the average capability score across the sector is 67. Now that's a number. How do we come to that number? I'm happy to unpack that with you, and the report certainly goes into detail about that. Obviously that number varies across each council. So we can work to find. So what is capability when you're saying it's like 66? So the capability of my local council, what would just tell me in a nutshell, what that is? What does it mean? Is it how efficient they are? So it's a composite score. 40% of the score is based on usage. So how often are you using the tool? Okay. Which has a direct correlation to your capability. And then we have three risk scores that are 20% each that add up to 100% in total. So the three risk scores are firstly, and these are the three risks of any organisation. So we're tracking these scores. The first risk is do I know what's safe and what's not safe to upload to external tools, or to even internal tools? So this is all around data loss prevention and being safe and responsible with the data that we are custodians off within the organisation, particularly customer data and resident data. Second risk score addresses our ability to spot AI outputs that are incorrect. We're talking the hallucinations, the biases, all of that that we need to be trained on and understand that this technology is not perfect and that we have to play that human in the in the loop role and meaningfully not just read it and scan it, but actually interrogate what AI is giving back to us. And how do we spot those errors? Then our third risk score is around our ability to understand how to report a risk or a concern. So those that together creates a capability score which balances the risks and somebody's usage of the tool. So we saw that the sector wide capability score is 67. We wanted to understand what's differentiating the councils with the highest scores versus the lower scores. And we looked at council size, we looked at council type, we looked at all we cut the data in so many different ways. And what we actually landed on at the end is that there's this intersection of indicators that when they arrive as a package, the council tends to do better in their capability score overall. And the three areas that we saw make a meaningful impact when they arrive together is having formal policy and guidance in place. Having a named AI responsible owner in the organization that is actually there for you to go to and have those conversations with and can actually guide the conversation and coordinate it across the organization. And then the third one is around funding and investment. And that often will translate into training and upskilling and tooling and all of those other things. So when those three things intersect, we see a higher capability score manifest at the council level. So that's if I had to give people kind of a roadmap around, well, how do I build AI capability in the organization? It's really around making sure that you're doing these things together, not in isolation. And the other one that I probably would put to the top of my list is understanding that your workforce is not homogenous and that they all need different things. There's not one mechanism that's going to address this. We need to we need to tailor our approach. So you're in Victoria and you've but the piece of work you've been doing is nationwide. Are there I'm thinking as you're talking about, wow, this is so sophisticated. So so sort of quickly in my mind. How are there other councils that aren't yet at all doing anything around AI? And it's probably just down to individual staff members are dabbling and there's nothing happening at a council level. Or is the general vibe that councils all across Australia? I'm just thinking for my international listeners. Are we, you know, as a country? Are we, is there a percentage of councils that are using that starting to really think like this in in bad AI into their practice? There are certainly councils that are that are doing it and that are leading and that I have. Yeah. Okay, really? I've got some councils that I kind of I'm a fan girl for and I look to really wow, I love that. Yeah. And I will say that one council might be focused on capability and another council might be really focused on governance and then another council might be really focused on tech and so they're for me the three kind of ways in which we get involved in in this world. All of the networks that I'm a part of comprised of people across everyone from PNC or the people in culture and HR teams, thinking about workforce planning, thinking about training and development. And then you have your real techies that show up to the meetings. These are the developers, the coders, the guys that, you know, the wizards, I call them. That's Dan. First Dan. People, they drink Red Bull. They drink Red Bull in the morning. Those people. And then you have your real governance, focused people that come with the data and analysis and the risk and the privacy lens. I feel that we need to have all of these elements of the conversation at the table. We can't look at this conversational, this topic with one lens only. So I think back to your question Becky, because I could talk about this in so many different directions of the tour. You asked me, you know, how do we understand who's leading and who's lagging and what are the differences? Where are we? The answer is that we're all at different stages of our journey. The one thing we all have in common is that we all feel behind. There's not a single capsule that feels like they are doing enough or doing it fast enough. And because this space is moving faster than any of asking keep up with. Okay. And that is, I think where we're at perhaps in the community engagement industry right now, literally in the last week, hasn't it been Dan? I feel like we're just, we can't keep up. And I wanted to bring my question really to Lisa around because I know White Horse Council does have pretty good strong community engagement. So the process of involving the community in decision-making. And this is, I guess, where we're Dan and I are really looking out, which councils, and we don't need to name any, but our councils using AI to help inform decision-making, listening to the community. Are you seeing any use of it yet? We're starting to, I'm certainly using it in my own, like I'm an individual consultant and I'm using voice recordings and getting transcripts. And I'm writing reports now before I leave a workshop. It's incredible. But of course, I'm not within that big organisation. So, are you seeing anything happening yet in councils? Tremendous amount. Oh, really? Absolutely. Absolutely. So, let's revisit the primary goal of community engagement. It's about building trust, improving transparency and ultimately leading to decisions that reflect the needs of the local people in that community. Yeah, huge high-pays. So, that process has so many different elements to it. And if I think for myself, if I'm engaging my community, which is the organisation that I work within and the staff across that organisation, every point of me engaging with them, I've got to create really interesting, engaging, clear comms. That's my first kind of starting point. What's my comms plan? How am I going? Who am I targeting? What's my message to them? Am I asking them to do something? All of that, I have to think about upfront. The second part that I often have to do when I'm thinking that engagement is design a thoughtful engagement. So, if that's in a format of a survey or maybe it's a podcast, maybe it's about what questions are going to ask or be asked. Whatever that conversation looks like, I that thoughtful design will ensure that if I'm asking really good questions, I'm going to get really good answers. And then you have the fun bit of analysing all of that information. Right? If I'm running a workshop and I've got 50,000 post-it notes on the wall, how am I actually going to translate that information into something meaningful that I can then report on back to my executive or personally get something really helpful to guide my next steps in the task that I'm trying to do. And then I will then often have to close the loop. What did this actually result in? How did we go? What was the score? All of that kind of thing. When I think about just the development of this, right? So, this was about we started doing this at our organisation first and then we engaged councils across the stage to do the same. I had to think through all of those components to to complete this as an engagement exercise and then push out a report at the end. We used AI in every part of that process in really meaningful ways, but not in ways that are beyond the average person's skill set. We didn't need a huge project budget. I didn't need expensive tools. But what I was able to use AI to help me do was that comms piece, which we all know that everyone's using it for, right, helped me write this better. The survey design specifically stood out to me. I spent two weeks designing my questions. And what was the most helpful part of using AI in that specific instance was that I actually was able to get AI to generate dummy data. I then produced hundreds and hundreds of rows of responses of dummy data. And then I did a analysis on the dummy data to understand whether if my questions were actually giving me anything meaningful. Wow. Is it really interesting topic at the moment that I've heard I've been present for the conversations? We just have one out in February with the engagement by the beach by Darryce from Captive Air Consulting. And it was a large topic in the room, which is synthetic audiences is what we call that as being able to understand the personas that we're engaging. And then preemptively testing out our strategies, our communications, our questions to see that like, is this going to get me what I actually want? Or is there something like a hidden bias is there leading questions? Are there like those kind of things based on this topic? And it's really interesting because as a sector, I've heard there's a lot of conversations regarding the ethics of that and whatnot. But it is a seriously impactful tool. I mean, we have it in conversations where we can go, Hey, this question is quite leading. Just be careful in case what you're looking for at the end might not actually show up. Like if you're looking for unbiased, you know, unassuming results, I just love that you really hit on it. Like you spent two weeks on the questions designs because I think with, without additional intelligence and with some of our customers, we're solving problems of time and it's allowing everyone to take a breath of fresh air. You go like, now that you've got an extra few weeks, where are you going to spend that time? Like where can you actually then put that to have better, more meaningful, deeper, richer and ideally, you know, higher quality engagements with your communities. It's really awesome to hear that that's also happening in other spaces as well. Can I clarify for the engagement practitioners listening, because I always come in with that lens? Lisa, when you said so, you're creating your questions and then you're creating the dummy data and that could be using personas say it could be say, like this is the community we're going to survey. It's made up of older people, younger people, people, renting, whatever it might be. And then you're testing your questions to see that you're getting, you know, something that's good to analyze and is actually going to help you make your decision. I think that's an amazing process. Dan, in the community engagement section, I've seen a bit being talked about, and I think it was at that event in February about personas. And I was taking it to assume that that takes it that step further from what Lisa's just said, which is where we're using maybe personas to test our engagement process through to actually using personas to make the decision. That's where the ethics crossover isn't it, because we can definitely use it in our planning, but I wouldn't want them actually making decisions based on what we assume the personas are going to say. Am I making sense? Yeah, yeah, yeah. And definitely, like Lisa, you've mentioned a few times before, which is keeping that human in the loop, like really ensuring that for things that impact our communities, we don't hand it off to any model, you know, regardless of its exposure to our internal documents, regardless of its training. In fact, I mean, one of the big things for that is typically the frontier language models. I'm not sure about the open source predominantly Chinese models, but the frontier language models being trained on the internet, some source predicted something around the 62% of Reddit is where the source data comes from, which is predominantly weird data, which is Western educated industrialized democratic and rich. And so just by the proxy of these models, these large language models seem mythical, they seem magic and everything like that. And there is a there's a little bit of that, but at the end of the day, it's actually like f of x of y where the function of x is your training data, your context or import, and then y is your kind of token output. And we just have to remember that like this is a really complicated mathematics function, like a statistical modeling capability. And given its training data, given all of the kind of stuff that we know about what has gone into it, we just have to be careful when we're kind of we are generating content and what we're using it for because if it is for honing, if it is for kind of picking up our own biases and giving us some test modeling, that's fantastic. If we can actually help downstream communities that we serve brilliant, but if we are using it for and I think like Lisa, this was this was the main topic of the rumors synthetic audiences is this deep concern of it puts us one step away from replacing our communities and kind of doing it at hawk and that the question then becomes, well, how accurate can we do this? In fact, we're already seeing vendors kind of put out that like where this close to real community feedback and that does beg the question of well, in a time press situation in a in a brief general repressor's press situation, is there the inclination? Because there is the possibility of just replacing our community feedback for synthetic data. And then it goes back to like Lisa, I think you hit her on the head, which is why are we actually engaging? What are we trying to do here? What does that solve the intended outcomes? I'll go back to that that whole core reason Lisa started with a while. Engaging is like building trust. It's making it. This is all of those things. It's a minefield. I want to address some of those concerns right up front. The two biggest concerns that staff will have are going to be based on data privacy, storage and inference. Where is this data being stored? Who owns it? All of those kinds of questions. Organisations have the responsibility of stepping into that conversation and providing their staff with the tools and the training. So, just so you guys are aware, sentiment analysis is probably one of the top five things that staff ask me about. Hey, I've got a survey whether it's internal external and on any level. I need a hand really analysing this data set. What tools do I have available to me? And that's a conversation I'm having all the time because that's a very administrative process and it is a major challenge for anyone doing this work. And the beautiful thing about technology is that maybe 15 years ago in the engagement space we would get 50 responses and they would be rich and deep. But now the expectation is that we get 10,000 responses for the same engagement activity because we now have all of these amazing tools like your save platforms and surveying and a variety of other tools that enable us to expand our reach. So that's fantastic, right? What a fantastic opportunity that technology has brought to us. On the other side of that, we've asked our community engagement practitioners and people that got into this work because they really wanted to have that deep connection and that human element to be a part of what they spent all of their time on have now have to transition into spreadsheet users and you know, are drowning in quantitative and qualitative feedback that they received across multiple platforms. So how can we use these tools to assist them to do that? My interest is there is an opportunity to use these tools at every stage of this process that is ethical and responsible and addresses all of those concerns. So using the right platforms that protect our data, security and the public data that we're receiving and there's if you want to have that conversation, I'm happy to have that with you, that's a daily conversation I'm having. The second part of that is is the AI helping us analyse it or is it also an opportunity for us to better design the consultation in the first place. So as Dan's pointed out, when's the last time we reviewed a survey that we've run every year and then actually scan the questions to make sure that they're not leading or they're not bias in nature and AI is fabulous at being able to assess questions and design, right? So we can go back to that design phase, we can test it with dummy data because AI will do a great job. We can then spend more time in that design phase. So by the time we get to the analysis phase, we were actually already got an experience of what the analysis is going to be spitting out, which means that we have better surveys, we have better engagements. And then we get to that analysis point and then it's about using the right tools again, safe tools that are helping us code that qualitative feedback, that are helping us sort and shift and slice the data in many different ways. And having the human and the data scientist and all those really great educated and qualified people to be able to check the work of whatever the AI is doing, not relying on it because if we do make the mistake of handing over that critical thinking piece and that checking over to these tools and they do make mistakes, the entire point of community engagement essentially goes away and now we're actually causing harm because we're reporting on things that actually the community never said. Yes, now I want to add in there, it's only when these new tools come along that we start questioning the process of community engagement because I am nudging towards 30 years of doing sentiment analysis in this thing here, my brain. And never, and I produce these what we heard reports that sometimes our thousands of pages long because of all the submissions and I include all the original raw data. And I manually with my brain or an ex-SolSpritchie go through and tag them all in them and come up with themes. Never once has anybody said how do we know that you're not applying bias? How do we know that you've got it right? Yet with AI we're going how do we know it's not applying bias? How do we know it's got it right? So I kind of I'm so on board with our use of AI and I'm because we're now starting to question how do we know there's not bias? How do we know it actually is reflecting because it's a different tool and it's not a human we're questioning it but I've never been questioned on my bias and it's made me think gosh the amount of stuff that's like gone through parliament or you know and I'm like is that actually what the community said or was it my bias in writing out that report? I don't know because nobody ever challenged it. He's a he's a thought exercise for anyone listening in that's done a community engagement exercise the manual way and and spent the hours going through every row and feeding it coding it perhaps you had a team of people do that perhaps you built in safeguards by saying you can't do more than ten at a time because the human bias kicks in and all that kind of stuff maybe you've done all that work and you're really really confident what I would love to do just as an experiment is actually give that raw data set to a safe platform and actually have it assess it and then compare what you did manually to what the AI has done so you can try to have that experience of going how off is the assessment and what are the safeguards I need to build in so then if I am going to use these tools I've actually got some kind of personal experience with it already being checked. Now I actually did that now you're saying it I did it in a workshop a couple of weeks ago I was doing some strategic planning with a client I had the people in the room we did like a manual sorting activity of the survey results it wasn't a massive survey so I had something like 20 respondents with five different questions and I got the manually sorting and we came up with headings and we did themes and they didn't know that I'd actually run it through AI at lunchtime whatever it and got and we actually came up with the same themes it was an amazing process and so I suppose that's part of that building the trust and testing the processes isn't it and seeing where we go wrong or where the AI goes wrong it was a good good process to do I think there's an immense opportunity to have an experimentation mindset yeah there's nothing stopping us if you're using the right platform or you're using an anonymized data set or you're using public data or you're even using synthetic data I often use synthetic data to test a platform so I can be sure that it's working in the ways in which it's claimed yeah and these are these opportunities will teach you more than listening to any podcast because that firsthand experience I've had so many experiences where AI has gotten things wrong where it is definitely a biased one submission over another like whether you're talking a grant application whether you're talking a CV whether you're talking a tender response so it doesn't matter what you're talking about if you're using these tools to help you make decisions yeah that is absolutely critical that you're aware of the fact that it is and you're you've got safeguards in place to ensure that it is performing as it should and testing and experimenting is a fantastic way to learn that and you quickly become familiar with which tools do a better job because they do not all operate consistently if you've got a Claude to do a certain thing and the new ask chat GVT to do a certain thing and the new ask copilot to do a certain thing or you've got a purpose built AI platform to do that exact same thing that's actually an activity I encourage everyone to do because then you'll get to see how they perform differently where the individual platforms strengths and weaknesses are and that is where the learning is you're absolutely right and I think it's that natural curiosity that certainly those of us on this podcast right now probably all have and that's how we're at the point we're at is because we just started playing Dan and I were actually on a call with a council in America earlier in the week and and the guy from America he was saying you know they're finding that that distrust because of what's happening at the federal level in terms of use of AI on all sorts of things and people can't trust what's real what's fake and I guess the same implication could happen if you are suddenly yeah appearing speaking a different language you know it's not going to do any favors is it in terms of building trust and setting an example in terms of yeah using yeah that's why transparency is extremely important yeah the data supports that in terms of the Australians trust scores with the latest OVIC survey results yeah Australians want to know how you're using their data they want to understand how AI has been used in that process our responsibilities to be as transparent about that as possible yeah been plain language at the right points in that process a really great example of this is in the recruitment process you know there are councils now starting to say on their their portals. This is how we use AI in the process and this is how we expect you to use AI in this process. So we can actually get better quality applications. That's a yeah, sorry. Oh, I just I love that example because it's one of those things of actually that that piece where it's upfront and we're going like this is how we use it and how we expect you to use it. It reminds me of I kind of remember the extension but there's a story I'm pleased to give me for those who actually remember this story correctly. But there was an exchange that kind of was built off. I think it was the new stock exchange because people were starting to like companies were starting to inch and inch there like their automated trading service close from closer and they were trying to get like milliseconds of speed over their competitors. And this startup basically took like 17,000 kilometers of fiber optic cable such that there was a reliable delay between the circuit change. Yeah. And it create and people were like, why would you want to have this artificial exchange that levels the playing field and actually massive amounts of traders wanted it because they wanted to actually trade on there not just speed or cost of like service to things. It was actually like no, no, our algorithm is the best for this and it leveled the playing field and it actually kind of feels like we're doing that now where it's like no, no, no, no, no, like let's let's put that delay in here. This is how we're going to do it. This is how you're expected to do it. And again, it does level that playing field. But I do wonder, has it exposed? I mean, like, have you have you heard of in your travel so far and opportunity where we started to go like, hey, resume and couple letters were really good for people. But fundamentally now we need a different system and I'm not immediately going to go and so we need to do psychometric testing for everyone. But like, is there kind of even like the start of the discussion of the evolution of like, we need to completely revolutionize how we do this because at volume, it doesn't work anymore. You know, or maybe it doesn't. I don't know, you know, but like curiosity, like, and maybe it's not in recruitment, but is there now that we've had the opportunity to step back and kind of see it from a 10,000 view almost? Like, are we now starting to go, hey, that worked for a time and it was absolutely great. But we need to evolve this. I think the best example of this is the education system. So back before I was in customer experience and then moved into transformation and now working in AI, I was a, I worked for an RTO as a trainer and assessor. And so part of my role was to understand how do I demonstrate competency for this particularly individual. And there was many assessment methods that I could choose from that would lead someone to being competent or what we call incompetent. So that process is completely having to be reimagined as a result of this technology across every form of education. And it is the biggest challenge for anyone working in that space. In the same way that every school and every university and every RTO is having to re-imagine how we actually assess these things because this technology has completely changed the game. The same conversations are being had around recruitment. They're being had around tender applications. They're being had around grant applications, award submissions. Anything that we have had to traditionally use some kind of criteria for to establish weightings and competency or value is now having to be reimagined because if you're a people leader and you're opening up the inbox of CDs that you receive, 95% of them are written with a PhD background because you know they've had AI assistance. So how do you now make those distinguishments between what's a quality application versus one that potentially isn't being all that truthful, all that accurate? I agree with you, Dan. We completely have to bring it out of these process. Can we then translate that to the thing that happened this week? I don't know Lisa if you saw it. I shared it on LinkedIn and I don't expect you to have seen it. And this is how community are using AI to get involved and have their say, which is a phrase I hate. In Bondi Junction, just this last week, which is why we said we've had this hectic week of just all the commentary around AI. I believe a group of residents there have created their own AI system to encourage other residents in Bondi Junction to have their say about a proposed, I think, it's an apartment block. And they created this tool to be able to kind of select the things that they're worried about and then it would formulate a letter that they could assume instantly send into the council to approve to sort of have their say. Basically, now I was actually quite excited by that because I'm all about community being activated and getting more people involved in this. And if it's a thing, a tool of process that makes it easier and people go, actually, yeah, it's the childcare that I'm interested in or not keen on or it's the the hubbielding high to whatever. I see that as a really good thing because it's community led and it's community getting other community to think about what's happening in their neighborhood. And it doesn't make me feel any different to the way that communities previously have just had a copy, like a Microsoft word, template of a letter and they send it around and then that you get, you know, 50 of them get sent into the council and they're all kind of similarly written. And let that's happened before or a petition that people just sign for the sake of it and it's all prewritten for them. I don't see any difference, but it kicked off quite a lot. I wondered have you seen much use of community yet using AI because this is going to start happening more and more. And I next time I have to campaign my council, take this as a warning, I will be using AI every possible version of community feedback or requests or complaints, whether it be appeals, whether it be request of information, council questions, all of that now has the ability to be AI assisted. I'm curious, Becky, what was the response from the community and data and practitioners around that bond dye experience? People didn't like it because and I can't wrap my head. I think it's because you can't prove that it was individual people. Now I'm assuming that the letters you would have had chance to put your name in and your address, surely. And then it sends it. So, but they're saying is it genuine? And what I heard, I think on the news report and the reporter has actually joined in the conversation on LinkedIn. What he was saying was that the council haven't said how they're going to use this yet, but I'm not seeing it's any different to just normal letters that get sent in. So that the engages are just worried that it's not real, that it's not a real process and that people haven't really thought it through, I suppose. They've kind of checked boxes as to what they're worried about and then it's formulated a letter for them. I'm really curious about that. Just I kind of share your premise around someone's built this thing that has led to activating residents that has led to more thoughtful or more nuanced or deeper ability for them to quickly and easily share what's important to them with their local council. Yeah, in my mind, I think how brilliant of the person that generated this idea for the purposes of activating, for the purposes of generating a broader region, making that process easy for the person to be able to share what's important to them. Fundamentally, in a principled level, I don't have a problem with any of that. This is the kind of stuff that we're going to have to respond to in the same way that councils had to respond to social media joining the conversation. That was something that we didn't have an option to join in on in the end because if we didn't join it, it would be happening without us. Yeah. So we have had many times in the past where technologies have come in that have ground swelled at the community level and that have then come at us without us being involved. I think the opportunity is to go, okay, well, if that's working, how do we build that into our engagement practices? So then that's not led to left to a resident to design. So we can address important things like for the first thing that you've raised, is it actually a resident that submitted that fee? So that comes down to authentication, right? Is it just a bunch of bots pretending to be residents or is it an actual resident? How do we authenticate that piece of feedback? How do we ensure that the feedback within the council's ability to actually create an impact? No point receiving feedback about the education system if we're not the right level of government to be sending that to. So there's an opportunity for councils to actually get involved in designing scenarios where communities can be activated, send more thoughtful feedback through, that actually relates to the decisions that are affecting them at a local level. I completely agree. And so it comes back as we start to need to wrap things up, it comes back, doesn't it, to the core, like engage well in the first place, and you won't be getting this. And I actually, you know, I think even if I'm engaging in a one-to-one way, let's say I'm planning to put a shed up in my back garden and I need to go to my council and get planning approval. I now as a community member, I'm going to have done my research, I'm going to have popped whatever planning and design code my state my council has. I'm going to have popped that into Claude. I'm going to have popped in my shed plans, I'm going to have shared like a street view or aerial view of of my block of land and I'm going to say give me the reason I am allowed to do this and I'm going to be turning up to that planner knowing the exact specifications and everything. And so I guess it's I think maybe Dan Orlyce you said it is leveling the planning field. It's giving us the intel without having to spend that would normally take me five days to do it. And I probably wouldn't because I have ADHD and I zone out, but it's giving us that you know that concrete ability to just rise to it basically is community. So it's coming isn't it is here is happening. Certainly is. Yeah. I think one of the things to point out as well is Cloudflare which is one of the world's largest kind of it's a proxy. It's a web application firewall. It's a lot of things that are in the urge speak, but effectively like they help to host websites or they help to protect websites from bots or or monitor traffic and whatnot. And most of the web I didn't say these days runs through or on Cloudflare. They just reported I think was last month that more traffic is now done via AI bots than humans. And so this is the world that we live in that like this I think believe it is the CEO of the CTO of Cloudflare said this one happened to 2027 and it's already happened with wild pasta and that number is just going to keep going. So I'll just if I can add like I actually think it's really good in terms of from a community perspective that they have the ability to engage more but I feel like we've missed an opportunity or it's helped us highlight an opportunity that we have community members that feel that it's too hard to have this a and for whatever reason that is and they've opted to go down this route which is further upstream of while I'll press a few check boxes and buttons and whatnot. And it's interesting because I was chatting with a colleague and I think we were both clearly using AI and it got to the point that we just both went like do you want to just send the dot points should we just send dot points and be happy with it. And we got really happy with that. We were just like yeah great cool. There doesn't need to be the expectation of formatting is out the window here in my house. And sometimes I feel like it's just like just give me the dot points like it's okay like I just I want to know genuinely and especially given like the the concern of like authentication and whatnot we are getting to the stage where I don't want everything to have to be authenticated and authorized in order to prove that it belongs within the the set of data that is to be analyzed because I think that really I mean that's just another barrier that we could put up in front of community members to have their say in people that I have time or you know whatever it is it really hampers their ability to have their say sorry Becky. But we do have to know that like even if we even if we don't have these these proactive community members creating these kind of like pseudo portals so that they can encourage their communities to do so and again I applaud the effort that's awesome that's really cool from the community members to have that proactive nature. And what a community minded sense as well of like hey yeah this worked for me does this work for you. And even if it's just a existing have you say portals or emails or whatever it is the fact for the matter is it's only matter time if not already here that agents will be submitting on behalf of people. I'm quite a bit on behalf of people if not a campaign as well. Yeah. I think you're touching on some really important things Dan authentication is a difficult thing to achieve and it is a major barrier but it is so incredibly important. Imagine digital elections where we couldn't authenticate people and so the downstream effects of this challenge are extraordinary. I understand that we're in this grey zone at the moment where if we got 50 pieces of feedback from made up Gmail addresses would that actually be a fair representation of what the community is asking for if it's just one resident trying to generate get attention. And the answer is no. So authentication is important and we are going to battle against that and we're going to have to factor that into our design. I think at some point we're going to have to solve this probably not councils individually are going to have to solve this is probably going to have to be solved at a much higher level than just local government. But it is a really important factor in the way that we receive feedback from a digital perspective. And I will leave you with this one thing. I was asked to submit to my council my opinion about the parking signs on my street which I deeply care about. But I absolutely, yeah, I did not submit because the requirement for me to authenticate myself as a resident was outside of anything I had time for. And so there's many issues with this particular challenge. You might get fake representation or you might just not get participation at all. So I feel like we absolutely do need to address this and hopefully technology can aid us in that process. And it's actually not a new thing because this same conversation was happening back when the, you know, bang the table started back in 2007, 8, 9, I was working with them back then. And it was all about why the community need to log in to be able to have their say online. But actually it was kind of authentication because you didn't know who are these people contributing to this conversation. Same goes for commentary on Facebook. Like how do you know that's not just the same person with lots of different, like authentication very important. Look, before we wrap things up, I thought we might just do a little whip around on our favourite use of AI personally. And I'm thinking this might go for the people. These aren't the, that, that was first quarter. These aren't the naysayers that, you know, my daughter's not going to be into this because she doesn't use it at all. Not, not interested. But maybe the people that are dabbling or curious. And Dan, do you have a favourite, you know, I can only imagine. Do you do use it for anything personal? I've got one in mind for myself. You see, that's why I'm asking this question. Yeah, meeting summaries. Oh my goodness. I was thinking about this the other day is I cannot remember the last one that I took notes that I'm meeting because I was terrible at it. So the advent of meeting recorders has just been the godsend that I've been looking for. I was terrible holding a conversation and trying to drop something down and go, I missed three conversations in between. Yeah. And now I can't read my chicken scratchings. Yeah. Oh, I'm really good. I'm really good. I'm really good. And I was actually going to add, in my workshops now when I'm delivering, I do like, my standard housekeeping, you know, I do my acknowledgement of country, I do whether toilets are. And I now always say, by the way, my laptop is taking notes. So I have, you know, it's doing an audio transcript. And so, and I always tell people, and I think that's one of those ethical things, isn't it? Yeah. That we need. Absolutely. Class. Absolutely. Class. Absolutely. Class. But also like artificial intelligence usage. I have customers that don't like, don't want to be recorded and used artificial. And that's fine. Yeah. That's good to know. Yeah. I will say, the tip that I've had is that with all of these meeting recorders, if you jump onto a meeting before the person actually gets there and their recorders there, you start screaming hallucinations, just start going like, hell, hell, I'm on the Titanic and it's sinking because the meeting summaries that come after are absolutely hilarious. That's so good. Lisa, do you have a favourite personal use at the personal work or at home? Ah, too many to mention. I can only imagine. Lisa, are you open-clawing? Are you going that far as to have open-clawing or whatnot? I haven't gone that far. No, but I have made transition to Claude from ChatRbTB. I've never used Copilot. I'm a Claude. Claude is mine. He's mine. We're married. Go on, Lisa, give us one thing you do with Claude. I will give you one thing that I do with every AI platform that has changed my life and that is the dig-tech button. Oh, yes. I use that exclusively. I almost don't type anymore. So I find that I'm able to provide an incredible amount of context. Well beyond what I ever would have bothered typing and it speeds up my entire experience. So it's not unusual to see me at my desk speaking to my computer and not being on a team's call and all my colleagues are very used to it now. I love that. So I don't use dictate. I use the voice. So when I'm in the car, I chat to Claude. Oh, the voice mode. Yeah. So he talks back and he wasn't very good. So I was still on chat GPT but now I'm on Claude because Claude is much better now voice wise. So he's like, "I make Becky." That's brilliant idea. He always says, "Oh, this is amazing. Sometimes he sighs at me." I will jump on that because dictation has absolutely changed my mind. I've just subscribed to Whisper Flow. Unfortunately not sponsored or anything like that. I have in the last four weeks, I've dictated 66,000 words. It is revolution again for emails, for Slack messages, for prompts, and to chat GPT. If you're not dictating, you have to be dictating. I do 147 words a minute instead of my measly 30 typing. It's incredible. It's incredible. I will get the last word on this four week close, this conversation, which has been amazing. Although I need to know how people keep in touch with you, Lisa or come to that. So I say, "I have ADHD and I'm fine at work. I'm focused on everything. My biggest nightmare in life is the supermarket." I guess I overwhelmed with the choice of tuners, the choice of Latin trying to remember. And I've got a list, but it doesn't quite work. Anyway, last weekend it was my birthday and I had a little gathering and I decided I was going to cater it myself. And so I put together. a grazing table, but for normal me, I would be so overwhelmed with that. Just knowing like, what have I got? Have I got the right cheeses? Have I got the right meats? Have I got enough biscuits? Me and Claude went to the supermarket together. And oh my goodness, I took photos of what? I had in my trolley and he was like, oh, he wasn't actually speaking out, thankfully. Oh Becky, I see you've got the E-DAM, I see you've got the cheddar. I noticed you haven't got the blue cheese yet. Make sure you get the blue cheese next. So I get it, take a photo. And so I just did this whole process in the supermarket. It has changed my life in terms of I can now cater for events. That's me. Now, Lisa, thank you so much. I am absolutely fangirling you from here on. Thank you Dan for introducing me to Lisa and to my listeners and my listeners to Lisa. Lisa, how can people stay in touch with you? Is LinkedIn the best place to find you? Perfect place. Yeah. And for those that haven't had an opportunity to download the report, this is available on the allergy pro website. And I'm sure there's a link that we can pop in the show notes as well. We certainly can. And just for the people listening and not watching, I know some people watch some people listen. That report is called. I've got it somewhere in my notes. What's the report called Lisa? A.I. adoption in Victorian local government sector baseline report 2026. And it is a really good report. I had a good read through and it includes all those things we've talked about in terms of the different user profile. Doesn't it? And yes, some fabulous things. So thank you again for joining us. We really appreciate it. It's been a pleasure. Thanks so much.