Evo DK #356 - Why AI Governance Is Necessary In Driving Business Value

Welcome to the Evolution Exchange podcast, a melting pot of ideas and inspiration shared by some of the most successful technical leaders in the world. The views expressed by the speakers on this podcast are their own and not necessarily representative of their organization. Hello and welcome to the latest episode of the Evolution Exchange Denmark podcast. My name is Tom and I'm joined by a panel of industry experts to talk about AI governance. Before we get into it, I'll let the guests introduce themselves. Jonas, would you mind kicking us off with a little bit of an intro, please? Of course, my name is Jonas Björk, I'm an expert manager at Brain & Company and in my day-to-day work, I work with I guess the biggest companies in the world and help them build and implement exciting AI solutions and yeah, I guess that's it. Amazing. Tell me a little bit about how you got to where you are, Jonas. I originally went to America studying international business and then I came there in like the early boom of data science and just fell in love with the concept so I did my second master's degree over there as well and got a job at Google and then I guess it just sort of spiralled from there and coming back to Europe was quite a shock. It was a bit like going 10 years back in time and I guess we'll get into that further down the line. Sure. Looking forward to hearing what you've got to say and now welcome to the podcast. Björn, would you mind going next, please? A little bit of an intro. Sure. My name is Björn, Björn Brøs, I'm the VP of Product Advisory at the 2021 AI, which is a company focusing on software products around AI governance and I have a assistant professor position at CBS Teaching Risk Management and Corporate Finance and I've been in the field of finance risk management for the last roughly 15 years now. I've seen the whole AI governance agenda developing over that time, which I think is particularly now when we're looking to agenda AI and all the LLMs, pretty interesting and companies more and more also not only from an IT perspective but also from a business perspective start to adopt it and I think that's a very interesting field to discuss with a lot of still open questions for the time. Absolutely. Thanks very much and welcome back beyond. Giovanni, are you okay to go next? Yes. My name is Giovanni Leoni and I'm happy to be here on the podcast to have this conversation with the fellow participants. I've been a small introduction myself, I've been deep into business for closing up on 25 years both in operations, development and strategy, both seeing how to improve our conversations with our processes, like mainly making it more data-driven and seeing how to use analytics and AI, specifically the topic of responsible AI for many years. Came into the more focused on the AI governance topic for the past six, seven years and I've seen it from everything from within larger corporates in developing it but also from the perspective of technology and advisory and I'm very happy to looking forward to the conversation today and to see it, to just intern the topic of AI governance with the other gentlemen. Thank you. Amazing. Pleasure to have you. And last but not least, Mikkel. Yeah. My name is Mikkel Cedrusen-Tvinge, I'm currently the head of engineering at MASC Training. On my day-to-day business, I work heavily in development, oversee architecture and implementation and then, I mean, we have very much practitioners of this whole AI-agentic order and how to apply it to our systems and operations. So where you guys are on the more theoretical part of things, I think I'm going to be pulling us towards a more grounded experience. So I'm really looking forward to that. For MASC Training, I was a tech lead at Danish Healthcare and then, before that, I have been a consultant. So I've been in code and in development for a decade, a bit over a decade. So yeah, that's me. Amazing. Thanks very much, Mikkel. And pleasure to have you back on. I think we can see that we've got a fantastic panel with us today and the title of today's podcast is Why AI Governance is Necessary in Driving Business Value. Bjorn, I can see you've got some fantastic questions and things on your agenda. Are you okay to kick us off today? Of course. And I think to kick it off, I think one of the first questions I would like to put out here to the group to discuss is AI governance, often from organizations similar as risk management is referred to as costs and yeah, what is that about? How does that relate to business? I mean, is it just costs or is maybe having proper AI governance also helping a company to flourish and create value? Because if we look at the business value in itself and we look at it as a risk-adjusted value, then probably a thorough governance process can help having a high risk-adjusted value without it, even though that it, of course, adds something to the P&L sheet. Yeah, and I'm happy, I can start off with a comment to that one. It's a really great question because often when you see AI governance coming into practice, it becomes either driven from the risk and compliance team, it can be driven from the legal team, it becomes an almost tax of bureaucracy that is later on top, and on the other hand, we see a reluctance from technical teams to see it as another heavy stage gate to drive innovation and launch it. But I think you're onto something here, how do you bring in both the narrative of business value rather than just being another added cost element? And what I've come across companies that have succeeded in bringing the value element to it is also encompassing value realisation a part of the governance process. So while being able to intake use cases of AI to be launched, whether that is internally developed or procured, they also at the same time of having an evaluation of risks, feasibility, the demands of compliance, the companies that I've seen also bringing in very clearly up front, a part of that process talking about value realisation, talking about what is the benefits of launching it. It associates the governance team towards bringing value to the company, so it doesn't become something disparate to different. It also makes a good balance between, okay, what are the risk appetite towards the value that the use cases can bring, and it can more easily manage going through the process of governance to see if we're not having these solutions launched, what do we miss out from a business value point of view? But I think that's one aspect, but the second thing is also, of course, to see how can the team, overall, the governance team, who member is engaged, be a department of yes, rather a department of no, because I think that's fundamentally a way of seeing how do one associate a governance team of being positive to bring business value with establishing AI, but having an efficient enough process to cater for the governance need that are necessary. So maybe those are the two things that I come to think of on that topic. Yeah. No, I can relate to that, actually, with seeing actually similar things that, I mean, right now in a lot of organisations that is seen as the bureaucracy, the burden to do it, right? No one wants to do it, and the larger the organisation grows, the more you need to do it. And then, especially on what we in finance will call the first line of defence being basically the operational people, being operational management. So the ones that actually do the reporting, they obviously in, maybe it's also a bit of the question of the way it is done these days, but they have this burden of doing all of the reporting. And then, yeah, and then what I think if you look to a lot of people, a lot of developers that I met, there is already, especially in Europe, quite a good feel for where they're like, yeah, it is important to make sure that the model is not biased, that it's good enough, also because they're proud of what they're creating, but this linkage often to making also business understand and making what you're saying, Giovanni, and having it also transparent in terms of what is governance, you know, delivering for the business, that's often a bit lacking and still a bit more the cost element in it. Also, when you implement it from a process perspective, where the risk department is usually a bit seen like accounting, we need to have it, but we're not necessarily like it a lot or we want to have it very cheap, yeah. And as you know, it's nothing bad and having cost control and that, but yeah, I definitely see a value at risk being quite an interesting point of that, because that could in the end of the day also be a good quality stamp for if you do that for your services, especially once you would use AI for outside services. I think I like your way of looking at it, Giovanni, you say it's a cost, but I think the real value in having AI governance internally in the company is if it means you save yourself from a massive file, like we are all in the EU, right, like there's a lot of regulation coming out and it's ever changing and growing and some would maybe say it's been a bit detrimental at least to the innovative side of AI inside of Europe. I think it's not a stretch to say that most companies that could leave have already left, which I think is another cost, right, the brain drain that has happened inside of Europe because of EU's harsh stance on it. I'm not saying I'm against EU having a hard stand on it, but I think the way that they implement the rules could have been done in a way that made those costs less impactful to the region because when they make rules that are so hard and let's say harshly phrased, it scares companies. So risk departments, they immediately find solutions elsewhere, right, and you can always go to America and always go to Asia and the rules just simply don't apply there, which of course has another cost because if you can just have the models run rampant, I mean, if the Gini comes out of the bottle one day and the worst thing happens, you can't really put it back in, right? So we need governance to avoid having to pay that ultimate price, right? I feel like a lot of the cost aspect of AI governance inside of Europe has kind of like the price that we've all paid is that we now no longer have a hand in play. We are at the mercy of whoever comes up with the big model first, right? And I think that's much more scary than if we've taken a lesser hard approach and or maybe even just been better at explaining the rules, because the rules are in reality not that harsh. But when they were introduced, they sounded like it was like, oh, you're killing all the innovation. It's now impossible. And then everyone fled. And of course, that's a lot of cost that we're associated with leaving. So none of them are coming back now because it's just, it doesn't make sense once you have already left. A small comment to that, Jonas, I think you're bringing up something really important. And that is in terms of kind of what are, how have my, how has one approached the burden of compliance in regards to AI? And what does that, how does that impact organizations? And I would say that on one hand, the EU have been progressive and early to the stage, not only with AI, but with digital regulation overall in promoting it. And of course, many companies that seek a principle or approach of least compliance strategy, of course, will try to move and find opportunities, not only for a more lenient compliance regime, but also in terms of the US, there's availability of funding in a completely different level. But I think here is also to see what are like the foundational reason for compliance, whether that being the EU AI Act or the forthcoming harmonized standards coming out of Stenlyck. It is mainly just following the same route as any other technology in terms of standardizing and setting the rules of the game of what is expected on the market. And I think what the reason why it becomes a hurdle at this stage is that we have a bit of an unevenly pace development where we have technology developing much faster than organizational maturity and adoption of the technology in an organizational context. And because of that discrepancy in pace of adoption, we see that when bringing then requirements of compliance, whether that's being regulations or standards, being something a hurdle because it impacts mainly how the organization's maturity can manage that to design smart processes to do it by design in utilizing and developing the technology it had. So I definitely believe that there is a movement of trying to avoid compliance, but if an organization looked long term by establishing processes and ways of working to work with the compliance efficiently and effectively, they're actually aligning with just normal ways of working with technology. And this will be as everyday life going forward, we're just at the early stage of the maturity of the organizations to be able to manage it going forward. So I think you have a good view on it. But I think this is mainly that we're just liking the organizational maturity in Europe right now and we're catching up on that. Oh, I'm sorry. No, please go ahead. No, no, you should go. But I fully agree actually with the maturity, I think, as you also put it, Jonas, when things are being written in a way that might scare people, and not intentionally, but when phrasing like you can take a lot of the governmental things, they're written in a way that maybe not is so friendly to mere morals. And unless you have an expert on your team, it might become something where you'd rather avoid it than actually learn about it. And I think that's something you can see easily here in the EU. I think also that a lot of the tools that has been sort of showcased in the recent years to sort of help people has been more or less a band-aid. And again, then it becomes sort of a last-minute stamp on a project like let's just do some governance real quick so everybody's happy. And it becomes flawed and becomes immature again. And I think that's the issue that I most likely run into the most is about all the way back to even talking about the data quality of things. You need to understand the entire process of AI, and that starts at the culture of people. So I fully agree with everything you guys have been saying. And I think to put it quite provocatively, the cost of governance is a budget line, right? And the cost of non-governance is an obituary. Basically, that's what I had to put on it. I mean, we can laugh about it now, all right. But the reality is we have, and when I say we, I mean, Europe, I have a bachelor's degree in law. So I feel like I can say even for people with a legal background, it's completely embraced some of the stuff that they're putting up. It's so obvious that they have no understanding of what they're trying to regulate. And I'm so afraid that they'll make some kind of crazy amendment of it five years down the line and then just completely kill the entire industry inside of Europe, which so far at least is thriving, right, or at least growing not at the same rates that you see in the rest of the world, right? I think it's, like, I think of GDPR and how they made all of these massive threats and scared every company in the world senseless. And in reality, it just means now there's another box we take yes to before we enter our website. Like, it hasn't really changed our lives that much, right? But it's scared everyone in industry. And if that happens, I don't think our AI industry will come back or recover. Yeah, I would actually also like to add or maybe broaden the perspective on it a bit because, you know, we'll start to look a bit on the regulation and say governance equal regulation in a way or compliance, but we could actually also say, well, governance could already also start without the AI, right, just simply from the perspective of getting into control what is running out there. So if you have a development team that starts to build, I don't know, a couple of hundred agents that interact with each other, there should be a fundamental business aim of having control over it. Similarly, as you want to have control over your firewall and your APIs that you run and all the other applications, right? And then in that sense, I agree that sometimes from, yeah, maybe they did not so like some personas, some organizations that are not educated enough in the space start to make AI a bit of a, you know, different or more than it is, at least for now. I mean, it's not a crazy robot. Yeah, there's some technology that needs to be guided and need to have some quality stamps probably, depending a bit on the application and the act is also following that and the ISO standards are following that. So yeah, but I think AI governance, from my perspective, should already start way, way early and should also, I mean, also when I talk with a lot of companies now, the reason why they're looking into that is often much more business driven, like, let's get control over it and we like to know what's going on rather than any of the regulations, which where anyway, don't really have a clear answer whether how it will be audited in the end of the day, who will approve it, I mean, a lot of a lot of holes still in it. So, so question mark also with regards to when it will actually start happening. Yeah, and to bring a comment both to you, Bjorn and Jonas is that I think kind of one of the biggest challenges is that we are meeting a future of uncertainty in terms of how the technology is being developed, what the capabilities it has and how it meets its interface of application in context of organizations in society. So I think maybe the greatest challenge is that not specifically the regulations or standard themselves, because that's an ambition to set the rules of the game. But I think the main challenge is how to bring that in to make it modern in applying the governance according to it. And I think there is a great need of innovation within the space. You can't, I usually use the analogy that you can't govern 21st century technology with a 20th century kind of way of doing things, because then you'll create the bottlenecks, the limitations to the capability that the technology can bring. So I think that that's where maybe I see that the greatest challenge is not specifically kind of what is determined as preferred or not or the demands for it, but rather the need of how can one govern technology and AI processes and systems smarter. How can that be done efficiently? And as you're on to Bjorn, how can that be done from day one in terms of designing, in determining how to set up both processes and development and procurement in order to set a good base going forward? And I think there's much to be done to mature those practices, but also to mature technology support in order to support the processes such. Sorry, I'm going to have to jump in there, guys. It's been a fantastic discussion so far, and I just want to say thanks, Bjorn, for kicking off the podcast there. Mikkel, I'm keen to hear what's on your agenda today. What is it that you'd like to discuss? What questions do you have for the rest of the guests? Yeah, so maybe following up on this whole discussion, compliance gaps. So basically, in my day-to-day, I kind of often see organizations become compliant only after the fact that a new regulation drops, or it's sort of a very reactive state that you're always in. And when you look at different tools like policy as code, it's still a reactive band-aid, which means that there will be moments where you're actively and non-compliant. And I guess my question is sort of how do you design governance so that teams are never temporarily non-compliant while waiting for policies to catch up? I really love this question. It's in my mind the simplest thing ever. Like there's two ways you govern, right? Either you start outlawing every single gun ever developed, and then you have to create new laws every single time a new gun is developed, or you just outlaw gunpowder, and then every single gun that will ever be invented in the future that uses gunpowder is now illegal. It's not really a new technology in that sense, right? We know it needs electricity, and we know it needs data. So you regulate electricity, and you regulate data, and then as long as the companies don't break those rules, you know they're within the limits or whatever it is, right? But we're choosing not to do that. Instead, we're making up these concepts of like, no, you're not allowed to use this on medical devices because it's not 100% correct yet. And then we ignore the fact that no doctor in the world has ever had a 100% success rate. Like, it's ridiculous to think that a model should never be used until it also has a 100% success rate. If we can already prove that it's better than humans, then why won't we use it together with humans? I think you, Bjorn, are at your hand in person. Go ahead. Hi. Thank you. I took it down again because now I know I have another point, no. I think that there's maybe one thing where I would say, well, probably there's still a kind of challenge with using the AI system, even though it's maybe a bit better than humans. And that will be the responsibility of it and how to manage that. Because if you have one person, yeah, let's take driving cars. We can take the doctor example where you have a doctor making a mistake, right? Heart surgery went really bad. Okay, that doctor is obviously that person is responsible for it. If you, as an organization, you build a system that is an average way better than most doctors in doing that kind of practice. But obviously, you will have an error rate or you will have some unfortunate cases where it will not work the way it is tested or things like that. Then I think there's a responsibility question. You cannot really blame the machine. So maybe you can blame the company that puts it on the market. But if you really do that, very harsh, then you will also kind of limit innovation in a way where an organization will say, well, we're not ready to take that risk unless we have a legal ground and protection for that type of innovation. And I could also imagine now that would be actually interesting in the US case where court cases, you have way, way higher costs, higher fees and fines after court cases. How that will develop ongoing, speaking about innovation and with the otherwise in terms of regulation, maybe a much more, I would say dispersed legal body, which is even on a state level rather than on the whole US level. So I think that's one question where I'm maybe saying, okay, having some sort of regulation is probably not too bad. But I think generally also, and that's also a bit pointing on, building on top of what Giovanni said earlier is also a bit the whole process of it. I mean, there's obviously a technological tool question on how to do the governance, so to keep up with development and AI and things, but there's also a process question on that. Like, how should that an organization be chopped into pieces? So development teams are not totally swamped with things that the last day, let's do some governance and let's maybe copy all of our code documentation into a Word document and send it to the lawyers, which might have worked in the old days where you train the model and they use it for 10 years, but it doesn't work if you have a model where it's still a bit unknown how it will be used or it is retrained every two days or something like that. Yeah, it's a great, great point, Bjorn, and I would kind of want to kind of spin further on it in terms of, like, why do we need governance at all for the topic of AI? And I think what is fundamentally different is maybe on a principal level is that what is uniquely different is that we are, to a higher degree, implementing a technology where we are designing it to be as autonomous as possible, and we're stepping away, figuratively, stepping away from the machine to let it drive processes and where we wanted to have desirable outcomes. And as we're stepping away from the machine to say so, we need to create a layer of control because accountability structures will not change. Currently, we have a legal regime across the world where entities and individuals have responsibilities connected to them. And that will not change for us foreseeable future. We can't blame the machine, which means that we need to retain control, we need to retain accountability and responsibility, which will demand more because we have technology that is highly complex that will be adopted to a higher degree across organizations, which means that we will, to a high degree, need to have a smarter way to maintain control and maintain control for non-technologists, which becomes a challenge of interpretability and transparency for leaders that have accountability, but not the technology understanding to go into the details, but need to be able to control that the algorithms and the processes and systems behave as they should, but also that they deliver the outcomes desirable. But I think we're in a paradigm shift of kind of stepping away from the machine. And that creates a demand also in seeing, okay, how do you create the structures so we can maintain accountability and responsibility while we are adopting this technology? Yeah, I would fully agree with what you said there. And I think the difficulty is now, obviously, so to say, agreeing on the contract or the policy and what you want the machine to do, I think that's in the first place, probably pretty simple because I think everyone will say, oh, if the machine is uncertain, then don't do the heart surgery or if it's a driving car, don't run over some pedestrians and things like that. The question is then, how do you, as you also say, kind of, if we want to control that, how do we enforce these controls in efficient ways so it's not too manual, which would otherwise mean that we need to interfere with the machine ongoingly when it makes the decisions. And that is probably even more the case if we look on these agent-to-agent protocols and setups where the idea of the whole thing is that actually the models are informing each other, making certain decisions and interacting with two links without a human in the loop and without the, well, maybe in the end of the day, some oversight, but definitely not in between the prediction and the actual decision. I think this is where, from an individual company perspective, the real value is in AI governance because this is where you can build systems that stop hackers from, for instance, doing typosquadding or injecting malware into your system through the yellow lens produce code. And I think there's a lot of good practices already, but we haven't really found the solution yet. Like, right now, as we're recording this podcast, there's a worm that's, like, it's replicating itself across react-native libraries. So it's now spread, I think, as of this morning, it was, like, 800 different libraries. And it's like, whenever somebody uses these with access to other libraries that are also really popular, then the worm spreads. And all of this is essentially an example of how you could also inject LLMs, because if you give LLMs execute permissions on your server, now it runs code. And as soon as it runs code that it writes itself, this can happen. Like, it can happen through something as simple as typosquadding, which I think just for, like, simplicity sake is when an LLM very often makes a mistake and calls a library something wrong, and then a hacker will use that name as his own library. And then when the code runs, of course, instead of pulling the correct library, it pulls the malware in. These sort of attacks will become a lot more frequent, because frankly, they're not difficult to make. And they're to me super scary for a company. Like, if you build something that actually runs as one of these agentic systems, and all of a sudden you're compromised, because the LLM itself creates code that you not necessarily can see after it's run it, right? I mean, you can see it in the logs. But if you don't think there's anything wrong, why would you go and check? I think that's where AI governance really has a lot of value for companies, because you need to have systems in place to catch these things. Yeah. Yeah. You need to probably catch it on that note, so that, for example, you brought up the example of these libraries, right? You could, for example, say something like, oh, you're whitelist only certain libraries and the rest of your blogs, so if it's misspelled and weird library, then that is prevented from entering your system. But it also points out a bit more general question. When we look at agents, there are two things that you should maybe think of controlling, and one is the actual action that is generally allowed to perform with whatever software tool is connected to this system. So for example, is it allowed to make me a travel diary with suggested flights? And yes, maybe it is. Is it allowed to actually book the flights and judge my credit card? Or maybe not, or maybe not for everyone, and then the other thing is, and that's, I think, because that's just purely automation, right? I mean, that's not necessarily that different from having some generally automated systems that do things connected to each other, but because we have this AI element in it, we also need to make sure whether the AI is interpreting the signals that gets correctly. So if I, for example, write, and I'm not really agent to agent, but more MCP kind of style, I write it a text that I would like to go to Austria, but I actually meant Australia, and the agent then books me a ticket to Austria, all that's cheaper than Australia, it's not too bad, but if it would be the other way around, that would be probably really bad. So how do I, throughout spelling mistakes, also generally hallucinations of the model get there and misinterpretation, and the actual action that should be performed on the end software, that's the second thing that needs to be kind of controlled, ideally from a mixture of engineering setup, and then on the other side defining the right controls and policies from business. Is it too simple to kind of associate everything with a process first sort of mindset? I mean, AI agents are built around processes, right? And if you think that each process has to be digitalized in some way, and if you take a look at that process from a maturity sort of perspective and say, is this process ready to be digitalized or not, then you can also at that point have a, you know, a minimal very lightweight, but big enough sort of discovery phase to take a look at that process from a governance perspective as well, and look at it and say, what should this process be able to do before we actually digitalize it? I think that could be a pretty okay way to go with it. I think it's also how most people do it, right? Like there's either people do it with having a human in the loop, or they make like fixed rules, and it cannot go over a certain spend limit, or it cannot access anything that isn't on this IP list, or like it's the only way to really do it at this point, right? But all of this takes into account that it is doing what it is supposed to do is just doing it wrong, right? Where I think if you have malware involved, now it all of a sudden gets to a point where they can maybe even turn off these limits. And all of a sudden you have an LLM in a chatbot that's selling cars for $5, which has actually happened. It's ridiculous, but if you build your system without enough AI governance, someone will take advantage of it, just like if you build a website today and you don't have a proper firewall or whatever we want to call the cybersecurity that we put in place, someone will take advantage of it. Like that'll be the same for LLMs in the future. I think these are all really, really interesting topics. There's some fantastic input here, and I just want to say thanks, Mikkel, for keeping the podcast going there. Giovanni, you're okay if we come to you next for your agenda. What topics, questions? Happy for that. We have already covered a lot of ground already on areas where I have interest. And I think one of the things that I want to kind of spin further on is the last area where Mikkel touched upon. And that is specifically around, it's on the element of maturity of the organization. I think there we're underestimating the maturity of process, understanding and insight with organizations. I think we have been in a place of where we have, of course, a practice of incremental development and fine-tuning our processes within organizations. But in the light of what the capabilities that AI systems can bring, we haven't properly redesigned and re-engineered and created the good foundation to really take the lead going forward. And I think there's a high necessity of seeing the foundation of proper AI governance and proper AI adoption really being foundational process re-engineering in the face of what can be done. And that will demand also from a people point of view, what should people do? What are the interfaces they should have? And what is purposeful from what people should do and what technology are capable of? And I think we have a way to go in terms of taking that forward. And I think we have a great limitation of adopting AI and adopting AI governance, kind of standing on the maturity of process maturity and process re-engineering. And so, yes, a comment to what you touched upon, Mikkel. The topic that I would want to kind of proceed with is specifically on the maturity of AI governance. What I've seen across different sectors and geographies is that there demands a lot in terms of maturing from a people process and technology point of view to have efficient AI governance. Having leadership involved, having high awareness within teams, but also having specialists from a technology, risk management, compliance, ethics point of view being engaged and some from a people point of view. And I would say that it starts, of course, with leadership, but it's not until the specialists can come together where it actually sees a fruitful and progressive approach to AI governance. On the process part, it's tightly connected to process maturity overall. But having an AI governance process that is mature enough to be efficient, to cater for internally developed and procured solutions, being efficient enough to be tiered in terms of risk management to see that it's light for what is low risk and more substantial for those that are of medium and high risk. Being able to be well integrated across the organizations and cross functional teams is something that usually takes time because it demands more of the organization. It demands more to connect teams across cross function. And I think here's where AI governance is hard because it's not only a technology domain. It becomes social technical when it demands more of cross functional teams and it demands from non-technical participants to be engaged. In terms of the technology aspect, what is critical is building on the great foundation of data governance, being able to see that we have good control of whatever data we're managing today. Furthermore, there's building on the capability of operations around AI and to see that we are having that being in control of what we're using in terms of AI systems, whether those are procured or internally developed. But I think we're, be able to properly mature AI governance to the next level and from a technology point of view, is also being able to have that oversight and overlay of bringing in also the non-technologist teams, whether that is being risk management or compliance or ethics and the non-technologist business stakeholders and leaders so they can have the necessary oversight and control to be fully accountable. And that takes capabilities from a technology point of view that have not been introduced at this stage because we haven't had that level of AI adoption and AI scale going forward. And I'll hand over to my fellow colleagues to comment and reflect on that. So for me, the scariest one, at least internally in companies, is definitely the people aspect because I 100% agree that people are so far away from being even at a level where I would say they've adopted any part of it really. Even if you use the chat GPT chatbot every single day, I don't think a lot of people are aware of the capabilities that you can do outside of a chatbot. And I think if we compare it to when hackers use more traditional means like phishing emails or spear phishing or whaling or any of these concepts, people still fall for them today. Even though emails and spam emails have been a thing for decades. But imagine if one of those emails now has a voice recording of your boss telling you in some detail what you need to do or asking you to send him a password or there's a lot of very terrifying aspects of this right where we're at a point where people need to understand that even if it's a video of him telling them to do something, you should still probably verify it through a different communication means that it actually is this person wanting you to do this. Having those sort of processes instilled into people's minds in a way that they actually know that they have to think this immediately after they see these things, I think is going to be really, really important and I hope we find a way to get more widespread adoption than we have had with let's say email or even like just the internet. If I sometimes I get questions from my parents or even my grandparents and it's things for them like have you even attempted to Google this because it's like you would get an answer immediately like click a random thing on the first page and you'll get an answer to your question right but they still don't do it because it's not top of mind to them and I think when we get to have a world where fake news is now also fake video news and the videos are longer than 10 seconds and they are let's say coherent there's no fingers missing or something that's a scary world if people are not adapted. I do think that the scary reality of it will force people to adapt faster but I also think especially inside companies, we really need to be vigilant with upscaling our employees because giving them access to chat GPT or a local LLM is not going to prepare them for all the other things that malicious people can do with this technology. I think that there are really a lot of good things you're mentioning there from my perspective they're all tying a bit into the creating this awareness and the sensitivity of what is what is possible and what you should do and not do. I would like to also add a bit of point where coming from the financial sector I will use to always say oh yeah but the financial sector is pretty far ahead in terms of risk management of quantitative models where you could say AI is kind of related to that. Now with the speed and the change in that regards a bit the slowness that I can see in my both the institutions I used to work for and now the institution I work with it's often too manual from a process perspective and it's because of the increased burden of some compliances and internal policies also becomes too much so it becomes actually more more blocker than an enabler in that regards and you could say the measures taken for controlling our eyes often very much old school so it's a bit like asking a lot of questions to the developers and they'll think oh I already documented that 10 times in my code and instead of having an automatic solution to extract that information what is actually there. Now I think that's also a question in terms of implementation because on one side obviously no one likes to do that and we should find a more efficient means of how to enable that through automatic tracking, automatic gates and things like that but the business community both not only from a user's perspective but also from ownership perspective of these processes and these decisions that are taken in the end by the system they obviously want to have some transparency and I want to have transparency both in terms of what's going on as well as in terms of accountability and that's something to build into along this process by hopefully making a bit leaner than what it is and more modern in terms of you think it was you Giovanni mentioning it with the old school tools so don't do it with paper maybe and the word document for me is pretty similar to paper even though it's digitized so maybe there will be maybe some things that still need to be on paper in some sort of sense but other things can maybe happen through certain metrics that are collected from the systems it could be through certain configurations in gateways or whatever you could imagine and then tying that into the process and having some logs around it that can then be visualized as a reporting standard. I apologize I'm going to have to jump back in there it's been really interesting stuff so far but Jonas I'm aware that I haven't officially asked you for any of your topics questions just yet I'll let you take over from now and you know guide the rest of the the red sorry the rest of the podcast okay and yeah I think so I published the book last year well my publisher published it but I wrote it it's called the early career professionals guide to generative AI and in this book I talk a lot about what the future is going to look like and how let's say you I don't think that you should be as afraid of this technology as you might think if you watch the news nowadays so I'd be very curious to hear you guys's perspective on this and if you open a newspaper today or honestly watch anything really it's very often mentioned that like AI is coming and it's going to take all of our jobs do you guys actually think that's a realistic image of what's going to happen. I think AI is like it's a tool like anything else it'll take some jobs and it'll you know produce some new jobs it's like as Brian mentioned with the paper once we transitioned away from paper we had word and people were afraid of I guess word so so they were stuck in the paper world and that of course is a lost job on on on the record right but but no I I don't think that AI yeah I mean I guess it's a it's a perspective sort of thing because of course some jobs will be lost but I think that's a very natural sort of progression of any new technology it's again back to the whole adoption if you don't change you know with the with the tides then yeah then then you'll be lost in the in the history so yeah maybe it's a bit I don't know pragmatic view on it but I'm not really afraid of my job for example or any developer job per se I think that yeah AI can definitely build a program for you but if you know just a little bit about what you're doing as a developer you can also easily see where the gaps are and then again AI becomes a tool to help you build boilerplate to help you you know investigate or read upon things and then you you know double check it triple check it and you know it becomes a tool to to effectivize your day-to-day and if you know how to you know power that tool then I think you're pretty safe I really like this answer because I agree and I think the reason I asked the question is because this whole narrative about AI governance I very often feel like the purpose of it from the EU side often feels like they feel like they need to protect these jobs and in my opinion I think that's like saying we want to stay in the stone age like people will find new jobs and I think one of the biggest industries that are probably all going to lose their jobs is stuff like transportation and driving and these things that we already now have self-driving cars in San Francisco and Texas and like they are perfect like in the sense that there are no accidents almost right and they are significantly better than humans so I don't think AI governance needs to play that role but I feel like a lot of the governments around the world are actually playing that role actively especially inside Europe but yeah I'll let you to the next one yes and I can happily spin further on on that reasoning so I think there's it can easily seeing regulations can easily be seen as kind of wanting to preserve what is I want to kind of control the kind of to keep it as it's been successful so far but I might take in my interpretation of not only the EU AI Act and the forthcoming harmonized standards and neither on the digital regulation is rather that the EU has the ambition to steer the development so it will see we'll see progress going forward and to answer your first question Jonas is that I definitely believe that we will see a lower change or rate of change in the short term but a massively greater change in the long term so what we will see over time is a radical kind of implementation of technology across all elements where it can be automated and towards that long term from perspective I think the main question we need to ask ourselves what should we do and what should we contribute as as people in society not only as consumers of products and services but also as contributors and creators to work and value that needs to be done and I think why this maybe is even more important for countries as Denmark or Sweden or any of the Nordic countries is that we are in the periphery both in terms of of of amount of people we are the power of purchasing power generally as countries which means that we are more vulnerable to a global market where where work can be executed in the digital realm anywhere in the world so it becomes even more important that we steer towards what are kind of the ambitions we have as what we want technology to do and AI to do in the future and where we would want to maintain a certain level of control of what is desirable or good but the change will come slow at this first start and I think we it's mainly to see okay where do we want to land in 10 or 20 years in terms of this optimization journey we're on yeah I would I would also maybe add to your question Jonas that the I think it depends a bit on who you are so first of all the what the in game will look like I think no one can really say that and you know it can go slower and can go faster and I think for now at least in the in the next couple of couple of years maybe it's it's going to be a bit unrealistic to have only only robots working for itself and we're living basically a perfect capitalistic world but the only thing you can do is owning a robot that works for you um that being said though even though we'll not necessarily have that in the next next next next decade yeah maybe I'm wrong but from a personal perspective I could definitely see people seeing that their jobs might well if not disappeared and change quite a bit so and and what I mean is that that maybe you have picked a certain job because you you because of the type of work you do there so let's say you now Amigo you said developing yes I agree it's it's probably not as simple as making a quick dash up with a with a LLM and that's it but maybe there are a lot of coming up a lot of more tools that that will do or will change the the way of working with programming quite a bit yeah no and then you can decide you know are you then the controller of the tools that reviews the outcome that improves it that there's thinking more creative giving the right direction of it if you see yourself and that as your job then you will that will be probably secure for quite a quite a long time however if you saw your profession more and actually writing the code hands on actively and typing then and you're a bit reluctant into the change and you don't want that then it's probably going to be a bit more difficult and you can then now copy that kind of pattern to a lot of other jobs as well where I would say that the the more creative elements of the jobs there will still still be there and you probably have much more interactions with kind of assistance agent tools whatever you might call that and and you know you need to start learning this and then making use of it for for your own personal endeavor yeah I think that sort of wraps it up beautiful all the way back to to people and adoption rate and maturity right if you're not ready to adopt the change then I gotta say then I think you're gonna be you know looking at your job I fully understand what you're saying with the whole do you want to write the code or do you want to produce it in a way and and I think that for me it it comes out to the same thing it's it's the journey of you know designing it figuring out how to to build it and then do it and that doesn't have to be the same way as I did it when I started it's perfectly fine that that changes and yes then my job changes too and it's adopt or you know get left behind I think it's a bit harsh but but I I feel like that's it's a pretty fair statement to say that that times change yeah I think it's about maturity yeah and it's it's probably not not now we'll talk about AI and how that changes jobs more mostly white collar jobs you know there are a lot of jobs that went through similar changes before you know if you build the cars and you really like to build cars right that changed quite a bit from when you probably started in a in a kind of you know I don't know 50 50 persons factory thing and then at one point you stand there with a with a couple of thousands and you do one specific thing and then you you can decide what what you like more yeah or if you invest in stocks you went to the stock exchange a lot of loudly noisy conversations with people and trading stocks now you sit on a desk doing it there are some programs helping you which maybe some enthusiasts didn't really liked a lot but in the end of the day that's that's the change unless you're in New York they're very much still yelling or in Chicago with commodities yeah yeah no but I really I really like your answers and I agree obviously I've asked this question to a lot of experts and everyone agrees and the thing that scares me is if you ask experts 10 years ago they also all agreed that the first jobs to go would be blue collar jobs and not creative jobs and now it's like we have some very very good creative tools coming out immediately right and you could argue that they're only coming out now because because the fact that they said that those were the hardest ones to make 10 years ago they got the most funding and then they got developed first right so maybe it's a bit of a they all predicted the same thing and therefore they were all wrong I don't think there's a big chance that we can all be wrong now I think there's a pretty vast alignment across the board that like if you really want to become a doctor then you should still become a doctor even though being a surgeon seeing as you always do the exact same knee surgery for instance a robot would be able to do it a lot better than you I very much still and this is probably actually why I wrote the book originally I still think you should choose from your heart if you if your dream is to be a developer or your dream is to be a doctor then do that and then you can still be close to that even if you're not the one doing the surgery in the end you'll still be maybe your teaching maybe your guiding maybe your I like the word you use migl directing um but yeah I guess that's that's it over to you Tom yeah I think on that positive note it'd be a good place to wrap up this podcast um I just wanted to say thank you to you guys for for taking the time out your day to share some of your ideas and your opinions on this subject it's been really interesting listening in the background there once again the guests on today's podcast have been Jonas, Bjorn, Giovanni and Mikkel if you are hiring for new technical roles or looking for a new role feel free to get in touch with us here at Evolution or if you or anyone you know would like to be featured on a future podcast you can drop me a message I am Tom Catherwood and you can find me on LinkedIn or email me at [email protected] or visit us at evolutionjobs.com/nordix/ one final massive thank you to you guys it's been a pleasure hosting you but for now bye bye