Evo Finland #80 - Thinking About The Problem Before Creating The Solution

Welcome to the Evolution Exchange Podcast, a melting pot of ideas and inspiration shared by some of the most successful technical leaders in the world. The views expressed by the speakers on this podcast are their own and not necessarily representative of their organisation. Welcome to the Evolution Exchange Podcast, my name is Luke Vickers and I'm really excited to be joined today by Robert, Marcus, Mikael and Nico and we're going to speak in about properly thinking about the problem before building the solution. Before we go into the podcast today, I just wanted to hand over to the day speakers to give an introduction to themselves. I'd love to come to yourself first, but he's Robert. Yes, hello. I'm Robert and I'm a strategic designer part of Corners' Genie I Lighthouse team and my main role is the identification of new use cases for where we could leverage AI and then test these with the users. So I'm happy to be here today. Thanks Robert and Marcus. Hello, I'm Marcus Seppala, head of data and analytics and AI at camera working for the IT function in Helsinki, a long history in camera working with data and the evolution management architecture. Nowadays, as my title says, I'm working with data AI and analytics and especially very interested in the design element of developing those. That's right. Thank you Marcus and Mikael. Hi, I'm Mikael Motishimi, Data Team Lead at Bethelor, Circular Economy and Mathematics Technology Company. My passion and startup heavy background is enabling companies to get value out of data, analytics and AI across the board. Other domains I've been working on include cross-platform audits, measurement and use analytics. On a personal note, I'm Finnish German, bilingual in the search integration. Yeah, thank you. Thank you for having me. Looking forward to the conversation. Thanks Mikael and last but not least, Nico. Hey hello, I'm Nico. I'm the CTO for S-Bank and being responsible for the banks' technology and, of course, making sure that our technologies stay competitive in the long run and also ensuring all the time that our customers' money data and identity are in a safe place and we are worth to trust that our customers and the society trust on us. Brilliant, thank you Nico and thank you everybody. And to kick off conversation for the day, I wanted to come to yourself as Robert and ask, how do you make sure you're solving the right problem before investing in building a solution? Yes, thank you, Luke. Well, I think, first of all, why it's important to start with understanding the problem is that quite often people tend to jump into the solution, as this is maybe also the more fun part to kind of create something new. And I think especially in today's world with AI, people quite often think that the company just use AI to solve this and create something that works and it's used everywhere. But even more so, it's important to really understand what is the problem we're solving for who, what are the experience and what are their goals, what kind of tools and data are they using. So I would always say that we call it deep dive before we start to build anything. We do a deep dive in which we would interview key users, understand how they currently work, what is there as is situation, so to say, what are the experiencing and what kind of pain points they have and who do they work with before we start to develop anything. And only then we can start to ideate that what kind of solution would make sense and what kind of technology would actually fit that specific use case. So to really assess what is it doing for who, is there a business potential, does it make sense for us to kind of invest in that specific use case and is maybe also the organization ready for it. So really trying to deep dive into understanding the user and also involving them from the beginning. So I think the real process is to start testing and iterating together with the end user as soon as possible to learn and kind of pivot the path and continue to do so. So yeah, that's one of the key things to start with a deep dive. And one thing we do at Kona is also what we call a proof of value. So once we have defined, okay, who's the user, what is their problem. We then kind of go into a proof of value, which is a process that we quickly test and build a small version of the solution, a prototype, so to say. And then again, test it with users to collect feedback. Let's see my go. Go ahead. Yeah, thanks. Thanks for the work. This really resonates and sounds really, really good. So don't build anything that doesn't make sense. There's no value behind that. So question, question. So what initiates the spark? So when in what case do you go for the discovery mode? So from where do we get the idea that there might be something interesting? Yeah, good question. So we kind of have to to funnels that we use at the moment. One is that we have an online submission portal where people, mostly business units, can actually submit ideas where they think or problems where they think they could leverage a new technology or where they require a solution. And secondly, we also do discovery workshops. So these are workshops. I facilitate with different teams to kind of show examples of how this technology could be used. What kind of problems are being addressed to kind of together with different stakeholders in the organization? Find opportunities. And we then try to bring them to a certain level that we can actually compare them to each other. And so, okay, which one makes most sense to start a deep dive? Because of course, it requires resources and time to then investigate that specific use case. Thanks. Yeah, I have a question, Robert, for you as well. You mentioned about the proof of value. And throughout the years working IT solutions, first we were emphasizing on the POC, the proof of concept, but we're gradually moving through proof of value here quickly for in your own terms, describe the differences. I would say maybe like a POC or proof of concept is really trying to test and validate the technical feasibility. So trying to create something that you can validate whether it's technically possible. I think a proof of value takes you to step further in the sense of that we actually test it with users in a somewhat real life context. And then we also set some KPIs to track whether we actually get the desired business outcomes as well. So I think it in comparison to proof to a POC, which I think would be a part of the proof of value, is that you would not only look at the technical feasibility, but also the desirability and the viability. So trying to get a bit more complete understanding of what is it that we need to do and a proof of value is still in the early stages. So it's not a scaled scalable solution afterwards, but it would give us good insights that doesn't make sense to continue investing into this solution. Or do we need to pivot or maybe even kill it? It's another option. Right Robert. I put a question about risks. So I come from the financial sector and we need monetary risk really on a daily basis. And I'm really focused on that. But in your universe, what do you do? So how do you manage risks and evaluate when you innovate and find solutions? So what methods, what are your approaches towards managing risks with new innovations? So a good question and also the easy one to answer. I guess there's a lot of dependencies on other experts as well. So I focus more on the user and the business side. And of course, when we select the use case, we always consider what kind of business impact could it have if it goes wrong. So that can be for like from, let's say when you're creating a tool that helps the ordering system that it would go to wrong orders. But then again, there's also the data side. So we would work with the cybersecurity team to make sure that it's safe that we do an environment, this initial testing with the right authentication. There might be from architectural point of view or so it's really use case dependent that we need to see. Okay, what kind of risks do we have here? And who would we need to involve from an early stage to ensure that we we do something in a safe way? Of course, this is always difficult because it can really slow down innovation. If you do everything according to the book, especially as we work for a large corporate, Konaz very strict rules on how everything needs to be done. So we need to see how can we do things in a safe way, but still got learnings fast. So it's it's a game of decisions. But again, the most important part here is to include the right people from the beginning and to plan it a bit ahead so that if you have any dependencies that this has already been like a agreed beforehand how they will contribute to the product. Brilliant. Thank you everybody. And Marcus, I want it to come to yourself next and ask is design thinking and process getting less or more important in the new era of AI and agenteic development? Yeah, this is a it's a good one. Kind of a bit overwhelmed by the AI now coming in after 23 Gen AI to the more wider consumption or consumers and it's it's landing to the organization. And I kind of was went through the digital transformation. there as well and there was a lot of hype out there, everything needed to be quickly on agile methodology built prototypes, POCs, most probably some kind of mobile app needed everywhere and then robotics as well and maybe we didn't totally succeed over here. I see the similar kind of error over here but now the hammer is much more clear and it's powerful and we're trying to have that hammer basically everywhere and maybe overcome the issue what we're hearing that AI, POCs or POVs are even failing. What's the kind of a problem, problem out there and I think we should be even more careful with really understanding the problem and now we can quickly, prior we had the mobile apps or local platforms or somebody good coder could build or whatever solution quickly and it do in some cases prove to have no value at all. It looks fancy with AI you can do even even fancy your things quickly nowadays and building out agents that make no sense so I think the kind of a danger of overwhelming us with unnecessary flado gimmicks that really don't work we need to pay much more attention so this is why I'm listening to Robert out there I think the systematic way of kind of pushing back a little bit on the hype of quickly develop something to have a very structured process which is underlining the value itself from the beginning it's very important so you need to understand what is the process you're attaching to how it's actually working you shouldn't be maybe sticking into that this approach we can't change it I think in the era of AI we should rethink our processes as well what are the opportunities not to just be able to solution and make some or kind of a correct something that should be maybe rethought with AI capability so all of this is very important I feel that we are we want to quickly get something out since everyone else is getting something out and we're kind of missing the the bigger point where design could be addressing the solution so that's that's maybe my we could take on that one maybe Robert you maybe stepping in at Marcus you mentioned it briefly so we're getting more and more tools that actually as a designer you can like grow a step further than the past it would be fully dependent on on developers but now you could use tools like Cairo or a cloud code to as a designer actually take a step further and already develop a certain application so how do you think this will develop over the years do you think that in that sense design will play a bigger role or do you think that just the responsibilities will shift somehow or what's your vision on that yeah that does a very very good point and actually the prototyping itself I feel that part of the POCO POB when you're kind of visualizing what could happen your mimicking the project that with the initiative with the demo I think there's a great of value especially when it's done with proper kind of a design analysis first and I think these tools are now providing a lot of opportunities out there they provide a powerful way of quickly building up something that we reflect as a solution as you mentioned those tools and I think we should take them into account I think the toolkit for designers is now expanding that's a great point but that when it comes to the final solution we should be very you know careful how we're how we're going to be building it and over integrating it I guess the lab vacuum work is easy to do and and that's maybe where the designers also feel more comfortable even now and be more powerful but when we're really deploying it to organization that's something we need to be careful it's something right yeah thank you Marcus you mentioned this kind of re-sinking of of the process so how how to do that so so understanding there's like a lot of very brownfield processes that are there and they work at the moment so so how there's any thoughts like how to go about this so from the technical side I've seen kind of solutions where you have like agentic workflows etc but how how to actually tackle an existing process and get that kind of to the next level any thoughts on that one I think it boils down firstly that good question tricky one I try to wiggle around here a bit so kind of a need to first be very understand what the processes like and and I think the design methodologies and whatever the tool it is it's provide provide the tools for it and maybe you would need to question much more now the human aspect over there so with robotics it was very much of a narrow scope we could optimize the process it was quite clear we could optimize a lot of things but they need to be very strict now we could be much more I would say inspirational around it but there might there will be boundaries of how much you want to apply AI to today intelligence part for example if we talk about customer service how much you would need those customer service agents who are the and in the front and I'm discussing with the customer how much you would like to bypass them and have them from instead of being the ones who are looking at was this order or request the order correct and before putting it to ERP making making the decision could you minimize them work and and just have them as inspectors of taking care of the outliers but at the same time engage them to the more valuable work like cross an upselling of of these products with product knowledge for example maybe that's part of the process but also role roles could be changing or you could do less people even better yeah I maybe want to build on that Marcus I think we need to re-evaluate a bit how we you like think about this technology and how we apply it because in the beginning it was often chatbots or kind of these knowledge retrieval tools that would help people do their job better or help a certain step in the process but I think now as we're moving to this agentech AI where these tools can plan and reason and take action on behalf of the user we need to actually think a level higher and not think about a role or specific process but maybe do we need that role altogether or what is it that we actually want to achieve and does it cross across across different platforms so it's a bit more scary and I think it has to sometimes challenge the current way the company operates and how it's maybe structured across different business lines that doesn't make sense how we have currently structured the organization or do we need to rethink what kind of roles we have and how the tasks and responsibilities are spread around so it's a really difficult question but I think there lays the true value in this new technology to really rethink how we do business and to revolutionize it from that kind of angle. Yeah to build on what Robert just said so I also think that the AI will help us in many ways and it will make the kind of the cost of trying out new innovations, new design and solutions much easier, much faster, much cheaper and I also think that it will change some of the job descriptions to roles how do we do things overall but then what I'm thinking is that it's helping us just to do more like mediocre solutions so how innovative it can itself be as a tool and then how do we actually utilize it to create more value and not just more noise so so how do you want to go see that do you share the view what's your take on that. Yeah good one Nicodera this exactly what we are in our company as we're assessing now the capability to say the low-hang improvements are kind of harvested and there's intelligent automation is something we we're focusing of course and in the times of IP struggles with the economics but but the where should I apply the AI first and what I think we're coming back to the the design part over there as well so that's that's a key of understanding where where we should prioritize our our efforts but is it is the vehicle you're saying that we're a little bit like we're a little bit missing maybe some elemental points with the capability to say or the company's in general they they can't find like the ground jewel is that something or we're we're working with mediocre things is that something you're making Nicodera or saying here. Yeah basically my thinking goes that the how innovative the AI actually is because it's just utilizing the information what it has learned and and read through and then making some statistical guesses out of it and of course it has superior capabilities to analyze the vast data masses and find trends and patterns that humans probably wouldn't find but then when it comes to innovation creating something new I'm I'm finding myself thinking that how far the current version of the AI can actually take us because I think the machines are still machines at the end of the day they don't care and they just follow the rules that they are set to. And then, at the end of the day, the AI could make more room for humans, be more humans and innovate together and find something new, something creative that can only be created by humans, looking at the problems from different perspective, from different backgrounds, and then finding solutions. And then we can use AI to kind of try out what the intelligence of those people have kind of created as some of their wisdom. So I think that's a bit lengthy. explanation or answer to your question. That makes sense for you. Maybe quickly, and Robert does well talk here. You hit there. I think there's a lot of expectations that if we would now build a megaprompt on how our company should make a new business with 500 million euros yearly. And we're expecting these kind of things to happen. I think it's kind of a tool that is firstly, it's taking the robotics to the next step. With intelligent automation, I think AI worked fine and we should find those core workflows to work with a little bit boring, but that's kind of a main job, I would say. Then it makes a thing expanding our thinking. So whenever we do innovation, whenever we are trying to do business creation, we need to understand and have more information data to support our hypothesis. That's where AI is still under utilizing complex. There are bright individuals who are utilizing it just a chat or everything, or building up some agents over there. I think we should be doing that a lot more, but it's not going to be a result. I think the humans are still there. They need to idea it. AI is just mimicking the old information, as you said. That's also my thinking there. Good point. Yeah, I want to maybe build on that. I think AI works as well as the tools and data it has access to. So if you have many automations in place in your organization and you make it in such a way that an agent could actually access those, what I think you're doing or what the true value comes is that you make it accessible and easy for the user. So they may have one place where they interact with, which would take action on their behalf on different systems. And what you're actually doing is accelerating the output of one specific human. So instead of them needing to take care of all kinds of back office tasks or maybe also certain expertise that they don't have. So dependencies they have on other individuals, you actually empower this specific person to take steps and go forward a lot faster. So you make sure that people focus their effort and their kind of expertise in the areas where it matters most. So could it be that they bring the empathy to a design, but they use the tool to kind of make the 3D modeling and then they are the ones that check it up, but they don't now no longer need to ask for different individuals to help them in the process, which delays it. So I think it really empowers people to be faster and perhaps also more creative to focus on the areas where they find most fun and energy into. And so to make, yeah, my thought on this, what it's more of an enabler. Just thinking along those lines what Robert mentioned, is it then so that better than an agent would report to an individual person, right? And basically be a helper, helper for that person. Thinking about data analytics and AI in fast evolving environments, what works so that the right problems are found owned and sold continuously. Yeah, thank you, Luke. So it really depends on what type of problem you are solving. So are you solving for scalability efficiency or are you solving, for example, a really hard R&D problem in a project? So what we have in the context of R&D, we have or much of science project team at the data scientist working really closely together. So that means that that problems surface, we see the project or that surface, we see the project can also be solved there and because you have this kind of expertise working together, so kind of you might come up with different project problems altogether. So that's kind of the one aspect. Another topic is kind of if you have developing or green field processes that come up up there, then you have of course a crawl walk run approach to building these processes. Don't overdo it. But it's a massive missed opportunity. If you now don't look in what is possible from the technology viewpoint with a bit of forward-looking view and for that you need again, you need the business expertise and you need also technical expertise working together. And then on enabling and surfacing problems, something that we have found to work quite well recently is these activities. For example, while coding tools to give people actually the opportunity to test this out, to surface some some problems. So it's much harder to describe it in text form. For example, if you want to have let's say in data science, you want to have some feature engineering, it's what kind of features to any it. If you have the UI, somebody thought about it, it might be evident already from that. Access might be a data model, et cetera. So that's kind of clearly useful, but in the end it go comes through collaboration. So that's important. You need to have that in place. Then of course you have like your technology stack behind that. How can you use it? And your balancing short-term impact and longer-term stability and capabilities, especially now also that kind of the capabilities are evolving really quickly. And I would say that one of these kind of real pressure tests that you have for a tech stack is if you undergo like a company bigger pivot. So if a large part of your problems actually change. So what can you still utilize and what still works? I can give one example. So in late 2010, internally takes we originally built like an audience measurement BI solution as a product and turned out that wasn't what the market wanted. They wanted basically bespoke custom insights and data feeds. And the way our solution was built that was really painful to do these things. So it was quite a quite a big job to work out the solution very good to utilize the data and work with that nicely. So I would say lessons learned is keep things more large and allow for different use cases where possible from the get go. And then also build for the proper scale. So if you're kind of think what Robert also mentioned the frequency points of the frequency is if you know that it won't be massive then don't overdo it. So keep it simple. And also one thing is kind of this build or buy solution. So in certain cases a point solution that solves a given problem right away can be the right chose. So that's that's quite clear. And it might be different if you're a bigger company but if you're in a sense you're evolving then then a point solution might be the way to go. So maybe you want to have a substrate that allows for problem solving it develops with the business and the technology. And when you have a new problem so you need to also have an owner. And I think this is also in line with what we heard heard today. And if you don't have it then you're basically built tech depth technological depth from the from the get go. And especially now if you have this kind of tools where it can pop up a lot of new apps quite quickly. So if you don't have an owner for that I think that would be a roots to a quite bad situation quite quickly because capabilities evolve in processes change and you need to an owner to surface the needed changes. Another few point on this ownership thing is to avoid handovers altogether when building the solutions. So a concrete example is in or case we have a setup where data scientists can build and to end data apps. This had a handover without a handover to engineering. So that basically helps helps quite a lot. So in a sense this kind of ties it all together so you need to have the ownership, you need to have like the technological site working top of that and then you need to have the ownership as well. Yeah interesting. I maybe have one point like from one of your earlier points related to the data. I quite often noticed that people like suggest an AI solution whereas actually in essence they have a data problem or a data analytics problem. And I was wondering if you have any learnings or examples maybe of where you feel like AI could help with this kind of data problem or perhaps gaps in the knowledge or the data quality because I quite often like I'm not a data specialist. So my first question is always like do we have the data? You kind of have like shit in, shit out. So if it doesn't work well. Exactly so so I think one thing you want to be cheap probably that the way I answer this but like having good data quality helps a lot. So if you have your data quality at the proper level then you can do much more with that. So so if you for example now think about the project and then you do a data cleanup then you preferably want to do it so that you have it cleaned up once and you can utilize it elsewhere quite quickly. And then of course you can utilize AI to do a setup where where this is kind of useful. Yeah. Then Nikko. Yeah Nikko yeah you mentioned about the solution ownership and I find this is a very important theme as well. So so I've been thinking about that a lot myself how how to do it so so I think the I think the current term for that is johon voidaan tehtyä, että haluaisin haluaisin, - että haluaisin, että haluaisin haluaisin haluaisin haluaisin haluaisin haluaisin haluaisin johon voidaan tehtyä, että haluaisin haluaisin, että haluaisin haluaisin, - johon voidaan tehtyä, että haluaisin haluaisin haluaisin haluaisin, - toiselle probleman ja haluaisin sellaista, - ja haluaisin sellaista haluaisin sellaista, ja haluaisin sellaista, - ja haluaisin sellaista, haluaisin sellaista, haluaisin sellaista, - -Mä oon hyvä. -Mä oon hyvä. -Mä oon hyvä. -Mä oon miksi Niko. Ja Niko, joo? Tämä on aika hyvin tullut yrityksiä. Tämä on tullut tämä initiatio. Tämä on ei ole seuraavassa tietenkään. Tämä on seuraavassa eikä tietenkään. Tämä on säädäntö. Tämä on säädäntö. Tämä on säädäntö. -Tämä on seuraavassa ja seuraavassa on seuraavassa. Tämä on säädäntö. Tämä on säädäntö. Tämä on säädäntö. ja mitä on jo yksi mennyt. in as bunk like I it's something that we haven't really seemed to be solving yet so at the moment for instance when I mentioned that we have an ID portal I've been targeting it more at specific business units that we know that there is a funding behind it that could actually bring the solution forward as well because we're in the corporate where does the funding come from so these are all complicated considerations so I was wondering how how that's managed that as bunking right yeah I think overall it's a good question and the balancing act that how much resources and time and money and people you can pull out from the daily operations or daily development work to try something really radical or uncertain out and I think that's one of the questions that is the idea doesn't look like it's it's worth the investment and I think that's that's kind of a the mathematical part and a business case part that if you have a sound business case then there might be it might be feasible to invest in that and then find other solutions to keep the development programs and the daily operations going on but then at the end of the day the idea is just an idea and it needs people so so I would say that the the people the team is the key so so if you have like a big less idea but eggless team then it might lead to something good but but if you have it the other way around then I think that's the problem so that the the team is the key and then just think if if the if the idea is so good that it's worth kind of a making room for the team to be to try it out and then see how it goes yeah this is not a like a silver bullet answer but that's the basic thinking what I have so think I what what's on your mind thanks really really interesting trusting topic how to enable enable like the larger organization to try out all those things I'm thinking you mentioned earlier you have really regulated environment and data so so do you do you have some sort of sandboxes etc or what kind of setups you have in place that people can basically try out with with a load threshold yes absolutely yes you you need to have a sandbox to try it out in in a safe environment and then of course you have places in financial sector where you absolutely you don't take risks and then that there is still like the innovation the possibilities for that are really limited and you need to have like proven solutions proven technologies that you just apply and use it and run them so but but then there are places where you can take more like risk maybe it's too big it's it's like around the you know customer experience so you can try out different concepts that that this kind of a concept be more pleasant than the option B or or C so there you can take more risks and I when it comes to kind of a concepts but then I think the the same rules apply there as well that you try you you fail fast often and cheap but but then then when you see that this concept is working then then you invest in that but then of course in in our sector the possibilities where to try out things it's really limited comparing to the many other sectors thanks yeah more news please you have something yeah just a quick one and this is known for Robert Anico as you talked about the maybe if we're well-mengued or well-mengued number of ideas for innovations coming in how to handle them but then also how to build on the on the good ideas maybe I again smuggle in the AI here for a bit so we talked about the white coding and things could you help with for prototyping but on they're not the wisdom but it's kind of a sparring partner in terms of the early design phases are you applying it in your organization on structured basis some kind of AI aids on on this whole kind of innovation funnel activities right yeah there actually no no we are not using that as of yet but but I think that's a really fresh idea and maybe we could explore that a bit bit further particularly when there is like a big mass of data ideas so what did the AI could come up with so that's kind of a a good thought that deserves a bit further exploration we're actually now like building jumping on that we're now building a solution where you could actually have a chatbot that kind of spars with you so it would ask you if you say like hey I have an idea to use AI for a contract evaluation that the chatbot would ask you okay what is the challenge that you're facing when it comes to contract evaluation for which business unit would that be relevant how often do you have that so that it actually does this step what I mentioned earlier to kind of retrieve the additional information to to make sure that that we have all the puzzle pieces before we start the evaluation and then I think this is a fun thing that I'm doing right now it's kind of trying if we can use Kira to quickly create a tool to handle these submissions but then maybe also team education so with us we work with different teams that focus on different technologies so an AI can quite well assess an idea that is this actually more automation or maybe data analytics or what it is and give a proposal that maybe this team should take this forward and assess it in more detail before we escalate it so I'm now playing around with these kind of how we could use these tools throughout this kind of ID submission evaluation process so but yeah still learning and I think here like confidentiality is a key aspect as well that we don't want this idea to just be everywhere available so it needs to be in a closed environment and then to create such a tool in the production level you would need kind of a commitment also from developers to build the integrations etc so I think it's now a good way that I kind of try to create how it could look like with a tool such as Kira to get the buy-in from leadership that hey this is worth investing some money to actually build in in a safe way so I think here it's kind of maybe starting from the from the beginning point where we as designers can use this tool to showcase what it could be in a more convincing way and a way faster way to then get maybe the budget and approval to build a production ready solution yeah good point and just to be clear we are we are not also not yet using it but good ideas Robert and I think it could be a value certainly in some cases here definitely we talked about it on whoever I think it's it touches all of us on the product driven model for for driving these things and they were mentioned the product owners I think they're still something we've been trying to educate the business on product ownership it's still hard do you still so I feel one of the things is very important in terms of when we're ideating new things or and then starting the design and it's to have the vision of whatever the end goal here is and and the the product owner at that stage is committed to it do you how do you how do you make it happen in your organization in our organization we we often still they're a little bit vague the responsibilities and roles and and the comment commitment might be not there do you experience similar kind of problems in your environment in any phases like yeah Nico yeah you had a boy yeah I was yeah thinking for your question then I think I've been well in my during my career I've been a couple of times as a role in the role of product owner and and see I've seen different companies from that role as well and I think yeah I've seen lots of variation also that what are the expectations for product owners product managers and then how do they respond to those expectations and I think it's a really crucial role for the success of of the organization so the product owner is like the CEO of the the the area that he or she is working on and I believe that the strong ownership of the solutions or the value creation of of the responsibility area I see some of the core of that role and then how how the team gets there what are the the right metrics and what are the actually the results they they are really important for the for organization and I think Margaret you spot on the problem problem that how do we make the performance and the ja sosiaalitokalueet ovat hyvin ja konsistitititititititititititititititititititititititititititititititititititititititititettiin. Minulla on se, että se on ensimmäinen, että se on se, että se on se on se. Se on tärkeää, että se on tärkeää, että se on tärkeää, että se on tärkeää, että se on tärkeää, että se on trophopistu. Tupuk. Like, I see it maybe from reflecting maybe on our earlier point on this kind of handover face and how product owners can be involved. I think that these are one of the roles that can be more continuous throughout the project. So that maybe the business owner and the product owner we try to involve them from the early stages that they know what's going on, what has happened and kind of give them these learnings and insights that once we move from approve of value to an MVP and then to productization and maybe the design research or the developers may change that at least the product owner and the business owner are consistent across these different faces of product development so to kind of keep that knowledge and expertise inside the product as it develops further. It's a bit maybe off topic of which here we're discussing but it is one way to look at it. There's nothing wrong with going off topic, Robert, the conversation's been brilliant and I have enjoyed it. And yeah, thank you very much to all of the speakers for their time today and sharing their views. It's been a brilliant chat with lots of insight shared and we even ended up going off topic at the end but very relevant to the theme of everything. If you've enjoyed listening and you'd like to get involved in any future episodes, please feel free to reach out to myself via LinkedIn or email and also please feel free to reach out to any of today's guest speakers and connect with them should you want to continue the conversation. Thank you very much everybody.