Welcome to Optimize to Innovate, a show where we help organisations stop wasting money on things that don't add value to their business and understand the technologies that actually will. Join us as we share practical insights into the latest trends in innovations, with industry experts across everything from software and phintops to cloud data and AI. And we decided that for a bit of a change we wouldn't have any guests today, just Alex and me and a whole bunch of agents. So let's get a second human in the loop and start this week's episode over to you Alex. Fun to just keep on rolling Jason, love it. It's hard to believe we're already on episode 6. It's gone past really quickly. So I just want to probably start by saying thank you very much to everyone who's followed along with us to the fantastic guests we've had as well who've been sharing their expertise. We thoroughly enjoyed it. Yeah we thought we might do something a little bit different this week and maybe take a look back at a couple of the themes that we've seen from our more recent episodes. Plus I know that you and I have been on the road a fair bit recently attending some really interesting events. Things like the London AI Summit, it was also London AWS Summit. I think there's a good one. Sorry, it was a good one. It was. I think it's a really good kind of a litmus test for the world of AI where it is. What's the zeitgeist? However you want to put it and maybe we're going to pull a few observations out of that. And for me, I think the very first one was at the London AI Summit. There was one really major theme, right? Two years ago I attended and it was all about AI that chats. So it was all about I can get all this really useful information or I can get it to write my emails or whatever. And now it's like we've given AI hands. So the theme was absolutely all about AI that works. I mean, is that what you mean to you? Yeah, the thing that really struck me at the AWS Summit in London this year was compared to last year. Last year everyone was talking about model context protocol, right? So they were talking about how agents could interact with tools, with data effectively giving agents hands. And a lot of the conversations then, the focus was around the standards, how they were going to be extended, what else was needed. When I went this year, it was really a focus on this is what we've built, right? This is how we've used MCP to actually make services available to teams to shorten the path to productivity and working with agents and really to think about building out an ecosystem and taking some heavy lifting away from teams who are working, building individual agents without that scaffolding there to help them. So yeah, absolutely, it's far more practical, real good examples of things which have been deployed. Yeah. I think there was a really interesting kind of takeaway, I think both of these events is a lot of the conversations, there was some brilliant conversations about use cases and genuinely inspiring stuff. But I would say actually more of the conversations were almost around that governance piece. And this is the thing that, I mean, just kind of harking back to a couple of our previous guests, like Alex and Sever when they joined us, we were talking about a GenTKi. They were saying a GenTKi is not really about agents, it's about the governance, it's your data foundations, your process design, it's all these things. And actually that was definitely coming through in the themes. I mean, somebody actually said at the event, running one agent is like running one server, whereas running 1,000 agents is running 1,000 servers as well. And actually you're running a data center then and you need to almost have that level of mindset as you expand this into your environment. I think a lot of organizations are running into almost that, not a brick wall, but certainly a wooden fence of governance and try and work out how they're going to handle it. Yeah. And if you think about an agent as an app, then you're thinking in an organization, you might have a couple of hundred apps. But that's the wrong way to think about them really. That might be how it feels like when they're being exposed if they are user-facing, but if they are agentic processes, then they're going to be multi-step. There's going to be lots of them. If they're specialised agents running in that chain, performing particular tasks, the challenge here is the control plane. It's having that visibility across all of these things and understanding what are they doing, when are they behaving in the normal way and when are they doing something you don't expect. So when you get to that kind of scale, you've really got to switch to exception reporting. Otherwise, you just, you can't keep handle on things. And I think it's interesting to see that what Microsoft have done, they've released their E7 licensing, which is very much focused on the agent control plane. One specific offering, agent 365, which is not just designed to manage agents within the Microsoft stack, but it's also being built as a way for you to bring third party agents under control as well. And you see what they're doing with Entra, which is their identity provision. And they're giving effectively non-human identities. So we're talking about how do we give an identity to an agent, how can we control its permissions, how can we govern it as we would do a person from the point of view of policies and controls. So bringing it back to the idea of control, I might use saying governance, that control plane, which is, you know, what have I gotten in my technology landscape that's going to help me to actually get my arms around all of these things, given that they're going to be coming from different directions. You won't be just working with one vendor, with one platform, you know, you need to somehow think about the flexibility to cover it all. That's, you know what, it's really interesting, you say, around the whole Entra ID thing, because we are at this kind of point now, aren't we, where if we think of it in that governance perspective, we need to manage these agents in much the same ways. We manage any traditional service, you know, you talked about applications before, so you used to have service accounts and we'd lock them down, we'd give them permissions, we'd allow them access, you know, role-based access to certain services, and then we would track them. We would have, you know, a logging layer in place to actually ensure that we're monitoring and identifying what is actually happening. So that observability element, all of those exact same things we've done with humans, we've served as accounts in the old days, those are the same kind of ideas we're going to have to have with AI agents. And in fact, I'd go so far as to think of it almost like a logistics supply chain, if you will. You know, you start with that data right at the beginning and you need to understand, especially given the way that governance is going, or sorry, regulations going in many different parts of the world, you're going to have to legitimately be able to explain, I think, in the future and already some of these regulations are describing this. Exactly how a particular decision was made, or how did a particular outcome from an agent occur, and you need to be able to roll back through what did the agent do, what did it touch, what sources did it, you know, was it utilizing in that kind of logistical supply chain of knowledge, if you will, that then came out with an outcome. So you imagine you're a, I don't know, like a financial services organization, you're doing, say, loan, you know, loan decisions. If somebody challenges that loan decision, how do you legitimately say, well, this is the reason why our agent made this decision. And I think that kind of regulation is only going to get stronger. So I think it's going to be, you know, the hovers as our good American friends would always say to, to have an even stronger internal logistical supply chain, if you will, on that data and the AI agent. Yeah, you could expect that, that to come. So if you're not sort of building and thinking with that in mind, I mean, you don't necessarily need to be working in a farmer, you know, really regulated industry like farmer or financial services to, for this to be at the top of your list is explainability. You need to think, how can I ensure from a tax surface point of view, you know, in terms of my cybersecurity, my security posture that I understand what agents are authorized, which are behaving to norms, which are doing abnormal things. Then also there's the liability aspect, right? So if you've got agents interacting in the supply chain and something is happening, which an agent shouldn't be doing, well, what guardrails have you put in place to actually prevent that from happening? And if you haven't, then is there an issue of liability? So we know that the technology capability is, as always, running way ahead of the legal frameworks, you know, and actually what we see at the moment is the kind of almost like the operational frameworks are being built. It's like, you imagine the trains running and somebody's laying the track in front of it is it's going along. It's that kind of thing. Oh, come on guys, we need to, we need to build this pathway faster. Absolutely. And interesting you say that because that was another kind of key theme that I was certainly seeing was around, you get, you know, every bit of AI related content you see will often use this phrase, human in the loop. And one of the key points that was really coming out in several of the sessions that have outlearned some of it was it's not about necessarily the human in the loop because human in the loop becomes a phrase we can hide behind. So you know, we say, oh, this committee over here, they're the ones who are responsible of this group of people, they're responsible for the outcome of this. And actually, if we make something, multiple people responsible for something and we always know that, you know, humans are humans, nobody's actually responsible for that. And so the human in the loop, I think moving forward needs to be much more about a named individual, a named person who's in charge of a particular process. You know, they design that process, they see the outcome, the, you know, the, sorry, the input and the outputs of that process. And they are the ones who are sense checking, is this actually doing what I expected? Not only that, but actually controlling the costs of that because it's so easy for us to, you know, utilize this technology to do.
amazing things. But sometimes I don't know how much you've observed this with some of your experiments, but sometimes you find yourself doing something like I did something with Git recently. So I did a very simple update into Git and actually measuring the cost of it using the API. I saw that a simple push cost me 60p. Now that doesn't sound like a lot, but if you're doing that multiple times a day, that's a very simplistic task which doesn't cost a human very much, but actually in the end up by doing that using AI cost quite a lot. And so there are plenty of ways that we could optimize the way that we are using AI, which ultimately for me comes down to the responsibility of that human, the human who owns the loop, not the human in the loop, if you will. Yeah, and maybe that's a move from thinking about the human in the loop, or the human owning the one who's effectively bringing, if you like, the business wisdom to it, the understanding of how process should run optimally, then that's everything from quality of data to the verified outcome and agents producing, but also taking some ownership for the business value created, because if it's costing 10,000 pounds or $10,000 a month and you look at the business value and it's only half that, then there's something fundamentally wrong with the approach. And when we talk about having an AI operating model and defining an AI strategy, the one of the most important things up front is to look at the business value, the business feasibility. And there's an element of cost forecasting that's really important. And I think because so many organizations have been stuck in PILA, I don't think they've had to go through the hard yards of actually finding out what it costs to deploy and then scale AI. And that's when the real inefficiencies become apparent. Yeah, AstraZeneca, if you say that, they were referring to that. They termed it pilotitis. The whole challenge there. You know, you can now develop things so quickly. The development is actually the fastest bit often. It's your rollout of your environment. It's the trust, the change, management, all of those parts in rolling out a proper production system. That's where a lot of the work is lying. I mean, arguably it was always there. But some people like to avoid that part. But if we don't have a focus on one of the one of those three really important things to most businesses, right? So what's the return on investment or what's this kind of cost impact to this or what's the performance improvement that we're going to bring for our team? Then we're probably just going time after time after time, doing endless pilots, not actually gaining any business value. So it's really important to actually pick a single outcome. Work out what is the ROI and that work out all the fundamental foundations that you need to get to that and drive that thing to completion rather than just experimenting endlessly with all different all different options. And I remember in several, we had seven Alex on them. They were talking about agent to AI processes and how you build them and how you run them. They were effectively saying you designed for failure. You designed for failure because if you don't design for failure, then when it happens, it becomes something that what do you do when you're not prepared for it, where you throw people into the mix. And I've always had this question in my head of what point does the amount of effort involved when something breaks? Where so you sit up a great process, but it breaks somebody gets involved trying to fix it. If the cost of trying to fix it is more than the value created for the time it was running. Obviously, it doesn't even balance out financially. But there's also the challenge that as organizations step into this world of embracing agent to AI to drive business processes, core business processes, once they actually rely on operating. They need that confidence in the tools, in the understandability, the outcomes that there's visibility and observability of what's happening. And it's explainable. So the team is running it, operating it, understand it. And if you have a process which breaks and people spend 24, 48 hours looking at it and they still can't understand why it's broken, you can imagine the confidence dent that that organization is going to have. It could set them back a long way because fundamentally the people who are the ones making the decisions at the top and signing off the budgets and shaping the agenda for technology, they may steer away from that, it might take them quite a long time to come back to it. So I think it's definitely one of these areas where you want to make sure that you really have understood the risks and that you have like a risk management plan in terms of your operational response to things not going as you expect. Yeah. And that's an interesting one because again, massive theme that we are seeing all the time is around trust. There's a, was it they say, trust is built in drips and lost in buckets. And I think we see that certainly with AI when it first came along, everybody was a bit like, what is this thing? And just within a couple of years, you see far more people almost trusting it beyond where they should, I would argue, in some cases, and not having that level of, I'm just going to make sure this thing is okay before I actually put it in production. So I think that trust element is something that as again, we go back to human nature as so many things often are, isn't it? We start with little trust in something then we build very quickly often, perhaps more trust than we should or certainly, you know, a comfort with a particular technology. Perfect example. If you've ever used an autonomous vehicle like a Waymo, Waymo, I don't know how to pronounce it. But if you've ever been in one of those, you know, the very first time you're in and you're sitting there and you're like, I'm not in control here and this vehicle is in motion and there's other traffic around me. And then after two or three times, it becomes natural, it becomes normal. You forget about those trust issues. And I think the same thing applies when we're using AI around we understand some of these constraints and everybody says be careful when you use it and then you start to get in, you know, you get familiar with it and you start to trust it and you trust the outcomes. But then maybe you, in that case, you're forgetting about the guardrails that you should be putting in place mentally speaking, if you will. Yeah, it's interesting. When we think about trust and I think about AI, one thing that comes to mind is consistency, right? And you think about technology and you know, just think about something simple like an Excel spreadsheet, right? So people are using Excel or Sheets and Google or whatever for years, right? And you know, if you're using in particular, using a route based tool like SAS base tool, then you know, you enter your data, you expect it to be there, right? You don't expect it to disappear. So we've become comfortable in working in certain ways and we don't question them. And I think the thing is when, if you think about the pace of modern business, is we move at speed, we're all being asked to move at speed. You don't have the chance to question things when you're moving at speed. So very quickly, we move from questioning to just that's just the way it is. It'll work. I'll just accept it. It's part of my norms. I'm thinking about if you know, if you're a technology team in an organization and you're affected, if you're trying to build trust, you know, by operating an AI platform, you know, it's doing various things to the business, they're starting to rely on. If something happens, then it's not necessarily that everyone will lose faith in AI itself, although that could happen. I've seen an organization where they had a bad experience with workplace AI and then they just retreated a lot from AI, you know, big picture. But what I'm thinking about here is the reputation of the team, right? So, you know, you're building an AI platform, people trust it or they think it's trustworthy, something goes wrong, you can't fix it. That lack of trust and belief is going to be focused on the implementation of it by the technology team. That's the thing that concerns me, right? And how you recover from that, I'm not quite sure because everyone at the top of the organization is going to be thinking, AI is the way to move forward. But at the same time, you've got this challenge, which is I'm never getting in that waymo car again. Because it just pushed into a lamppost, yes, exactly. Yes. We should be clear for the purpose of legal proceedings that we are not aware of any particular waymo related incidents. No, and you should be safe getting into one, honestly. Trust me, you go. But yeah, so the other part though, and when it comes down to that, that trust pieces, I think there's a, we're in a really interesting time right now as we scale this use of AI. Because as adoption is growing inside of organizations, a lot of that has been on the back of, let's call it fixed price AI access up until now. And so I think one of the, one of the looming challenges is AI is an incredibly expensive thing in the background, setting up data centers, providing that infrastructure, the research and development that's going into it. I don't think it's a sustainable, let's say, for us to continue to see AI being made available at those same record low costs, if you will, to the consumer or to certainly smaller businesses. Yeah, so this question of the cost of running AI is an interesting one, because first of all, it's still such an immature area. So when businesses started using APIs to drive some of the models to make applications intelligent, one of the struggles that the Finops Foundation had was that a lot of these
If you like levers to execute, they didn't have the kind of financial metrics they weren't giving you the visibility and observability into what's happening in terms of driving the cost. So the reason for the option involved and they have a scope services scope for AI cost management is because it has a fixed element and a variable element and the variable element is around the use of tokens or what you're driving through in terms of inputs and outputs. And also very importantly not just inputs and outputs but what the model is doing you don't see so the reasoning tokens that use is the internal computations that it performs and they can be really really high. Especially if you're doing things like coding because a huge amount of computation being done within the model before you get something back. So the challenge around the the economics of, and let's call let's let's make that a workable description. Let's talk about forecasting. So forecasting, tracking because you mentioned being able to track the ROI of using AI. And the only way you can track the ROI is if you understand the value of the problem it's solving and you understand the full cost of what it's actually costing you to solve it. And the challenge is that the cost of the tokens is not the complete cost to an organization operating AI. In fact, it's anything but the complete cost. But if we think about what's going on in the in the market in terms of the large players of the frontier organizations like Anthropic and Open AI, they've got this strange thing going on which is almost like a structure imbalance. So you've got individual subscriptions. So like you and I might have a Gemini Pro or Anthropics Pro or something like that. We're paying an amount. I pay $20 a month for using Google's subscription. So I get a certain number of tokens for that. I get a reasonable number. But lots of people who subscribe don't use their allowance. So in a way, it's subsidized because they don't really use a lot of subsidized and those who do. That's the insurance policy model, isn't that? Yeah, exactly, exactly. But you see that that only makes up a small proportion of what the tokens which are being consumed through that company. So roughly 15% this is more of a global stat, roughly 15% of tokens are estimated to be going through the consumer tools. And about 85% are being consumed through APIs and organizations who effectively plugged models and made them available for intelligent applications and other things that you might be doing with it such as coding, developer workflows and things like that. So if we think that the majority of the tokens that have been consumed out there are being consumed by organizations who are chasing them for business use, proper use, all kinds of things. The cost of those tokens has been dropping and dropping and dropping and dropping and dropping. So the cost of operating models and the cost of token has been dropping. So financially you could say that the supply cost for all of the organizations using these things has just been dropping continually. And it's the frontier companies who've been driving that cost down. And the reason they wanted to do that, the reason it's become a bit of a pricing war between companies like Anthropoc and OpenAI is they're chasing market share. Yeah, for land, yeah, a land grab and they've been able to do this because they've been burning, funding, right? They've been burning investment funding and they've been doing what every company does that tries to win by scaling up is spending inefficiently or let's say less efficiently to win customer share, mind share, because they know that as they mature their products and people get more comfortable with them, they'll settle on them. Then as it becomes if you like a formalized business revenue, they've got those people, they've won that space. So we've got the strange situation where the majority of tokens are being consumed as a commodity or using commodity pricing. And it's only I think when we see Anthropic and OpenAI go public and they've got shareholder pressure on their returns that we're going to actually see that behavior change. Now that in a sense is what's driving the overall engine of consumption. If we think about tokens, a pool globally, imagine a huge pool globally of compute. There's a certain number of a certain amount of inference, which is when the model's running on chips because the model is an algorithm, it's software, it gets run on compute. It's a certain amount of compute out there. Yes, people are trying to build data centers, but we know it's taking time for them to line up the extra capacity. We see already problems with power, with power grids, with even the skilled power engineers to help enable this data center scale out that's happening in the US and the UK and other places. So that's a bit I think that's a bit I'm worried about because we talk about the cost coming down, but you've just described there. We have a limited pool, don't we? Right now it's limited and yes, the pool will get bigger. But that build out will take time. Some of that build out is going to be on Mother Earth. Some of that build out, if you follow certain individuals is going to be extra terrestrial. But you and I both you and I are both sci-fi fans, right? Exactly. We shouldn't go down that tangent because that'll be the rest of the episode. But that capacity, whether it be on Earth or extra terrestrial, takes a long time. So given the popularity that we're seeing of a given the growth explosion that we're seeing in the use of AI, my biggest concern is that those tokens become scarcer and scarcer because you have a larger and larger pool of people who are going after the same volume of tokens or a very slowly growing number of tokens. And we have that there's that thing called Jévin's paradox in there, which is the cheaper you make something. Actually, it's not the people who save money, they'll just buy more of it. The perfect example is things like light bulbs. We move to much lower power and cheaper light bulbs. And what happened, people put lights everywhere, then they put lights into devices. And now there's way more light than there ever was. And the actual quantity is going up year and year in terms of power and lighting, despite it being lower power. So now we take that to AI. We have this AI explosion and a slowly growing pool of tokens. That's where I'm a bit concerned that we're going to run into challenges. What do you think? Yes. I think what you've described there is we could call it token scarcity. I'm not for I'm not calling that phrase somebody else has. The token scarcity is really relating to the fact that we're moving into an era where you'll be wanting to consume something and it may not be there on tap as you've expected it to be. Right. So you hit a cap. And how can we see this happening? Well, let's think it's really quite simple. It we've got companies who are driving demand and they want to because they want to grow market share. So part of that is they want to grow a bigger piece of an expanding market. We know that AI is a mega trend. And as we see a gentick AI running business processes, the complexity of that AI operation is going to increase. What does that mean for tokens? Well, let me let me give you a simple idea. Humans interacting via an NLP natural language programming chat window typically generate 500 to 2000 tokens for interaction. If we think about using something like Claude code or custom enterprise tool sets, single command can process between 50,000 to 1 million tokens in a in a context block. And the thinking token requirements for agentic processes is also really high in is that the challenges is what you wouldn't see because it's behind the scenes. You'd see the cost, but you won't get that if you're measuring, you know, from the final point of view, tokens in and tokens out, you wouldn't necessarily see it. So the the challenges as we as we move on this on ramp to wider AR adoption, which means more token consumption. It's going to come from business use, business use is what's going to drive the higher requirement for tokens. That's not going to be curved initially by the costs because the companies are still focused on the land grab. So then you're going to hit that ceiling. And it you know, the kind of the suggestion in a way is that we need to start thinking of tokens as a not a limitless resource. We need to start thinking in terms of how can we make our AI processes token efficient and actually stepping back from all of that would be do we understand what's in what what's where to understand where values being created and do understand what cost is being incurred because not all tokens are equal. Right. So if you're using a really high value or say a high very high quality model versus a lower quality one to do a task that doesn't require the high quality, you're overpaying for it. So those tokens are actually costing you a lot more than the ones from the lower quality one that couldn't give you a very adequate response. And you know, sort of brilliant quote the other day from some legal brand or talk who is saying I've seen the same processes operated and you know, one person running them using different tooling or different parts in that process and it costing eight times as much to operate effectively get the same outcome. And this is a challenge because if you think about it,
from a financial management point of view, how can we understand enough of this to see where the waste is? So that's sounding awfully familiar Jason. I seem to recall many times in the past 20 years having conversations with people about why did I size a virtual machine with 8 CPUs when I only needed one or why did I put 32 a ramen to something when I only needed 8, etc. This is sounding a very fin-opsy conversation to me. Yeah, absolutely. Absolutely. And I think it's interesting you say that because when people's sunk investment in a capital into on-premises kit, the only kind of conversation or argument happens once as getting the purchase sort of signed to my, whereas when you move to the cloud and it's fin-ops, then you have people coming back and repeatedly asking like, okay, why is this is change, what, why is this so high? And it could be down to inefficient usage. It could just be down to growth, right? So it's trying to understand it, but you're absolutely right. So the fin-ops foundation, which was formed 2019, has as one of its services a focus on cost forecasting and management around AI workloads. But to put in perspective how important this area of tokenomics is going to be, there was announcement on the 3rd of June that the Linux, Linux foundation is now launching something called the tokenomics foundation, which is going to work in close partnership with the fin-ops foundation and it makes sense for them to do that. But the point or purpose of the tokenomics foundation is to help shape open industry standards, benchmarks and best practices for the economics of AI infrastructure. This is absolutely needed. It's a garner of forecasting that by 2029, 50% of cloud workloads will be running AI. So we can see that AI is going to dominate both from the point of your driving business outcomes, but also shaping technology platforms and ultimately the cost. If you think about what's in the IT budget, a large proportion of it should be relating to all of the parts of your ecosystem that are enabling AI to drive your business processes. That's the same challenge that we've seen again. History doesn't repeat itself, but it's certainly rhymes, doesn't it? We've gone through the same thing as we had the shift towards cloud. And let's say the discomfort of financial professionals and organizations with releasing budget for something they're not entirely sure what it's going to cost them. As soon as you move from that, static capex spend to something that's a lot more metered, it becomes a challenge. And therefore, I'm going to use the word for the second time today. It behooves us as IT professionals to manage that spend and to actually put those wrappers in that governance in. So I think tokenomics absolutely is something that anybody who is touching AI in any way, shape or form, should be considering. And who owns the token bill when you are running hundreds and hundreds of agents? Where is the responsibility lie? And it can't just lie with one individual, one that doesn't scale and two that leaves no sense of accountability and responsibility in the organization. So I think in just the same way as we saw in Finops, the idea of tokenomics is going to be a cultural necessity, if you will, inside of organizations, not just a technical necessity. It's not just about observability. That's very useful. It's useful to have the data to back up where we're looking at. But it's also fundamentally important that we train and enable people to understand the impact on what they're using. And if people don't understand, you don't know what you don't know, you don't know what the impact is going to be on token usage, it's just going to be on anything. So I think that's going to be a really key thing for organizations. And what's possibly more of a challenge there, I think, for some organizations is they're still trying to wrap their heads around this today in the expert teams, the centres of excellence, whatever you want to call them in your organization. They are still trying to wrap their heads around this. But if they're not already thinking about how do I enable others in the business, then they're potentially setting themselves up for these larger cost implications, if you will. Yeah, I think there's an element of maybe technology teams are still chasing, like, running to catch up themselves, because it's been a lot of focus on trying to understand the platforms and how you operate them. And we talked to everyone about control plane and how you get that governance. And so we're kind of like, you've got the squeeze at both ends. You've got trying to find the business cases, the right ones, the best fit for the models, the technology that deliver the value, that are feasible based on the data you have, the industry you work in. But then you've also got the other side, which is, do you really understand all of the financial metrics involved to be able to give somebody that full view of return on investment? And if you don't, then you are effectively some part of your organization is subsidising that process. And you could be effectively running less efficiently than you were before you put it in place, which is a kind of scary thought, I guess. Absolutely. And it's interesting when we talk about, you know, education, skills, etc. One of the other things that we're seeing in terms of these trends, and this was actually brought up at the summits as well. I think both Amazon and the AI summit, they were talking about the same topic, which was around the impact on roles and hiring. So as it's the elephant in the room, but the impact of AI on jobs will agents cost jobs. So there's been some really interesting study starting to come out. And I think what we'll do is probably post a few links in the show notes about this. Let people read up and kind of start to form their own opinions on this. But certainly, what the data is showing at the moment is in technical knowledge management roles. So let's say engineering, developers, those kinds of things. There absolutely has been a reduction in the number of junior positions, quite a significant reduction. I think it's around eight or nine percent, according to one of the large American universities. But interestingly, the flip side is, there's an almost identical increase in the total number of roles in senior development roles and senior engineering roles. And so when you look at it from a technology standpoint or technology role standpoint, the trend is not down. It's that it's shifted from junior to senior. And now this is not going to be the same in every vertical, quite clearly. There'll be other ones where agents are like for replacing processes. And therefore, if you think about an organization is really a set of people and collective processes combined. And so if an agent can do that process, then it will have an inevitable impact on roles. But certainly, as we say, we're seeing this more of a shift from, let's call it a top-down triangular shaped organization, or a pyramid shaped organization, with a smaller number of seniors and lots of juniors in a high bottom end intake to more of a wide middle. Yeah, I'm thinking of a diamond shape. Yeah, I'm thinking diamond shape. Diamond shape. That's far more polite than what I was describing myself. So I think there's a huge challenge there that we're going to have to face because where do we get the talent for the next generation? Okay, well, you've touched on a couple of points here. So let's dig into them more. Now, are we talking just software development, developer teams, or are we going broad on that in terms of general market? Either way, I mean, I'm interested in your opinions on both. Okay, so I think you were asking a question around the developers, for example, just coding skills still matter. Because if you think about that program of juniors on boarding, so we've got this challenge which has always been the skills gap and skills transition. So when people come out of university, they've learnt, say, do software engineering, they've learned how to code, they come out, and they might know 80%, 85% of what happens in a coding team in an organization. But there's always a gap, there's always that gap of the way we do things, of some of the smaller tools and the processes. So while they might understand some of the really big building blocks of the whole software development motorcycle, there are some specific things that they're going to learn and develop and shape within the workplace. So if we talk about what they gain from coming in at their junior level, what they gain is they fill in those gaps. What they gain is they get to work with senior colleagues who can coach them, who can help them mature, they deepen their understanding of processes and they become more efficient, more effective. Also there's that communication aspect as well because no developer works on their own. So working teams, communications is a skill, you learn that skill by working with other people and you're going to do that most in the workplace. So we think about do we need developers, do we need junior developers? I think there's
There's two aspects to this. One is do we need them now, right, in terms of do we need what they could bring in a team coming in to join versus having some agents operating and running some of these development tasks. So I have a colleague who I can't see the company he worked for, but his measure was if I have got some extra development work I need doing, he was looking at whether he would go to an offshore team. He said if it's less than six weeks worth of work I won't bother on boarding new people because it takes too long. I'll just look to actually start sitting some agents in place to do the work. And if it's longer term than he would consider going to an offshore team and spending time with them getting them familiar with the project code-based requirements, that type of thing. So he'd already started to work out this equation in his head of when is it not worth bringing the person. If we think about something you said in the past, if we don't have those juniors doing that work and building up that scar tissue of the challenges and all of the difficult things, who is the middle level or let's say the senior developer who has the understanding of all the different aspects of the stack and the software development lifecycle to actually orchestrate those agents to set them up and to run them. So that effectively he's got 10 different tasks running at the same time as if he had three or four devs sat there working for him. So there's the productivity at the start which is when the junior comes in, they're going to contribute something, but let's be realistic. Most people in the joint in organization, it takes some months to get up to speed and yes, AI can help that. But then the challenge of not having that pipeline of people coming in is you're not growing the mature competent seniors. Who you're going to need in 10 years time? Exactly. Exactly. And that's an area that we understand well, right? The area development, you could probably repeat this conversation for legal, for scientific or genetic research or something. I don't. The point is it's the younger people, it's the people who are looking for their first role in work who are really, really being impacted. Back to your point about the diamond, it's the bottom of the pyramid that's now becoming the diamond, that's the issue. And interestingly, Andy Jassy was saying quite publicly, he made a statement about this just a while back and I don't have the quote in front of me, but he was effectively saying that AWS intend to continue to hire developers at the same rate they ever have from those junior roles. And arguably from the same perspective, we talked about that, Gevin's paradox before, well actually if you are now reducing the limits or the barriers to entry to be able to create code, then actually what it does is it will increase demand. So a lot of people are talking about, you know, the SaaS apocalypse that was in the news and was impacting share prices and all sorts of things. Yes. You know, a few months back. But what may actually be the reality is that because this becomes so easy, because it becomes so cheap to do, actually demand continues to increase. And so you still need junior and mid-level developers to be driving that, you know, at the wheel if you will. So we could find ourselves in a position where more software demand equals actually still junior, there's much junior hiring in the future once this thing is settled out. Yeah. Yeah. There could be a lot and then there could be a peak, actually. And I can see that happening because if you think of the power of, you know, vibe coding and certainly like we were talking with geo and yes, like about vibe coding with the scaffolding, right? So think about those really core skills of defining requirements and, you know, then the testing and things like that. It's entirely feasible to believe that, you know, subject matter experts in line of business functions that we could train them, we could give them the skills to be able to, you know, create 30% 40% of the value in an application or even more. But it still needs somebody to come in and then look at the code base to work out the inefficiencies, to work out the vulnerabilities to formally, you know, take it through a bit of a sheep-dipper productionisation, it's safe to let loose in the wild and it can be supported. And you're right because then that would scale the need for developers. They would be doing a different type of thing and it actually got me thinking when I was thinking about the episode, do you remember like agile when agile first became a way of project managing, you know, there was a role for agile coaches. Yeah. And it was always to scale other people. There's lots of project project managers out there that the challenge was how do we get them all to onboard agile ways of working, how do you bring the, if you like, the process support around them, that's not just the tools, it's the way we do what we do. It's making sure it's implemented in the right way because agile is a certain way of doing things. So it's part of it, part of it is a cultural practice. Yeah. It's actually how you see people do stuff. And I was thinking, well, maybe these, you know, the kind of senior devs will end up being almost coaches to some people out in the business who are shaping their own applications because they're so close to the need, you know, that if they can think of a really bespoke application that could do something incredible for them, for their team. But, you know, we can't say, well, I'm going to say no because the genial be out of the bottle from the point of view of this is something you genuinely could do. So the question then becomes how do we do risk it? How do we do risk people wanting to move out speed and do that? I think you nailed it there. I'd just add one kind of additional almost thought on top is that you mentioned the profile of those juniors. I think that's actually the really interesting point here because I'm going to I'm going to put my, I don't know, stake in the ground. And I'm going to say I think that given the rate and pace of improvement that we've seen in the capability of models, not even just the foundational models, but actually some of the open source, open weight models, etc. to deliver high quality code. That's only going to increase to the point where I don't think it will be about the ability to code or even the ability to go in their incense check and peer check. However, you want to put it the code or peer reviewers, it's it. I think that those juniors then it becomes far more about them being the problem solver and also the communicator. So you talked about going out into the business and working with individuals in the business. If you have somebody who's that junior to mid-level developer who has a fundamental understanding of the code, but actually the AI is doing a very large proportion of that, then it's up to the junior almost becomes more like an architect. So they're translating those requirements, they're translating the meaning, you know, that vertical knowledge, if you will, from the individuals, the subject matter experts in the business. And they are the conduit, if you will, that helps that to become a reality in the system and be done in a way that's sensible because as we know, the larger the context, the more challenging it is for AI to handle. And that's one of the reasons why AI is fantastic, writing you a module, it's less fantastic at writing you and into our software suite. And so then it becomes more of that architectural mindset. And I can tell you from experience, actually, I've run a number of years ago, we actually did a graduate scheme, another organization I was working at, and we brought in people straight out of university and put them into architecture roles, which is, sounds crazy when you think about it, because they don't have all of that backlog of knowledge, but that also meant they didn't come with a load of baggage. So they didn't come with a load of hang-ups about this is the way we used to do things. And so they were thinking and they still do, because it's fantastic to see, you know, where they've progressed in their careers. They are thinking in a very cloud native fashion, rather than say somebody like me who has a mixture of, I've done things in data centers and I've done things in cloud. So I think there's also going to be a huge benefit to bringing in these juniors, who just have a completely, you know, a blank slate if you want a different way of looking at it. And they're going to be at this native generation. Yeah, now there's a really good point that, you know, if you introduce somebody to the workplace 15 years from now, there are some things they just will not bother learning, right? We'll have no value to them. And there'll be things that we've thought about. Tokenomics, for example, will be, you know, think 10 years from now. It'll be very well defined. In the same way that if you think about Finops at the moment, you know, Finops tooling, processes, there's so much available. And, you know, to some extent, even the hyperscalers have stepped up to bring their capabilities to try and make it easier, you know, in terms of following open formats and bringing their own tools, improving the rune tools. So over time, yeah, it's definitely believable. So maybe as our closing point, I want to finish on something that's a bit of a thought provoker for the audience. Because the data we're saying suggests the impacts of AI aren't predominantly on junior employment. That it's happening now and it's structural. And, you know, we see two things that are affecting, if you like, the appetite or the desire to recruit juniors into organizations. So one is task-level substitution where we've got people who were doing things within their roles, such as modeling data entry, document drafting, preliminary code generation. And they are the exact capabilities where
LLMs connect cell. Okay, so it's very easy for an organization to see how they can substitute the role that the junior will be doing those tasks within their role. And the other is when we think about programs where organizations bring people in as a junior, they rotate around an organization, they progress, they mature, and then they move them up the stack. And we talked about developers there as an example, right? So how people come in and they then start that career path moving up. And because they're seeing a different way of doing that work overall, almost rethinking, reengineering the way they do the work, they're not seeing the need for that pipeline. So there's both a removal of a need for the somebody's come and do the work right now and be a contributor. And potentially they're thinking we don't need to grow that person for the future. And it creates a paradox. Because if companies do not hire juniors because they can do their work, then at some point our organizations will face a severe shortage of experience middle and senior managers, right? Whether it's a decade from now, 15 years from now. So to what extent do we think organizations are, I was going to say making short term decisions, I'm actually going to say sitting on the fence because I feel like a lot of organizations, because they're not sure how much AI is going to reshape their future workforce needs. I feel like they've just hit pause because they don't know what decision to make. I'm interested in what people in the audience are listening to that. That's fascinating. I think that's a fantastic point to end on. Actually, if you would like to share your opinion, we would love to hear it. As always, you can obviously get in touch with those via our socials. We're at software one just about everywhere. But also we have an email address. So if you want to email in, we would love to hear from you. And perhaps we might even read out a few of those comments on a future episode. You can email us at
[email protected]. That's o the number two
[email protected]. And just before we go, I'm going to plug an absolute shameless plug. We actually published a blog post literally just a few days ago from the London AI Summit. So I'm going to drop that into the show notes, which covers off some of the topics we've been covering today. And also we just ended up having time to get through everything. We had all bunch of other things we could have talked about. So definitely encourage you to check that out. And with that, we're going to wrap up and just say thank you again, everybody for joining us. And we look forward to the next episode. Please, if you did like this episode, please do hit subscribe on your podcast app. Leave us a little review. Absolutely. It really genuinely helps other people to find us. And yeah, as I said, if there's something you want us to cover in the future, please don't hesitate to reach out. Until next time.
PodcastTranscript AI