AI in your delivery teams.

Hi, welcome to the Delivery Manager podcast where are you host, Mario de Cristofano? We talk about everything agile delivery, technical delivery, whether you're an agile delivery manager, you work at technical leadership level, maybe you're a head of IT, maybe you're a CTO. I talk through all my experience, everything I've learned over the last 25 years, so hopefully I can share some things with you, I can learn something, we all all have a conversation, we'll get guests, so I'm going to be amazing. You can get involved in the conversation and mary out of ways on X, DM underscore daily or delivery knocks or Mario DC or Midlands delivery, there's a telegram channel, you can use that, it's still a bit empty at the minute, I'm working on it. And you can go to marioesblog.co.uk or medium.com/mario de Cristofano to find out everything about what I'm talking about, what I'm doing and show notes for all the episodes and more. It's The Delivery Manager Podcast. Hello and welcome to today's episode of The Delivery Manager Podcast, I'm your host, Mario de Cristofano, and today I wanted to talk about, well I wanted to talk about AI again in project delivery and how you can leverage AI in delivery, the things that I'm seeing, but this has come from a chap who reached out to me all the way from Australia, I'm hoping to get him on the next episode, but he reached out to say, hey I've been listening to the podcast, which was obviously lovely for me to hear, he's a delivery person and has asked me what are my thoughts on sort of how I'm seeing AI being used in delivery. The guy's in Sydney, so I'm going to try and arrange a chat with him, but interesting to hear another delivery manager ask about that. So I thought what I'd do in this episode is talk about what I'm seeing, how I'm using AI, and I've kind of touched on this before, but I've got a few practical things that I'm doing at the minute, and maybe if you're in delivery, this will be interesting for you to listen to. So grab something hot to drink or cold, whatever you would like. Take a seat, get comfy, and let's get on with the episode. I think I'll start practically with the way that I'm using AI to what I hope enhance what I do, but also I think the underlying question when a lot of people ask about this is what do you think about it being AI taking our job? And I think like a lot of roles, there's an argument where you can use AI to heavily augment what you do as a digital delivery manager. What I'm using AI for at the minute, and I'll talk about the specific tools that I'm using, is predominantly for automation, information gathering an organisation, assessing risk, and having it as a second pair of eyes to my work. So we'll talk about those in particular, but before that, let me just talk about my workflow and the AI tools that I'm using. So I'm using the paid version of chat GPT, which I use quite extensively, including its deep thinking, Bellamann, and I've started to use the Atlas Browse too, so I'm bringing it into my daily workflow. I use Claude, so I do a lot of vibe coding, and we'll talk about the app that I've built and I talked about it a few episodes ago, Planner. So I use Claude if I want to, in effect, run a software delivery team, a virtual one. I'm using, in conjunction with that, cursor as an IDE, and I use N810 for automation and kind of a gentick sort of development, and a bit of Zapier that I've used for years now, sort of middleware, Boolean, Logic, if this, then that, and between all of them, including in things like my Google Mail and things into chat GPT via an API, I can do quite a lot as a delivery person. I'm using some additional tools too, but that's all for R&D. I find that there's so many tools on the market you can get lost, and I found a combination of tools that work for me, and I suggest as a practical tip, as a delivery manager, kind of you find the tools that work for you, but I don't think you can go wrong with cursor and Claude as a combination from a development software delivery perspective, chat GPT for general kind of sort of AI augmentation on your phone via the Atlas browser, kind of plugged into everything that you can, and then for a gentick and automation, Zapier, N810. I think that's kind of, for me, a sweet spot, for image generation and other things. You can use tools like Benana Nano and from Google, I think, and VO, but again, I'm playing around with those particular tools that I think in terms of what I've weaponized for my own use. That's my kind of tool set. I'll be interested to hear what other tools are the people are using, so do let me know because I'm super interested in that. So that's kind of the tool chain that I use, and then we talked about some of the use cases that I use it for. What I tend to do is I find that a lot of delivery is not cookie cutter, but I certainly take a similar approach, an approach that works for me and that I know generally gets good output, you know, and good results. So I find, regardless of the project, there are some foundational things, and you'll do this as a delivery person to how you kind of information gather or do it as is assessment or, you know, you start with a retrospective maybe if you come into an inflight project. So what I tend to do is use AI for looking over information and sorting it and spotting trends that may be I've missed. I use it to create recommendations that I can then kind of drill down into further. And I also get it to look over project plans in terms of risk, and obviously there's an issue with security, you have to be really careful when I've talked about AI security and kind of a couple of episodes ago, we did a cyber security special, but I'm always careful about that. But broadly speaking, I am using it as a desktop partner in effect. So for project management, it's really useful. I'm not using it to create anything because it's just not good enough. If you try and get it to do any kind of sort of imagery or any kind of sort of spines of PowerPoint decks, it's pretty useless and I'm quite into kind of how my work looks and feels. So I don't use it for that because it just can't do it. But it's certainly as if I had someone sitting next to me, I've kind of trained it on a corpus of my work, my tone, my approach for things like risk and my kind of risk appetite level and the way that I do analysis. And I've set up a project in chatGPT, kind of giving it quite a big corpus of unsensitive information, but you know, things like my blog content and things and all the things I've talked about before and that's pretty good putting like a junior version of you next to you. I find that really, really useful. Get it to look over project plans, look at kind of risk assessment to see, you know, if I've missed anything and I think that's quite useful too. And then in terms of automation and I think this is the thing that you could really leverage for yourself, it's what things are you doing across the course of a week that you could really use AI to help you save time. I think writing up minutes and meeting notes and all of that kind of stuff you can use AI really effectively. And again, I'm not getting into the security of what you're using to record your notes and all of that kind of stuff, but to pull out kind of summaries and trends and things that I think is massively helpful and just saves so much time, that alone. If you did nothing else, I think that's a really good use case. In terms of other things that you can automate, you know, I tend to find, I've got my notes, I run a protocol where, as a fancy, fancy word, isn't it, protocol, I just do something, right? I take my notes, I pull out all the key trends, I summarise those and then I email them to myself as well and I kind of do that at the push of a metaphorical button. So if I'm launched into a workshop and, you know, I need to get stuff done quick, I can rely on that workflow that I've automated and there are a few others too, but that particular one for note-taking and project administration and project hygiene is really good. It's the same for automating and setting up JIRA projects or creating a spine of ticket templates and confluence sort of templates, I find myself, if I'm setting up confluence sites, I generate the same spine and the same kind of set of initial pages, it's consistent. So I've automated that to a degree, I need to do a video version to show you and I just don't have the time or the inclination to do it. So I feel like I'm not doing these things just this that I talk about, but you can Google on the internet how to do it just like I have. So I tend to find that saves me time and if I look back through all the kind of approaches and techniques and things that I've learned in terms of doing delivery and then look at how I'm using AI, I can definitely see where I've saved heaps of time. So that's really, really helpful. The other thing I do, which the notion of a hackathon isn't a new thing, right? But what I tend to do now is get people in a room together and if we've all got something to achieve or deliver, sit and literally vibe out that idea, everyone sits in front of chat GPT or an AI tool of their choice and actually works through a bunch of idea analysis and ideation and kind of looking through a problem deeply and talking about it round a table. It's kind of changed ideation to a degree. I think you can do it quicker and much more in depth if you're brave enough to sit with your laptops, with AI open and actually sort of hate the term vibe it, I know, but I'd get battered on LinkedIn for saying anything like that. But that's what I do and it works and if you can get people that are willing to kind of do that, I think you can get to, it's almost like 5Y analysis really deep but really quick and you can really drill down into a problem or an idea and actually along with kind of your delivery approach, you know, post it notes on a wall, quick synthesis, being able to articulate ideas quickly, you can take all of that along with automated note taken and actually do a huge amount of work in a very, very short amount of time. So I've worked for agencies and consultancy firms where this would be a week long process. I reckon you can get it done in a couple of days now with fewer people too. So there's some ideas of how you can use AI in delivery. It's not necessarily what I'm seeing elsewhere, I think there is a thing of people not wanting to put the hand up and say I use AI when they invariably will be. I'm more into kind of talking about how I'm using it, the problems that I'm finding, etc. The other thing I tend to do if I'm going into a vertical that I'm not familiar with or less familiar with, I'll get it to do some kind of initial kind of sheet dip for me in terms of, here is Mario who maybe doesn't have a deep knowledge of, say, fast-moving consumer goods. You know, like I'm a young university graduate in turn, and give me an overview of the sector trends, you know, those doing it well, those doing it bad, that kind of thing, and do like a one-page report. I find that quite useful too, again, for kind of self-learning and sheep dipping myself into areas that I'm not quite familiar with. So that's a good use of AI. And then moving on to kind of software delivery, so a lot of delivery people, I'm not sure if this guy in Sydney is in the software delivery sort of field but I reckon he probably is. And actually, how you can, and it's not really vibe coding I'm talking about here, but certainly I've thought as a delivery manager, what are the tools that I could spin up that would help me? And I talked earlier about, you know, I found myself doing some things again and again, kind of rinse and repeat. I talked about creating a brand for myself, kind of a corpus of work that I can constantly refer back to, documents and templates, and all that kind of thing. I then underpin that with a conversation that I have via the podcast to talk about and underpin this learning continuously, and then I can refer back to it. I can also use the podcast as a launch pad for conversation with clients or teams, that kind of thing. But then I thought, well, I'm talking about these things, how can I kind of bake that into what I'm doing too? That sort of, in probably four hours, have created this tool called planner. And the idea of it is that you can give it some metrics and parameters, and it assumes a start point of you dropped in as a delivery person in an organization, it doesn't really matter whether you're a full time or a contractor or part of a delivery team. And you can plug in various things like budget, kind of risk, politics, psychology, state colder mapping, and a few of the parameters like the type of project, it takes a few minutes to kind of fill in them. What it does is it spits out a sample plan, an approach, a technology approach, or the kind of key baseline documents that I'd look for, kind of high level architecture, software, technology, platform and stack recommendations, links to those solutions, rationales to why. It starts the spine of various business cases to get investment. And it kind of creates a launch pad for me to move forward with my delivery. It's not that something I just kind of go to feed it, sausage me, and take the sausage that comes out, but it gives me that starting point and reduces the think time that I need to do. But also, it points to various podcast episodes of the delivery manager podcast that talks about that type of project, I've done nearly a hundred episodes now, so there's a good chance that I've talked about, a project variant sort of type in one form or another at some point over the last couple of years, recording these episodes, so it actually refers to a podcast as well. And when I'm thinking on how to start something, I find it really useful. Now, it's probably still quite ropey for a kind of even a beta release, and I've put it online before, and then I took it down, but I'm still kind of developing it, but actually I split it up again today, and again, I used cursor and clawed to build it. And it's actually really quite good. So again, this is something that would probably lean more to doing a video, and I don't really want to do that at the minute, because I don't have the time, but that has been a really good way of sort of building a tool for me. So I think what I would recommend for delivery managers, and what I'm going to predict I'm going to see, is people building their own kind of discrete toolkits to kind of save them time for me. This is it. It's my Swiss Army knife, so when I go into a project, I'm going to pull this tool out, and I'm going to feed it some parameters, and it's going to give me a base set of recommendations and an approach. And I might disagree with that approach, but if I think I've done it right, and I think I've kind of done it how I want it to work, it should give me a fairly balanced, non-biased, industry, safe project approach with rationale and data behind it too. And then from that, I can move forward really quickly. So if you've got this kind of launch pad generator of how to start your project and get those foundational bits right, whereas before you might be scratching your head and thinking and pontificating on how to do it, and then you're doing your as is sort of assessments, what I said would have taken a week, now I can take two days, you've kind of shaved off a huge part of the project inception bit, and I reckon it's such a good way of working, and we're seeing this already big, consultancy firms are going to automate that bit. So I think the days of sending in delivery teams to do those as is assessments, and kind of ideation with the client, I think are probably going to dwindle. I think that's the bit in the project life cycle you will see go first and be lost to AI with fewer people doing it. So I find that quite interesting to talk about, let me know what you think on that. So some of the challenges with having access to AI and AI stuffed into your project and everyone using AI. So I talk about self-organising teams, and if you're a delivery person you'll know that it's really important that you get your team right, and you get information hierarchy sorted, and you have information passing through the team appropriately, you're open and all of that, there's a bit of rigor and structure and logic around how you plan, how you ideate, how you kind of work through problems and issues, and how to get something done. And what you want is you want as minimal chaos as possible. You can do that with interactive sessions, you can do that. With deep dives, you can do that with ideation sessions, and again, we've talked already about how you could use AI for that, but there is a problem. The problem is that, just like you have kind of shadow IT, you have shadow AI, but you also have the shadow behaviour around it, and because access to these really clever smart tools are more ubiquitous, and the barrier of entry is non-existent, you often get teams actually in quite a lot of disarray, because everyone's come into the table and they've already done it in AI, and dependent on their prompts, and dependent on their approach, and dependent on the lens to which they look at the project, you get wildly different opinion and approach, and people have some cost fallacy, they use these tools, and then commit to the output, and I think my concern with delivery is you lose that human critical thinking capability, and I'm seeing that more and more, it's very easy, you see it in yourself, because you see what the tool spits out, and you think, well, it sounds reasonable, it's not a million miles away from what I do, let's run with it, in an effort of saving time, and of course everyone's under pressure to do more, with less and save time and therefore money, so it's almost a compelling impossible to avoid temptation to just overly rely on the output from these tools, and what that means is as a delivery person, you'll now find yourself in the middle of developers that are using AI to write code, testing teams using AI to kind of go through code reviews, instead of doing it themselves, you've got project planning being done with AI, using all the techniques and things that we've talked about already, and none of this is, oh, and then you've got the stakeholders as well come into you, they've, I've coded the solution, so instead of a client saying, well, I think we want this, or I know we want this, and then taking them on that journey of discovery, they'll already come aggressively, even more so to the table with an idea that they think is right, because they put it through AI, and it creates quite a mess, and I see that in teams right now, as of today that I'm talking about this, I'm seeing it and I'm in it, and AI is massively responsible, and it's creating even more project uncertainty and instability. So, on the one hand, what you see in being reported is that AI is going to, you know, remove thought workers, jobs, consultants and all that kind of stuff, and to a degree, you know, I think that there's a certain element of truth in that, but that's for another conversation, but what isn't being necessarily noted or talked about is actually having AI present in your project actually just creates such an unstable foundation, an unstable landscape that increases your mean likelihood of failure. So what I see before projects getting better is they're actually going to get worse and project management in a pretty bad state is it is when it comes to technology projects, I mean, most fail, right? So it'll be interesting to see the impact AI, AI has, but it's almost that superpower kind of mentality of, you know, with great power comes great responsibility. I think we'll see a lot, a lot of mistakes and immaturity around its use, governing in it within your team, the governance and assurance of AI, I think is a very, very murky, difficult world, and I'm not quite sure what the answer is yet, but I'll talk about what I do in my teams or try to do, and maybe you can take this to your teams too. So what I try and do now is part of the delivery standup, whereas before we would stand up a Giro and Confluence site and project, we would, and I would populate it with the same kind of way of working information that I've used for quite some time, kind of agile ways of working, story point estimation guidelines, how we'll do planning poker, diorization of all the agile events, et cetera, et cetera, you get the team together doing initial project overview, go through the commercials, the targets, the milestones, all of that kind of stuff. Now the thing that I bolt onto that kind of project setup or that kind of team setup is now AI use and how we will and will not use it to basically put in some team agreed guard rails on what tool we'll use to make sure that at least some of the output and approach is consistent. We'll use consistent prompting, thorough, deep prompting as well. So another area on the Confluence site or whatever project tool that you use, now think about some lightweight guidance on prompting, the security around that, what data you feed it initially. So in chat GPT, I would create a team project that's trained on some security safe data around team approach, risk appetite level, nature of the team, all of that kind of stuff in an effort to get more thorough and deep responses and get a team commitment that will be using the one tool and the one approach and we'll err on the side of caution making sure that we use AI and play with AI within the project within the constraints that we've all agreed to to hold ourselves accountable for. That becomes ever so more difficult when you're working cross teams, cross providers, cross consultancy firms. It's just really difficult. People now have that extra tool in their toolbox to be able to pull out and use it and weaponize it as an opinion, as facts, but I try and find going sort of best foot forward with at least an intent with the team is helpful and I would really recommend that you think about doing the same. And then finally, I refer to this resource a lot, it's because I'm a really big fan. The Gov UK website, including a blog on defradigital.blog.gov.uk, includes commentary on using AI to accelerate project and product delivery, especially during the discovery phase. Now I'm a big fan of the UK Gov's GDS framework, have worked alongside it for quite some time and used it as a personal model for how I deliver digital product and service and I thoroughly recommend that you do check that out, but it's interesting to just kind of go through some of the things that it says it's using AI for and some of the things that it finds AI is not great for. In particular, the reality check it mentions is, and I'll quote, "While AI has been a powerful accelerator, we quickly learn that it's not a silver bullet. It's not yet suitable for tasks like stakeholder mapping, planning or managing risks and issues. It's a great assistant, but it can't replace critical thinking and expertise." And that's interesting. I agree and concur with that, although I do use AI to sometimes create the collateral. It also talks about that AI tools not being created equally, so it mentions and goes on to say that while we have access to Microsoft 365 co-pilot chat, we found that other tools sometimes produce better outputs, additionally tools like Google and OpenAI, have limitations on the amount of data that they can analyze, making them useful for analyzing smaller data sets, but not for large scale analysis. If you've ever worked in public or central government, you'll know that typically, especially around data, normally playing around with massive data sets. So again, I agree with that. And then the article just summarises, and I'll include the link to this in the podcast at Show Notes, that they've generally learned that AI is really good for pace, so AI-enabled teams can move quicker, and administration for things like core recordings and transcriptions. They do say a mark that as a note of caution that they learn the hard way that the note taking is not always accurate. And they've said, "This is why we've made it a rule, always manually check transcriptions for errors before using them with any AI tool. We also make sure we check any AI output for accuracy too." But they're generally saying that from a delivery perspective, certainly in that discovery phase, that pace is increased. And they're building traceability paths to show how AI reaches its conclusions. And I think that's interesting and important, where you're using AI also include your prompting and rationale and data that you've fed it, rather than just parroting and citing the output. And I think having that traceability rule set in your AI governance framework for your delivery team is probably a really important thing. Efficiency, obviously, it sites as one emerging observation is that the work is surface in opportunities to make some of the wider organisational processes more efficient, which I think is really interesting. And also the measurement of that too, not by speed, but whether the findings genuinely reflect the user's reality. So using AI to kind of go through, I think it mentions kind of user discovery phases and talking about doing initial, using it as like a research assistant and drafting user research materials. And it categorically says that it's saved a significant amount of time. I don't know what significant means, but I know how long it takes to do user research. It was a constant bottleneck in projects that I've been involved with. So just because to do it well, you need to spend time on it, right? So that's interesting, but really, really good article, and then just finishing on the big takeaway. So it says that our experience is shown that AI is an incredible tool for accelerating specific parts of the discovery process, especially research and understanding of the problem space and policy intent. The key is to see it as a powerful assistant, but not a replacement for human judgment. But combining the right tools with a collaborative mindset and a strong human in the loop approach, we've been able to make a discovery phase more efficient and effective. I think that that is really, really useful and a great blog article. So check that out, and I'll include the link to that in the show notes. So I think the main takeaway I would like on this episode is to just revisit the things that I am reminded myself that I will do next time, but also what I'm trying to do in the various projects that I run. So to recap, some of the things that we've talked about, and I'd really encourage you to think about this too, is when you're at that point of putting together a team or putting together the inception and ideation of a project, to outline as well as including things like your agile way of working manifesto and diorizing all your agile style events and putting together how you'll collaborate your estimation efforts and share information across your team. And as a delivery manager, you put all that rolling together, hopefully in a collaborative way that everyone is agreed to and brought into. Now also, adding on top of that, how you'll use AI. So almost like some AI governance guardrails specifying the tools that you'll use and why thinking about how you'll prompt AI and being sure that where you're using it for data and research discovery and analysis, the outputs that you cite, you also include how you've prompted it and the rationale behind that I think is really important, especially if you run a data and research team. I think that's really important. Thinking about the tooling that you'll use and the rationale behind the tooling and also the quality of your prompts and maybe putting in a side, training the team and sheep dipping them in consistent AI prompting within the constraints of your project industry and the kind of language that you speak. And then of course, it's setting up dedicated projects on enterprise grade AI tools, making sure that that AI is trained on relevant and suitable kind of sentiment and risk appetite and kind of how you do projects. And you can do that as a project manager. I'll be thinking more how to do that, maybe through some kind of a gen TKI. So every time I use enterprise, sort of an open AI product or an enterprise kind of anthropic product that I've got a standard kind of set of training to be able to at least give it a starting point. So we get in some kind of consistent results the same consistency that I'd expect from just doing it without AI really. And maybe you should think about that too. And then finally, again, that security and governance and you'll probably now need to be working more with the IT teams more than ever. And the legals and the contracts and if you're involved in bids and contract negotiation and contract management and authoring them and you're now going to have a big old AI section around sort of GDPR and PII data, but also now how you use in AI, how your teams will use AI, how your past data, how you'll use it for analysis and maybe start thinking about having some boilerplate copy written that's not aspirational, that's accurate to how you work as a team. So you've got that ready because that will save time. These are the things that I think are going to create bottlenecks and inefficiencies in projects and project delivery teams certainly over the next few months. But I'd be really interested in understanding what you're seeing, now you're using it in project teams and hopefully on the next episode where we talk about this, I'll have a guest with me. So thanks for listening to this episode of the Delivery Manager podcast. I'm your host, Marya de Cristofano. And as always, see you on the next one. Thank you for listening to this week's episode of the Delivery Manager Daily podcast. I hope you've enjoyed that. Do share far and wide with all your friends, colleagues and peers who you think might be interested. Go to marriesblog.co.uk to read all the additional content that I tend to post. And find me on x@marryobc. If you've got questions or comments or you want to leave feedback, do please subscribe. Click that bell notification so you get notified every time I release a new episode, which is usually at the minute, every Friday at about 8 a.m. So do please subscribe. It really helps me out. Thanks for listening.