Generative AI in 2024: How to Move From Experimentation to Implementation

[Music] Welcome to Gardner Thinkcast. I'm Karen Stokes-Lockhart. Today, we're taking a look into the future. 2023 has truly been a transformative year for businesses and technology. So let's take a little deeper into some of these emerging trends and take a peek into what we can expect in the coming year from one of our favorite Thinkcast topics. You guessed it. Generative AI. What you're about to hear are two episodes of our video series, Top of Mind, with Gardner Global Chief of Research Chris Howard. First, we'll hear Chris Live from Gardner IT Symposium Expo North America, where he shares three key themes from the Gardner Top Strategic Technology Trends report. In the second half of our episode, he'll cover generative AI in 2024. How businesses can properly move from experimenting with AI to a full-scale implementation plan. Now, here's Chris. [Music] One of the sessions I've done here is the Top Strategic Technology Trends. I thought I'd take a couple minutes and just walk you through that at a high level. The Top Strategic Technology Trends fall into three main themes or categories. The first of these is the protection of the investment that you've already made, especially as AI becomes more part of that story. The second of these groups of trends is about supporting the rise of the builder, both inside IT and outside of IT. And then the third set of trends is about delivering value from these investments that you've made, in platforms and in people, in AI and other innovations. As we think about protecting the investment, it's really about what has changed because we've started to insert more AI into the work that we're doing into the systems that we're building. And AI creates a different kind of attack surface that requires different types of approaches. And this is everything from mitigating bias and grounding with better data to taking care of model drift and collaboration using model ops and even LLM tools, LLM ops tools as you're building. The other is that it's caused a different kind of threat that's happening all the time. So Gardner talks about this in terms of continuous threat management or CTEM. And this is a both a process where you're interacting about the business surface itself, the business as an attack surface, and negotiating with the business to say, what do we need to protect and how is that changing at any moment in time? Then we talk about sustainable technologies. So it's really about protecting the future. And so sustainable technologies is a set of engineering technologies, sets of data and how it's used, how it interacts with systems, helping make better decisions that actually have long range impact on the planet itself. There are also social aspects to that as well. And there are implications from large language models that were starting to really understand in terms of their impact on the planet, how to minimize that impact. And so paying some attention to how those technologies are evolving and how you're actually bringing them into your environment is really key for 2024. The second group of trends really about supporting the rise of the builder, one of this is about platforms. So you think of a platform as something that makes hard things easier to do because it standardizes, stands, you have common capabilities. Maybe you have wrapped regulations and the ability to actually do things that fit within the way that you're supposed to build and supposed to deliver. One of these is the idea platform engineering, where you have groups that are both on the business side and the technology side working together to figure out what is the set of capabilities that enables us to build and deliver solutions that are both technical as well as very business oriented. And this idea of platform teams, platform engineering, being encapsulation of standards comes with platform engineering. The next piece would be the idea of intelligence becoming part of the tools themselves, but also the developer experience, the extended to Genai, sure we're paying a lot of attention to this in terms of cogeneration, but there are other ways that Genai can be used to impact the whole development lifecycle from requirements gathering through documentation, through tests and so on. So this is a great productivity help for developers themselves. The other thing that's happening is that the applications are becoming more intelligent. And so vendors are starting to include these kinds of capabilities like Genai capabilities into the tooling to actually help support workers at all different parts within your information, workers or frontline workers, wherever they may be, in helping bring data into context for them to make them more effective in their jobs. The value theme really is about how do I use all of these things to actually produce a value for everyone. Now what I last talked about was this developer experience, employee experience, turns out that this is one of the really great ways to introduce value into the organization to improve the employee value proposition. So if employees are getting to their goals faster, they're actually enabling customer citizen patients, you know, in their goals too, which binds them to you as an organization and actually goes towards growth. Another thing, which is an interesting topic here that is maybe a little bit more future for some of you is this idea of machines as customer, machines creating a different value vector for the organization as machines actually get more involved in procurement. So for example, a car ordering its own oil changes or refrigerators orders these are some we've all had some experience with that, but this is going to become more ingrained in supply chain and you're going to have to start figuring out ways to actually market more effectively to the machine who doesn't have emotions, but is actually very specific about price and supply chain issues. So it involving set of things. So again, you have a core that you've been building over time and there's a need to really protect that in even more effective ways, giving the introduction of AI. We're supporting a whole new type of building with software engineers, new tools and approaches for that, but also looking at ways to produce value because ultimately that's what all of this is for. There's some new techniques for doing that. So that's some highlights from the top strategic technology trends for 2024. This is a really interesting time of year when the midst of so many big gardener conferences or IT symposium, our HR reimagined a number of other events that are taking place over the end of the end of the year. And what I've been watching for is to say, well, what are clients really doing? What are they struggling with? What are they planning for for 2024? And something becomes very clear because AI is still such a dominant topic across all of these conferences, is that people are really starting to move from a period of long experimentation in 2023 and preparing for more of a implementation year in 2024. And the numbers here are quite interesting. If I look across my clients, not 100%, but a large number, like north of 90%, happen doing that experimentation, but only about 10% say that they're moving into some kind of a skilled production. So I thought it might be interesting to talk about some of the reasons that are keeping people from moving quickly into production. It's not that they don't believe the value of generative AI and AI technologies. It's just they've realized that it's harder to do perhaps than they thought it was going to be, so winter has come. One of the examples here is grounding. So grounding we've talked about before, this is the technique that you use to create a more accurate, generated response by grounding it in your own data. So the large language models, the big horizontal language models, teach the system how to speak. What grounding does is it teaches this system what to speak about. So you're using your specific data in that, in that instance. The actual technique of doing that actually takes a lot of work and lots of experiments in terms of how do I create even more accuracy. And there's interesting innovation happening in the input and output filtering of prompt engineering, which is creating more detail in terms of validation within the prompt itself. And what's happening, of course, is the accuracy is getting better, but a lot of executives are still concerned about, you know, does this have to be 100% accurate? What cases can I use it for? What should I not use it for? And so there's some clarity coming with the experimentation, but it's still really hard to do. Another thing that clients are starting to question are the productivity claims around generative AI. Now there's no question that in certain use cases, we're seeing significant productivity gains like in things like code generation or even just creation of materials that are used say for marketing purposes, those types of things. But what we're looking for is like real data about what that productivity is, and is there something different about artificial intelligence that changes the calculus of productivity that actually makes an exponential change. That's one of the things that partners can be looking at in 2024. Something else that our clients are experiencing is that they're as they move from experimentation to implementation. Experimentation was often done by tiger teams or people that were doing it kind of in extra time, but we're moving into a much more dedicated teaming environment where maybe you require center of excellence, or some other change in the way that you operate in order to make the use of these technologies efficient. So this is something that happens with technologies all the time. So we're in this transition phase to see how should people work differently when machines are actually part of the team. How do you operate differently and trying to come to terms with that which may mean some hiring or some restructuring in order to create that team that is specifically dedicated. The other thing about that operating model is it needs to be cross the enterprise because so many different parties within the enterprise are trying to see how to use AI within their function within their products within IT itself. And so this team really needs to span the entire enterprise regulation. This itself is a big topic and it seems like there's news about this every day every week. Recently we had in the US in executive order. There are laws coming in Canada in the UK and other places. And so many companies are kind of hesitant to see well, how are those regulations going to shake out so that I know that I'm working within the boundaries of what is deemed to be safe and responsible. So that causes a little muting of the innovation kind of waiting for the regulations and policy to be put in place. Just a word about REGs and policy. I've spent a lot of time thinking about this and interacting with governments over the last several months. Regulation and policies are actually essential for innovation because it calms out the space within which you can operate. And knowing where you're going to stay safe or where you're across that line is actually very helpful in terms of experiments and scoping use cases that you're going to pursue. So keep an eye on this space and we'll certainly talk more about it as regulations come into focus. Security. Another reason why there's some hesitance to move from experimentation into production. Part of that is the REGs and policy. But actually when you're using AI and especially Genive AI, some new things emerge as either attack vectors or risk vectors. Things like model drift, whereas the model as it's being used, the data actually starts to drift and become less useful potentially or potentially more dangerous. There are more nefarious things like model poisoning, where you know, Molly gets injected with perhaps intentionally fake information. And then it actually changes the whole dynamic of the model itself. And so there are techniques within model ops and the emerging LLM ops. So at the time of development, you're building more secure and model sitter less burdened with bias, those types of things. But this whole attack surface of model from development to runtime. Is the emerging space that people need to get their arms around related to that? Of course, is responsible AI knowing how decisions get made, being able to track and do attestation. Genitive AI isn't really great for those types of things yet. But what that means is that there's a whole lot of innovation happening inside the prompt engineering space where you can actually use techniques to validate the sources that were cited perhaps. Or to interrogate the generated output before it actually goes live in generated output. So a whole lot of interesting innovation happening there that we're also covering. And then finally, the biggest question that my clients have is, what's the value of all of this? What's the value that's going to be produced by applying these technologies? And this value comes in multiple forms. Sometimes it is actually creating a new revenue stream, or it is reducing the amount of effort required to produce something. So the margin becomes richer. Certainly those productivity gains that we talked about earlier. But there are other values that were pursuing here that may be social values. So things around diversity and equality, things like healthcare or education, those are a form of value. And so what we're doing at Gartner is actually tied to articulate that's those different types of value and how they line up with use cases. And that'll be a big part of what we're doing over the next several months here at Gartner. So to recap, what we're seeing is that maybe the new car smell has started to come off of generative AI, which we expected. And that's just fine because what happens is when height burns off, it settles into the trough of disillusionment. And you've heard me talk about that trough is simply being the place where work begins. And I can tell you that across my client basis is exactly what I'm saying. Instead, the work is beginning. What Gartner is going to do is try to help there in terms of deep implementation advice, which becomes very technical. But also strategic advice in terms of how best to apply to what kind of value to expect. We've created a new framework for you to use that I'm going to introduce here, but we'll talk about more in some detail in a future episode. And it's called an AI ambition radar, very simple tool, which you can use to say, where do my use cases fall and where am I going to apply what we're calling everyday AI? So things for automation and back office, perhaps. But also game changing AI. One of those things that AI enables is you've never been able to do before that creates an exponential value. So more to come on that. I want to encourage you to keep experimenting, but also choose those places where implementation seems likely to produce value. Think about everyday AI and game changing AI. And share what you're doing with us and with your peers so that we can evolve together as a tech community in the application of this very important technology. I'm Chris Howard. This has been Top of Mind. Thanks for joining me today. That was Gartner Global Chief of Research, Chris Howard, from two episodes in our Top of Mind series. You can catch new episodes of Top of Mind every two weeks on Gartner's LinkedIn and YouTube channels. And if you'd like to listen to the full top strategic technology trends presentation, you can check that out in a recent ThinkCast episode as well. ThinkCast will be back where we listen to podcasts two weeks from today. In the meantime, please rate, review, subscribe, and share with a colleague. So neither of you will miss it. ThinkCast is a production of Gartner. This podcast may not be reproduced or distributed in any form without Gartner's permission. It consists of the opinions of Gartner's research organization which should not be construed as statements of fact. Content provided by other speakers is expressly the views of the speaker and/or their organization. While the information contained in this podcast has been obtained from sources believed to be reliable, Gartner disclaims all warranties as to the accuracy, completeness, or adequacy of such information. Although Gartner research may address legal and financial issues, Gartner does not provide legal or investment advice and its research should not be construed or used as such.