AI and engineering productivity: Debating the headlines

Welcome back to the Engineering Enablement Podcast. I'm your host, Justin Riyad. This episode was recorded live at DX Annual. Throughout the day we'd been hearing a set of assumptions that are circulating in executive teams and boardrooms right now. But AI will mean fewer developers, that adoption has to be mandated from the top, the code review is the new bottleneck. These beliefs are rarely black and white, and so we close the event by convening a panel of senior engineering and research leaders to debate these questions in the open. On stage we were joined by Rafe Coburn, Chief Product and Technology Officer at C, Jessie Adametz, who leaves platform engineering at Twilio. Irene Calli Ambaku, a developer productivity researcher from GitHub, Colin Green, a senior staff UX engineer at Google, and Brian Hauke, co-author of the Space Framework. Each was read a series of statements and asked to weigh in. Let's get into it. Our final panel here brings together five leaders from five different vantage points. We have two practitioners, Rafe Colburn, the CTO at Etsy. We have Jessie Adametz, who leads platform engineering at Twilio. We have Irene Calli Ambaku from GitHub, who is a developer productivity researcher. And then we also are going to be joined by Brian Hauke from Microsoft, as well as Colin Green from Google. Welcome to all of our panelists. I'm going to be in a mic issue fix. Maybe while we're waiting for Colin, I'll just kind of explain so everyone can be aware. This is a little bit more of a unique format. So the way that we're going to do this, I'm going to be reading a series of statements. And our panelists are quite literally going to give us like a thumbs up, a thumbs down, kind of where they're at on this topic, just as like a gut reaction. And then we'll get started. Okay. So our hope is that you all will walk away with some new perspectives. Hopefully that felt pretty on theme for how the whole day will today. Just like the last session with Uma, there's not going to be any QA Q&A during this time, but still do the same thing. Feel free to put questions. And again, you'll be able to talk with our panelists during our reception immediately following this. So let's get right into it. So our first statement, an AI first SDLC means fewer engineers. Rafe, will you start us off? I'll give my answer for you. Give us a take. Sure. So I think, totally to tell, I think probably it's true that, you know, a job is a bundle of tasks. And for sure, a Isaac Lee Institute for some of those tasks and people might not be doing those things anymore. On the other hand, the demand for software does not seem to be going away. It seems to be going up. The price of building software is going to go down. And so I don't think that less engineers is likely to be a scenario in the near future. It's just, I bet against it. I think if I could just like add something to that, like I definitely agree it's not about fewer engineers. It's about doing more. But one nuance to be, you know, maybe a little controversial is like in the future, what do we even define as a software engineer? And so like the bundle of tasks that we think of today that make up a software engineer, I'll be and I talked about it in the opening where it's, you know, writing code is a core part of the identity of a software engineer. And so like the people who do that, the things that we think of make up software engineers, that may go down, but the total number of makers of builders like, I see. Yeah, no, yeah, we're on the same. Yeah, yeah. Yeah. Any other shakes? I think, I think I could add that you'll be able to do the same amount of stuff with fewer people probably. In the future, that doesn't mean, as I said, that the demand will go down. And I think actually small companies that want to accomplish something will be able to get sort of over that barrier to entry faster. So maybe actually increase the demand. We're agreeing too much. We're going to make it harder on you. Okay, next statement. AI is currently creating technical debt faster than it is helping us refactor it. Give us your quick reaction. Yeah, Irene, can we hear from you? Well, because you said currently, so I do think that at the moment things with cogeneration are moving as such a pace where like the rest of the system hasn't necessarily evolved enough to match it. So we're definitely seeing that. I think we're also seeing the anticipation of technical debt kind of struggling. How much developers are using agents. Because it's certainly something that we saw. I get to have a some point because they're trying to balance the risk of something with the velocity of something. And it's kind of like mental math that needs to happen in that moment. So I do think that there is a trend there, but there are also ways I think we heard. I think it was the Airbnb talk. There are also ways to have workflows in place. That's one solution that we're working with. I'd get how you can try to manage against the accumulation of debt by having agents that are event driven or run on a schedule that try to do a lot of maintenance and health and hygiene for code bases. It's not a perfect solution. And I'm sure that eventually things are going to evolve differently. But for now, it is something that is working. But yes, the accumulation of technical debt is certainly a big issue. I just think it's too definitive statement. I think we look at our engineers in quartiles and our highest performing engineers who are our highest performing AI engineers. We'll hear this a couple of times because it's on my mind. They'll be like, "As an amplifier, we saw that a few times today." So if you've got high performing engineers, or if you don't, it's kind of garbage in, garbage out. I may get quoted on that now, but our highest performing engineers weren't putting garbage into the system and they're still not. I think I was going to jump in and say, "I've worked at the same place for 14 years at Etsy." And I would say proportionally, "Are we creating more tech debt now than we were five years ago, 10 years ago?" No. I've seen the old code is still there. And so, of course, we're creating a greater volume of code. But I'm not sure a greater proportion of it is tech debt. I think automated code reviews, I think, are helping a bunch and you can ask a coding agent to look at the patterns. There are a lot of things you can do to write better code, I think, with a coding agent. So just a greater volume of code for sure probably brings in absolute terms more tech debt. But I don't think it's an accelerator for tech debt at a greater rate than it's an accelerator for anything else. I am envious of all of you. Because my experience is different. I think that you have a couple of issues. You get the outcomes that you're optimized for. And how many talks that we've had today have been about, "Oh, increase PR velocity, increase PR velocity." We are optimizing for speed, not necessarily cleanliness. And I think with that, you do sort of get an absolute terms, maybe not in relative terms greater tech debt. But sort of a concept that we've also talked about a little bit today is this notion of cognitive debt. And I think that is a form of technical debt, whereas if we understand our systems less, then they are less resilient to us being able to modify them and act on them in certain ways. And I guess maybe we'll be able to kick that can down the road. It's like, "Oh, the agents will take care of all of that." And so I think eventually the statement will be true. But in the short term, I actually say that tech debt is not, it is in fact going up, I think, relative to our PR velocity. >> It's an interesting bit of what you just said. Technical debt being incurred is not strictly an engineering decision. Right? So, if we have a decision about why do we need to go fast now, what do we need to achieve, what are we trading off for later? So if businesses still have the same overall tolerance for that risk, and if they have the same market pressures to incur the debt to get where they're going faster so that they can be first or be better, whatever, we'll incur the same total amount, I think. It may be a little more volatile, especially right now, because we can go faster. And if we view development and agentic development as zero cost or low cost, that increases the risk that will not make good decisions, prudent decisions, about technical. >> The choice of what even are you building becomes so important, yeah. Okay. In five years, more than 50% of code in most organizations will be written by AI, that reactions. Colin, let's start with you. >> I'm interpreting that as like new code, not 50% of a code base, so I think that's important because the answer might be different otherwise. But I think more than 50% of a new code, maybe even quite sooner than five years, I think it's worth thinking about whether that's code that would have been written by humans otherwise, versus there's just a larger volume, and now the agent generated code is starting to sort of take over a bit. I also think it's worth talking about how much of that code will be rapidly rewritten after it's written. >> There was some, was there a dissenter? >> I was interpreting it less than the other way, right? I think like, because I think we've seen those headlines too, right? It was like, well, all the code now, it's just all, it's like, there's no way. There are too many companies that are too old to have too much legacy software. We're not rewriting all of it. There's no banished rewriting all of it. So yeah, if you interpreted that way, I think those claims are. >> Yeah, I think so. >> Yeah. >> Yeah. >> Yeah. >> Okay. The future of software engineering is more about managing agents rather than about writing code. >> Yeah. >> Yeah, I don't know. >> Brian, let's start with you. >> So I think in this like idealized world, that is true. As it turns out, as humans, we aren't actually great at multitasking, and I think that we don't, at least in the short term, have the skill set necessary to do this. I think what we're finding is people who had prior management experiences, an example, who were used to delegation and used to sort of being able to context switch between lots of different things. They are going to be very successful at this. But we've actually seen with some of our internal research at Microsoft that even went under time pressure. Developers will often even experience developers. We'll revert back to sort of sequential, agentic workflows. Because our brains just don't really work very well that way. And I think it's going to be a really tough learning curve. And it's a different set of skills that we're going to have to be hiring for that we don't necessarily have today. >> Yeah, I think I put my thumb up. And even that is good. And I think, so I guess maybe the plural is the interesting part of it, right? Well, you manage an agent. But what's really funny is that even if you're using like Cloud Code of today, you're managing multiple agents. It's just kind of doing it for you. It spins off some agents and it does things for you and it comes back and reports back. And so I think that's another thing that's a little getting abstracted away. I think probably this idea now where you have people in their hobby is like building a crazy harness where they're managing 15 agents at the same time and doing all this stuff. I don't think that's the future. So the tips you need to do it AI will help you do that too. You're not going to have to do that. I think we'll a really large chunk of your work be mediated by one agent at minimums. >> Yes, I think that's true. But I totally agree. Like, you know, agent orchestration is not a thing for humans. You really is as an old age. >> I think this question goes back to the handful of topics that I think have been coming up throughout the day. It's like so one, as a director, I have a propensity to use AI in the way that we're suggesting. I'm managing agents and that's just where my brain goes. But we come back to like the identity of a software engineer that the coders, we're talking about changing job descriptions potentially. And it's almost like maybe in the future we find that that role, how we have it today is actually useful for this type of work. And the shaping it differently is this. But I think it's maybe goes back to the other topic it touches for me is the whole like agent experience versus people who's good, it turns out again, the bright thing here is what's good for the people. Right? We can't just assume that, okay, everybody's now just going to be a man. There are people that inherently don't want to be that are not good at it. Don't want to be good at it. And so I think, again, at least maybe it's too early to say, but like the blanket of like this is what it'll look like. You'll all have to manage. I think that's crazy. It doesn't mean that you won't have to work on your communication skills, which has always been always been true. The thing. I think there's a lot of always been true here because I think even when I was an IC engineer, I was at managing agents. But would I run a slow running task in one window and a fast task that I had to think about in another one 100%. Like now do you run two agents in different tabs? So this one's going to take 15 minutes going through the code base and writing some research on this one. I like I'm an active mode 100% and so I think that, you know, people have always juggled. I think they're still going to juggle that the management word is weird. I don't think it's right. Managing agents, you're just communicating through a different kind of UX. I was going to say something to that effect. So the managing, I guess I'm taking it a different way. I'm reading it to management as like supervision. And that seems like not as substantial a role as we see it be in practice. So I've been interviewing developers who are like really advanced with their AI use for some time now. And over time they describe their role differently to the point that the ones that are at the at the frontier, they do see that what they're doing is orchestrating agents, but not from the multitasking aspect and not from the baby sitting aspect. They're looking at all the work that they have to do to set up an agent for success so that it can do good implementation. So everything that has to do with defining intent probably has to do with the communication skills. Yes. So yes, defining intent, setting constraints, setting guard rails, giving context. All of those things are really important and then verification after implementation happens. And that's real engineering work is not just sitting back and checking out the agents, right? Is real engineering work that is happening at a different level of abstraction? But still engineering, different flavor, different abstraction. But yeah, so that's that's what it looks like. Things are headed in terms of the professionally very much. Like, that's what we meant with the sentence. That's how I interpreted in my head and that's definitely a thumbs up for me. Definitely, micro managing agents is failure mode, right? We don't want to see that because then you run the task for checking the bottlenecks. Yeah. That is kind of what Stewart, in his talk about how he's hearing feedback from some of the engineers at Netflix. It is kind of how he characterizes them being like a little bit needy. So it does sound like it could go down that route. Okay, leaders need to mandate AI usage as a way to make sure that adoption is moving along as it should. We all strongly disagree. Jesse, let's start with you. It's just not what we've seen. You know, we did, I think we did a little bit of enablement. If we didn't have a top down mandate, we did a little bit of enablement because there's no reason to reinvent all the same wheels. So we enabled like, here's how you should install it and say in defaults. And then after that, it took off. I do think maybe the thing I would comment on this one is, you know, if the people who don't come to this conference who are in the rest of the parts of the company, definitely think we need to mandate AI usage. You know, by and large, you know, people met my boss. She's wonderful, but, but it's a very common sentiment. And people are really afraid of falling behind in the AI arms race. And they feel like if you don't really pressure people to do it, they're not going to do it. And I think one we're seeing a lot of crazy good hearts lost stuff with that, where people are just optimizing, you know, token maxing or whatever people are talking about all the time. And like, that's what you incentivize. That's what people will do. And I think the other part is, you know, you get the shallow, not meaningful adoption. And then what I really think is, you can't run away from the technology adoption cycle where you have like the innovators and the early adopters or the majority, late majority. And I think, you know, how quickly can you accelerate people meaningfully on that curve when it's everywhere? I mean, we're having a, you know, if someone said we're going to the DX conference a year or two ago, and we said, and they said every single talk is going to be about AI adoption AI transformation. People would have been like, no way. But like, is there anything else to talk about, not in any room I'm ever in anymore? And so, so I think like, oh, and also we need to tell software engineers, AI is going to be part of their job. It just like, already feels like you're fighting last year's battle this year. And people just need to move on to removing the friction and, and making it easy to create value that rather than trying to push people on it. So you touched on something there that I think is really important is like we have selection bias in this room, right? Like we are, you know, collectively have a fairly sophisticated take on developer experience. And so I'm actually running a study right now where I asked about 600 engineers and engineering managers, which metrics do you think are suitable for measuring individual level performance? And so, you know, do you think lines of code is a good measure of individual performance? No, of course not. Do you think feature velocity is? Well, like, yeah, that seems pretty reasonable. The place where engineers and managers disagree most like most strongly is on AI usage. The majority of engineering managers think that AI usage is a reasonable performance metric for individual level performance. And like, that is, I think like a myth that we need to dispel in our organizations because it is like the way that people view the world. It's like, no, no, no, like a count of activity is interesting to understand output patterns. But like that's why I love looking at tool usage and things of that nature. But only is a way to understand what helps get us to the outcomes we want. What if we reframe this statement and said most organizations have an unspoken and that AI adoption is required for your job? I think we are seeing this in reality. It's, you know, like lots of companies and I'd actually be interested to hear at the reception afterwards if this is your organization as well. But you have usage dashboards and you are tracking usage. And like, even if you're aggregating at the team level, it's like, oh, my team has 99% usage. And your team only has 50% usage, thus we are better. And it's like, yeah, they will likely have better outcomes, but that should be sort of measured separately. Yeah, maybe it goes back to, there was talk earlier, right? Like if we're not being crystal clear about how we'll measure and how these things will be looked at, then people are going to come up with the answer from their biggest anxieties, right? So, I mean, the question for us definitely comes out. It's been asked of me multiple times, for example. And, you know, so far, I know we've heard a couple of different takes today that some people are putting it in their career development frameworks, some people aren't, etc. I don't think we're at a place where we're putting it in anytime soon. That said, it's a big asterisk of, as always, you're compared to your peers. And if your peers are in a different place than you are, like that's going to be the conversation. But it doesn't start with the AI topic. If you can use no AI and keep up, like, I think, yeah, I totally agree with that. And I think part of the reason why I haven't felt the need to mandate it is, I think it's completely inevitable. And I think so, why would I need to mandate something that's inevitable? Maybe it speeds it up a little bit. But, I mean, I think probably the first time I did a Q&A when I got back to Etsy last year, someone asked this question. I get it every single AM. And I said, I think there's a point maybe we're already there where if you went to a job interview and you said I won't use an AI coding tour and AI coding agent. It will be like saying you're a programmer who refuses to use a text editor. Well, text editor is an essential tool of the job of Zoplin engineer, maybe less so now. But, but it would just on strange to people and I think even now, if you went interviewed for a job today and you said, I don't do that. Yeah, it's going to be tough to get in tough to move you to that. That's that's definitely the harder part of the conversation that I think we're not talking about a lot. Maybe that maybe it's behind closed doors kind of things. But I've seen examples from friends who are managers elsewhere and either having the conversations with the folks that are against it for reasons. Like against using AI and the best response so far as like, are you are you willing to career change? Yeah, I mean, and like, yeah, and just this interest is probably not acceptable. I, you know, I heard about a lawyer and they interviewed for a job on a legal team and they're like, yeah, I'm just completely disinterested in AI. So, you're probably not a good fit for the job. Yeah, you know, it's going to imagine for software engineers. Now, one thing that I do think this whole conversation about AI usage at like directly as a performance metric like highlights is like as an industry. Always sort of sucked at measuring developer experience and like, AI is just continuing to bring that to the forefront. It's like, oh, well, this is an easy thing to measure. And so like I'll measure this activity. And this is just, you know, sort of the merry go round of that conversation, I think. We'll need far fewer junior engineers in five years. We all disagree. Rafe, can we hear from you first? Yeah, sure. I think I mean, it kind of goes to the same thing as we need far fewer engineers period. I just don't think that's the case. I do think beyond that that. I mean, you know, again, like I like to look back at the past and see what the patterns are rather than always focusing on predicting the future. I think. You know, I can remember when we went through the transition of of like to the web and away from client server applications that's hold I am and. And you know, people who are web native and use the web for funding these things at home could just code circles around the people who were, you know, power builder developers or whatever else people used back then. And so I think the idea that people who, who this is native to them are not going to have this unique value to provide is crazy. You know, we're going to intern class this summer. I'm going to be really excited to see the kind of fresh perspective that they bring to our team. And then I think the other thing far fewer is an interesting one. Maybe this is a little bit of editorial. But you know, really since the pandemic, it's been a bit tough times lots of companies have done layoffs, you know, and you know, I know there's been a lot of big public statements about hiring junior engineers, but it feels like junior engineering hiring has dried up anyway. And so it's not like we're hiring huge amounts of them now. I'm hoping there's a resurgence. I mean, I want to. This industry to have a real future with young people in it. And so, so like I know so many companies that are like we just not only hired junior engineers anymore. It has nothing to do with AI. So, so I'm hoping it can go to the opposite direction. I hope we're not also not so short-sighted that we don't think we don't like senior engineers because they don't just like spring fully formed from. Yeah, of course universities. You just graduate. And similarly, we don't need fewer people who like just understand engineering from an educator. Yeah, one of you. So, yeah, it's just, yeah, you might not need them to your project today. But if you want sort of long term value sustainability in your engineering population, you need them. And you need to grow them properly and it's short-sighted to neglect the pipeline. I think. No, totally. And I mean, you know, we've been really lucky to have people who stay at C for a really long time. And some of the best senior engineers we have started as junior engineers. I'm sure you've been at Microsoft 20 years. You know, that's true as well. And so, you know, you need to grow people through the whole through that whole cycle. It's really, really valuable. So, the question is not do we need fewer junior engineers in five years is do we think we need fewer experienced engineers in 10 years? Right. And I don't that we've already answered that question. Hey, the bottleneck of software delivery is no longer writing code. It's now reviewing code. Finally, we're like not feeling the exact same column. Let's start with you. I think one of the first things that I heard come out of Brian's mouth this morning was that developers only spend about 14% of their time writing code. So I think the idea that the bottleneck is writing code in the first place is maybe maybe not the right place to start from. And I think more broadly, we also heard a fair bit of discussion today about the other parts of the STLC being essential here to think about productivity. The implementation is probably the place where we've made the most progress. So I think the bottlenecks are going to be decision making, they're going to be prioritization, they're going to be product design. They're going to be support and operation. Those of it. Yeah, again, amplifier. Right. So it's whatever your organization's bottleneck used to be. It's still it. The code was never the thing. Fortunately, our data actually showed us this. So it's a good place for us to start. Like, you know, turns out we've had deep work as an issue for quite a long time. Still an issue. I don't know. Something about the conversation has changed. Again, like, we're going to find some earlier, but it's like the, the, okay, well, we need the agents to X, Y, Z. What are we doing about agent experience? I was like, well, the good news is we know what to do. We're doing the whole time. So create, make a better developer experience, agents are good better, etc. But that comes down to decision making meetings, coordination tax, all that stuff. We've known not to do for a long time. So to be just like really clear, reviewing code is still a bottleneck. Right. It's just like, you know, no one wants to have to wait for a couple of days in order to get, get your merge in. But I definitely agree that even if it is increasing as a bottleneck, you know, as we are producing more code, and particularly if we don't have more sophisticated ways to sort of like do risk assessments and use that to tailor the depth of review, like reviewing is an increasing burden on us. You know, I know the earlier talked from Uber was talking about feature velocity. And when I go and measure the feature velocity at Microsoft, it's, you know, I can go and look at features that took us two or more years to get from our planning phase out into the hands of customers. And so going from, you know, two days to three days on the review process, that's not the long pull there. It is the planning. It is sort of the prioritization. It's like all of the things that were always a problem. Yeah, I think maybe thinking about this like the perceived bottleneck of code review is what's really interesting. And you know, when it took me two days, let's say to write some PR to generate a PR and they go check it into a repository that requires a code review. And I didn't know whether someone gets flagged automatically or I need to go find out if I need to select somebody and say, hey, can you review my code and they're actually going to do it today. It's super annoying, but it's like boy, it's, you know, it's not as, you know, I had to spend a lot of time on the code anyway. It doesn't feel quite as bad. If like writing something meaningful takes me 10 minutes. And then there's a not only a delay to get my code review, but I have to go to the person and so you please do me this favor and review my code. Like, why can I not just use my agent and not put up with this stuff. And so, so I think like what I'm seeing from people is like, you know, the, interruptions of these human processes, unless they provide real value that is clear or just like so grading and, and that is why we hear so much about code review. It's not just the code review volume, but it's also like, you know, it was already kind of the worst part of your day to have to get a code review. And now it's like really the worst part of your day. The perception change, I think is the thing for us is like our highest quartile AI developers, they took a major hit in the perception of of code turnaround or code review times with some like 14 points or something. But compared to, and that was compared to our like the first quartile, right. But they're median merge time, the high performers decreased by out multiple multiple hours. So it's like pure perception. And it feels more to me. It's like, okay, let's get, let's get personal for me. Like the thing that my significant other hate most about me is when we are out at dinner and like the check comes and I sign it like once I have signed that check, I am ready to go. And it's just like it's the exact same that there's something just psychological about it, where it's just like, no, I'm ready to be finished now. And so I want to be finished. Every second you wait is just like, there's this psychological. Yeah, I feel so much bigger. Yeah, I mean, you're with your partner, you can just go and watch TV. Why are you still in the restaurant? Yeah, right. Let's get out of here. I'm ready to be done here. Move on to the next thing. Okay, we have two statements left. The only thing holding engineers back on AI adoption is unnecessary worry about risk. Okay, Arrini, let's start with you. I don't know about the unnecessary part. I guess. So I kind of hinted at that a little bit earlier when I was saying, how like the mental math that our developers were doing trying to like figure out how much to use agents and how. The maximum velocity was not necessarily the optimum when something was high risk. Right. So I, this is, if I take that statement, this is something that I have seen in like interviews in surveys, like it comes up again and again, where there's something holding people back. And I think it has to do, or like, that's how we see it in interviews come up, is the accountability. Right? The engineers are feeling very accountable for what ships, perhaps this is something that will change the future, I don't know. But today, they feel very much accountable. So if something is high risk, it's hard to undo. It also has a lot of review burden that sits on them. Then that is something that kind of, they pull the brakes a little bit. Right? They do throttle like how much they're going to leverage the the agentic rhythm of work and the velocity that comes without because there is the accountability. Kind of barrier there. And that's a very human behavior. And I think that there's probably down the line going to be some AI mitigations to that. But the human behavior is like hard to get over. So I don't think that their risk is like anybody would consider it unnecessary at the point of time that we are at. And if they do consider that there is risk, it changes the behavior of how much not in general, they're going to adopt AI. But how fast they're going to go with it and how much they're going to leverage it. It seems like it comes into their decision making. Like only is sort of an important part there. Like it's the only reason it is certainly a big reason. So I published a study at the beginning of last year where I asked developers, you know, what are your biggest concerns with using AI. And the number two most frequently cited answer over, you know, fifth of developers say they are concerned about introducing defects vulnerabilities into their code. And so like it is a very real concern that does limit adoption. It is certainly not the only concern that limits it. It was AI, sorry, it was AI increasing that though because like they were doing that already. It's really not as fast. Yes, but it's like even like real or not, it is the fear that it will. The bugs that it means. I mean, I think it's interesting. You know, much like. Many things are changing. I think what the risks are and how they manifest, you know, is just the landscape has completely shifted. So, you know, in one hand, you can say, well, you know, if we had like a whatever, you know, a butter knife and now you have a chainsaw, you know, the risks are different. The risks that you won't be able to cut down a tree have gone way, way down. But the risks that you might, that you might really make a mess are a lot higher. And I think there are probably some redisense. On both sides, like, like, you know, we have a lot of people who are using these tools now who don't really know how a chainsaw is different than a butter knife. They're like, what someone just gave me something cool. And, and you know, in the other hand, I think there are engineers who are like, yeah, I don't, I don't want to use the chainsaw unless I really understand everything about how it works. I think that's. I think beyond the risk to there was a talk by, I think Stuart from Netflix, we're thinking about it just enablement and smoothing the path to adoption. It's not just people worrying about risk. It's not just concerns about quality, you know, helping them choose, helping them configure, helping them get started. Yeah, like these are, and they're not AI specific things. That's just getting people to adopt a thing. You have to solve those problems. Yes, Stuart had a really good quote. I think about it. It was just like, we just eliminated options because options was another reason to, you know, not get started. Our final, our final statement, our final takes most AI adoption problems are really culture and management problems, not tooling problems. Oh, I'm the man. Hello. Yeah, go ahead, rave. So I wouldn't 100, I mean, I'm in management. It can't be our fault. No, that's not true. It can definitely be our fault. I think there are a lot of culture and management problems that make it hard. And I think that other panelists will talk about it. But I do think, you know, it's just a series of like working through the impediments. And I think sometimes the impediment turns out to be a tooling problem. Or do we say, or especially I think I'll cheat. And I'll say like I think process is also a form of a tool more than it is a culture. Even a management thing where, you know, there are a lot of processes in a company. I mean, again, I work at Etsy. We have a 20 year old code base. We have really 10 year engineers and we have all the stated processes. Maybe that is what culture is. I don't know. And like, and you know, so it's like, boy, if you don't jump through who she can't do a thing. And I think that that hurts adoption and just about you of it as well. So for sure. And that's why mid, I think it's kind of all the above and probably in every organization. What is the next blocker that's really keeping you from getting to where you need to go may or maybe tools and maybe. I think the last question sort of foreshad this one a little bit. I mean, I think most problems in the end are like human problems with adoption. So yes, there are tool impediments for sure. And that's not to be neglected. But, you know, helping people have the right incentives, lessening their anxiety, helping them learn, giving them space to learn like those are big barriers. And think about how huge the transformation that is happening is right. How many things that we touch on we touched on bottlenecks that used to not be bottlenecks are now processes that are not exactly working how they we thought they would metrics that no longer mean the same thing. People are having an identity crisis and for good reason. So like there's so many things happening all at once and the pace of change is brutal. And so it cannot be just a tooling problem. It has all of these other things attached to it is culture inside of organizations. It's like human motivation that comes into the picture as well. And of course, yes, tooling and infrastructure are absolutely things that need to adjust and be smoothed out. But yeah, it's all over the above. I could imagine there's a compounding thing going on to where it's like for folks who haven't gotten started, it's harder to get started. For myself, I feel like I'm in obviously in the conversation every day, but it's like I told my VP a couple weeks ago, like I have to quit to be able to like keep up with the industry. Like that's the day job. So yeah, if folks haven't gotten started, I can only imagine how much more daunting it's getting. I mean, even if you have gotten started, right, like everything is moving so quickly, like everything I learned yesterday is now going to be relevant tomorrow. And there is this notion of how do we come back sort of long term burnout and sort of you're falling behind in all of these things, like those are all sort of very basic human constructs. But at the end of the day, this has always been true in the software engineering industry in my opinion, no matter how complex the technology, like it is always a human problem at the end of the day, as long as we are still operating all these tools in a human context. I do think I could just reinforce one thing, Colin said it's that, you know, if you're a leader and you're not giving people time to learn, you know, you are doing it wrong. That has always been true in this industry. I mean, I can remember asking engineers like, you know, like are you working as fast as you can? I don't know. Are you learning anything from what you're doing? No, then you're working as fast as you can. You're not taking any time for that. And I think now more than ever, you know, whether it's like hiring junior engineers, if you hire junior engineers, you know, give them time to learn, they're not going to come and see your engineers. You have senior engineers who have really, really deep expertise, but you're not giving them time to learn how to adapt that to the AI world. You're not going to get the value that you want. And so, so I think, you know, that's, I just think learning is the whole thing almost and making space for it solves 90% of things. And persistently see when we survey developers that learning a new technology, learning a new process is one of the barriers to productivity. And that's not a new thing that that's not about AI. That's just a thing for engineers. So I think one, it's important to set that time aside. It also, I think it demonstrates from a leader that they're like in touch with how the world actually works. And they're not just, you know, sort of sitting in a bubble mandating things like, oh, this will take time for you to get good at. I understand that. Here's the space to do it. Because I understand what you do. Actually, like I can quantify, I can look at the relationship between teams that set aside dedicated time for learning and their long term sort of like coding velocity, which like, you know, is not necessarily the best metric to use moving forward, but it does show that, you know, forward progress doesn't come at the expense. The learning doesn't sort of come at the expense of progress. It sort of super charges it. We're also seeing a difference between doing the learning as a team versus as individuals as well. So in like the data that we're looking at teams that took like two weeks hackathons are not really hackathons, but they've decided that they're going to do everything using agents. Nobody's allowed to use anything else. This is how we do work for the next two weeks. They did as a team have seen much better results, both in terms of adoption and engagement and like actual outcomes. Then other teams that have dedicated time, but its individuals need to do kind of like their own. And then they're sharing and community of practice forming around that. But the other version where teams are doing like one, two, three week kind of group learning has been much more effective. I've been pleasantly surprised today by how many things came back to the sociopath of the sociotechnical world of software engineering pleasantly surprised, I would say. I want to say thank you to all of our panelists for taking the time to share all of their their takes today. Thank you so much for being here. Thank you.