Microsoft Killing Tech Debt with Agents

[MUSIC] >> Hi, everyone. Welcome to another episode of the Tech Distropters podcast. My name is Anurag Rana. I'm a technology analyst at Bloomberg Intelligence, part of Bloomberg's research department with 500 analysts and strategists working across all major world markets. We're delighted to have Amanda Silver as I guess today. Amanda is a corporate vice president ahead of products, apps and agents at Microsoft. She has been with the company for over 24 years. With an intense focus on all software products related to developer and tools, including Visual Studios.net and GitHub co-part. On today's podcast, where we focus on how AI is changing the entire coding landscape, and how Amanda and her team are embedding agents throughout the entire DevOps space. Amanda, welcome to the podcast. >> Thank you, Anurag. I'm delighted to be here. >> To start, could you please give us an overview of your journey at Microsoft and how it has led you to the different products you oversee? >> So I actually started at Microsoft about 24 years ago. At the beginning, I was focused on tools for developers and platforms for developers. And actually, that has been my focus for the last 24 years. I love having developers as my customers because basically, all of the future of technology really starts with developers. And they're very enthusiastic users. And so that means that I can basically ship a new product in the morning and get feedback by the end of the afternoon with a very sophisticated perspective on whether or not it's good software. Most of my focus has been on tools that developers use to write software, which means basically the act of writing code. So that might be Visual Studio or Visual Studio Code, or GitHub Co-Pilot that now helps you write code in an AI-assisted way, as well as DevOps products that basically help you collaborate with other developers as you get that code ready to ship to production. But over the last couple of months, I've actually transitioned a little bit. And I'm now starting to focus more on how we think about software for the future is going to get written, which we believe that basically the future of all software is actually going to be based on agents. And that while today you've been writing code for how you would imagine how software is going to get built, in the future, a lot of new software, a lot of new workflows are going to be automated using natural language prompts. So Amanda, the coding is definitely the biggest area what people discuss when it comes to AI and AI implementations. So please give us an idea as to for those of us who are not developers, please explain to us how software was written, tested and deployed before agents, and how these agents are now collapsing the entire software development lifecycle. Yeah. So, you know, in the in the previous world, we had basically this world, we would call it waterfall, where there would be this process, this handoff process, where you would move from requirements gathering to the customer that you wanted to be able to serve, to then specifications, then designs, then further, you'd hand it off to software developers who would implement it using code, and then they would even further hand it off to QA engineers, quality assurance engineers, that would make sure that it was at the quality that you wanted it to be at. And then eventually, it would be handed off to operations that would deploy it, or in the really old days, rip it to, rip it to disk, and then ultimately it would reach the customer. You know, about 10, 15 years ago, we all in the industry started to transition to DevOps, where we collapsed the latter part of that lifecycle, where essentially, rather than doing the coding, and then the quality assurance, and then the operations all separately, we actually started to merge that into the responsibility of a software engineer, so that they would basically have continued, they would have responsibility for continuous integration of new code, and continuous deployment, and then facilitating the operations of the actual software, as it was reaching customers, ultimately through the web, through SaaS services. But the first part of the software development lifecycle didn't collapse in the DevOps push. Really, what had continued to happen is you would still go through this phase of requirements gathering, speccing, designing, and then ultimately handing off to developers. And so what we're seeing now is that AI and these co-pilots can make it much easier to go from natural language ideas into the specifications, the scaffolding, and ultimately the code generation, and then further agents can take on the process of building tests, making sure that those tests are comprehensive, dealing with the technical debt, as, for example, the libraries that you depend upon in your code evolve, and you need to update that, or take on more of the responsibility in terms of the runtime operations. And so the result is that we end up with a faster cycle from idea to impact, which is really lowering the cost to iterate with faster transitions and a lot more freedom to continue to refine the solution so that it fits the business or that you can find product market fit more easily. So I might have you seen any new kind of problems emerge when the developers are using co-pilots to write code or see any other rationalization or so. Or for that matter, if agents are looking to check the code and verify after it's been written. Well, I think that's a really broad question, so I could probably answer it, you know, a hundred different ways. But there are two dimensions that really come to mind. And, you know, this kind of reminds me of the early industrial revolution when we had factory lines working on machines, and they were steam-driven machines, and then, you know, you'd see in those old black and white movies that the machine would all of a sudden break, and steam would start popping out, and somebody had to go get a wrench, and they had to go fix, you know, what was actually happening to the machine. And I think that in some senses, part of that is what's happening in the software development life cycle, because when you have the ability to assign tasks to agents, it shifts the places where your focus goes. It used to be that the very costly parts of doing software development was the act of actually typing all of the code that you needed to then check into the system. But when you can actually assign a task to an agent to manufacture that code, then the cost where you spend most of your time actually starts to be on things like reviewing the code changes from what the agent had submitted. And so in a sense, that's the gasket bursting. You know, our shift is now that we now we see many more change requests coming through. We call them pull requests in the industry, but we see many much higher volume of pull requests coming through, which means that developers really need to shift how they think about their day-to-day tasks. You know, they might spend a lot more time reviewing code changes and making sure that it fits what they wanted it to, rather than, you know, doing the typing of the new code that they want to check into the system. The other thing that really comes to mind is, you know, software traditionally has been based on deterministic processes, meaning that you had a static description of the code that you wanted to execute, and it would execute the same way every single time, unless there was some kind of external change that it encountered. But for the most part, it was very predictable. What's challenging about this new world of agents is it's based on probabilistic methods is stochastic. And so you can't write software in the same way that you used to write it. And so that means that developers actually need to acquire new skills to be able to create and construct software in this way. And so there's a huge new practice emerging, which, you know, you could think of as evaluations, which is in some ways the new way of testing these kinds of solutions to make sure that they are continuing to behave the way that you want them to behave. And that's a little bit of both an art and a science that represents a new skill that developers need to go to develop acquire. It's very comprehensive. Now, how does it change the type of apps, you know, developers are delivering to the business? Well, I think the easiest thing for everyone to immediately recognize is that modern web applications, especially those that are consumer facing web applications, whether it's a reservation system for an airline or your insurance company or something like that, almost all of them have a virtual support engine, some kind of virtual agent assistant that pops up on the home screen that can look up the information about your account and basically assist you as you're trying to get the job done. But I think that's that, you know, over the last five years, those have existed, but they weren't very good. They were very frustrating and they were hardly used. I think the most immediate thing that everybody will start to see is that those are actually getting to be quite good. And that's further, you know, when you think about making a call to a company that you do business with, you might get routed to an automated voice operator or assistant. And again, the, you know, everyone hates being on those kinds of calls with those voice assistants because again, they weren't, they're just not very good. They were based on static systems. And I think those are going to start to get a lot better. But I think what's actually really, really interesting, and this is kind of the unlock for most of most businesses is when you think about back office business processes. A lot of back office business processes are basically workflows that have to be codified in some kind of automation, some kind of software to facilitate the workflow. And what that means is two things. Number one, when it's automated to have it be automated, you have to have a software engineer help you do that. Number two, when the business process workflow changes, again, you need your software developer to help you make that change. In this new world, where you can use agents to create workflows, or where you can use agents to, to adapt to changing requirements. First of all, there are going to be many business processes that are going to be able to be automated and codified, where you will not need to interface with a developer to get that done. And secondly, even once that workflow is codified and automated, if your requirements or policies change, you don't need to do a new software deployment to have your system adapt to those new requirements. Yeah, I think that's going to be profoundly impactful for almost all businesses. Oh boy, I'm thinking so much Amanda because nobody has talked about it in this framework for me so before. So should I assume that software developers will basically move out of the tech department and into the business, different, I mean, what should I say, different departments within a company? Yeah, I think that this has been a path that we've been on for multiple decades as I've seen it in my experience. When I enter the industry, everyone was building systems level software and over time, we started to build SaaS level software. And in some senses, I think about it like the shift to the public cloud. Before the public cloud, teams had to go wait weeks to procure hardware and commit the capital upfront. And with the cloud, the environments can get spun up in seconds and you pay for only what you use. AI really brings that same elasticity to the product definition and the delivery. And so it removes the friction at the front of the life cycle and it lets the teams iterate based on real feedback really quickly. And so in a way, you could think about this as the cloud removed the friction from infrastructure, but AI removes the friction from the intent to the implementation. I mean, very well said, I'm going to use that. You know, you did just talk about like what kind of different tasks that developers may take, for example, code checking. Are there anything else that they would now do more than compared to coding? So I think, you know, if we think about how automation had been done historically and where the most developers in the industry spent their time, it was, you know, in a lot of a lot of the last two decades, it's really on central IT processes that needed to get automated. And I think what we're going to actually see is that more people who don't necessarily have a computer science degree or a formal software background are going to be able to automate a tremendous amount of back office operations or what we would think of as business to employ types of applications, right? And therefore, you know, I think more software developers are going to focus on business to consumer applications or, you know, maybe more sophisticated systems, or systems that are more mission critical that need to be hardened through the lens of systems thinking. But, you know, I would say that I think that that actually represents an unlock for the entire industry because the every IT department has a backlog that is way longer than their engineers can go tackle. And I think it's what that means is that they're going to be able to focus more on what's differentiated for their business. And even further, every software development team, it doesn't matter what kind of software you're building, has a huge amount of technical debt that they're saddled with. And so the other thing that I predict and expect is that the overall technical debt for the entire industry is going to shrink dramatically. Whether that means that, you know, you have a Java 8 application that you need to upgrade to Java 21 or a.NET 6 application that you need to upgrade to.NET 9, you know, that kind of task is going to we're seeing 70 to 80 percent of their toil that's associated with that getting eliminated using these kinds of tools. And so just imagine what happens when you take your IT department and their backlog is just burned through much faster than ever before. What does that mean, right? And I think that a lot of CIOs are having to make really, really hard trade-offs. They have to decide between whether or not they're going to take care of technical debt or whether they're going to have to go they want to go after their innovation agenda. Technical debt represents basically its opportunity cost, right? It means that you're not spending as much time kind of building the future of your business because you're spending all of your time on maintenance. Amanda, what you just said is music to my ears because this is one of our biggest thesis that application modernization or code modernization is going to be one of the bigger drivers of cloud in the coming years and rightly because of what you just said. So that's cool. Now how is that shifted the way Microsoft's internal developer teams have operating now? Well, I think, you know, in a lot of senses, Microsoft teams are probably some of the largest software projects that have ever existed. And our teams are actually some of the larger larger teams in a lot of senses. And I think that that, you know, there's kind of two ways that this impacts Microsoft specifically. First, I would say that some of the tasks that we have to do inside Microsoft is to drive standardization and consistency across all of our code bases. So for example, we might want to make sure that we're all using the same authentication library from just to make sure from a security perspective that we're confident in the way that we're doing authentication so that we can have confidence in the security of the products that we're shipping. And we think about those as standards. Historically, when we want to drive that kind of a standard throughout all of Microsoft's code bases, we actually file tickets. And those tickets are assigned to individual developers that then have to go respond to every ticket. And sometimes these things can be in tens of thousands on the order of tens of thousands that we have to burn through over a period of a year or something like that. But it represents a huge amount of manual toil for the developer teams, right? In this new world where we can actually go create these specialized agents that understand how to deal with the technical debt or the migration to a common library or whatever it is, the security vulnerability that we need them to respond to, we can actually handle a lot of this more centrally and basically just issue automation that then the individual teams review the automation, then the agents that we essentially deploy into our software systems, they take care of the code changes and then each of the individual teams review the code changes that the agents are making and basically get to check it into their code bases. So that's one way. The other thing is just this point around the front end of the life cycle, you know, the idea that you can go from an idea to an implementation to operation very quickly, that's getting reduced very, very considerably. And so with that, what that means is that any new project that we want to go explore or an experiment or a new feature that we want to go implement in our code bases, the cost of actually turning that from an idea into an actual implementation in the code base has been reduced dramatically. And so that means that we can actually ship new capabilities more quickly. And again, as we talked about at the top of the hour, you know, there's less handoff, formal handoff that has to happen from these different specialized disciplines. And so that also changes the way that we think about the collaboration amongst the different disciplines. Whereas it used to be much more of a formal handoff, we now have more like pods that are working in smaller teams working on different kinds of initiatives throughout our different code bases. You know, matter any tangible benefits or productivity numbers, you could share in, you know, different teams that you are seeing from some of these tools, you know, whether it's on the code review side or the code writing side. So we talked a little bit earlier about the migration modernization tasks, kind of dealing with technical debt. And what we've seen is that the Xbox team, for example, moved from.NET 6 to.NET 8 and they saved, they had an 88% reduction in the manual migration of the code base. The quantum team, the discovery and quantum team also had an application that was based in Java. They saw an 80% reduction in the time that it would have taken for them to upgrade their Java library. But what's also super stark and interesting is in a lot of our open source code bases, when you think about, you know, something like.NET Aspire or VS Code, which are some of our more more popular open source code bases, the entire world can see how much co-pilot is actually contributing to those code bases. And even for, you know, our GitHub GitHub repository, which is basically the repository that has the source code for the GitHub.com product, GitHub co-pilot is the number one and and number three contributor to the overall code base. And then further, you know, when we look at things like SRE agent, which is where we apply an agent to the response of life site incidents. Like when software goes down in the middle of the night, who do you call? You call an on-call engineer that has to go respond to that, investigate it, diagnose what's going on and mitigate and fix the issue. What we're seeing is that the SRE agent can actually, first of all, reduce the mean time to remediation quite considerably because, in a sense, it assists the investigating developer in fact finding to understand what's going on with the system. But even further, it can actually take on autonomously complete mitigation of many issues. And so we're seeing tens of thousands of developer hours being saved through this. And this is really a quality of life issue because for a lot of these engineers, they're on this on-call rotation where they, they eventually, they effectively have a pager and they could get woken up in the middle of the night to have to deal with one of these websites going down or one of these services going down. And so if we can have an autonomous agent that can mitigate the issue, that really means they can sleep through the night. And obviously that that's a that's a quality of life issue. Now, Fer, you talked about autonomous agents. I'm very curious as to how do you decide what functions you can allow an autonomous agent to, I guess, take care of from, you know, the problem versus where do you say, no, the human has to be involved. How do you make those decisions? Yeah, so our overall philosophy is that agents are really kind of here to help humans, right? And so, you know, in software development, it's common practice to actually have multiple developers review any code change that goes into your production code base, multiple human developers. And so when we have agents offering code changes, we still have multiple human developers review those code changes, right? And that's just because in the same way that we don't, we want to have trust but verify for any single developer, we also want to have trust but verify for any agentic developer as well. But then when you think about something like the SRE agent, where maybe it could mitigate issue in in deployment by, let's say, increasing the amount of capacity that's available for a certain service or restarting effectively, you know, rebooting a cloud, cloud service. It's really a question of what are these safe operations for the SRE agent to be able to do? There are a category of different kinds of operations that we consider to be relatively safe. And so it's just again, human judgment that goes into that initially that basically kind of gives the agent limits and restrictions in terms of what kinds of capabilities it actually is permitted to accomplish. You know, quick point of reference is I was talking to a CTO and was basically saying, like, listen, I used a co-pilot or, you know, right code and it was like 50,000 lines of code and a good developer would have taken a lot less than that. I mean, is that something you grapple with or, you know, what's your view on, you know, the quality of the code that flows in? Well, that's actually one of the most important measures that we actually use to determine the quality of our co-pilots themselves. We basically look at the GitHub co-pilot coding agent, which is the asynchronous co-pilot that you can assign tasks to. We actually look at that in comparison to human authored code changes. And we look at things like merge rates, which is basically the measurement of when you have a code change that you want to submit into the system, what percentage of those code changes are actually accepted by other developers in the engineering team. And so we can look at the merge rates for these kinds of assignments that are given to co-pilot and compare it to a human engineer on the team. And kind of, you know, we think of it as hill climbing in a sense that basically we want to make sure that we're continuously getting better and better and better at the at the merge rate. And so we think about that as like, you know, a kind of foundational way to actually improve the quality of the code generation over time. And I will also say that like, you know, I think a lot of people may have looked at GitHub co-pilot or some kind of autonomous agent in for any kind of application a couple of years ago, but things are moving so so quickly and the agents are basically learning that we're building learning systems into the into the agentic flows across the board so that we actually have signal as we deploy them to continuously improve these systems as we build them. You talked a little bit about the quality of life. If you, you know, any other feedback from your teams in terms of, you know, what are they really doing with a lot of the times that this is saving from from all those, you could say, mundane activities or, I mean, critical activities, but now they are, you know, helped by agents. Every, every developer really wants to spend most of their time in the joyful act of building and creating, right? And so, so I think if we're taking care of technical debt or helping with code maintenance or helping with operations or platform engineering tasks, security bugs, you know, generating tests or fixing flaky tests, all of those kinds of tasks are the kinds of tasks that developers really dislike. And so, you know, if we can take care of those kinds of things, that means that developers can get back to the focus on their innovation agenda and kind of, you know, imagining what the future could look like for their organization, whether that's through improving the operational costs and kind of improving business processes, or if it's, you know, creating new products for, you know, their, their customer, their target customers, I think all of those things are very compelling. But also, I think people can then go build new agents as well when they think about deploying agents throughout the software development lifecycle to take care of the tedious mundane and repetitive. That means that developers can actually go and spend their time learning about how they can build their own agents to go solve their problems, right? And so, just as some examples, Levi Strauss worked with Microsoft to build a super agent that they embedded in Microsoft teams that really helped them improve their IT, HR and operations for their organization. And that really helps them, you know, get better insights and automate a bunch of these repetitive workflows inside of their team, which allows their, not just their developers, but all of their employees to really focus on these higher value tasks. Another example is, is, you know, Eon. Eon really needed a faster, more reliable way to do risk analysis and policy reviews across a bunch of different data sets while also making sure that they maintain compliance and governance. And so, they built the Eon GPT, which is a reasoning agent that's built on top of our our Foundry agent service that really orchestrates the policy review, external research using web search and risk analysis while also having this enterprise grade governance as a part of it. You know, you just talked about agents to fix software bugs in production. So, you guys are using it. How about your client base? Have, you know, have they started to embraces? Is there a particular industry that's using it before? Or, you know, where are we in that adoption curve? Yeah, well, I mean, GitHub, GitHub Co-Pilot has been adopted by more than the 90% of the Fortune 500. So, we're already seeing, you know, huge amounts of adoption. We have, I think the last number was 26 million users using GitHub Co-Pilot. And so, you know, overall GitHub itself has 180 million, you know, Visual Studio Code, which is the most popular coding experience that's out there today is about 55 million users. So, 26 million users using GitHub Co-Pilot is quite significant. And so, I would say that there's been huge, huge, huge demand and it's been growing very, very, very quickly. We're also certainly seeing the coding agent grow very significantly, you know, in terms of getting the assignment for asynchronous task completion and kind of, you know, this more autonomous agent that can come back to you with those code changes. See, coding is probably one of the most important areas that has been impacted by LLMs in the last couple of years. But at the same time, it's also lowered the barriers of entry for new companies, you know, companies like Curses are popped up. How's the competitive landscape changed? And, you know, what do you think is Microsoft's strength compared to some of these other, you know, new LLM bond coding tools? So, I would say a couple things. First of all, you know, I actually love the competition. I think it's great for developers. I think it's it actually means that there's innovation that's happening in the industry and there's broad recognition that this is a strategically important audience that we absolutely have to serve. And so, I think, I think, first of all, you know, in a lot of senses, Microsoft, and this is one of the reasons why I've worked at Microsoft for 24 years, we view ourselves as a platform company. You know, we build platforms so that basically other people can build more value on top of our platform. And that's one of the reasons why developers have always been so strategically important for Microsoft itself is that it's absolutely core to the way that we think about our overall business value, business structure. And so, you know, I think it's that, it's that focus, that relentless focus from MSDOS to the acquisition of GitHub to building GitHub co-pilot as the first broad application of AI into this space after the partnership with OpenAI. I think all of that really reflects Microsoft's strategic focus on developers as an important audience. And I think that, you know, when it comes to our structural advantage, obviously, like I think that, you know, we have a huge community of developers, of over 180 million developers on GitHub, over 55 million in Visual Studio Code, that use our tools day in day out to get their jobs done. And I think that on top of that, you know, obviously Microsoft is fairly well known for the confidence that people can have in us because we understand what it takes to operate Mission Critical Software for really important companies, important missions. And so, I think all of those things are really, you know, kind of the wins in our sales. What I think is exciting at the same time is that there's a huge amount of innovation that's happening in the foundational models. And the other interesting thing about Microsoft's strategy is our approach is that we actually want to be able to support multiple models, multiple vendors, and become basically the marketplace for all of the best models that are out there. So Microsoft Foundry actually has over 11,000 models in our catalog. And many of the ones that apply to coding are also available in GitHub Copilot to actually make sure that GitHub Copilot can actually get better and really get state of the art for any kind of category of development that you need to do. So developer choice is also really central to our overall strategy, both the platform level and at the level of really, you know, the tools for creating code. Great Amanda, we're almost coming at time. And I wanted to finish up with probably one of the most important questions. You know, what does this do to an engineering department? The pyramid is there going to be wage deflation? Are you going to see big layoffs? So I'm very curious as to how you have thought about it that, you know, if you go back in history and say, well, if I was going to do 100 projects, it's going to take me 500 engineers. But does it mean that this entire, the way we have thought about a particular department is going to completely change? I think, you know, if we look back at the industrial revolution, we certainly saw shifts in terms of where, you know, people would spend their times different kinds of jobs. I'm actually reminded of my grandfather who was a mechanical engineer. He worked on aeronautical jet engines, actually. And one of the things that he had to do in the 50s was he was designing a ball bearing for a jet engine. And to be able to replicate the equivalent environment for what represented the atmospheric pressure of 45,000 feet in the air, he actually had to have a computer that would do the floating point operations for him. But in that era, a computer was not actually a machine. It was a human who would actually do all of the number crunching all day. It would take them all day to do that set of floating point operations that would then come back to him and tell him, you know, how we needed to construct the hyperbolic pressure system to be able to test the jet engine on the ground instead of 40,000 feet in the air. And so I think, you know, I don't think we would go back to a world where we have humans doing floating point operations, right? But I don't think that that means that there's not a job for somebody to go work at some higher level point of the abstraction ladder. And I think that, you know, that's really why I do what I do, right? I believe that technology is a force for good. And that, you know, if we make the act of creating technology available to more people in the world, then we're going to be able to use it to solve more problems overall. And so I think in a lot of senses, there's there's an unbounded amount of applications of technology that have yet to be applied to problems. And really, in my mind, the best thing that we can do is to make it easier to build software and to make it so that more people can build software to solve their challenges. Great, Amanda. That's a fun conversation. Thank you so much for your time today. All right. Thank you. For our listeners, we want to thank you for tuning in. If you like the episode, please subscribe and leave a review and check back to hear conversations with the leading disruptors in the tech landscape. If you want to learn more about our research including deep dives and topics like AI, check our work on the Bloomberg terminal at BI Go. We also want to thank Aditya Samani for his help in editing this podcast. This is your host on Raghurana signing off.