If you can imagine it, you can probably build it | OpenAI's Satya Tammareddy and Rowena Westphalen

When you're actually rethinking the way work happens, that's where it starts getting really interesting. 75% of enterprises have said AI has allowed them to do things that they weren't able to before. The word they use to describe this type of AI is they call it probabilistic, right? So it means every single time you get an answer, it's going to be different. All the IT and tech systems we've had up until now have been deterministic, like there's always a black and white answer. And so humans are used to expecting the computer to give you the same answer and it for it to be very black and white answer. And these systems are completely the opposite. Technology that's changing at a pace that we've never seen before. I've been thinking recently about the things that AI has upended. Like, obviously, we've all had lots of conversations about the technical side, you know, like how AI has represented this huge leap forward in what we're able to achieve with a little bit of data and a web interface, how it's transforming industries and tasks and causing us all to question what humans are going to be doing in 10 to 20 to 50 years' time. But at the same time, at least in these relatively nascent stages of its adoption, there are actually some amusing ways in which AI has led to a return to the old. One of those ways is the return of services. I was thinking the other day about the contrast to when I was building my own software company about 10 years ago. And we were adamant that we would never invest in a service wraparound for technology. Not only was service-based revenue found upon and basically written off by investors, but it was also kind of an embarrassment, you know, like we saw it as a sign that your product wasn't user-friendly or intuitive enough to be able to nail the product-led growth everyone was searching for. But that is now a vestige of the past. Perhaps because of how transformative AI actually is, or maybe because people still love their hands being held, services are back, baby. We heard this first actually through the provocative language that Jess Richmond from some day offered up on the podcast a couple of months ago. And it's a big theme of today's episode two. Interestingly, an episode with two execs of one of the world's foremost AI companies, open AI. In this interview, I chat to Rowena and Satya, both on the kind of go-to-market side of the house, on what on earth a Ford-deployed engineer is, and what role these humans are playing in supporting open AI's big customers get the most out of their technology. We also talk about adoption more generally, what they think is driving at, and perhaps the surprising job roles that are the most reticent to use AI in their own work. And yes, we also talk about cybersecurity and risk. This is an episode with two of the region's leading players on these topics. And I hope you enjoy listening. Rowena, Satya, super excited to have you on Wildtarts. Thanks for being here. Thanks so much for having us. Thanks for having us. So I'm going to start actually at the hopefully easiest point, which is actually to ask you both to describe what your job is. So you both work at open AI. You've had these storied careers in tech. What are you doing at open AI? It will start with you, Ro. Sure. Well, thanks. So I'm Rowena Westfalen, a lot of people call me Row. And I have a great privilege to lead applied AI architecture for open AI for Australia and New Zealand. So I joke about having hackers, hipsters, and hustlers in my team. It's engineers, architects, specialists, a wide range of technical skills. And what we do is we partner with customers to help them really deploy and uncover the value that they can achieve in their businesses and in their communities with AI. Cool. Satya. Yes. So I lead our go-to-market for Australian New Zealand. Me and my team, we work with customers. So I'm also out with Ro's team. Partner with them. Customers of all sizes. So it can be your large enterprises, like Commonwealth Bank, Coles, Canva. It can be very digital native customers, such as employment hero, bright, and then startups as well, such as our relevance, Heidi Health, etc. So really customers of all shapes and sizes. And my team, our goal is to really understand what their business challenges are and how AI can make a real difference to what they're doing. Amazing. OK, I want to unpick a couple of things that I'm curious about both of your jobs. I'm going to start with you, Row. I understand that you have four deployed engineers, also known as FDEs, in your team. And I'm going to be honest, I don't think I really understand what the job is of an FDE. And importantly, why that's different from what we've known for decades of IT implementation teams that have been a part of consultants' teams and all that sort of thing. I feel like it's different, but I want you to tell me how. Yeah, so we have both deployment engineers and four deployed engineers. So that's probably the first distinction to make. If you think of an FDEs, just the equivalent of professional services, you're probably missing them out. So and every organization tends to use four deployed engineers in a slightly different way. So the way a big part of what my whole team does is open AI is essentially a research and deployment company. So we have a huge amount of AI research that happens. And then we apply it by turning into products, putting it out there, and then a big focus of what my team does is the deployment side. Now, what's interesting is the researchers love learning about and being actively involved in the deployment bid, right? Because that's like research in a vacuum theoretical research doesn't really get you anyway. So once it's being used in a real life scenario, that's where you get the best feedback and that sort of thing. And so that's what my whole team does. Like we have this great ability to kind of feed the insights and learnings we're doing with deployment and what the customers need back into the research team and then provide that research engagement directly back to the customers as well. For deployed engineers work on our most high value opportunity areas. It's generally something that's quite transformational. It's generally something that's quite ambitious. And it can often mean that they'll end up kind of almost building a new product. So they're kind of starting to bring that up a bit into the deploy at the same time, if that makes sense. And so the two of you would work together like, imagine we took Combank as an example. They might be like, I can see an opportunity for OpenAI's technology to improve our business, but I kind of need your engineers to help unlock that value. Is that the right way thing you might have? Absolutely, yeah. And they have, they've got a huge and amazing engineering team as well. But they love to partner with us. We work side by side. And so sometimes the projects are, I wouldn't say straight forward because every project has its nuance. But sometimes the projects are more rinse and repeat. And other times it's actually like, hey, here's a dig area that's really interesting. The real transformation comes from digging into work and changing a workflow or really like rethinking a value chain. And that's where the more speculative and experimental it is, the more opportunity there is for us to explore things together. And so then it becomes a really high caliber opportunity. We've got huge demand for four deployed engineers in Australia. We're hiring a lot now. She says, plug, if you'd like to. If you consider yourself to be a four-deployed engineer, that you'd love to work with us. There may be people in the audience who are thinking, yes, please. If you think you've got the capability to be a four-deployed engineer from OpenAI in Australia in New Zealand, we'd love to hear from you. Yeah, great. And so I'm quite interested then in what it actually looks like to partner with OpenAI. So you've described kind of like the FDE role. But maybe Satya, you can tell me from a kind of go-to-market standpoint, walk me through the interaction. Because I think one of the things that's curious, you've both done SAS as well. And one way of thinking about this is actually, SAS was the layer where you built technology and then you sold subscription services to that technology. It was done in-house and then offered as a service to a particular customer. Interestingly, you're kind of before that, which is all of the where's the research layer. And after that, which is, let me get in your team and help you unlock the value, sort of skipping the software layer as it were. It's a full adaptation into the customer. So talk me through, I guess, how you think about your job and also how it is both similar and different from the time that you were doing go-to-market in SAS at Stripe. Yeah. Yeah, it's a great question. I might say the scene a little. We launched in Australia in December last year. Eons ago. Eons ago in the AI world. I feel like everything's changed. I mean, we were just inundated with businesses wanting us to help them through that process and SAS products that we all worked with. There is a level of familiarity in the customers. And the pace of change wasn't anywhere like what I'm seeing here. And so you've got a few different, really interesting things happening at the same time. You've got completely new technology. It was only into the hands, into the wild in 2022. The world is still catching up to the incredible progress we've made since then. You've also got a technology that's changing at a pace that we've never seen before. Even in my one year here, like every couple of months, there's been some major change. We wake up in the morning and there's just like, what's happened overnight? And so if we're feeling this way, you can imagine the customers and the businesses how overwhelming it is. And so a lot of what we started doing was just walking our customers and businesses through what is happening. in the industry and I mean the technology is there. It's incredible even in the last six months the how much it's advanced is incredible. And so how do you as a business make use of this to really make a big difference to what you're doing? Whether that is solving a customer problem, whether that is helping your teams work better, there's just so much there. And so to your question around what's similar and what's different, what's similar is relationships. So building trust is really important. Us being in market, just turning up and we've been working with customers like in their offices and that makes a whole lot of a difference. The other thing is just proving value. There's a lot of hype to AI, but ultimately it's are we bringing value to our customers? That hasn't changed. What has changed I think is the pace and just our ability to abstract a lot of that complexity away and talk to our customers and really help them understand the potential. I have to say such has got a bit of a super power. I think when it comes to hiring 'cause she's got the most amazing team. As a solutions person, as a technologist, right? The quality of the answer that you can produce is a direct reflection of the question that you ask. And so discovery and like making sure that you're actually answering the right question is incredibly critical. And we're so lucky with the team that such a has I'm blown away by their ability to kind of be curious and explore things and frame things in a strategic way and that makes a big difference. Thank you. They're amazing. Well, I'm actually curious to get beneath that because you sort of alluded to this idea like the technology is emerging so quickly if those of you who work right up close on the frontier in one of the frontier firms feel a little overwhelmed at the pace, imagine what your customers feel and then imagine what the kind of customer that's just about holding on to the next development. I think they have an understanding. They're reading about in the news. They're sort of trying to pick things up here and they're trying to have interesting conversations. But frankly, a feeling of like they don't really fully understand. There must be actually a massive gap that customer discovery needs to be very subtle about like both presenting what the frontier looks like and what is possible while also not alienating and giving space to someone to be like, I'm sorry, I don't even understand that acronym. Can you talk me through what you and your team do to think through that sort of customer empathy in discovery? Yeah, I mean, firstly it's understanding where they are on their journey. And so if they are very early, it's really us going back to basics and walking them through the technology and in some ways demystifying what can and can't happen. I mean, an example is hallucinations work. Yeah, six to 12 months ago, there were a lot of hallucinations. We've seen that come down significantly. And so I'll go to a meeting and a customer is asking about hallucinations. So then it's about explaining hey, this is what's happened in the industry. We're still not there in terms of 100% accuracy and we will never be because of the nature of LLMs, but just walking through and explaining that. And then on the other hand, we've got our most advanced customers that where we've got FDAs, we've got rose team in there and we're talking about evals and pushing the boundary on what their company is doing. And I mean, those customers are helping us define the roadmap and we've got a lot of customers here that are design partners for us. And so you've got that entire spectrum. You mentioned empathy. That is still really important. You know, we say to our teams all the time, I really understand what the customer is trying to do. It's not about selling or like talking them through use this product. It is how can we tailor our solutions to what you really need? What are the characteristics of the companies that end up becoming a design partner? Can you predict them ahead of time? Yeah, I'm curious for your thoughts. From my side, they're pushing the boundaries. I mean, Rory mentioned earlier around when we involve our research teams. It's when there's use cases that are not in production anywhere. And so there's customers that are pushing the envelope on what that means. And so that would be one thing I look for. The other one is just ambition. An ambition to either be the leader in their field, or their industry, whether it be in Australia or globally. And we've got amazing companies here that are doing that. But also having impact beyond what they're currently having, and there's a lot of headlines around cost, but the most ambitious leaders I'm seeing, they're pushing the boundaries of, "Hey, what can we not do currently that we will be able to by partnering with like now?" Anything you'd add? Yeah, I mean, I think you've framed it really well around. The leadership team, particularly the CEO, but the whole leadership team, vision and alignment, and kind of not just vision and alignment, because you get lots of leadership teams that are like, "We're going to be AI," but also like hands-on. The great leaders that we worked with, and we've had the privilege to work with some amazing organizations in Australia. So like Lyon and Deputy and CBA, and many other really great ones. All those leaders, they use the tools themselves, and they often-- As individuals. And they often start using it personally first. So they have this like hands-on drive, but then they've also got a brand aspiration that's kind of interesting. And so like a really interesting element for us in kind of tempering the solution. I often talk about like kind of different levels of transformation. So one thing that's true is that you can see the same organization using the same tech and getting very different outcomes. That's been the tech thing for years, right? So you start looking, peeling that back and kind of going, "Oh, OK, what's the difference?" And so one of it is, I sometimes talk about, you can have projects that are like renovations, right? Well, you're like, you're going to just replace the tiles and put in a new sink and maybe add some new tiles. But like the plumbing's pretty much the same. And then you get a renovation where it's like actually the bathroom's moving to a whole new place. And so depending on the organization's brand aspiration. And like there's no judgment there. Some organizations are still very, very dependent on like a traditional manual on-premise workflow. And so actually taking that exact workflow and automating it to some extent very quickly can have huge cost-art benefits and simple and can get, you know, so there's nothing wrong with that sort of project. But when you're actually like really rethinking the way work happens, when you're actually like, "OK, I'm going to change the value chain here." That's where it starts getting really interesting. And then there's also a fundamental thing of like, you want to answer the right question first of all. So you get a quality answer. But then there's also like, is the way we're answering the question, living up to the brand and leadership aspirations that you have. And there's a big thing that happens with any engineer is the can you versus the should you advisory because tech can do anything, right? So the courageous thing that my team has to do when the customer says, "Hey, can you do this?" You know, at what point do they go, "Yeah, but like, should we do that?" What might be an example of that? We don't have to name the customer, but like, I mean, it could be anything to do with, you know, the way you're thinking about making a decision around a workflow, right? And like, maybe there's a stakeholder that's not being considered as part of that workflow. You know, in the early days of marketing, there were lots of ways that you could kind of look around privacy issues and kind of extend the, you know, you could still officially be regulated, but actually like, "Hey, if that was having your mom or your son, you probably wouldn't like." Wouldn't pass the pub test. Yeah, so it's kind of like, "Hey, yeah, you can do that. The question is, that doesn't feel like that's actually aligned with with your brand or what your leadership aspirations are." The more complicated thing, I keep thinking this, certainty is a little bit overrated, right? But like humans love certainty, right? So good consultant is gonna come in and say, well, there are three steps and only three steps. And once you complete these three steps, you will have all the answers. And of course, they're trying to sell you something and wouldn't it be wonderful if the world actually worked like that? Unfortunately, it doesn't really. So getting the balance right of earning the trust and providing the correct advice, but recognizing that a new model comes out every six weeks, and the technology changes, so like six weeks later, we might actually have a slightly different recommendation for you, or in some cases, an even different one, and being able to have the trust to say, "Hey, I know we advise this." And I suppose the last thing I'd say about transformation, I love your view on this. The word they use to describe this type of AI, they call it probabilistic, right? So that means every single time you get an answer, it's gonna be different. The opposite of that is deterministic. All the IT and tech systems we've had up until now have been deterministic, like there's always a black and white answer, and so humans are used to expecting the computer to give you the same answer, and it for it to be very black and white answer, and these systems are completely the opposite. That's why we get hallucinations. And it's a really good example of a strength overplayed becomes a weakness, or you know, your weakness is also your strength, right? So these AI systems would not have learned or evolved as quickly as they had if we didn't have the, you know, that old type of technology was very brittle. - Yep. - This flexibility is what gives us our strength. Then when you put that into workflows, we currently have businesses that are reliant very deterministic workflows. And so there's like a whole range of things to rethink when you if you actually want to take full advantage of the probabilistic side of things and so that's where as Satya was saying evils become so important because the key thing to getting this right is like having a consistent way of ensuring that the range of answers that you get is acceptable, but you're still catering for the range. How many customers are capable of designing an evil like because it isn't new skill right there so many new skills developing to get the most out of I like you know I remember wasn't that long ago when this concept of having a prompt engineer and what is a prompt engineer and now we've got all of these additional new role types like how many customers are in a position to be able to sit down and say gosh now that you ask me what is the range of acceptable. Like responses here are we finding is that an area where you're also having to do most of the educating is the market ready for that. It depends on the customer there are lots of there are lots of organizations that I think are ready for that you just have to. I think they need the permission from their leadership and they need to be open minded but you know when they go back to basics I think they can do that. It's worth saying prompt engineering is not going to be a thing for much longer I don't think you know the reason you needed good prompts and prompt engineering is so that you got a you got the right answer and it was unambiguous and these latest models are incredibly good at inferring context. And so maybe emails will just be a short live thing what do you think. The interesting prediction I think it'll take time to your point around context even with the move from organizations using AI purely to respond to things so think back to chat right. We've back to chat like or we knew it to be like a year ago. Yeah and now we're we've moved to agentech right and so to your point the agent goes and finds that context and comes back with an answer for us or has completed a task for us. I still think emails will be important particularly for really high stakes work and humans will also be very important for that kind of work but I think day to day to your point we may not need them for that that much longer. Or at least it'll be easier to template it'll probably prompt you with like you might want to think about this as this. Oh yeah you're right that that does seem like right to me. Yes. So actually I want to come back quickly Satya to the aha moment. I think you were sort of describing actually wrote the aha moment for a lot of customers is like when they finally start going from a sort of interior design version of a renovation to an engineering and you know demolition style of renovation. Really interested in what you're learning from particularly enterprise customers as they deploy these sort of first and second you know use cases into their business. Is there a measure of success that you find is just radically faster at building adoption and ease. Is it like time saved or like you took a bunch of really painful and repetitive tasks off my team and they hated them and so they're happier or is it money saved. Is there for you a sense in what is likely to drive that aha moment first or is it as simple as like a storytelling and this it touched the CEO's work helped that CEO and that's when you start to get that buy in. It's a great question on the question around measuring and what is driving that there is no one answer there. I do think someone getting to that magic moment and if that is the CEO or a leader in the business that I think is is the pivotal moment where they get it and then they're then encouraging their team or teams or company to experiment and and get to where they are but I do think it's a journey. We see a lot of leaders that have had that and you know you can tell they're just living and breathing the tools even though their day job is completely different and so I do think that has a lot to do with inspiring and encouraging organizations to get the hands on the tools. Now to your point on what are the metrics we're seeing it really varies and there is the element around saving time saving maybe dollars of investment in building a product getting to market faster like Lena teams driving output. But also what we're seeing is 75% of enterprises said AI has allowed them to do things that they weren't able to before and so often that may not be is it a measure. There's a really interesting customer that I'm working with where they've actually asked tragedy work to help them quantify various tasks around what is the ROI on this based on what my employees are telling me. And so I think on the measurement piece from what I'm seeing it is a journey and a lot of folks are struggling around how to actually prove ROI on this yeah I'd love to then kind of get into the other part of our ways cost. And you know the big topic at the moment is tokenomics satire you have a degree in economics started your career as an economist you'd be the best place can you define what tokenomics is for part of the audience that may not know. So tokenomics is how valuable a token is in terms of delivering a task and so I would think of it in terms of cost per task. And so we're seeing a lot of attention around this topic given the cost of using AI has for a lot of businesses has become exponential yep and so that is both a combination of the tokens they're using. But also the task and then the complexity with which those tasks are delivering outcomes so it's super interesting if you think about the cost of the token has actually decreased significantly in the last year. It's like this two stories going on because such is absolutely right lots of organizations are like how am I spending all this money and at the same time you look at the individual token cost it's gone down quite a lot actually efficiency and tokens and ensuring that our customers have the most cost effective access to AI and ensuring that all of humanity can access and benefit from AI is like a huge focus of ours. I imagine there would have been a time when they made a light bulb affordable to run to like run one light bulb and be like hey that's so cheap and then they decided to switch on an entire city and they were like wait that's lots of money. So it feels a bit like that we've actually been working on efficiency and effectiveness of our models for like the last 18 months to years I'm really impressed at the vision that open AI had around this because even if you look at. So our current model is five six will have a new model coming out imminently so maybe by the time this podcast is out we'll have a newer model but two models before that was five four not that hard to work out the numbers. If you look at the performance between five four and five five it looks almost exactly the same like in terms of achieving an outcome but literally the token usage is a third from five four to five five. And it's got even more efficient with five six and and then on top of that we've also we're delivering different size models so in five six we talk about. Soul terror and Luna and if you think about like the moon the earth and the sun and you think about the sizes there that kind of gives you an idea so soul is our is our flagship model premium. Terror is the the medium size model and and then Luda is so affordable yeah compared to what it was it's like Luda has now reduced costs by about 80% it's like it's so affordable but even our most premium models are much more cost effective. Then all the other frontier labs like that's where we're really leading the market I'm sure they're going to I'm sure they're going to catch up like that's the way the AI market goes but right now that's kind of where we. We're seeing a big differentiation and we look at it on a cost per task that's right this is so we mentioned that cost of tokens has come down significantly but also. It's more efficient and so to actually get a task done yeah I mean the whole token things not that useful because it's like each AI lab has a slightly different definition of a token it's not exactly like a commonly understood unit anyway yeah. And so it is it is much more useful to look at you know what's the business outcome that you're achieving. And that's where it's it's incredibly exciting to see the sorts of things that we're doing with the likes of chat GPT work so. You know we've had chat GPT for years chat GPT work is the agent equivalent where you can actually delegate tasks and like the typical use for that is coding right helping a software engineer deliver and develop code and so our coding just in Australia. The coding output has increased six times since January and in fact for woman three times for women which is which I'm really happy about lots of women out there coding. But we then realized that about 40% of what we thought was coding work was actually non being done by non coders so it's like people are producing PowerPoint slides and creating spreadsheets and chat GPT's creating websites to describe things and that's really exciting kind of output and delegations. So yeah it's it's been really exciting to kind of see how these use cases kind of evolve into into new opportunities. I do want to come back a little bit to tokenomics. because I mean, it's a thing that's very top of mind for a lot of our portfolio companies. It's been in the news, obviously, a lot recently where, as you've described, there's so many forces happening at once. On the one hand, you've got firms like yours who are constantly pushing out new models with new capabilities. And as you alluded to, Roe, I imagine one of the most interesting strategic questions for Frontier Labs is, what am I optimizing this next release for? Is it efficiency or efficacy? Or am I looking for moving further into the Frontier of answering the hardest questions at any cost? I mean, all of those things have trade-offs, right? And you're always thinking about what should be done there. But I think one of the interesting things is if you are a customer of OpenAI or any of the others, or indeed, you're building technology that then enables your customers to be users effectively of the OpenAI models, you're having to think about, like, are they calibrating their use of those models to match the task and the complexity required? So most of them, we can choose, right? You mentioned Roe, the sort of sole Lunar and Terra models. And there's an instinct for people to say, give me the best of always. Even though it's like I'm asking for a recommendation for which cafe to go to around the corner, I'm really interested in where you think this is going to go from a product standpoint. It seems obvious that AI will get better and better at defaulting to the model that it needs, saying, like, this is not that hard. You're not asking me for an mRNA new vaccine. You probably don't need the frontier model to do that. But also recognizing that you want to be demonstrating the full capabilities of the model, knowing that the cheaper models by definition have to cut some corners. How do you think this is going to play out in products? And maybe I'll start with you, Satya, and then come to you, Roe. - Yeah, this is a very top of mind at the moment. In terms of product, I think two ways. Firstly, currently you have to select both model but also how fast you want an output. And so there's a world where we abstract all of that away. And so it's really easy for you when you ask your tool to do something, the choices are made behind the hood. So that's one. Second is empowering businesses to. So there's an education piece, but also empowering them to set defaults for certain, it could be certain. So coding, obviously, is the most advanced use at the moment in terms of what we're seeing. And so enabling those that are using coding, more powerful models, versus there might be tasks that don't need, you know, the soul. - I mean, I think there are some coding use cases that you use a medium sized model for. It just depends on the complexity of what you're doing. I feel quite strongly about using the right model for the right task. And there's kind of two analogies I've got for that. So the reason I care about it is because I'm always thinking about the resources that we're using for our planet. And so like, if I'm angsting with chat GPT about like a recipe replacement, I don't really want to be thinking about using premium resources to do that. But then on the more high-stake stuff, then obviously I want to leverage that correctly. But then it's also this element of like, if you thought of these models as individuals in an organization, and it's like, would I really go to like that senior consultant and ask them, can you staple these printouts for me, please? You know, meanwhile, I've probably got an intern who's excited to be in the office that is prepared to just do anything that probably doesn't mind sapling these things for me. You know, and so it's that kind of side of things as well, just kind of like use them for what they're best for. And I think about the AI is it doesn't mind either way, it'll do it. But I think there's a responsible element there. - As long as you say, Franks. - I'm interested. Do you say thanks? Do you say, please and thanks? - Yeah. - Didn't Sam say like it's gonna cost the planet X amount if you keep saying, please and thanks, but it's very natural to want to do it. - I don't necessarily say, please or thanks, but what I do do is everything we're doing in AI is actually replicating the knowledge we've had in the world. I think to get the most out of the model is probably everything you would do to get the most out of the human. So a human that feels appreciated and nurtured is gonna be better. And I've got this weird theory, I don't know if it's true, but I've got this weird theory that if I'm more constructive with the model, it'll be, it'll do better things for me. - Are there any other funny things you do with your use of chat JPD or the other models that you think is like a quirk? - I've been using voice a lot more in the last few weeks. - Yeah, should we show you? - Should we show you? - Yeah, go for it. - The biggest benefit here is, I mean, I'll give you an example. We had a business review a month ago. I was walking home from the bus stop and using voice, I prompted chat and work to look at a deck I had from last time and then populated for the update over the last three months. And I got home, looked at it later that night and it was 90% there. - Amazing. - I didn't have to type anything. I just spoke to it. - Yeah, it's great. - So I'm kind of then interested to kind of go into a bit more around how you are responding as leaders in this AI wave to the way that AI is expressing itself. I mean, you mentioned earlier Satya, you've been in markets since December, which is a very, very long time in the era of AI, but no time at all. We are all still learning what this is going to look like. Maybe starting with a kind of customer perspective. What's something that you think customers are focusing on that is probably not a thing that they need to worry about? And maybe what's a thing that you would love to be getting more attention in this next wave of AI to build the best version of the kind of AI version of life that we want to see? Roh, can I start with you? - There's a lot of theoretical debate around AI. And like, so there's a lot of discussion about the potential risks and not to minimize them in any way, but there's almost too much theoretical discussion rather than actual doing. Like the sooner you start to prototype and use these things in a safe way with guardrails, the sooner you'll actually understand what it should actually do and how it should actually work. I think there's a little bit too much discussion about the potential risks of what if and maybe not enough discussion about the potential risk of if we don't adopt it. And like, what will it mean to Australia if we just ignore all AI? And by no means saying, let's just take all the holds often just do anything, but. - What do you perceive as the biggest risk if we don't embrace it? - I think there's a real concern that I have about relevance and productivity in Australia. Like the value chains are being changed. Ways of working, new service offerings, new ways to engage customers. Traditional enterprises that aren't really thinking how to fundamentally change their organization are potentially at danger of having a startup come in and completely replace them, completely change the way the customer service is being delivered. And then at a global level from a community perspective, Australia needs to. I would love Australia to stay at pace with where the world is at. I think we're doing that actually. I don't want to have it interpreted that. I don't think we're doing that. In fact, I'm really proud at like the level of adoption that we have in consumer here. Like one or two adults in Australia use chat GPT every week. - Yeah, I mean, this is one of the most interesting things is that actually adoption rates in Australia and New Zealand are amongst the highest in the world. - Yeah. - I think across both open AI and anthropics platforms actually higher per capita use than the US. The last time they had a look at it. And yet the tenor of the debate does feel different here. Well, I just wonder how much it is people using it as a consumer and then actually like involving it in work. Like one of the things we did notice was 40% of the consumer conversations was actually work related. So then you have the situation where you know, you've got maybe a regulated industry who's being super conservative and has gone like, "Okay, I'm just going to switch up everything "because you know, we can't risk anything happening." And like the essence of what they're trying to protect isn't wrong, but like just switching everything off isn't useful. And so as a result, you wonder how many people are just like doing their work stuff on their personal machine because they know it's going to help them be more productive. - Yeah. - So getting that balanced right is important. What do you think starts? - I completely agree. One thing that's also surprised me in all of this is actually I'll ask you a question. Can you guess outside of coding what function or industry has been, we've seen the most adoption. - You mean in a business setting? - Yeah. - Customer service? - No. - Legal, legal, interesting. - Which is really interesting. With legal, we've seen a hundred times increase in how they're using agentic AI. Across the globe, anecdotally seeing similar trends here in Australia. Gilbert and Tobin, for example, is one of our customers and you would think legal being regulated, usually a more conservative voice. - A very conservative privacy, confidentiality, all of these matter a lot. It's interesting seeing how Gilbert and Tobin has evolved and their leadership has been very, very innovative but also keeping in mind that they are in such a regulated industry. And they've taken a quite a measured approach to how they're thinking about rolling out AI and I've been really impressed. And to the point where they told us that their clients have actually asked them to be using AI more because it reduces the number of billable hours. - Well, it's interesting. You should say, that I actually heard a very senior partner in a law firm or a name that law firm described, this is probably close to a year ago. So again, you know, the dark ages relative to now, but describe one of the challenges that they were having managing clients that they actually had to keep two very distinct lists was almost bifurcated for them at that time. They had the customers that were saying, "I want you to promise you're using AI wherever you believe AI can deliver the service for me faster and more efficiently and with fewer billable hours. And I won't pay to have a junior person do that if there's realistically something else they could be working on the higher order and an AI could do it." And then they had, excuse me, another list which was "I don't know circumstances, can you use AI at any point in managing our work?" And so internally, this law firm was trying to manage two quite distinct customer expectations. And I just thought it wasn't an absolutely fascinating, likely temporary move as everyone gets comfortable, but this really, really stark difference in what their own customers' level of comfort was. And it would make for a really interesting experiment on like in 50 years time is the first set of customers still around in the second set, not because their mindset on adoption of new technology is just very different. Yeah, that's, I mean, matches what we're seeing as well. And it's great to see law firms be really mindful of that and be progressive but also cater to their customers. Yeah, so I'm then interested actually, you've both spoken about the opportunity set and you work with large enterprise customers across Australia. Is there an industry or a sector where from the perspective that you have about knowledge of what the technology can actually do? You'd think this is obvious. This whole industry has a huge opportunity with AI but it's surprisingly lagged and what's it about that. Are there areas where you think why hasn't it taken off faster there? From my side marketing and when you look at industries that are embracing, like I wouldn't say marketing is a lagide but given the potential, there's content, there's multimodal. So images, text, there is analytics around campaigns and what they're generating, there's so much that I think marketers can be doing and so I've been personally surprised that marketing hasn't been the leader like I was saying with legal. You must have a hypothesis why? It is interesting, speaking to marketers, there is definitely a strong, I mean, obviously human judgment and creativity are very important. That is coming into play where maybe there is an over emphasis on the fact that we need to be really human and when you look at marketing these days, yes, there is an over stimulation for all of us around content and so that human touch is really important. But I do think the best marketers that we've seen, they are leveraging AI but also remembering that human touch, the judgment, the taste, all of those things that make marketing really human and make that emotional connection for us. If you also see cases, I think we're marketing is now being done by non-marketers because they can, similarly to design. But that is sort of resulting in us all being drenched in AI swap. Well, I mean, I suppose there's an element of that, but I think there's also an element of like, it's not just content, right? It's like programmatic that's right. Piped gen, stuff that you would normally have to have to rely on someone else to do, you can just be like, actually, no, this is important for my bit of the business and so I'm going to run with that. The other emerging bit that's super interesting is the ads in chat GPT. I mean, people are using chat GPT, it is equivalent of search anyway and so we've launched ads in Australia and the first few companies are lining up for that. So that's going to be a really interesting area to see how that evolves and changes. I'm really interested in that. I mean, Google for the longest time built, one of the world's most successful businesses over figuring out the answer to a very similar question. It's right in the line of side of the big front-end models. Do you have a prediction for where we'll go? I mean, I think it's going to change the way we think about things and I think it can become so much more useful than the way we think of an ad today because the ability to be hyper personalized to your context and so it is much, it can become much less of an ad and much more of a realistic recommendation. I mean, we all use chat GPT for recommendations of what to buy. I mean, I'll give you an example. I've got my friend's first birthday coming up for her son and so I ask chat, hey, one year old, my friend is very into these things and then I got a list of toys and books on Amazon that I could purchase and so it just made my life so much easier because I don't have to go and research all of this and it's just so nice to have a discussion around that because then it's like they suggest it but and it's like oh, I didn't tell you that actually they're vegetarian or like they don't want anything with leather or you know what I mean and then suddenly it's like instantly it's like oh okay, I understand. Here you go. It's going to take a lot of friction out of some of the consumer processes which is just going to be fun and interesting. I mean, I think one of the things that's curious is like context is king at the moment, right? Like an actually whoever can really own the maximum context and then of course have the memory to hold onto it. Those are some of the biggest questions we experience at the moment is going to be able to win that and of course in the past all of the other big players have sort of purchased context from others to be able to make the best recommendation but you know you'll be in an interesting position to draw on thousands and thousands of conversations and tone of voice and a whole bunch of other things that allow you to do things differently than the kind of you know bygone era has allowed us to do. I'd like to move to the sort of founder perspective and you've just done a founder event last week as I understand. So I want to get your sense on who are the founders that are really getting the most out of your technologies. Like what do you think distinguishes them? Is it the type of questions they ask, the type of team that they have, the type of problem they're trying to solve? Because there's so many people in our audience that are early days founders trying to think about how to make the most of this set of tooling. Satya, can I start with you and then we'll move sure on. We're seeing probably two types of founders. One, what we would consider aionative. So what they're building is entirely built on the technology at hand, right? So the thing Steve from Laura Keid, Jackie from relevance, Heidi Health is another one. We're also seeing founders where maybe they've got businesses that can leverage AI. Let's say a more digital native company that are building with AI now. The best founders we've seen, I mean we had founders day last week, we had 150 amazing founders from around Australia join us. I was blown away both by the types of founders. We had a few year 12 students that were building companies which is incredible, but also some of the founders I just mentioned and they're all, I think the best founders are really thinking about what customer problem can they solve rather than how do I embed AI into what I'm doing. They're starting with the customer and the problem. Really going from there, how do we also help our teams build faster? How do we have impact faster? And so I mean Jackie from relevance mentioned to us. He's spying up a new team at relevance. Let's focus on helping customers with their internal tools. And there's five people on that team and he was saying two or three years ago that would have taken 70 people for him to dedicate to that. And so they're able to go faster because hiring 70 people, like just the time it takes, but also they're able to just go and have that impact. And so I think it's really amazing just to see everything starting from that customer problem and working backwards. Right. I mean exactly what such has said, I can't go past a founder with a vision, like nothing beats vision. And I don't mean vision for AI, I mean vision for customer outcome. I think of Ben from employment here. You know, they've got like this really strong perspective on like what employment could be and what it could mean to individuals. And you see it across any sort of leader who can, who has just got like this clarity on like how we can get here and then is able to then slot in and be flexible enough to to try the opportunities. It's a lot of this is a mindset shift. And the person I follow a lot and we're very likely to have access to him in open hours. A guy called Pete Steinberg. So he's the inventor of OpenClaw, which is that viral agent that went viral end of last year and Pete's amazing. So like he, you know, he's got agents that you see these pictures of him. He's got like 12 screens that he's working off coding and that sort of thing. And then and so then he's like, well, I've got an agent that does this and an agent that does that. And then he was updating his agents and then he's like, wait, why am I updating my agents? Like I should have an agent that updates my agents. And then like it's this whole kind of so that that was just a mindset thing, right? Like it's like actually I could have something else predicting this and doing that. And so it's that same of ability to really think in a big and aspirational way and then translating that into the technology, but increasingly the technology can do it right. It's there. It's about having the imagination. I love that hydrogen. Yeah. So one of the big topic areas of course with AI deployment and adoption at the moment is hacks and the sort of cybersecurity and trust element of it. It's probably top of mine for a lot of your customers. How's that playing out in customer conversations? So I'll start with you. Yeah. There's been a lot of interest around it, particularly how can businesses defend themselves productively from these threats? We're working with a lot of companies across Australia and New Zealand to help them with that. And that can be as it can start from, let's say, a developer that's building their products or their is behind their code. We can actually help them check the code that they've built. And so there's those micro moments where through our cyber models they're able to check and strengthen the code base that they're building, but also it goes much much more macro from there as well, you know, into defending. So how are you seeing that play out? We've got a huge amount of interest and I think it's a really important area for us to support Australia. And like we're getting the wonderful response and partnership from the, they call themselves the chief security information offices that we deal with. As as such it mentioned, so the package around it we actually called daybreak because it's a combination of not just the cyber capable models, which are a very intelligent and very clever, but it's also the harness that we put around a harness as a fancy word we use for the coding platform. And then we also have these kind of reusable models. And so as such I said, the simplest bit which we call daybreak blue, which is much more kind of a proactive defensive perspective. And there's the much more common thing that we're doing in Australia, which is, you know, as we scale coding and as we reduce backlogs and produce more stuff, we're ensuring that we're not increasing vulnerabilities along that way. So the engineers who are, you know, being made incredibly productive in producing new solutions aren't actually creating vulnerabilities along the way they're empowered to to check the work that they're doing and ensuring that that it's safe. And then for certain organisations into the right circumstances, then we can help them with what they call the red teaming and the more proactive stuff. And that's incredibly important for critical infrastructure and that sort of thing. And it's actually great that we've got the ability to to leverage models like that. And what's the part of the debate that you can expect coming next? I mean, you see things coming down the line well before your customers. What are the sorts of things you're advising them to start thinking about now, even though it hasn't become topical yet? I don't know if this is necessarily a new thing, but cyber is not a technical problem. Cyber is a business problem. So I guess the organisations that are still kind of putting it to the side in some like small IT team here and leaving it to the problem of the tech leaders, that's the thing you advise them around because as we've seen in businesses that have had issues, it's the whole business suffers from it. And so treating it as a holistic business approach is probably the most. What might that look like? I mean, why involve other teams in that problem set? Well, because of awareness and responsiveness and the general build that you can do to be you know, it's an everyone problem and then it's also integral to the way you go to market. It's not just an afterthought or something like that. So it's a responsibility of everyone and it should be led by the business in my opinion. Yep. Satya, anything you think the forefront of the customers should start thinking about? So I would agree with Row and making that everyone's responsibility. And so I mentioned earlier around when teams are actually building the products, having that kind of vulnerabilities checked at that point. So it doesn't just sit with the SISO. It sits with all the teams building. The other thing I would say is with the proliferation of agentecoding, we've seen that previously the build phase was the bottleneck. And now we're having a lot of product leaders say, hey, actually it's like the product managers that are now just bottlenecked in terms of the drowning under review of is this the right product. And so how do you think about building your teams so that they're flatter and they take into account the shift that we're seeing. So in some ways it's organizational structure. And this is a question we're getting a lot of companies that maybe are facing these bottlenecks. How do we adjust flatten maybe align PMs with directly with the business, right? So that they're no longer that bottleneck. Yeah, it's interesting actually at so much attention over the last couple of years has been placed on like who's not going to have a job from AI without thinking of the next step of like, well, if AI makes certain tasks and capabilities ubiquitous, who then becomes decision makers that then you know, we may need more of certain roles. And actually product is an interesting example that you know a few years ago people would have said, you don't need product because you can just go straight to build and you can do customer discovery so much faster through prototyping, but we're increasingly hearing from our founders that actually those roles, the decision-making, the taste around what to build for whom and why is actually becoming its rising again as the skill set that they need more and more of. 100%. I love that. Better things for humanity. My final question for both of you. What's something you've changed your mind about? We'll start with you, bro. Change my mind about in anything personal, professional, something you used to believe, and now you believe something different. What have I changed my mind about? I do change my mind a lot actually. Yeah, I think I've changed my mind quite a bit about hiring recently as I'm like looking for people. So like this, there's stuff around hiring that's signal around hiring that you always need like in my case, you know, you need to check the technical capability and they need to be coachable and need to be self-aware and that sort of thing. But I think it's going back to that vision a bit. I'm now like really testing for like, you know, what do you imagine could come from that? Like, yes, there's a bit of hunger and yes, there's a bit of obviously there's confidence and capability, but like having a really wonderful aspiration about what you want to do is something. Not so much change my mind, but have added along the way. What do you look for there? I just ask them like, hey, tell me some stuff that you're building. Like, what are you curious about? What do you think this might be? And then just see if it's just kind of, you know, the same, same stuff, or they really stretching the thinking and kind of increasing curiosity themselves. Satya. Something you've changed your mind about? Six to 12 months ago, I mean, we spoke about hallucinations and how much the models have improved in the last six to 12 months. For me, I still, I value writing and communication a lot. And so I used to be skeptical about the outputs and, you know, I always edit and review both what AI has given me, but also what my teams get giving me. And I remember six months ago saying to someone, hey, that reads like, it's very clear. You used AI to write that email. But what's changed for me, I think, is the fact that you can actually, there's someone on my team, these very good with the tools, but he trained codex, which is now chat work, to mirror my tone. And he sent me a draft the other day, and I did not have to change your word. And so what for me, has chat was uplifting, not creepy for you? No, because it saved him time. But I think we all know that, hey, if you're sending something, it's still coming from you. And so you should make sure it's authentic and like, you own it. And so for me, what's changed is the models are there. And I was skeptical earlier about the more creative and expressive parts of, you know, leveraging AI to represent you. And I still think there's a strong need for that human judgment, taste. We talk about these words a lot. Taste is really important. But I've changed my mind around leveraging it for writing, which I'm really passionate about. Actually, on a practical note, we were talking about this in the office just last week about everyone's working hard to build prompts to get the models to really replicate the way that they write. And there was an extensive discussion amongst our team about like, I've written this thing and I've embedded it in into various parts of the, the stack so that it really knows it's like every piece should come out like this. And it's not just about the ellipsis. It's more than that. Have you got any tips on, like, maybe this is your colleague who did this for you on like the ideal way of prompting to get an outcome which feels authentic and is consistent over time? Yeah, I think the context memory is important. But also my agent that I use, I've actually trained it on my voice when I get an output. It's almost there, but I still like to add my own touch, and so I think you can train it to get almost there, and in some cases there, but all of those things, I think, are important contexts, the memory, but also you providing feedback, and just like you would talk human. Yeah. Yeah. The feedback, I think, is that that's how you train it, right? You're like, "Hey, here's the stuff." So go back and say, "These are the things I changed." Yeah. After what you did. Yes. Thanks for what you did, but here's what I did, and this is what I'd love you to learn from. There's two kind of chat-yepity tips that are good around that, so you can easily create a skill around that, so that's just basically a text file that you associate with it, but if you just say, "Hey, I want to create a skill around that," and so feeding in all your historical stuff, all the stuff that you think really represents you, I mean. And actually, on that, it was interesting, because it created the skill for me without me asking. Yeah. Even compared to three months ago, I feel like we've come a long way there. Yeah. Which is nice. And then that makes it much more easily shareable. Well, thank you both for sending us. Thank you. I know you've got loads and loads on your own. That was fun. That was great. Yeah, thank you. Thank you all so much for joining us for another episode of Wild Hearts. If you want to learn more from other ambitious people building, designing and creating the world that we all want to live in, then please hit the subscribe and follow button. It would mean the world to us, the founders, the operators, and the investors who join us on Wild Hearts. This podcast is a labour of love from Blackbird and our production team. The show is produced by Joel Connolly of Blackbird. Our marketing genius is Laura Coford, and our editor is Andy Jones. Thank you all so much for listening, and we'll see you all next week. [Music]