$300M Platforms and Synthetic Audiences with Eren Celebi, AI Lead at WPP

Exploring WPP's AI Leadership and Hybrid Role Welcome to another episode of AI for Operators. This week we've got Aaron Celebi, who's the AI lead at WPPA, big media and communications holding company. He's going to talk to us about how he does AI projects both for external clients, but also helps to lead an internal AI group and how those things cross fertilize each other. He's also going to show us two different demos, one of one of the tools that his company of 140,000 people uses internally, which has lots of interesting features, and then another demo that he built for us specifically that is really interesting. So you'll definitely want to tune in for this one and hope you enjoy. Welcome back to AI for operators. I'm Tom, your host, and today I've got Aaron Celebi with me. He is the he's the AI lead at WPP. So welcome, Aaron. Speaker 2 Hey, hey, Tom, good to be here. Great to chat. Speaker 1 Yeah, great to have you. So maybe just to kick things off for folks who are familiar with WPP, explain what the company does and and what your role is there. Speaker 2 Yeah, I'm doing the best with that. It is a giant company, does a lot of things and everyone has a title of some kind. So it's, it's a little bit hard, but I am part of the Ogilvy One CTO group. So that's reporting to the global CTO and, and the, and the local USCTO. And that group does a lot of things as well as the broader WPP, right. The broader WPP is, is a communications conglomerate. They're amongst the publicities and omnicoms in the world. Of course, I'm biased. We are the best in that. But we are also numerically quite, you know, well established in that market. And what is that market, what used to be? I think we shouldn't divide ourselves like an ad agency. We're trying to actually stay away from that a little bit. And you know, and we have divested away from that as well. So under WTTR, organizations like the classic ad agencies, which will be, I guess used to be part of, but not only one. And then there are groups like Satalia, which are mostly physics PhDs, really smart people building AI models and servicing those, as well as, you know, more of data engineering teams like choreographs. We're also well established across the globe around like CDPS and, and this idea of like, you know, how data is the source of all things marketing. So I guess it's a communications marketing verticalized conglomerate that includes everything from the data, the consulting to the services to the media and, and most importantly, actually a lot of the business like where and how that media served more than that, but it's also a very creative company from its DNA. So that's that. I know Google V1 as part of that is more of like the customer experience arm of of Ogilvy and yeah, and that, but that includes doing crazy stuff. Hopefully some of none of which I'll be able to share with you around, you know, my day-to-day since I'm innovation and AI, you know, it's, it's very different actually from day-to-day. It's not like build another image for an AI ad, right? It's not really like that even though that's cool too. Yeah. Speaker 1 Cool. Yeah. I mean, it's, it's not the days of of Mad Men anymore. It's, you know, marketing has evolved and you've got tech and data, but the creative stuff's always going to be there too. And AI can help with all of that I'm mentioning. Yeah. So I think you're mentioning in your role, you're doing stuff that's both client facing. So for Ogilvy or WPS clients, building tools for them, helping them, but also helping to lead the internal AI community. Yeah. Eren Celebi's Hybrid AI Roles and Core Philosophy In general, kind of what are clients saying they want in terms of AI and and how sophisticated are they? And I know that's kind of a big question. Speaker 2 That is big. I'll have to do the same if you like starting from the top again actually. So within what I do, I do about 3 rolls as well. I wear like 3 hats, one of which is the Yankees hat that I'm very happy to wear. But the the other three are that one, I used to conduct research at NYU for a couple of years around now. Now demand is too much work. So I'm kind of focusing on that. But and that was around AI simulations of people. So could we use large language models to simulate people and how they and most importantly from a commercial lens? So there's a lot of papers around this from Stanford about how you emulate people. But when you translate that to a commercial setting, like, oh, instead of an AB test on a website with real humans, could you do this with another one? Could you, you know, understand the click through rate of an ad? Could you understand price sensitivity of someone, right? All of these tests, could you do them with a synthetic palm instead of an actual right? And there's actually really strong data around that. So that's one thing I do and that's also one thing that clients ask for. Second thing I do is to be a more traditional engineering resource within the CTO group of only one. But as far as part of that, I'm also as you, as you described an FDE, right, like a front deployed engineer, which I think is a great model for anyone trying to do AI know your network isn't just engineers. But I think for anyone who's trying to solve a business problem that didn't used to be solved that way or that didn't even exist before, the best way to do it is to put the person who's going to do the solving with the people that need the solving right we. Speaker 1 Just create a deck anymore and and let them figure it out, right? Speaker 2 No. And that's the biggest crux of AI to me. I mean, there's like data security is in this field to me, not as much because I know the inside. I think people don't know what's inside. Everything is scary. What to me is scary and I believe to clients as well as believability. It's like the use case I just described. Yeah, instead of doing a market research or consumer insight study on the Toms of the world, if I do that with an AI, do should I believe that study, Is that study real? Is it hallucinating? Is just making things up? So increase that believability is to just put the person who knows what they're talking about with the client. If you don't show them a POC, if you don't show them a product of some sort of it working, you just build a deck, put some nice photos on it. I don't think that's viable. I don't think that's solvable. I don't think that's something that can happen. So that's kind of what I do with an ugly one and a little bit above the demand signal that we get. And then the WPP role with the community is a little bit of this product thinking and a little bit of this like, oh, here's a business problem. How can we solve this better? How can I float in and out of different groups and talk to smart people and and connect. Like I said, we just have like insanely capable people with pH DS that I don't have, even though I have a research position as well, which is another story for another day of how people let me do that. But but yeah, basically connecting those resources with the needs, because a lot of times for tech organizations like us or our subgroups, we just have so much capability, we don't know where to apply it. We try to find an application for a tool or for a technology, whether it should be the application that demands the technology, right? It should be the other way around. So I tried and be that connection a lot of times that sounds like a very product role, but I think someone we can code as well is required in steps. So lots of words. I'll I'll go into more specifics of like, you know, what kind of specific use cases I hear about that people want. But hopefully that gives you an overview of like kind of the frameworks that I have of like how I listen for problems when I jump in, how I try to connect with business problems. Yeah. Cultivating a Product Thinking and Hacker Mindset for AI Yeah. No, that that makes sense. I think one of the, you need things about your role that strikes me as like the, the product universe that you could build is just massive, right? Because you're not only creating for this big global conglomerate that has lots of different businesses underneath it. You're also creating for clients. And they could be anybody or anybody big and, and rich enough to afford you guys, right? So how do you, what does that building product process look like? What it What lessons have you learned there? Speaker 2 Yeah, I mean, and by the way, I'm glad you're taking this into a product discussion. I want to like open this up to a wider range of people. So correct me if I'm getting too technical or into the wrong directions here because I'm a big believer that a lot of what I do is inspiration. Like a lot of what I do before we got it popped on this recording, we're talking about how, you know, there was some, there was the exco happening last week for US. WV recently had ACEO change. There's a lot of these like high level discussions that are happening. And I think for someone like me or for anyone interested in technology, it should be their job to just come in inspire. And it should be, I think anyone's job to think about technology, right? Like, you know, fire is a technology, a hammer is a technology. Like everything we do is a technology. Like this idea that you have to have a PhD. While I respect those folks like you don't have to, right? You could, you could think about this and, and you could generate revenue, you could generate efficiency for your, for your, within your company or for the people that you service or the products that you sell at any scale. And it's like this thinker mindset, right? It's this hacker mindset. So by the way, the I don't know why they left me, but the AI community that I lead is called hackers AI community. So it's just called HAC actor, that community. And, you know, I hear a lot of different kinds of demand and there's a lot of learnings from it. The first thing I try to do when I come in is to inspire people to think like engineers, like put their engineer hats on. Whether they finished, you know, what level of mathematics they studied at school doesn't matter. It's actually really simple. It's like you just, I mean, this is classic product thinking about you look at what's my problem? How's it being solved today? How can I solve it better? The issue is for people who don't know how can it be solved better, like the the raw tech capability, they actually can't really make that connection. Like there needs to be a conversation with someone who's experiencing a problem versus someone who knows the kinds of outputs you can get out of these elements, who knows the kinds of outputs you could get from an image model, etcetera. So there's going to be some translation there, and it's a good skill, if you have that, to be able to do some of that translation. Speaker 1 Yeah, that makes sense. Yeah. And it's like you've got to have somebody who who knows the problem really well and can describe it and can figure out like, hey, this is, this is what a good enough solution looks like. But you also need somebody who's up to speed enough on the capabilities of the technology to actually figure out how to make it better. And sometimes, you know, that is the same person. But more often than not, I guess that a lot of our listeners fall into one bucket or the other, right? And so that means you need to team up with other folks in your organization and either listen and really understand their problems or find somebody who knows this technology better than you do so that you can figure out how to what the make it better looks like. Speaker 2 Yeah, I mean, I'm going to bring this to. So the references that I make, can you still hear me by the way? Yeah, the references that I make are generally in like either from psychology or from like physics. So I try to keep quite scientific. I use something like that called the Hicks Law a lot. It's like a UX thought actually of like when you give someone 1000 options on AUI, right? Like they don't know what to click on. So if you gives a business lead 1000 different things to look at, thousand different things to do, like they won't know what to pick. So that's one idea and then another. It was going to be this is something. So we had the extra last week and I got to meet with some of the, you know, highest level, smartest people at my organization and I always listen to like their management ideas. Something comes up quite often is so the Ogilvy 1 and and vertical leadership sky outdoors the vertical CEO who built the company out of nowhere. I have nothing. His thing is like when you go into a room, have the ability to think that the other person is smarter than you. Like open your mindset, the possibility that the other person might be smarter, right? You don't have to already believe that going in, but like open the open to that idea. And so that's what I expect is like when you go in as a tech person, right? You go in to talk with someone who has a business need, like open your mind to the fact that, yeah, maybe you're super cool and you have a page THD and this and that, but the other person knows what they're talking about. Make that assumption in the beginning. And I think doesn't it's not the same experience for I believe, executives or business leads. When they speak to someone technical, there's usually a level of respect of like, wow, they did the schooling and they know what they're talking about. But a lot of times it's they they can't communicate because like they don't actually believe that the technical person can can take on the business problem, can really merge, merge themselves in it. It's like, you know, it's also like another psychological self, right? If you tell yourself, well, I don't know maths, like I can't understand. AII never did. I was never good at physics. I was never good at maths growing up. Then you won't understand it. Like if you just tell yourself you won't get it, you won't get it. So I think going to meetings, going into stuff with an open mindset is, is a huge skill to do any kind of any kind of innovation, which I guess lends itself to to what we're doing. WPP's $320M AI Investment and Internal Productivity Tools But yeah, that's the question. Yeah, but what? Maybe. Speaker 1 Just to get like really tactical here for a second. What where have you found that your your work is made more efficient by AI or like what are the highest leverage things that you do with AI on a day-to-day week to week basis? Speaker 2 Yeah, I've been like, I had this in my back in my mind since I believe this is going to go live quite soon and it's still relevant. This whole like MIT study that came out I think was talking about how like more than 90% of AI POC don't generate revenue and don't turn into like useful products. Speaker 1 Which I think is BS BS by the way. I mean like on what time scale idea I don't buy. Speaker 2 It if you read that study, so I think a lot of would look at the highlight if you read the rest of that paper. First of all, they have an incentive to say a certain thing. They're basically every paper. I know a lot of scientists. There is no such thing as no bias, right? You'll do your best to not have a bias, but everyone does. If you come from a, you know, and, and their bias or, or their point, which I might actually agree with, is that they're saying there's this, there's basically shadow revenue. There's this shadow unrecognized profit that's being generated. So they're not saying like AI isn't helping the workforce. They're saying I is helping the workforce a lot. If you just read 3 papers 3 pages down, it's like it's super useful, but it's what the, it's what each person is doing, right? It's them like using ChatGPT in a next tab. And it's not the product that they built, which again, may be I'm biased for my company. We're actually doing something really cool. Maybe it's a model for other people is from day one. We're very early with this. I mean, the number changes a lot. So I might be wrong here, but it's somewhere in the $320 million range is how much we invest in AI each year. So that's a ridiculously large amount of money. And the the first thing they did was, OK, ChatGPT, this GPT, whatever it might be, let's take these individual Productivity Tools and let's make it available to people. So we built our own version. We built an individualized productivity tool that also actually does collaboration. It has every model you could think of, from image models to language models. It has different contexts, different databases that it's been connected to, and it's called WB Open. People can Google it. There's a lot of knowledge about it externally. I could even pull it up here in a second and then show you very quickly what it is. But but yeah, and we just gave it to people and that is the first step, right? It's just giving people the individualized LLM generally, because for us we have people who need image models. But in general, I think for most industries, images and video aren't really important as a media source. People are writing emails and they're doing white collar work, which generally involves text and that's that. So that's what people do. I think. Yeah, it gives people the individual tool is the first step, but there are other steps if you want to go into them. There are other places we can apply stuff and the learning to the mighty studies, like start there. I still think there is usefulness to the POC. There's still usefulness to build your own custom builds. Like that's why I exist. So that's my bias for you. Like I'm here to build tools. Mind you, open was also a tool, but there are more complicated things you can do and in that way, the best thing to do is now that you have the basics, now that you just have things available to you and you've experimented. Now go back problem like reverse engineer from the problem. I start with the problem. Reverse engineers say, OK, these base LM that I have access to. I keep doing this behavior right. I keep going to GPT 4 and I keep asking it to edit this e-mail. I keep asking it, So what are those repetitive motions, repetitive things that OK, perfect automate that like go with that. Don't, don't just say this is like you don't have to mentalize and think this is a good place for me to start, start and then work backwards. Building AI Simulations for Healthcare Consulting Over a Weekend Does that make sense? Speaker 1 Yeah, you Speaking of tools and things, you mentioned that you had a demo to show us. You want to show us a tool? Speaker 2 Yeah. So I'll show you open very quickly. And then I actually had, if we have the time, I have another use case that I want to show you. So maybe we'll look at like, what's the first thing and what's the last thing you can do? So maybe that's a good comparison. Let me pull up open unless you want to start with the most complicated. In the meantime, I'll describe to you what the more complicated example is going to be because I'm about to just throw you into a use case that I've been building. So health consulting is a big part of what we do, right? There is a lot of this type of communications issues that happen in the healthcare industry. I'm not actually an expert in it. The beauty AI and machine learning is that if you know your tool, you can actually apply it to stuff that you're not expert, you're not expert in, right? So I'm not, even though I'm not an expert in healthcare, the idea was what I've just described to you about the NYU research right around how could you create Toms, right? How could you create synthetic beings that then you could S messages with, you could do consumer insight with, understand and communicate with. And then that was a perfect place to apply it because we heard the signals, right? We heard from our Consulting Group. And by the way, I just want to caveat, this is not a built product. This is not a paid process. This is me just tinkering over a weekend. And I do think that's a good model for it. Yeah, how things can go if this becomes a paid for tool and it becomes deployed and it's ready for the world, I will tell you about it so we can we can communicate it that way. Right now it's just an it's just something I cooked up over the weekend to show. Speaker 1 You. That's how all good things start. Speaker 2 Right. Yeah, yeah, exactly. That's what you need. We did that. We did the simulation thing, but we did it. Are doctors, right? Because, right, We hear it a lot. Smaller companies that spend most of their money on R&D, right? These guys are solving cancer. They're solving like very important problems. They don't have the money to spend sometimes hundreds of thousands of dollars to go talk to doctors and understand how to communicate with them, which is very important by the way, because but then doctors are saving lives and they're a very expensive resource. To convince a doctor to sit down with you and have a conversation with you for 30 minutes is very expensive. So this sort of like consumer discovery thing is very, very hard in healthcare. It is it's a bit easier for the consumers over CPD, right? Like those you kind of find people that want a bottle of water out in public, but the doctors are hard to speak to. So we, I basically just take care of the weekend and prepare this for you where we have uploaded a bunch of interviews we've done already with doctors and then we're going to speak to them and test stuff out. Demonstrating WPP Open and Strategies for Scaling AI Adoption So I have open and both of them both available. This is open. It's what I've just described. It's different in terms of, you know, it started off, I don't want to say the wrong thing here, but it started off as a rapper, right? A lot of people call these like rappers, but now it's gotten so much more than that. And this is the this is the creative studio sub platform under open is actually too much for us to go to. There are ways that we've allowed our people to generate their own agents, right, where they can bring their own context windows, you know, do things like search grounding. So this place has become like huge, like there's so much you can do. But a good starting point is something like this Freedom Studio has always been a, it was, I believe, one of the first platforms in this suite. And it's things like you come in, you have a strategy question, you have a product, you want to look at a current strategy, you want to get an insight from an audience. You get that out right? Or it's something like, you know, you can generate podcast, you can generate video and when you come into chat, it's as simple as this. So there's a lot of frameworks underneath of more complicated things you could do. But I think the first thing to do for people is to inspire you, your management, people who are the general public of your company, is to just just use it. Like I personally was a skeptic. I come from a background where I was using models like Burt that like the first transformer models when they first came out. You'd think I'm an early adopter. I wasn't, but people were criticizing me for not using it enough. And and that's it. You just give it in the hands of people and it can do a lot. So this is a bad intro of of Creative Studio, but there you go. It's the simplest thing you could do. Get people moving, give them some frameworks and prompts to get started, and have some models make it easy for them. And then the more complicated thing you can do is probably something like this. Speaker 1 Well, let me pause on this for a second because I think there are a couple of things here I want to highlight One, yeah, is just the kind of a couple smart moves here and not, not surprising 1 is like a bunch of custom agents that are specifically for use cases that you know that the WPP employees deployed on a yearly basis, right. So one, that's just useful, 2, it gets it makes adoption easier because suddenly they see it work in the context that they're already operating in. They don't have to stare at an empty screen and say, OK, like what can I ask? What should I ask? Etcetera. They're like, it's going to create like a creative brief for me, right? So that's super useful. And then the other thing I just call out is probably a lot of folks listening to this don't have 320 million to invest on an annual basis at their company to build something like this and spread AI. But most listeners probably aren't as big as WPP either. And so you can approximate some of this with even off the shelf Tachi PT with like custom agents or same thing with with Claude or with some tools like LVX or other things that like allow you to create some of these things and share them amongst your team. So if you can do something like this, amazing, great, go invest your 300 million. But if you don't have 300 million, it's not doesn't mean you can't play in the sandbox. Speaker 2 Yeah, a lot of that money is the scale actually. Like I don't want to diminish how difficult it is to build this, but like building Apoc, what I'm just going to describe to building a version of this that 10 people, 20 people, 100 people can use is actually like extremely simple. The problem is this has 50,060 thousand Maus because WPP is 140,000 people. So when you give a tool to 140,000 people and make it free to use, it ends up being quite expensive to upkeep, to scale to the underlying models being pinged, right? Those APIs like that becomes expensive. So I don't think that should scare away anyone. I agree completely. And as you were described talking, I was just clicking through some stuff. You're like something as simple as writing a JD. Yeah. Speaker 1 Yeah. Speaker 2 Right. Like I hate writing a JD, I hate the recruiting process. I hate being in the one side of it I had beating in the other side of it. Like it's a tough process and I think just simple things like that could be a great starting point and building frameworks, like you said, like get people started somewhere and then build on that. Advanced Demo: Extracting Insights and Questioning Synthetic Doctors Like there are way more complicated tools that happen. There's a lot of this like creative stuff that actually happened on this platform that we won't have the time to go into that is also very cool. Like, you know, creating ads and creating communication that way. But yeah, good. And then you're the more complicated thing you can do, right? So what I just described, right, let's maybe so you could upload into this like example data. I'm going to refresh just so everything is brand new. I'm going to upload just a few doctor transcripts. So it's like 4 docs of talking to doctors, right? And it's and the beauty of maybe if we're thinking about AI product building today, one thing I'll bring to your audience is that data used to be structured right. Like, and this the beautiful thing for me with AI is you could now just use anything. Just use what? Yeah, like, Oh yeah. You have APDF of a transcript of a conversation of a doctor. Oh, in this other version that you asked this question, but you didn't ask this question. You needed to be very pragmatic and and uniform back in the day with this kind of data and data out. Nowadays, the beauty of AI is you just put anything and see it magically work with that, right? Like, oh, you have a job description, but it's not in the right format. Upload it. It'll understand. It'll get the output to you. Like that's the beauty. And that's what I think unlocks so many use cases. All of these four documents are like different conversations that people had. It's in different formats, it's in different places. It's just stuff that I've taken. So we're going to do now is we're going to extract some information. Like I just said, we're going to take these long form texts and we're going to say we're going to ask basically a couple of tags to be taken out. It takes a second. I don't know. I will try to get a little more technical for your technical audience. What it's doing is in the background is it has one agent that looks at all of the docs and says what are common themes that I can extract instead of extracting everything from every doc because they're so non uniform. It first asks what is a common thing that all of these documents share? Are they all talking about for these doctors, how many years of years in practice they were? What's their name? What are what's information that I should take out? And then it goes and does the taking out of that information right. It's this beautiful idea of agency. I'll send you an article and so you can share with your with your. Yeah, network. It's a paper written in 1960, the exact year of this guy who kind of talks about agents back in that day, about artificial intelligence, about how the machines will start asking the questions of, you know, they'll start preparing the questions, not just the answer, right. It's always been that we have the humans, the questions and the computers give us the answer. Now the computers can come up with the questions and then come up with the answers to the questions they came up with. So there you go. It extracted all this information file name using practice, their practice setting, whether in a Community Hospital or academic geography, like where they are in the world, etcetera. So it just extracted all this info from these very dirty data sets. It's also visualize it for you very quickly. So you can look at the results overview, you can look at years and practice what that looks like, right. And again, this is all done. I mean, we have a, as Ogilvy one, we have a history of building human simulations. But this version of it, this building a little POC like this was a couple of days. And then what we're going to do is ask these people questions, ask the doctor questions. So I'll add 1, and if you want to add a question, you can as well. First thing I'm going to ask is what is your name? It's like a test question. So we fed these interview data with people's name in it. So if they're giving us the wrong name back, then we're probably not doing a good job of simulating these people, right? It'd be really bad, Tom if Tom didn't know his name was Tom, right? So we got that. Another one I'm going to ask is how important is page and quality? I could type better quality of life in pretty well. And then I'm going to say I'm going to give a category, I'm going to make this categorical and I'm going to say answer from 1/5 with five being extremely important, right? And then let's just stick these out quickly. By the way, you could upload a document. Again, the beauty of AI is you could build UIS like this, user experiences like this, or you could just let AI figure out the what the interview questions that you uploaded and doesn't have to be in the right exact, right format. It'll hopefully do a good job. So we don't have that many doctors. So we're going to select all four of them and then we're going to simulate their answers. Again, if you have like cool techie people in the background, this is kind of built on the Stanford framework on how to do this. And it's doing it concurrently in the background, which means, which means it comes back very quickly. So it's asking 2 questions to each agent at the same time. So it makes it come back way faster. And then you can see that Christina Lopez says, I'm doctor Christina Lopez, I'm medical oncologist and colorectal and lung cancer. You know, Maria via Alto Villar says that's she does. She's at the Bal de Hebron hospital in Barcelona. Or if that's a bad pronunciation, and then boom. And then you have the responses. So most of these doctors said five, I really care about patient quality of life. And I believe just one of them said four, like I care, but not this much. And the most important part here is the confidence interval, right? It tells you how big of a jump am I taking from the interview data you gave me to making this inference, to making this guess? It's a very high interval because pretty much this question is already in there. Like almost every doctor was asked in these interviews. Like do you care about quality of life? But so therefore, you see this confidence is a little bit lower. It's given a four and then I'll give an explanation as well. Varying timeframe based on patient age, scientific approach is prioritizing early treatment. So I think this person, this doctor is probably like a bit more about the science of things rather than like the feeling good of things and. Speaker 1 Is, is the model here like we could ask it a question like do you prefer Nike or Adidas? And it would provide an answer, but with a much lower confidence interval. Sorry. Speaker 2 Yeah, if you prefer. Well, let's let's do this. Do you like Nike running shoes? But if you ask that, let's make it categorical as well. Let's say yes. Speaker 1 Yes or no or no? Speaker 2 Hopefully it should say something like, like responses. It should say something like, I don't care. It's like I'm a doctor. This is not what I care about. And yeah, and you know, testing, by the way, is the best way to increase filling ability, whether you use the right frameworks because like these are built on on solid research. But so that it's all saying no and it's saying the provided text focus is on professional life. There's no mission for personal purposes on footwear. It's like, don't ask me this question. Like perspective practice focuses, like this is not. It's actually pretty confident saying no. It's like, yeah, I don't care. I don't care about this, you know, do you like this? We asked it like, do you like this? And it's like, no, I don't care is the is the negative of that. So there you go. Cool. Lots of cool things you can do, one of which is this. Finding Your AI Use Case and Episode Conclusion I'll try to pause. There you go. Yeah. Speaker 1 No, that's awesome. Yeah. And then you can. Speaker 2 If if. Speaker 1 Whatever company you're at, you can, you can see lots of interesting use cases for simulating people in in AI, whether that's customers or recruits or your team to see how a certain policy is going to land or a strategy pivot or something like that. So. Speaker 2 Yeah, that's, well, probably over time. I'll leave everyone with this. This happens a lot when I go into conversations. I'll give you a more meta point, right? I just showed you guys something that could be built. That's my job is to show people what could be built, but the hope is that by showing you a thing to build, that inspires to build the B thing, right? Like we showed you that you could simulate humans, but like just go build something else. Just go think of another use case. Look at that. You just spent a lot of time here, but you got to immerse yourself in so many different use cases. Finally you'll be able to like, oh, I want to build this Z thing. You look at ABCD. This is one thing you can do like this chat wasn't about how to simulate humans and this tool right? It's like, how can I do my thing? And the best idea is like, be inspired. Go look at a bunch of things and then find your thing to solve. Find your thing to do. Yeah, yeah. Speaker 1 That's great. Well, thanks for sharing. Thanks for coming on. And we'll we'll add a couple of those links in the comments so that people can find them as well. But great. Yeah, I appreciate it. Speaker 2 Thanks, Tom. Thanks all for listening. Bye. Speaker 1 Thank you. Thanks for listening, and thanks to Aaron for joining us on this episode. If you want to find out more about Aaron and his work, we'll include a link to his LinkedIn in the show notes. And otherwise, please tell a friend, your smartest operator friends, about AI for operators and make sure that you're subscribed to the newsletter as well, if you're not already. Thanks and we'll see you next week.