Season 6, Episode 18: Meta's Business AI and the ad-product divide

Why pay platform fees when you can go straight to the source? Universal ads is Comcast's self-serve TV ads platform, powered by the same ad tech that powers the TV industry, now available directly to brands of all sizes, with zero fees and across premium inventory from NBC, Paramount, Warner Brothers Discovery, Roku, and more. Visit UniversalAds.com to cut out the middlemen and get your ads on TV in minutes today. The problem is that the distinction needs to be drawn between the confidence of the economists and the correctness of their analysis. Welcome to the Mobile Dev Memo podcast. I'm your host, Eric Sufert, and I'm joined today by Simon Whitcomb. Simon, welcome to the podcast. Thank you for having me. What's nice to have you here? So we met at Metta's Performance Marking Summit in May, if I remember correctly, in San Jose. That is correct. And then you visited us in New York a few weeks ago. That's right, yeah, I was in the New York office. And so yeah, it was really great to get to know you, and I really enjoyed speaking at the Performance Marketing Summit. Like, you know, there's not many opportunities in life to feel like a rock star, if you're not actually a rock star, but for me, that was one of those moments, just like a 2000 odd sized group of like-minded people. It was a really fun event. You were absolutely the head of the event, and so many quotable moments coming out of your talk. So thank you for doing that. Yeah, and that's actually where I coined the phrase commerce at the limit, which I then just fleshed out in a longer form blog post, but that just popped in my head at the time, so I'm grateful for that. All right, we're going to talk all about Metta, and maybe I just kind of spilled the beans a little bit, but before we get to that, can you please introduce yourself to the audience? Yeah, of course. So my name's Simon Whitcomb, and I lead Metta's Global Business Group in North America. So I and my team essentially work with all of our large advertising partners. So I've been with Metta for a little over 13 years. In fact, my anniversary was a couple of weeks ago. So I joined the company in 2012 to lead the gaming team in North America, which was about five weeks after we launched what I would consider to be our first great ads products, which was our mobile app install product. So I could not have walked into a better situation to join to lead gaming about five weeks after we launched that product. So before Metta, I worked for various traditional media companies. I was also an entrepreneur in the first.com boom of a video game website called IGN.com. I don't think I knew that about you. That's really fascinating. So 2013 was just such a wild time in mobile gaming. I was talking to Ari tomorrow about it. I had this event in New York like two weeks ago and it was just so frenetic and exciting. There was just this electricity. You had this new gaming business model for you to play. You had mobile. It was just the confidence of those two things. It was just striving so much investment and there were so many fascinating new innovations, like commercial innovation, technological innovations that were coming to market all the time. That was a really fun period. There's amazing period of time. Of course, in that period, the industry was buying installs on a cost per install basis. CPI was the metric. We launched a product that wasn't competitive on CPI. It was competitive on LTV. But we almost had to wait for the market to catch up with that change. It was fascinating to see the early move is here, the pivoted to an LTV or a PLTV model. Then the industry just skyrocketed from there. This could end up dominating the podcast. I'll make sure we move on from this topic. This is near and dear to my heart. We got to stick here for a little bit. It's funny to say that because I wrote about this a while ago. It wasn't a dedicated blog post, but I think it was just a point I was making in the blog post that mobile gaming took off when the advertising infrastructure was in place to support it. Because mobile gaming predated even the app store really. There was the J2ME games. But the reality is the paid model, if you think about like Temple Ron or Angry Birds. Those are a few of the games, but they were not like sustained long lives. Businesses really, its success was kind of built on the free-to-play model when it pivoted. When you were able to actually see the massive growth in the mobile gaming category was when you saw all the advertising units and the measurement infrastructure come to market. You look at the history of that and it was all the attribution companies come into market like 2012, 2013, a lot of the ad networks, Facebook, introducing mobile app. You needed those things to be in place before you could build a billion dollar franchise out of a mobile game. Yeah, undoubtedly. Mobile measurement partners were a huge step forward. There's actually a missing element here, I think, which is the old Facebook canvas gaming. When I joined the company Zinger and Farmville and Mafia Wars, these were huge games on our platform. Again, I think marketers were just beginning to figure out how they could find that 2% of the paid audience that would drive 80% of the revenue. But yeah, no, I think you were exactly right. We needed to wait for the pieces to come together. Then the talent needed to be there as well. There had to be the right talent that could kind of lean into this opportunity. Yeah, I mean, just because Zinger, I mean, it deserves a lot of credit for incubating a lot of those ideas, because they were very, very sort of like analytics forward. I mean, you hear the stories about their PM process was just an analytics exercise, right? I think they incubated a lot of those ideas and then a lot of those ideas propagated out of that company and then seeded other successful companies. Absolutely. Yeah, you could see almost like a coaching tree in the NFL, right? You could see where a lot of that talent went and then those companies got traction and it kind of built from that. Yeah, and then in Europe, so I worked for a company called Digital Chocolate, which I think it probably doesn't get enough credit for incubating a lot of the talent in the European mobile gaming market. Digital Chocolate acquired a company called Simea, which was like the precursor to Supercell. And they kind of left the leadership of Supercell left to start Supercell. In fact, they were like in the same office building. We used to share an office building. They just moved literally next door to the start Supercell and that has skated across all of Europe. But a lot of like the big European gaming companies, a lot of those founders either started at Digital Chocolate and had the same kind of effect. But okay, I'm glad we took that detour. I always love talking about the formative stage of mobile gaming, but I want to talk about the business tools that met and announced it ad week, right? Going into ad week, I think it was just before ad week meta announced that you're expanding beyond ad surfaces into what I see as like advertisers own tools and workflows, you announced business AIs and the meta AI business assistant. Can you provide some color on what those tools are? Yeah, absolutely. So I think this is actually a really good question because despite similar sounding names, these products are very, very different. That distinct products, the quite different audiences and roles. So let's start with business AIs, which is essentially a sales concierge, almost like a white glove AI agent that can be present on your website if you're a business inside of your meta advertising or as an agent experience in messaging. Now we are very, very early in our journey here, but we're incredibly excited about where this can go. So they're currently available for small businesses in the US for agents on ads and websites. And then in Mexico, they're available as agents in WhatsApp and in the Philippines as agents in messenger. I think the beautiful thing about business AIs is it allows you to answer very, very specific questions from the consumer about the brand, about the product, about the service and here directly back from the consumers regarding that business. I think it's really, really interesting for longer consideration purchases where naturally there's always going to be more questions. You know, furniture and home goods would be an example. I think auto's over time would be an example, even booking, you know, a two week vacation. So the small businesses who were testing here, I think they're already seeing some eye-opening things, even with what is quite basic AI assistance, which is going to get much, much better over time. So one of the areas that we're seeing is these agents in order for them to work, they need really clear, well-organized product detail and businesses are discovering that much of their information on their website is actually outdated or incomplete by going through the process of building these out. I think the area that interests me the most is that many of the questions that consumers are asking these business AI's are incredibly basic. So I was speaking to the CMO of OG who was telling me that as basic as is this a blusher on a page where he would say most people would know that that's a blusher, but I think as humans, we often set the baseline understanding significantly higher than reality. So I think oftentimes these are the things that we argue should it actually even be on the website or should it be in our ad creative where what we're learning is no, everyone doesn't know that an AI agents are really beginning to prove that out. And I think a final area is insights. Another retail brand we work with found the prospective clients were asking about best sellers all the time, even though a best sellers collection didn't exist within that individual business, but now as a result of the questions they build one out. So early days, but I think really interesting for long consideration as well. On the other hand, our meta AI business assistant really helps marketers find better performance or sort out and fix account issues with personalized AI recommendations, integrated via AI-driven chat within MetasAdTool, such as Ads Manager. So you might remember when I saw you last, you brilliantly said, wouldn't this kind of thing make a great product? And little did you know at the time, we were actually well down the path of building that out. So it's basically like having an AI-powered advisor embedded in Ads Manager. I think the great news here about MetasAdTool platform is it's able to drive outcomes at scale, right? We are consistently the ads of work. I think where MetasAI Business Assistant can help is really helping advertisers and agencies navigate the complexity of our platform and land on the next best thing to do or to figure out a problem when they inevitably occur. So again, it's really early here. We're currently testing in beta with small businesses, but you're going to hear a lot more from us on this topic as we go into next year. Yeah, so it's funny because when I made that comment, I saw a bunch of people in the audience like smiling. It's kind of a skill. Eric, isn't it? We'll say. So the best seller kind of data, that's really fascinating use because we'll circle back to that topic later about how you can like services insights that's I think like a pretty valuable opportunity here. If you think about MetasAI, I mean, it's obviously front and center for the company. It's been a core topic of the earnings calls. And I had Matt Steiner on the podcast maybe six months ago. And we talked about Jim and Dr. Maddress, which by the way, that episode has had like a second win. That got rediscovered a month ago because of the Andromeda update. And so that's actually had more listens in the last month than it did in the first month after publishing, which he was interested in. So it's obvious that AI has been front and center for the company and with ads. Yeah. I kind of want to talk about how else you're bringing it to bear for advertising clients in non advertising use cases. You talk about business AI. That's the agent you integrated into the website. And what I see that as is kind of like a handover. You're handing over the baton. And that's like a personalization approach. That's a conversion optimization approach. What are the other opportunities there with giving the advertiser that kind of technology that they probably couldn't build themselves, right? Yeah. So obviously we've invested a huge amount in ads ranking and delivery, which is what you were just talking about. But there's like three other areas present a meaningful opportunity. One unsurprisingly is creative, right? Leading into AI for creative generation. Another would be audience insights and then campaign optimization would be a third. Millions of our advertisers are already using our creative generation tools. And interestingly, two million advertisers are already using our video generation features alone. So we're beginning to see, well, more than beginning, we're seeing a surge in businesses really leaning into these tools. Now all of this is part of our longer term journey towards one to one personalization. The creative tools that we're developing today are really steps along this path. MetaSystem is incredibly good at serving the right creative message to the right person as we continually improve our ranking systems. But when partners find a creative that works, they invariably do exactly the wrong thing, which is they create a thousand iterations on that one idea, which effectively means that they're reaching the same small audience of people, time and time again. What we need and what we've been pushing for a long time is this notion of differentiation. And what we mean by differentiation is differentiation of formats. So use all of the surfaces that we develop of themes and of concepts. And if you do that, you can actually reach many more people more efficiently. And I often say to our partners that they should look at unique people reached and the connection to that metric with creative diversity. Now, while creatives love building concepts, they don't love the idea of creating variations for every single meta surface, which is where these tools can really help. So really leaning into them in order to power differentiation is something that we're seeing. I think we're also seeing larger partners being much more reticent to lean into these tools. And again, I think in periods of change, businesses often focus on what could go wrong rather than what could go right by being an early adopter of many of these solutions. So I think it's imperative for all advertisers of all sizes to begin experimenting with Genai for creative. Even if they don't run the results in production, it really allows them to test and understand how far away they are from the creative being production ready for their business. Because we expect those gaps to close really, really quickly. I think another focus is improving the predictive performance of advertising models. So better AI models lead to a better rate of conversion and allows partners to invest more to grow their business. So an area where we see this unsurprisingly is with our advantage plus shopping campaigns. When you turn on the products AI-driven targeting features, on average, they deliver 22% more row-ass than when you don't turn them on. So I think one of the great stats we have here is when you put a dollar into the ASC AI machine, it spits out $4.52, which is really compelling to get businesses to move in that direction. - Yeah, I think so, but the business AI piece, I wrote a piece about this 'cause I thought it was like really profound, right? And I think just as an evolution of the idea that personalization is gonna deliver better outcomes, right? Because if you take that personalization on platform to the limit, right, you got an individualized creative for the person that's most likely to resonate with that individual, right? And then they click through and they see the same landing page that everyone else has seen. Yeah, so there's an obvious next step here, which is a, well, then let's personalize that too. And again, like that idea of the baton handover, you need some information there. Like what do they see? - Right. - Now you can encode some of that in the UTMs if you're talking web, but if your app is harder, I mean, if there was a vehicle to sort of receive that person with some knowledge of what they clicked on, that is really powerful. Then you have the ingredients to personalize the entire funnel and not just the top of the funnel, which if they hit that landing page that everyone is seeing, like it may not actually benefit that much from the personalization drug to clickstown, but it didn't actually do that. At the end of the day, you want the whole funnel to be optimized and not just any given segment. - Absolutely. It's a collapsing of the funnel. I think the opportunity here is enormous if we can get this right. And again, the experience we're going for here is imagine having somebody, a real person who is incredibly informed about everything to do with that business or product. And you have the ability to ask them any question that comes to mind. And with very high confidence, you'll be able to answer any questions they have, which will take them further down the funnel and to your point, the ability to personalize the experience when they eventually click through. I think the opportunity here is huge. I think it also opens up digital marketing for businesses that have maybe struggled a little bit because they're higher consideration or they've traditionally been very brick and mortar based. - Well, yeah, so just kind of going back to the top seller's idea. I mean, you're going to imagine that, if you're an E-Com retailer and you've just, you've got some storefront that you set up on a platform, like probably not a whole lot of easy customization there. I mean, it's some framework that, unless you're going to hire a dev who's sort of proficient in that framework, I mean, it's going to cost you a lot of money to build something like that. Whereas MetaNose, I mean, they serve the ads, they see the conversions, they know. And so why not just layer that on? And I can imagine that actually having a pretty significant impact on conversion rates, because it's like, well, let's just give that information. It's actually pretty basic in a sense, but it would be really difficult for the E-Com retailer or whoever to implement. - And that's the whole idea with this entire product. It's turnkey, unlike having to go seek your own model, build out your own personalization, this entire thing can be very easy for millions of businesses to onboard and see that benefit right out the gate. - So what about extensions of that? Are there other obvious ways that your tools could be used for optimizing the products themselves or understanding the performance? And now I'm not talking about the ads piece, the top of the, like the actual, you know, the ads platform. So if I'm talking about their actual products or their actual landing page, are there other opportunities there? I think there's two other opportunities that come to mind. It's not so much landing pages, but I think one obvious way that Metas Tools can really help partners is to optimize their products and understand performance through catalog, right? And, you know, marketplaces have been leaning into the benefit of catalog ads forever, but there's definitely an opportunity here for smaller businesses as well. You know, catalog ads allows our partners to showcase a dynamic selection of products that leans into not only our ability to serve the right ad to the right person at the right time, but our ability to serve the right product in context. So catalog advertising is the lowest funnel product that we have within the Metas Suite. So it performs incredibly well because we can predict with a high degree of confidence the product or products an individual consumer is gonna be interested in. So even a small business with a catalog of 100 products or less has an opportunity, I think, to lean in here more than they would have thought about. historically. And then I think the other area where we're leaning in and making a huge investment is really on this idea of value optimization. So going beyond traditional conversion optimization and really focusing on like, can we optimize for value, which gives our partners a choice when setting up their campaign so they can choose to either optimize for a volume of conversions or they can choose to optimize for conversions with the most row S. So like I said, we have a big big team working on this value optimization is the most performance way to go in instances where a business has a lot of conversions of vastly different values. So if you want to optimize for large basket sizes, value optimization can do exactly that. We're also expanding into things like profit. So maybe a small business could indicate to us that they are actually selling a $10 product is better for their bottom line than selling a very similar $30 product and we can optimize for that. So we're essentially telling our partners, you tell us exactly what you want to do with your campaign and our system will go and optimize for that specific outcome. So this product has been getting better and better due to improvements in the model over the last 12 months. It now represents a large percentage of our performance marketing on the platform and I think it will only grow from there. So a lot when it comes down to value, more to come from us on this particular topic. So I kind of invoke the the Converse of the Limit podcast earlier, but in that podcast, I made the case, you see we talked about metal launching M.A.I like 2013 and then AEO was 2016 and then BEO is 2017. These were these sort of evolutions of that idea. I like to drive these actual outcomes that were more kind of closer to the commercial purpose of doing the ads, which is to make Rhoads. And I make the case in that podcast, I think there's maybe a little bit, oh, it's not there's certainly a lot of uncertainty around AI and the impact on employment, right? Labor force, like being restructured and displacement and stuff like that. But I make the point in that podcast that like, look, this is an extension. If you look at the the goal that was being achieved through all these this evolution of that product, it was automation, it was automating a lot of stuff. And you know, AI is just another means of doing that. Probably more effective, sort of scalable way of doing that. But like as those products were being launched, the 2016, 2017, 2018, EO, and these are principally launched for mobile apps. The mobile app economy was booming, right? And not only that, but employment in the mobile, I excluded this from the podcast here, the numbers were a little flimsy. I mean, I just couldn't find good data. But you overlay the employment numbers on top of this just the overall turnover in the mobile app economy. They were basically tightly correlated. And so this idea that more automation just means fewer people, I think is mistaken or certainly it's not justified with, you know, just the historical data. And so like, I think these things, the things that are being pursued, right? The objective that's being pursued, which is automating more stuff, taking more of this off of the plate of the marketer, that's not something historically that's led to fewer people needing to be employed. And in fact, it was the opposite, right? And it freed up a lot more. And for all the canonical reasons that people cite, it frees up more opportunity for the higher leverage, higher value pursuits that a human being is better at than a machine. I couldn't agree more. I've been with the company for 13 years and I've had a front row seat to the economic growth that these tools can generate, you know, incredibly small businesses that build themselves, leveraging our tools that became publicly traded companies. And you know, digital advertising, personalized advertising is a means for growth. And again, I think it's democratized. The industry, it's allowed very, very small businesses to do things, the historically only a few businesses could probably do as a result of many of these tools. And these tools are going to get even more sophisticated over time. And to your point, you know, ASE is a great example. It automates the targeting bidding and creative. We can run billions of permutations of those three things to land on a pocket of performance that no matter how big your team is, you couldn't replicate that, but those business outcomes drive economic growth and employment to your point. So I could not agree more. You ever run a holdout test, wait between two and five weeks for the results and then realize you just wasted a big chunk of budget? Yeah, by the time the data comes in, the campaign's over, the money's gone and the insights basically a post-mortem. That's why incremental the always on measurement solution exists. They give you always on incrementality measurement, no experiments, no holdouts, no delays. You get answers while the campaign is live, so you can actually make changes, not just analyze the wreckage. With incremental, you don't just learn what worked, you do something about it. Check them out at INCRMMTAL.com. So Meta just published an extensive white paper on gem, which is the ads recommendation system. Yeah, I wrote a thread about it on, on, well, everywhere. That white paper struck me as being more detailed than past white papers. Rossi and Drom and white paper was pretty brief. It kind of gave an overview, but it didn't go into it nearly as much detail as this gem white paper went into. I think this was published maybe two weeks ago or something. I mean, let me know if I'm reading too much into this, but it has met as attitude toward transparency with these types of systems change. Is there kind of like more of a willingness now to go into more detail and to provide more context around how they work? Because we did see a lot of speculation on social media about the changes that were made to Andromeda. And I felt like a lot of that speculation was very misinformed or not really rooted in any sort of understanding about how these systems work. So has there been kind of an attitude change within Meta? You know, I think the interesting thing, and you might actually smile at this. I mean, for the longest time, we debated internally as like, are people going to be interested in this stuff? Right? You know, if you think about how we would go to market from a consumer perspective, like generally, we would focus on the outcomes and what you would need to do to set it up. And we had a debate about whether we felt partners would be really interested in the nuts and bolts of how this worked. I think we tested the water a little bit with Andromeda. And I think everybody was taken aback by how much interest there was in this. So I think as a result, we're doing much, much more in this particular area. So these systems are getting more and more sophisticated. And we're seeing our back end innovations like Gem and Andromeda is a critical part of the performance story because when we talk about it, people seem to respond in the right way. So, yeah, I mean, I don't think there was anything more than would people be interested. And the overwhelming feedback we've heard is yes, we are. So give us more of that. I tell you though, the Andromeda thing was amusing, I thought, because I think Andromeda became the source of ills for all human kind. I saw a meme on X that really blamed Andromeda for all of life's problems. You know, while Andromeda is efficient at scaling ad selections from a growing volume of ad creative saw system needs to choose from, it is quite distinct from creative diversification, which I talked about earlier, which is directly impacted by creative ranking. So I can say with absolute confidence that no recent Andromeda updates are linked to any performance issues, if anybody still won't drink about that. All right, well, that's very good. That clears it up. That's good to know. Okay. I want to kind of move back to the creative piece. So if you think about the business case for AI generated creative, you have the opportunity, like from an advertiser, perspective, you have the opportunity to save money on production, which I think that's kind of overvalued. I think people overestimate how valuable that is. And then you have the opportunity to more exhaustively test new concepts and, you know, just explore the concept resonance space, which I feel that that piece is undervalued. But there's probably a progression from moving advertisers from recognizing the former, which is just obvious, right? It's obvious why that saves money to recognizing the latter in terms of like the timeline of that progression. Where do you think we are right now? I think we're a ways down the path. And again, these two things are intrinsically linked, right? I mean, I think most of the partners that I speak to, we speak to, they want to test new concepts because they understand the value of this. And again, one of the things that I talk about, a lot is this notion of a creative supply chain, which is going to be so critical as we think about where personalization is going over the next five or so years, the barrier to any kind of creative supply chain is the cost of production, which is holding people back. And again, I think this is one of the reasons why the creator ecosystem has thrived so much is because creators present a lower cost way of being able to produce a creative supply chain. So yeah, I think we'll well down the path. And I think, you know, our GNI creative tools really focused on empowering advertisers, especially small businesses, to be able to do exactly that, a diverse set of creative scale, which is needed in order to kind of follow our recommendations. So we have our video generation beta, which I think is a real game changer for small business because it allows you to transform multiple static images into dynamic multi-seam videos with text overlays music. So essentially, give me some static images and we'll turn it into a really compelling video. And I think that's a game changer. Background generation is another version of this, which automatically produces multiple backgrounds, which it's tailored for product images to allow the product to really jump out and allow advertisers and agencies to experiment with credit variations while saving time and effort. So yeah, I think we're well down that path. And I think the tools are essentially allowing that concept diversification to happen in a way that it's affordable, I think. I think that's underappreciated, right? So I wrote this piece of well back called like five value milestones from generative AI for ad creative production. And one of the milestones was video production distinct from just image production. And you can say, well, it tends to be higher engagement format. So I guess it makes sense to have that be its own distinct milestone. But the reality was, I think the reason it is a distinct milestone and the reason why there's more of a value unlocked with video is that if you think about static production, that probably just expands the volume of stuff that you're creating, but it is something you're already doing. So you've got this output and you increase the volume of the output with the static production. With video, it's actually unlocking that capability for a lot of advertisers. It's allowing them to do something that they were not previously doing, which again tends to be the higher engagement format. But nonetheless, it's something that they weren't doing before that now they can do. It was just awful limits. I mean, it can be expensive to get a video, especially if it's live videos, hundreds of thousands of dollars, even if it's just rendered. It can be very expensive. And the way I look at this is kind of like this dual access expansion, right? You've got people converting more, spending more existing advertisers because these tools allow that output. But then you've got the other access being advertising participation. So people that are participating in the advertising economy and coming online as advertisers only as a function of these tools. Because if you think about, well, okay, video production allows people to do a video for the first time, but those were existing advertisers. And then, you know, I think this capability given to you is allowing advertisers to advertise for the first time because there's a lot of them that had no capability to come up with an ad or even just de-optimization. Like this was just totally alien to them. I have no idea how to spin up an ad campaign on a platform, but now they can just kind of, you know, press a couple buttons. So I think like that ad economy participation piece, that expansion is not recognized as much as something that's unlocked with these tools. And again, you know, one of the things that we see time and time again is small to business is lean into these tools ahead of larger business. And again, here's a user behavior change that happened in social media a few years ago now, which is short form video. Right. So short form video is the preeminent content inside of our apps and services. Therefore, producing short form video ads that are native to that environment are obviously going to perform much better than a static image. And again, if small business can move in this direction by adopting these tools, they can benefit from that user behavior change very, very quickly. Right. The point I made at the performance marketing summit, which then got the gears rolling on the commerce at the limit idea was like, you think about some SMB that's just a service local customers. I mean, you're talking about a shop or something like that. Yeah. There's the chemical short form video. They have no knowledge of that. They wouldn't even know where to start. Right. Or even making a static ad or even understanding what a bid is or what a conversion is. Like this is just alien to like the way they run their business. But if you enable that with like a pretty sort of turnkey solution, that brings those people in as advertisers might have a transformative effect on their business. Absolutely. It's an enabler for growth. Mobile game developers no longer need to pay up to 30% in major app store fees. With XOla Web Shop, you can create a direct storefront, cut fees down to as low as 5% and keep players engaged with bundles, rewards, and analytics. And today at xsola.com, that's xsolla.com or use the link in the episode show notes. So what kind of feedback are you getting from advertisers about the AI business tools? I mean, you could expand that to all the AI services or all the AI enablement. Yeah. And then with that feedback, does the tone of the feedback differ by company size, by the category they operate in, by their geographic region? Are there distinct differences across those dimensions? Firstly, you know, the two tools we started off talking about business AI and meta AI business assistant, universally being used by small business today. So start with business AI first. I spoke to a client just last week who shared that they were impressed by how quickly the tool was able to adapt to their evolving business products and customer conversations with minimal money leffit from their team. So again, you know, part of the key to these tools is they will improve themselves very, very quickly over time. I think there's specific brand examples as well. There's a company called White Copemanilla in the Philippines that's in the medical apparel business. And I think they've seen a 5% increase in customer conversations company in New York called Julep, which is a beauty brand that were testing business AI on ads. So a 13% increase on row as well as a 10% lift in incremental conversions. So it's early and I can't stress this enough. This journey is really just beginning right now, but results I would say are really encouraging. On the business assistant, the meta AI business assistant front, small businesses and agencies are in our initial testing audience. And again, they're finding value in the ease of incorporating recommendations into their strategies. A client I spoke to told me it feels less like a tool and more like a strategic partner, which is a great thing to achieve so early with this idea that it's helping his business drive smarter business decisions that result in more efficient use of time and better allocation of budget. So like I said, it's early, but we're getting some really, really good feedback. In terms of the tools themselves, I think the thing that we're seeing between large business and small business is just small business as a desire to lean in here. Whether they've got more to gain, I do think an ongoing conversation with larger businesses is really a fear of what could go wrong versus the opportunity of what could go right. And I think that's an ongoing discussion here that we have with larger businesses. Like so the fear of stuff going wrong, is there a sense like by a very large advertiser that when they kind of quantify that fear or they scope that fear is the sense that we should not use these tools at all or is it that we should own these tools internally? Because I think it would be hard to justify not embracing the technology at all. I mean, that just feels like a kind of a lot like position. You know, you can't put the genie back in the bottle. This is going to be the future of human compute interface. But is it that we want to own it internally and we have to build it ourselves or we're going to avoid touching this for the time being? What's the kind of attitude? I think it's just slow to adopt. I agree. I think most businesses we work with large businesses are beginning to experiment with tools, begin to try things out, but maybe not moving as quickly as medium size and smaller businesses. Many of the large businesses are experimenting with their own agente experiences. I was in a room of partners just last week and I asked that question and every CMO in the room raised that hand and said, yes, we're experimenting, but it's all they're using them in production. Right? I'm beginning to test them in production. I think that's where it's been a little bit slower. Yeah. I mean, I kind of observed the same. You'll see companies are like, oh, you were absolutely embracing AI and relating to AI. And they just kind of look at it as like a component that swaps into their workflow and replace some existing component. And I think the reality is it actually redefines the workflow. If you're truly embracing it, it means you're kind of rebuilding the marketing workflow around it and not just swapping it in one specific discrete use case. Yeah. A lot of businesses have been operating in much the same way for a long time and they've built internal processes around it. Do you think about creative in terms of legal sign-off and such things like that? Many of these larger businesses are going to have to reimagine their workflows and what that looks like inside of a larger corporation. And so it's not so much about adopting their technology or being prepared to do that. It's really about changing how you operate as a large enterprise. And that can be quite hard for those large businesses. Maybe that's just a function of being a large business. Large businesses by definition move slower, more ossified in some ways. Okay. I want to take a big step back and kind of get a bigger picture update from you. I mean, you obviously have a front row seat to like the entirety of the digital advertising ecosystem. So how is the performance marketing landscape evolved since COVID? I think we're solidly existing with COVID in the rear view from a society standpoint. What are advertisers focusing on that they weren't a few years ago? How has the mentality changed? How are people approaching things in ways that are different from they were like 2021 era? Yeah. I mean, Fussy isn't astonishing to think that COVID was five years ago. I mean, it's just so many things have changed over that particular period. But right out the gate, I think two industries were really transformed. That would be retail and entertainment and how it accelerated the adoption of performance marketing in those two categories. And retail, I think it's pretty obvious right suddenly overnight retail brick and mortar stores were shut and it forced the entire industry to pivot and become e-commerce businesses overnight. And so, the result, this industry has undergone a massive transformation in terms of how sophisticated they are when it comes to performance marketing and how sophisticated they are when it comes to operating as e-commerce business. You know, it's interesting. If it wasn't for the pandemic, I don't think that we would have created our only channel ads product. which is a solution that allows those retail businesses to optimize their campaigns for both online and offline purchases within a single campaign would never have happened if it hadn't been for the pandemic. Likewise, I think entertainment similarly was affected when people were stranded in their homes and were at the mercy of their streaming services. The entertainment industry historically focused on big, grand moments for launching a movie or a TV show. That industry has really pivoted hard to become much more full funnel in their approach and much more sophisticated as performance marketers. So I think those two really stand out for me. I think on the consumer behavior front, I think creators, I think one could argue that COVID truly lit a match under the credit ecosystem. I think as people were confined to their homes, both create a content consumption and create a content generation surged. And that's fostered a new class of digital entrepreneurs who are absolutely here to stay. So in this area we're seeing exponential growth in credit engagement and something like two trillion minutes are spent with credit content locally across our platforms. So yeah, I think those are a couple. I think the final one that I would mention is just how the industry is now able to handle uncertainty. So I think marketers now have built uncertainty into their playbooks. So uncertainty now is the only thing that is certain. I would say as has been witnessed by some of the things that we've seen in the market over the last 12 months. I also think advertisers have moved away from this notion of siloed budgets to being much more flexible in performance-driven allocations. So budgets I think now are optimized across channels daily or weekly, oftentimes with a focus on profitability and efficiency rather than just growth. So I think those are some things that I would certainly say stand out. The media entertainment points are a good one. That's really interesting because I imagine also a lot of those companies are monetizing multiple service areas. They're driving people to multiple destinations, especially the film industry. Obviously streaming was growing a lot pre-COVID but COVID was a massive accelerant. And in fact, it cut off the box office opportunity. And just thinking about a lot of the marketing, and I think only a lot of the marketing I've seen, it seems more grandiose now almost for streaming exclusive content than what you would have seen for a major box office blockbuster pre-COVID. Again, it's so fascinating to see the evolution here because that industry has become really strong at customer acquisition. And again, this is a great example of where value could be really helpful there because finding a consumer that won't churn is going to be really important for that industry. But where I think the big unlock for that industry is going to be when they pivot to LTV, when they get to LTV. Because again, you think about the massive entertainment businesses. They have a TV arm and a movie arm and a theme park arm and a streaming arm. Really, the unlock there is understanding the relationship, the LTV relationship between each one of those businesses. Understanding, you know, it might not be as profitable to onboard a streaming service, but the LTV impact to the other business units could be that great. They should be investing more there. I do think that all of those businesses have a massive unlock if they can just connect the pieces and really begin to look at it all through an LTV lens. Well, yeah, and then especially the subscription businesses, right? There's obviously just an LTV component there just made up of the recurring nature of the subscriptions, but also like the beats, right? You know, you've got a big movie coming out. I'm blanking on the name, but it was the Robert De Niro mini series on Netflix about this cyber attack, whatever. Like, there was a lot of marketing that went into that, right? I think a lot of that was designed to reactivate dormant subscriptions, but I think having that that kind of LTV view is probably the next leg of growth for a lot of those streaming services. Simon, this was fantastic. I appreciate you sharing your perspective, your wisdom with us today. This was really great. I appreciate your time. Nope. And you know, I have been a huge fan for a long time. So it's amazing to be invited on here and to have a conversation with you. Well, it's fantastic to have you. I really appreciate you coming on today. Thank you very much. You're welcome. Thank you.