Solving Measurement

[MUSIC] Welcome to the Periam Podcast, Corey here, joined by Joe Z, though there are multiple Joe Z's in our life, which is the most hilarious thing in the world. They are spreading. I can see it. We got Zappa, we got Zido, there's a-- Yeah. You know, at least you have Swatsky-Wish-No one can spell as we've talked about before. So, Joe, I just questioned, his measurement has been solved for everybody at this point in time. Like, we got AI, measurements done, it's all solved. Oh, I can't wait to introduce our guest here because he is a fellow journey person on this quest to make measurements at the core of our industry and solve it finally once and for all. It's a hundred and five. Well, to have your one measurement to solve them all. Yeah. So, today we've got Andrew Cavado, founder of Growth by Science. Andrew, thanks for joining. Thanks for having me, guys. I'm excited to talk about measuring. I don't know if we're going to solve it once and for all, but we'll solve it. We'll make some steps towards solving it at least. But there's a good business trying to do it. There really is. Sorry, Andrew, you've been with Growth by Science a long time. How long have you run the company? So, the company's kind of morphed over the last few years. It's been around for about three years. So, three years. But you've been in the measurement world and the data world and the ad tech world for much longer. I have been in the data ad marketing analytics world basically since I started working in tech so over 15 years ago. So, why do you start growth by science? What problem were you trying to solve? Well, being in the measurement industry for so long and I've had the privilege of working for a lot of great companies. I started my career at Google actually eBay before that. Then Google I worked at Meta for a number of years Netflix and then let's measurement snap for almost five years. So, when you've got that kind of perspective, I was able to work with a ton of different advertisers, all shapes and sizes, really understanding how are the best and the brightest and maybe folks who are still maturing in their journey. How are they all thinking about evaluating the marketing investment? The common theme is I don't think anyone's really cracked it in that 10 and a half, 15 years that I've been involved in. I've only seen a few examples of organizations that have really made some serious progress. In the reality of it is it's hard, I would argue, virtually impossible to do perfectly and requires a really deep understanding of the limitations of what you can get out of the data that's reasonably accessible in marketing and also the math that can be applied to that data. So, really growth by science is an effort of consolidating all of that experience that I've had and others on the team have had and presenting what we think is the best approach and the most generalized approach to addressing measurement challenges that exist out there. Can we spend a second on your view? Obviously, the sweep of companies that you've been doing this with and for is August. But where are we today? Let's get right down to some of the three-letter acronyms and the portfolio of MMM, MTA, incrementality, how to create the scaffolding, etc. Then we'll get to activation and using it in practice and sure by the time we're done. But like, best practice today, what do you tell people? You're starting with something and where do you recommend people start and stop? Yeah, so I think let's break down that into a couple of pieces. The state of the state where we are today, I would say the expression, the more things change, the more they stay the same, I think really applies here, which is the problems that I saw when I first started my career are still the problems that are being faced by advertisers, which is the methodologies are not sufficient. The ones that are being used most commonly in industry. And I mean, this is hilarious to say after. And I'm sure even before I got into it, these methodologies were still being used. So people are still chasing, I feel in a lot of cases, this idea of a post-exposure attribution. So let me try to chart the path of a user journey with very granular touch points. And then let me use that, collect a huge data set around that, and then try to understand the impact that ads are having based on that data set. That's attribution, MTA, you're using only that, I would say, you're leaving, you're wasting a lot of money. I mean, it's alone in the short of it. In the last few years, we have definitely seen a shift to more advanced methods. And you can tell there's a ton of companies out there that are touting MMM, marketing mix modeling, right? Plus incrementality testing, geotesting, synthetic controls, things like that. And so those ideas and concepts have really gained a ton of more adoption and popularity over the last few years. I would still say there's room for improvement. There's still CMOs that I speak to that manage nine figure budgets that are reticent to lean into incrementality and have doubts about it. And some of those doubts are real and some of them are unfounded. But point is it's still a mixed bag. And I actually think this explosion of AI, which I'm sure we'll talk about is making things a lot worse because folks are thinking like, oh, that's fine. I'll just throw AI at it and it'll fix it. I mean, that's it's kind of like, you know, step one. It's like a board meeting. Hey, did you put AI on that data yet? You just put AI on that data, right? All good. Yeah, throw it in the eye. So yeah, it's not going to work that way, right? There's a lot of nuances and subtleties and a lot more fundamental things that need to get fixed. So to go to the second part of your question, Joe, what do we recommend? I mean, we recommend taking a big step back as a first, you know, first foray into measurement, especially if a company is, you know, had fits and starts with their journey on evolving their measurement stack and really, you know, try to ask yourself, I think every time I talk about this, I say the same thing. But, you know, we asked CMOs, you didn't spend money on ads. What would happen to your business? Right. It's quantitatively what would happen to it. And oftentimes folks don't know how to answer that question. They don't know, you know, what would happen to their business? They have some hypotheses. A lot of it's not founded. And so when we approach a client, that's really the core of what we try to solve. And we do that a lot with, you know, multi-channel holdbacks, which are a hard thing to sell to a marketing team, right? There's always a fear of opportunity cost and, you know, stunted growth. But, you know, there's it's a double-edged sword because if you're spending, you know, $900 million on marketing, but only a small fraction of it's working, by turning it off, you've actually done yourself a favor. And I would say, you know, you like a little little inside info here. Most of the time, the incrementality is far lower than folks think. And so, you know, it's generally a good idea to do this kind of test and maybe even do it regularly, I would say like a couple times a year. So that's the first place we start. But ultimately, you know, there is no special secret approach. There's no crazy cool modeling methodology that supersedes anything else. It's a few fundamentals that you need to solve. It's get your data in a good place, right? Make sure that it's organized in a way that's conducive to doing the right type of modeling. Have folks that really understand how to do incrementality. I'm not saying top data scientists who have PhDs in the stuff, which by the way, we have and so do a lot of other companies. But as people who have understand the business pragmatism is required in how to set up the tests, how to structure a measurement program, an incrementality testing program to answer business questions. Layer on a really good one of one model for your business, right? You know, there's a lot of great out of the box models, either SaaS or open source that folks can use. If you're not spending the time, especially for a more complex business, to tailor that model to really reflect the nuances of a particular business, you know, the model is not going to work that well. And so, coupling both of those two things together and ideally having a framework that wraps all of that around, that lets you do this kind of thing at scale and iterate on the model, you know, try to things and just kind of manage the whole process. So that's really the approach that we take growth by science with our advertisers. And we found a lot of success with that. I think people are really, really understanding the value of this idea of one-of-one modeling of the deep expertise in the in the measurement program setup. If your customers got measurement right, would they spend more or less on media? It's a great question. I think initially they would probably spend less because they'd realize that there's a lot of waste. And so, yeah. That's what you would go, yeah. Yeah, the journey is typically, hey, let's trim the fat, you know, find out what's inefficient, cut it off, get to a place where we're not sacrificing, you know, the minimum growth requirements that we have, but we're doing so efficiently. And we've eliminated the obvious waste. And now let's layer things back in intelligently. And at the end of the day, once you've got past that initial hump, you do end up spending more because you're confident in what you're spending, where you're spending, and all of that. And so, that can be a tough bump to guide certain advertisers through, but it is long-term better. I remember the history of this in a lot of cases where people that had religion, around being rigorous in sort of measurement would sometimes go down these exhaustive, huge data lake. We're going to make the model perfect. We're going to integrate all of our internal sales data to all of the activation touch points, whether it's going to open or walled gardens, et cetera. The business realities of getting there had people exhausted. Like the 12 to 24 months later, you're still working on getting all the right data in place and not quite getting there. You've got an interesting business, I think, in that you've got a mix of DTC businesses, where if you get to them early enough, you can almost build best practice into their initial go-to market. And there's nothing to retrofit in a lot of ways. That's got to create some good case studies. And I'd love to probe you on that a little bit. But do you often recommend the same thing with a larger organization where it's like, they're so saturated. You take a id and it's like, well, I've been running ads on TV, I've got billboards, I've got audio, I've got direct mail, I've got a whole bunch of things that have been the in-store shopper promotion, shopper marketing, et cetera. And it becomes really hard to sort through all of that to make cogent decisions, whereas inside of a PNG, if you had a new product launch or you acquired a small company that needed the scale of a PNG, you could almost build an internal case study of what it looked like to do right before you retrofit that best practice to some of the more established brands. How do you take advantage of that? There's a cleaner data set here, there's a way to build measurement into the business from the get-to. Talk a little bit about that if you would. Sure. So for context, we do have advertisers that span the spectrum. We do work with folks that have been spending ads for a decade plus oftentimes and their spend can be in the billions. And I would say that to be honest, the approach does not differ drastically. There are obviously a lot more complexities to an organization, like some of the ones that we work with that are, let's say, regionally they have even legal differences and regulatory differences on a regional basis. So you've got to have a model that captures that that are spending across multiple verticals and have been a household name for a long time. The model is going to reflect some of those challenges and needs to be adapted to that. But at the core, the framework is fairly identical. Like all we did in the instance of working with a brand like that versus another brand that we worked with that, spending an order of magnitude less, just under $100 million on their ads. And they're a pure e-commerce company, DTC e-commerce company. On both cases, we started by understanding the business goals, understanding how are they looking at growth right now? What are they doing to measure it? And obviously the answer for advertiser A is, we're more complicated than for advertiser B. But we still start over there. And then we help the advertiser understand what should they be looking at. And really, from our perspective, what we see often as advertisers are going to track tons of different things. What's my brand recall? How many clicks did I get from this campaign? What was my reach over here? All these metrics that on the surface, you give you the illusion of being worthwhile of chasing. But are actually orders of value to remove from what you need to be caring about, which is your top-line growth and your bottom-line profit rate. You need to look at real, true business KPIs. And all of those other things could be leading indicators of that. But you need to do the math to prove that that's the case before you start tracking them. And so most of the time, we will help call those OKRs from 20 or 30, down to two or three, maybe, all instances. And if it's a more complicated business, you replicate those OKRs across, say, different business lines and whatnot. But the point is, you've got a few north-stars that the growth teams rally around. And then everything else that's like, what do I do with my brand metrics? Well, let's first prove that your brand metrics drive certain true revenue or growth metrics. And then we can build a model that helps you understand the impact of your brand spend on those brand metrics and, transitively, how they impact the true business growth metrics. And so that's the kind of process that needs to happen. And so that's step one is getting all of that sorted out. Step two is designing a series of tests that will help answer the most pertinent questions. Typically, they are what is the overall incrementality of your marketing spend that we believe to be the domain question. And so we try to use that as the first entry point. And without question, like this, usually we solve that within the first month or two working with the client. And there's already like massive insights that are gained from that usually, usually a ton of other questions like, why is it so good or why is it so bad that we need to, you know, distill further. And that's where really having an understanding of how to apply a measurement process to a variety of different advertisers comes into play. And, you know, again, I feel like that's sort of our secret sauce and the expertise that we've garnered over the years. So we help we help follow along with that. And then it's really designing an appropriate model. So you're not having an incrementality test, constantly constantly constantly. I mean, you should be doing a lot of testing regardless. But, you know, you need to have, you need to be able to make more strategic decisions, do scenario planning to forecasting. And that's where like an MMM comes into play. And again, we'll build a one-of-what model for each of our clients to be able to do that. And really, again, comparing those two advertisers, the difference is really in the complexity of the model. If you're, you know, that one D to C advertiser, maybe they're spending on five or six channels all digital, very easy to, you know, at least get the data for that model. And, you know, still challenges in making it, making it right and making it aligned to the business. But the challenges are vastly amplified for that sort of bigger advertiser that has a lot more nuance to their business. And so we build the model to reflect that for each, for each of them. What's something you walk in and everyone thinks they're measuring well and you're like, yeah, yeah, you're not doing this right? You know, I think brand is probably something that that comes to mind. And it's almost, it's not like people are even measuring it well. It's that they don't understand how to incorporate it into a robust measurement approach. Performance really easy brand. Yeah, but like our perspective is it's all performance, right? It's just the degree to which and the timelines upon which it works. And when you're spending on building your brand, if you really increased your brand awareness, but it did nothing for your business, I would say that's a failure, right? And unfortunately, a lot of companies out there are still saying, oh, I got my brand recall up by two percentage points and this and then the other, okay, that's great. Now show me what that did to your sales. And they're missing that like crucial step. The best brand advertiser in the world, by the way, you're mentioning PNG, they have that math, they've done the math to figure that out. And so like, they've got a dialed, right? Like they're able to understand what like the change in in some of the brand metrics, how is that going to ultimately result in more sales? And so I think when a lot of aspiring brand advertisers look at some of those folks and they see these great case studies about, you know, we increased brand recall by blah, blah, blah, blah, they think, oh, that's what we should be doing, but they're not seeing is like, like, great, that connective tissue to what does that actually do to the business? This is the PhDs that are on there and doing like heavy duty intense causal analytics to understand the whole thing. Well, that's really complicated, though, man. Come on, you know? It's really hard. Don't worry about it. I appreciate it, like, you know, correct by science, sort of in the name, there's a systematic way to do some of this stuff. And the, you know, as sort of scary as it sounds to take a step back before you can sort of take a step forward. Like I appreciated the, you know, as opposed to this explosion of KPIs where people are, you know, different teams are pointing at different directions. And, you know, sort of B2B has its own set of wonderful chaos between like, you know, marketing qualified leads versus sales qualified leads and, you know, a multi touch complex. It's, it's nuts to really like spend a lot of time to think about, but you basically take a step back and you say, okay, what are the, you know, the KPIs and the okay ours are sort of the growth metrics that then connect from the CMO to the CFO, AKA what actually drives the business is sort of step number one. Just get everybody on the same page with what numbers are we going to look at and what, when those numbers move up or down, how does that change the core business? So that feels like belt and suspenders, that's wallet keys and, you know, phone before you leave the house. I love, I love that. And that feels like a big unlock in and of itself, because, you know, you, you can pull the average, you know, C suite and be like, how aligned is the marketing, you know, efforts to the actual business and you usually get people complaining. Okay, so fix that. How much, if you had another kind of want to, to wave, there's got to be something in the ability of the marketing department to syndicate their goals or to have the platforms optimized to those KPIs in ways that actually drive the needle of the business as opposed to, for example, simply increase margin or yield for the partner, right? Like, how do you sort of create that translation between, I've got a really a scaffolding or I've got a, you know, KPI framework that I truly rigorously believe in, but I can't necessarily share that directly with all of my partners because sometimes it gets works for me, sometimes it works against me. How do I sort of take that and allow the levers that I'm allowed to pull with the partners to be pulled in such a way that I actually drive my KPIs? Like, that's got to be another area of friction, right? Yeah, for sure. That's a really, really great and nuanced question, actually, Joe. So, let me try to unpack it a little bit. You know, I always say this, add platforms and I've worked at, you know, a number of them. Yeah, all of them. This is not, it's not a dig on them whatsoever, but like they're a business, they're in the, in the business of optimizing their own revenue, not advertisers outcomes. And, you know, I think I've said this all to the shock. Shocking, right? It's mind-blowing, right? And obviously you can't, they're not like fully orthogonal, you know, you have to drive out of some satisfied enough. Yes, but it's always just enough. And so, advertisers need to, you know, take the reins and need to take ownership over how do I use the add platforms to my optimal advantage? And that could mean not using them at all in some cases, right? So, there are some efforts being made out there by some add platforms to kind of make that connect a little bit more and let the advertisers a bit more control over optimization levers and whatnot. There's also the opposite happening where, you know, you've got all these AI campaigns that are kind of out there, black boxing, you know, frankly, in my opinion, in most cases, they're optimizing revenue really, really well for the advertising platform and they're not always performing optimally for the advertiser. But, you know, I think like rather than, rather than having this mental paradigm of, I know that this lever is going to, I've been trying to find a causal relationship between a certain lever on an advertiser platform and the outcome it's going to have, break that and treat the levers as kind of random, I don't know if it's going to do good or not for me, but I'm going to test it scientifically. I'm going to see if the combination of, you know, this bid and this budget and this targeting, if that's going to be good for me, if this is this other bid type and this other, you know, like I'm going to basically treat those as, I don't know if they're going to be good or not for me, right? And you test it scientifically and you can design a testing program to do that, right? And models to kind of treat that as well. But when you do that, you kind of like, it's almost like freeing a little bit because, you know, we've had some advertisers where, you know, they've turned off all of the optimization, they just try to bid and reach in frequency. And sometimes that works really well. Sometimes it works terribly for advertisers, right? It's not, it's also not like this, there's not a recipe. So I often get asked, you know, hey, what's the best like way to deploy medazz and likes to well? So one of one, right? Just like our models are one of one, you know, one of one for each business. Yeah, there's some principles that, that, you know, they carry over from business to business, but you have to try it for yourself. You have to test it for yourself. It's very product dependent. It's very creative dependent. It's very audience, you know, all of those things, it's dependent on all of that. And so, so really like, forget about trying to understand the mechanics of what drives incremental outcomes for you with respect to the levers in the platform and think more about how do I test it scientifically and let me be agnostic as to what the outcome is like or what the lever settings are. So we can't go without AI discussion here. So what are customers asking you to do with AI today, right? So you're there. And I mean, they're all, you know, everyone assumes AI is going to solve everything. The hype cycle is, you know, we're way in the, you know, the high point of expectation. What's the expectation you hear? But you know, AI is going to do what for them. I'll say like there's some interesting possible applications of AI, I think on the activation side, I think on the automation of certain marketing operations, possibly data wrangling, reporting, you know, all of those, we've seen tangible, you know, good use cases of AI. I think where it's not quite there yet is, we're going to just have the AI build the model for us and it's going to be perfect. There's a lot of nuances. The data is still too fuzzy for, you know, an AI to really understand. And we need to have, you know, a vast amount of business context to be able to do that. We obviously use AI very, very regularly. It helps us do things that would have taken us, you know, a week or a few days to do it down until like hours or minutes. And so we're able to deliver more value with fewer people to more clients. But ultimately, you know, there is no good AI replacement for an understanding of how to deploy a measurement program, but how to make the CFO talk to the CMO appropriately and get everyone aligned about, you know, designing a measurement program that is in sync with business metrics like all of that kind of stuff, which is foundational. That's the hard part and that's where, you know, I would say in most cases, is a prerequisite to do anything more sophisticated. But, you know, we've got, we've got some interesting thoughts and obviously we're building some things on our side as well. We're finding ways to to scale and to offer at scale the ability for advertisers to have an in-house measurement system, right, that it will be spoke in one of one. So that's sort of what we are focusing on building. And obviously there's some heavy AI componentry to that. But it's, you know, it's not, we're not going to be shipping at this time, right, AI-based model, right. And I think anybody that's promising that it's over-promising that we certainly do have AI-assisted model builds, right. But the model's helping us to pour code that we're managing in a more efficient way, just the same way that you can build apps quicker now, you can build models quicker the same way. But it's not, you know, you're taking your hands off the wheel. We're not at full self-driving marketing. We're not there and I don't think we'll be there for still a few years, right. It's coming, but it's not, you know, we're not there yet. It betrays my interest in this topic because I have a disproportionate number of my conversations somehow end up around, you know, measurements and the end, and sort of connecting outputs to incomes and, and, or, or, or, we actually, that's a, a Freudian slip. Outputs to like it. Yeah. Yeah. Outputs to inputs, but also to, to income, exactly right. Trade mark by Joe. Yeah. But I was not with Rex Briggs yesterday and he's another one of the kind of storied, you know, sort of practitioners in this space, you know, wrote a book called What Sticks. That was pretty informative and sort of my, you know, early days and then marketing evolution and kind of, you know, I've a lot of the one point I'll work and whatnot. We were talking about AI and exactly this topic of like the implications in the in the space today. And similar insight, the Korean I've had, which is like, on one hand, it's really good for like opening people's brains and organizations to like, hey, things are going to be different. And therefore we need to like expose ourselves to how different. But then in a lot of ways, like it's the the meat and potato stuff in the middle doesn't change. But the work to support that meat and potato stuff in the middle gets more efficient. I think that's what I, you know, heard from you here makes makes an absolute ton of sense. You know, two podcasts back, three podcasts back. We're like, hey, at what point do you have a marketing organization sort of so committed to change that they say, listen, we're going to rip out the, you know, want to make or I'm going to take a bunch of my marketing inputs. I'm going to try to sort and filter through all that stuff and make sense of it. And I'm going to be so committed to sort of an outcomes oriented approach to marketing that only the things that can be well instrumented. And I feel confident that the levers that as random as they might be, you know, produce the results in the model that the CFO agrees it warrant reinvestment. Are we close enough now that you could kind of like start over and just start with a model and then get to media as opposed to start with media and then try to jam it into a model. Interesting. Yeah. No, I think I think yes, you can build that organization. I've always said that that if if I were to build a marketing org from the ground up, that's how I would do it. I would build it based on the principles of science incrementality testing, you know, all of that. Like the all of the raw material is there. I think for an organization that's just getting started, you can start with the principles of science. Where it gets really hard is to change that. You've been using last click for, you know, 15 years, you know, your business. Maybe it worked in the beginning because it was brand new business. And now it's all it's all a mess and you've got two agencies that you're working with and, you know, multinational organization with regional CMOs. Like that's when it becomes really, really messy. And that's where no AI is going to at this point be able to help disentangle all that. There's still like like people related relationship related, you know, mentality related work streams that need to get accomplished. And so yes, if you're building from the ground up of the modern marketing organization, I think can operate a lot leaner and can operate with the principles of science and use AI to help facilitate the execution of those scientific principles. Maybe facilitate, you know, the delivery of the ads and the activation of the ads based on the outcomes of that. So you could build a modern marketing organization from scratch to do that. I think the reality of it is for really established organizations, that is going to be a long, long, long journey. I mean, you just think about some of the other things, you know, accounting systems or, you know, my hermitaties to work in EDI, electronic data interchange. And you know, you just retired a few years ago. And he told me that that a lot of the organizations he's worked with, like some of those systems like 25 years old, because that's so embedded, you know what I mean? Like, to 30, like it's crazy, right? And like I think the reality, we just have to temper the business realities of like upending decades-long process overnight. Things will change, it will be adopted. But even if you don't go down that road and you don't worry about like those automation things, the principles of science can be applied by any organization. It's just going to be a lot harder and maybe more manual to push it through in a bigger org than it is in a leaner, more nimble, you know, brand new org. But I think going back to that, right? What are the principles of science? I mean, have you, Joe, you earlier, right? What are they? Is create a hypothesis, design a test to prove or disprove the hypothesis, right? Conduct the analysis and then action on those results. And iterate, really. And there's also like some ancillary principles have it peer reviewed, right? Like to solicit the scent. These are all other kind of cultural elements that they need to kind of incorporate. Have a culture of experimentation in your org where, okay, I tried something in a failed that doesn't mean I'm going to get fired, that doesn't mean I'm not intelligent or not good at my job. It means that I had a hypothesis, I tested it out. And you know, let's be, you know, unemotional about it. We're at the point of a pod that I try to squeeze in one more question before Cory tells me that that I can't. So I'm going to do it. So for this, I think a little bit about like Vista, the equity, right? Like their whole PE philosophy to some extent is to take like, pretty good SaaS businesses and then deploy their, you know, playbooks to then, you know, hey, here's the blue sheets go execute a better sort of GTM around SaaS products. And they create a lot of value for the, you know, for the odors, they create a lot of value for themselves and doing that, right? Some PE goes to you and says, listen, go Andrew, brought by science, I'm going to give you a more chest. Here's a billion dollars. I want you to find brands that you think great products, great product development, great distribution in terms of like getting physical product on shelves, for example, or e-commerce. But their marketing deserves a material revamp and centered around science. Let's say you had a couple billion dollars to go buy some brands. And I think I could like really supercharge this, this business by taking control of their marketing department. Yeah, I mean, I would hesitate to call any particular brands and that would be a disparaging of marketing. I'm giving some characteristics. You know, I would probably look at brands that that have been around for at least a few years that have an established marketing program that by all accounts their, their P&L looks good, but maybe is not like stellar. And I would look at at ones where the ratio of marketing spend to revenue is maybe a little bit higher than the peers in their group. And then I would want to do a little due diligence and check how are they measuring it? If anybody says last click, those guys rise to the top. I'm going to pick those up because I can almost cut their marketing budget out and change their business materially. So I would almost certainly do that. In the next ones, I would pick or anybody doing MTA only. So there'd be a next on the list. Maybe even some cases higher than than last click. And also ones that have a very complex marketing org with like weird sort of matrix C type organizations where working with multiple agencies. But I think those I could almost certainly like turn off marketing and you know, rebuild it with like four people. And it would probably be just as good if not better than the crazy, the crazy team. So sounds like a private equity diligence checklist here. Yeah. If I want to reach out to me, I'm AC a growth by science. Let's make some deals. Let's go. Last click. Target on the back. Let's go. And you've also supported a point. Joan, I keep making, which is, yes, AI is going to be dramatic change. But changes are usually built on the back of what we did before. You know, I'm rip everything out and start over. And just just just a you made a really interesting point about your dad, you know, retiring EDI. My uncle retired recently as an IMS administrator. Do even either of even know what IMS is? I do not. No integrated something. The original sequential database from IBM that was released in the late 70s. He was still supporting it for a very large telecom company. And they were, you know, and hence a lot of data still live in there. I think Ed Texan is going to see the same. Yeah. Pieces of what we see today is going to be here for a long time, even if AI's put on top of it. So Andrew, are you a secret Canadian? I'm not a secret Canadian. I'm a very out and about Canadian, but I have a dual citizenship fair enough snowbird, right? You're a snorred at as a snowbird and ended marrying an American wife and having two beautiful American kids. There you go. I'm about there. I'm out there. You know, collecting the silence. It was not on my list. I'm on my list of questions. We're going to come up this pod. So, uh, Andrew, thank you so much for joining. Look forward to talking to you soon. Excited. Thank you guys. Great conversation. Appreciate it.