Scaling Meta Ads from zero to $100K a month — Ethan Ethier, Built With Science

Welcome to the sub-club podcast, a show dedicated to the best practices for building and growing app businesses. We sit down with the entrepreneurs, investors and builders behind the most successful apps in the world to learn from their successes and failures. Sub-club is brought to you by RevenueCat, thousands of the world's best apps trust RevenueCat to power in app purchases, manage customers and grow revenue across iOS, Android and the web. You can learn more at RevenueCat.com. Let's get into the show. Hello, I'm your host David Barnard. My guest today is Ethan Etier, head of growth and operations at Built With Science. The science-based fitness company he runs with his brother, YouTuber Jeremy Etier. On the podcast, I talked with Ethan about the myth that influencer apps market themselves, scaling meta from zero to $100,000 a month. And why they waited a full year before running a single ad. Hey, Ethan, thanks so much for joining me on the podcast today. Hey, David, great to be here. Big time fan of the podcast. Thanks, man. So I'm excited to have you on. Big fan of the YouTube channel, big fan of the app and everything y'all been doing at Built With Science. So I'm personally super curious about the growth of the YouTube channel, because you know, we are, you know, working on YouTube stuff here at subclub. But I think our audience will be way more interested in the growth of the app. So as a quick summary, Ethan's brother, Jeremy, started a fitness YouTube channel, Ethan joined the next year to kind of build the business. And now they've got over seven million subscribers on YouTube. But I wanted to kind of skip ahead from the early days to when you decided to actually build an app, why you decided to build an app, you know, how big the audience was at that time and kind of, you know, get more into the nuts and bolts of what our audience is going to be most interested in. We started the channel back in 2018. And throughout that process, we eventually decided to launch these web based courses, which about 100,000 people joined over the course of the years. And this was just a program that they would take on the web. And then they would be spreadsheets that they'd use to track their workouts and nutrition. And the reason we decided to eventually pivot that course into an app was because one, those requested by our users, a ton, they were tired of using spreadsheets on their phone, just to track their workouts and then a separate tool to measure their nutrition. So that was the first reason. And then secondly, and more importantly, we knew from RN that in order for us to create a more personalized product, we needed to pivot in that direction. So ultimately, it came down to those two things. I mean, it's so cool when when people beg you for our products, I mean, it's a nice position to be in. I mean, so many folks are building apps, just, you know, they're trying to create the audience for the app. It's like you already had the audience and they were begging for an app. So that's such a great place to be. How big was the audience at that point? Like how big was a channel? And you said 100,000 people had used a web product. That was at the time you decided to build the app or that was like lifetime. I'm just trying to kind of get a sense for like where were the, you know, the business and the YouTube channel was at that point where you decided to pivot to an app. Yeah, so at that point, I think there was around a million or so subscribers and it slowly grew over time. I think we're growing around a million or so subscribers per year. But at that point, we had enough members and the demand was strong enough that they wouldn't stop bothering us when it came to building an app. So yeah, we finally decided to make that as a key goal for us. That's really cool. So part of why I wanted to have you on is that I think there is a little bit of a kind of myth that, you know, if you're a big influencer, you've got, you know, three million subscribers, now seven million subscribers. I'll growth is so easy. You just, you just pitch, pitch the app at the end of every YouTube video or work it into the, to the video in the middle somewhere. And you're just going to like get millions of downloads and make hundreds of millions of dollars and everything's gravy. But that's not in the case. Y'all are doing meta ads, you know, you are growing the app in a more traditional way than I think a lot of people would expect for a influencer built app. So I'd love for you to kind of talk through some of that with me. Like, you know, even when you launch it with three million subscribers, was it like just a massive spike? That's a great point. And when we did launch the app, we had hyped it up. Obviously to our members and also to just our email lists and publicly. So we did see a spike when we officially launched it. That was about two years ago. But when it comes to actually marketing the app, like you mentioned, we've been slowly adding on new channels, such as meta, such as SEO, but for us, the core piece of content has really just been on education and building trust. It's been less of something that we try and pitch all the time. It's more of, okay, we're creating this piece of content. How can we position the app so it complements the like piece of content that we're putting out? So it's not necessarily we're thinking about, okay, we have this app. We want to create a piece of content that pitches the app. So it's kind of a different framing. And for us, that trust has been a key mo for us. And I think with the age of AI, when you build pretty much anything, that's going to be something we're definitely doubling down on. When you do pitch the app in a video and the video does really well, because you know, I mean, YouTube, it's like you can get even with seven million subscribers. I haven't looked at Jeremy's channel recently, though, I have seen a lot of Jeremy's videos. I was telling you, before we started recording, you know, I saw Jeremy's videos way before the app, way before you and I met. I saw videos years and years ago. But I would imagine there's, there's like, when a video does go viral and like, it's tens of millions of views versus like, you know, maybe a million views or whatever. Do you see that in the numbers? Do you see a big spike in revenue? Is big spike in downloads or maybe just like, is it more people just kind of checking things out? Where you see a big spike in download, but your conversion actually dropped kind of curious what the mechanics of that are for a big YouTube channel? Yeah, that's a great question. And it's definitely what you mentioned. There is a big spike when we do release a video in terms of we'll see a bunch of new trials come in. But what we've noticed is since we track pretty much every click coming from YouTube, both on the video and also in our description link, you can actually tell what types of videos lead to a higher trial conversion. So, for example, if it's more of a educational broad type of content where maybe we're sharing a specific diet or a workout program, a program that reaches a higher audience, larger audience, then that might not necessarily get the same amount of trial activations as to a video that where maybe we're telling a story about someone who started on the app. This is how they transform their body. So, when we create content nowadays, we're really also thinking about how can we tell a story that seamlessly ties into the product pitch again? How does that work as far as evergreen? I mean, you know, I mean, for me again, because I'm kind of a health and fitness nerd and have watched his videos before. And I think I'm now subscribed even his videos are popping up in my feed pretty regularly. Do some of those older videos that perform really well? Do they does it kind of provide a solid base of subscription revenue and installs? Where is it pretty spiky with like the release of a video? Really, you see it big time and then it just resets to like a much smaller baseline? Yeah, so for us, it's been pretty baseline. Like, we'll see the spikes when the video comes out, but throughout that, we've built a pretty solid bank of videos where we have a consistent number of trials coming in every single month. And I think with long form content, that's more or less the way things go where people are searching things up using YouTube as their own SEO and eventually these older videos are still getting traffic to this day. As a YouTuber, I'm curious whether Jeremy leans on like other YouTubers in the space or other influencers. I imagine you he's got friends and probably goes to some of the YouTube events and things like that. Do you lean on other influencers as well? Or is it just the the built with science channel? Right now, we're mainly focused on actually just expanding our internal content team. But Jeremy, I know he does have plans to collab with a few others in the space. I think he's actually going to Texas in about a month to do a big collab. Nice. I mean, I guess I'd imagine it's like it feels like every big fitness influence already does have an app. So I guess it's tough from that perspective, like Renaissance, periodization, you know, the macro factor, there's a bunch of other apps in the space that are that were built by YouTube influencers. And one thing I think that Jeremy does differently is he'll actually reach out to maybe they're not as known, but he'll reach out to some of the expert researchers and coaches and dietitians to collab with for videos in terms of promoting some of the research that they find. So that's been something that has added a lot of the credibility for the videos. Yeah, I want to dig into that a little bit more about the like building trust. and be more kind of science-based approaches and things like that. But you mentioned earlier, you really think a lot about how the app could fit into the content, dig deeper into that. Are you creating a whole video because you think it's a good picture of the app, and then how do you, you were talking to about audience trust and audience engagement and things like that, where you want to create a video just because you think it's a great video. Do you sometimes even leave off like you launch videos that don't even mention the app, or how do you kind of process all of that, balancing the app business with the YouTube business? I think for us, it still comes down to just focusing on a core and where we got started. So that's just the content itself and making sure that it's educational. We don't want to sound like a sales pitch on every video, and much of it is we're creating this piece of content, and okay, maybe at this point in the middle of the video, there's a smooth transition, not a hard pitch, a smooth transition. So for most of the time, it's very soft CTAs throughout the video, but we never really fully pitch a video unless it's maybe something on Instagram, like an Instagram short or something. So then what percentage of his kind of main video is to go out like the full YouTube video, have a mention all of them, or is it still a percent? Most of them now will have something in there, either in the middle or near the end. It's got to be tough balancing all those different factors, though, of wanting to maintain the quality and the audience, and then do you feel like the audience does get saturated too? I mean, this has been one of the things we've talked about on the podcast before, is like, okay, there's seven million people subscribed, and he's got plenty of videos that do tens of millions of views. And I guess that's the thing with the YouTube algorithm now. It's that it does kind of expand beyond just the subscription base, but does it seem like over time, the kind of peaks are a little lower, or like that the core audience who watches every video, like they've already seen it pitched, and maybe they just don't think it's right out for them, or are people like getting convinced over time the more times they see it? Yeah, so in terms of saturation, I think it ultimately still comes down to the content. You can have one person watch maybe a database video, and maybe they're not as interested in it, but then we post maybe another video that's more on the work outside of things, and that hooks them in. So I think it really depends on the content that really attracts to them, and we've seen it time again where if we really dig deep into creating an incredible piece of content that will not only increase trials from some of our existing viewers, but also just expand our reach to reaching more and more people, and then once someone sees one video, the algorithm, again, like we'll show a bunch of our other videos, and it kind of creates its own mini funnel inside of YouTube. Right, yeah, that makes sense. And then how do you think about like shorts and reels and TikTok, and UGC, are you working with any like UGC creators? I mean, you mentioned you're not really doing like other influencers, but are you clipping from the main videos to create more kind of ads like UGC? Yes, so for the most part right now, one thing we've learned from the past few years of going through businesses, focused as key, and for Jeremy right now, and for the business as well, YouTube Longform is the highest ROI I think that we can do, and we just want to focus on doubling down on not only increasing the output, but also the quality of the content that we put out. That said, we are working on a few other projects where someone, you know, maybe a third party or another person that we hire can do independently. So for example, we just onboarded a few clippers who are taking some of our existing Longform and posting that across Instagram Reels and TikToks, and that's actually been quite helpful because when they're posting, some of these videos will go viral, and we're able to see what sort of hooks that they use in those videos to inform some of our future videos. So it's kind of been a two-way feedback loop for us. And then how does that play into ads? Because I know this is like one of the main topics I wanted to talk to you about today. Y'all've scaled up from nothing to $100,000 a month in ad spend and profitably and are really making a big push in that side of things. So how do you think about the pipeline of ad creation? Are you using a ton of Jeremy's videos as source for that or is it totally different? Like, what does that look like? Yeah, so for us, actually, we're not relying on Jeremy at all for ads. I mean, besides maybe a few pieces of old content and he's done, old short-form content that he's done, we're really just trying to focus on how we can scale without relying on Jeremy. And in addition to our goal of just expanding our reach, we wanted to kind of solve the bus problem where everything was reliant on Jeremy's YouTube channel for our growth. And by focusing on meta and not relying on Jeremy, we're kind of building a new channel that can scale on its own. So everything we've done from that end is mainly through like B-roll cell videos. We've used some testimonial content, a lot of statics now with AI. So that's kind of the approach that we've taken with meta. That's fascinating. And again, I would have expected and I actually didn't look through your ad library in preparation for this podcast. I should have known that coming in. But I would have guest. It would be way more Jeremy focus because he is that kind of like celebrity YouTuber and there's so much opportunity to use him. But it makes a ton of sense when, and this is an interesting part too for an influencer with an app if the main business of creating content on YouTube is so, is very profitable in doing really well. That whole focus element of like how much time can you dedicate and I even love the way you're saying it is like trying to make it so that they aren't the limiting factor on anything because as the kind of star of the channel, there's only so much content he can do so much bandwidth he personally has and everything like that. So it's interesting. And again, I think a lot of people who just assume like, oh yeah, you know, influencers like that you can just do video all day and everything. But it's a lot of work and a lot of mental energy and everything else. And so I love that y'all are really trying to diversify away from him kind of being the center of everything. Do get me wrong. I would love to get more of Jeremy's time to make some ads because I know they would perform quite well with the trust that he has. But for him right now, he's already stretched to the max. So yeah, it's all about focus for us. Yeah, I mean, it's a really good perspective to for anyone pitching influencers to like really understand and internalize that is the influencer you think, you know, just sitting around or should be really excited about it's like even an influencer where it's their own app, you have to make really hard decisions about how to prioritize time. So as I mentioned, you started from like nothing in scaled up meta this year. Why meta, you know, talk me through the that whole process too. Yeah, so we actually launched the app in early January 2025. And we didn't launch any meta ads until this year in January. And the reason for that year gap was mainly because we wanted to double down on improving the product. We got a lot of feedback from our launch. And we know from just looking at some of the numbers that as long as we're constantly improving our trial to paid and retention, that lifts the floor for everything else. And considering that our organic traffic is already converting at a healthy rate, we know that any paid traffic that we send will be much lower. So we wanted to improve the floor for all traffic coming in. So we spent that full year just putting our heads down, improving the product. And once we felt like the numbers were right in terms of a trial to paid or retention and our churn, then we slowly started investing into these additional acquisition channels. So I love that perspective that that product really is key. And you know, so many people think if I can just scale up on meta, you know, if I just had $100,000 a month of blowing ads, of course, I could be as successful. But the reality is like, it still really comes down to like people actually believing in the product, people using the product, people staying retained to the product. Like that's how you build a great business. So how do you think about that part of the product versus growth balance of like how you put time into it, how you allocate resources and things? Yeah, no, 100% agree. For us product is everything. A great product will lift the floor more than any growth hacker tactic will do. So that's always been the core for us. Yeah, that's amazing. So starting from zero this year, what's your experience been like scaling meta? You know, what kind of attribution are you using? How are you determining whether the spend is profitable? Did you, like everybody talks about, you know, burn 50K, just kind of learning the ropes or what was that process like going from zero to 100? Yeah, so there was a lot for sure. We've never tapped into meta before, let alone myself. I mean, I had a quite a big experience in marketing and. copywriting, but the metal world was definitely new for us. And I think one of the biggest things that we needed to figure out right away was just getting more data, especially when it came to trial to paid, because like I mentioned earlier, organic, you have a certain trial to paid threshold or baseline because these users are generally a lot warmer. Whereas trial to paid, you're essentially tapping net new cold users who've never really seen the brand before. So we knew trial at the page was going to be lower, but we just needed to get a baseline of what that number would be in order for us to calculate how much we could actually spend to acquire a customer. So that took around a month to do. And once we had a good set of numbers to work with, then it was all about testing different angles and different avatars who would resonate well with the product and how our product can uniquely solve their pain points. So we had a lot of learnings from surveys, from past customers coming in, from warm traffic, and we pretty much just doubled down on finding specific avatars that work well. And right now we're in a process of constantly iterating on what's winning, while also slowly adding on new new avatars. So we're just trying to create a proper iterative and testing loop that allows this engine to keep on running. Yeah, you mentioned earlier the trust side of things and how important trust is and the research that Jeremy's involved in that lends trust to the YouTube channel and to the app ultimately. But how do you think about the creatives and kind of the way they shape the perspective of people coming into the app as far as like over promising or things that are going to get people to convert, but then maybe quickly turn or whatever. That's a big reason why we decided to actually do meta in-house rather than choose an agency is because we felt like we needed someone who really understood the product and was very passionate about the product and ultimately could promote the app in a way that isn't faking anything. So we wanted to make sure that whatever we put out there actually relates with what the product actually does. I'm sure you've probably seen a lot of companies that over promise what we can actually deliver and for us we just wanted to make that experience from marketing to the product handoff as seamless as possible. Have you seen certain ads perform poorly based on that hypothesis or have you sometimes seen the opposite where you see an ad really take off and then you're kind of like it's doing really well, but I'm not sure that's the message we want to send. Yeah, definitely. We've tested a lot of ads that are more like, I guess click baity style or encourage a click and encourage someone to start a trial, but we quickly see from the trial to pay that this is not the user that resonates the best with the app. So we'll see a huge drop in trial to paid. Whereas maybe we think a lot deeper about who this person we're trying to target is what their pain points are and then how the app uniquely solves those pain points that brings in a much higher quality of traffic. So it's definitely a balance of, okay, we want people to start the trial at the cheapest cost, but also we don't want just anyone to start the trial. What are you targeting against and what kind of data are you sending back to Facebook or meta for the targeting? Yeah, so for the targeting it's we have a custom one of our engineers built a custom dashboard where basically we can see and validate that every single new trial is coming specifically for meta just because with all our organic reach right now, I'm medic and often blend who's coming from where. So we have that specifically set up for just medic conversions and basically any time a trial comes in, we feed the data, we're able to see the trial to paid numbers and we're feeding the highest intent users back to meta. Gotcha. You read a post earlier, I think earlier this year, the biggest unlock to scaling your meta ads spend less time in ads manager. What did you mean by that? And then how do you balance that against what you just said about you do still want to have like the right targeting and the right data coming back and everything. Yeah, no, that's a great question. I think I think that must have been on Twitter. I recently started using Twitter and I just share any thoughts that come to my mind, kind of like a personal diary. But I think my point with that was it's super easy to get obsessed with setting up the right campaigns, setting up all these different bid targeting strategies and all that. But at the end of the day, I think what will really move the needle is just creating better content, same as same as on YouTube. As long as you create content that people actually enjoy, then you're, you're going to make a much greater improvement than tinkering around with different targeting sales, especially now with not sure if you've heard of the whole engine, but update. But a lot of the targeting on meta is specifically done through the creatives. So for me, that was just a reminder to focus on creating better creatives rather than tinker around and mess around with the targeting that's really out of your control. We've been talking through the scaling up on meta. But one thing we haven't discussed is that you send most of your traffic to the web or all your traffic to the web, not to the app store. Why do you do that? Yeah. So about 90% of our traffic is actually going to our website. And we have a really long personalized quiz that we send all leads to on our website. The reason we do that is because it educates this quiz that we have. We've tested multiple times. But it educates the user on the actual product. It overcomes objections. And it increases the perceived value of the product will also helping them understand how personalized the product actually will be for them. So for one, it increases the overall perceived value. But secondly, we actually use a lot of their questions in the quiz to build their personalized plan in the app. So it's kind of a twofold. And we've also tested the running traffic to the app store versus the quiz. And we've noticed that the quiz just has a much higher conversion likely because of some of the users right now, who are going directly on to the app store, have a lot of a higher intent. So maybe they don't need to take that quiz right away. But down the road, it is something we're considering on retesting. You basically matched the quiz almost one for one in the app compared to the web. But you saw a way better traffic on the web versus the app. Exactly. I mean, boy, better conversion. Right. You have talked to a few people about this recently. And you know, one of the hypotheses around this is that when you open an app, you're kind of just trying to like get to a solution quickly, like give me my workout plan. I'm going to go work out or whatever. Or like, yeah, just more kind of solution focus. We're on the web. You're maybe a little more like researching and open to kind of reading a little more. You're maybe not like checking out at a grocery store or whatever where your attention is split. Maybe even I mean, do you even track like the difference between being on mobile versus being like in a desktop environment or anything like that? Yeah. So the majority of our traffic is definitely on mobile. So we're just tracking conversions separately. And funny enough, though, we ran that same experiment with setting up the quiz on Android. And that actually is converting equivalent to our conversions on the web. It's just not the Apple store users. Let's dive into pricing. And I love the Jim buddy two person plans. I'm curious about that. So what what what is your pricing? And how do you think about pricing, especially with running ads and thinking about, you know, conversion? Did you change your pricing? Once you started doing ads or had you dialed in your pricing and feel like that's actually been pretty effective. Once you started rumping up ads. So yeah, so for pricing, we are slightly on the premium side. So about $189 for the year or $30 a month. And we actually just launched with that through when we launched the app back in 2025 through YouTube. And we pretty much kept the pricing the exact same the entire time. And the reason for that is because the way we position our product is is not as an app is a tracking app that most work out nutrition plant nutrition apps out there are. But it's more of like a personal trainer in your pocket. So the positioning of it is your comparing costs of a personal trainer versus a trainer in your pocket. So that way we can charge a little bit more premium. Also, we do have that added benefit of all the trust that's been built through the YouTube. And when it comes to approaching meta with this pricing, it's, it's kind of the same thing. We, we don't mention that the app is a tracker at all. It's more of a coach who's going to guide you on your journey. So I think with that positioning, we're able to actually do well with this pricing. Yeah, you have one really cool pricing thing that I don't see a lot of people do is a gym buddy plan where you get to for you bundle two together almost like a family plan, a gym buddy plan. How do you do that? Yeah, so this was actually an idea brought up by one of our product managers, but we got a lot of requests from existing members. saying, hey, do you guys have a family plan? I want to bring my husband, my wife. And this came up often enough that, okay, maybe we wanted to test something. So what we simply did was on our web quiz, we offered users the option to add a gym buddy, we call it to their plan and get their plan at a discounted rate. And that actually has been quite successful for us. Around 15% of anyone who starts a trial will actually add on a gym buddy. So not only has it increased just overall AOV, but also we found that when someone does select a gym buddy, then they're more likely to stay on the app. So that's also increased some of our attention. - Yeah, that's fascinating. I was talking to life 360 about this. And that's one of the things they said was that their highest retaining cohorts are the family units. And you know, that's very much a family app, but they're for them. And I imagine for a lot of folks, if you can get multiple people committed, and then it's a little bit like, well, maybe one's using it and the other isn't, but then you don't want to cancel, and don't want to switch to the single plan, because, well, I'll start working out in the future, whatever, a little bit like a gym membership maybe. But how much are y'all actually discounting to kind of incentivize that? - Just 15% both users will get 15% off. - Yeah, so it's not like a massive incentive. So you're, I mean, that's pretty amazing at 15% of your users are selecting that. And so you're almost doubling your LTV per conversion. - Yep, exactly. I mean, when you think about it too, there aren't many workout-related apps, either that offer the ability to, you know, work out with a friend or connect in that way, such as like Strava might do. So not only from like a growth perspective has this been valuable, but it's also helped us think more about strategy when it comes to the product and how we can maybe double down on this idea of creating more of a community within the app. - Does the app do that currently? So once you add a gym buddy, do you see their workouts, do you, like what, what aspects of community have you already brought into the app? - Yeah, so we actually just rolled this out, but for gym buddies right now, you're able to see what your partner, see when your partner works out, see what your partner eats, and you can send them a few nudges here and there. So there is that collaboration in there. - That's awesome. I don't think that would work for my wife and I, but I could see, I'm actually in a group text with my two brothers and we kind of hassle each other and you know, talk about like our fitness goals and stuff like that. So I should get them on a gym buddy plan and then we'll move our texts right into the app. - You can see what they eat every day. - Yeah. That's awesome. - Yeah, and how do you think about that premium pricing? I mean, you mentioned, you know, $189 a year is a lot. Do you feel like that's a barrier? How have you done any processing? You're considering doing some processing to see if lowering that barrier does improve things or has conversion been good enough that like it's not, you don't feel like it's super low hanging fruit right now. - Yeah. So I think for us right now it just hasn't been a focus. We've been trying to double down on just improving the metrics given our current pricing. And maybe down the road will experiment with pricing. I know pricing can have a major impact, but a lot of it right now is when you do change pricing, you can influence a lot of the product metrics. So we'd rather stick with, okay, let's focus on improving retention and trial to paid. And once we're at a point where we feel like that's healthy, then we're gonna start experimenting on these other things. - With that high of a price, and having looked probably at a state of subscription apps report, are you in the ballpark of the median or higher than the median on conversion? With that high of a price? - Yeah, so for our warm traffic, our organic traffic coming from YouTube, we are around the 35 to 40% range and trial to paid for meta is slightly lower around 25%. - Gotcha. - Which is super impressive at $190 a year. And what percentage of that does jump all the way to the annual versus like what percentage end up picking the monthly plan? - So right now it's about 75 to 80% choose the annual. - Wow. - And that was actually part of a recent experiment we did on the pricing page, which was a big one for us. - Yeah. And speaking of the pricing page and going back to the gym buddy thing, so you asked that question in onboarding so that you don't have to confuse the paywall for people who don't have a gym buddy. So like a person who doesn't get a gym buddy, doesn't even ever see that they can get that discount. So your pricing page is very focused on just the annual and monthly, right? - Correct. Well, on the quiz actually is where we ask, hey, do you have a gym buddy? Would you like to add them onto your plan? They can say yes, they can say no. But if they say no, on the pricing page, we actually have just a really small toggle that says a little bit about the gym buddy. So they have that option as well if they opted out on the quiz. - Yeah, 'cause I mean, I could see it becoming one of those like paywalls that just has way too many options and makes it confusing. So it makes sense to like ask it in the quiz and then not make it super prominent on the paywall itself. - We've been talking about meta growth and then the YouTube channel as a base, but I know you've also been working on SEO and now AEO and getting more and more people saying that ChatGPT sent them to the app. How's that going? - It's been good. It's an investment that we decided to do earlier this year as another channel for us to expand on. The reason why is because traditionally, we've had really strong traffic coming in from SEO, from all our old blog posts. So about 30% to 40% of our traffic coming in was from SEO, but ever since the rise of AI and people using AI search a lot more, we slowly started seeing that decrease. So we decided to invest in hiring an external consultant to help us with some of this AI search. And basically what it is is we just optimize a lot of our existing content, but also we double down on creating online tools. So for example, a calorie calculator, a macro calculator, a bunch of these tools that users could use in the web that not only added value, for example, they're able to figure out what calories they need to eat per day to lose fat, but also seamlessly transition into the quiz. So that's been working really well for us and also tools just tend to rank quite well with AI search. So that's something we're gonna continue to double down on. Although it does take a bit more time to actually see results just like SEO, it's something that we are gonna keep on investing in. - Do you see a difference in intent and conversion from the tools versus the blog posts or is it fairly similar or dependent on the blog posts? - I would say it's dependent on the tool and the blog posts, but for the most part, the tools have a slightly higher conversion. - Interesting. - We've talked a bit about your thoughts on experimentation, but I did wanna dig a little bit deeper into how you think about experimenting, what kind of experiments you've run, what kind of results you've seen. So how do you think about experimentation? - Yeah, so I think one of the biggest shifts we've had over the last few years of experimentation is we're obsessed with the problem and not the solution. And what I mean by that is rather than going ahead and just simply testing out ideas for experiments, we take our time to really, truly understand like what actually is a problem we're trying to solve. And the way we do that is like through three main steps. So first, we'll look at the data, we track everything on amplitude, and we know where this drop offs on our web on the app onboarding, and we'll look at what certain areas have the biggest drop off, but also that we feel have the biggest area to improve. And once we have an idea of what we actually wanna experiment on, for example, let's say it's a pricing page, then we'll go ahead and look at that pricing page, and we'll list out as many problems as to why that pricing page isn't converting before anything else. So maybe it's a lack of trust on that page, maybe the value proposition isn't clear. We'll list out as many problems as possible, and as a team, we'll decide on what we think the biggest problem is. And then only once we have that problem locked in, then we'll start listing out solutions. So if it's trust, then maybe we need testimonials, we need more credibility. And based off of the solution, then we'll just go up and roll out an experiment. So yeah, that's been the main process, and I think the biggest thing from all of that is making sure you document down everything, just because it's easy to send a message to your engineer on Slack and say, "Hey, let's roll this out." But once you actually take the time to write things down, you understand the problem so much deeper, and not only that, but if an experiment fails, you can go back to the drawing board, An unexpected. understand what exactly you tested, why it failed, and how you can iterate on it. Because there's been, I mean, probably 70% of the experiments that we run fail on the first go, and that doesn't mean it's a failure. It just means that maybe there's something wrong with the execution. Maybe we covered the wrong problem. So often we're iterating on these failures to uncover a winner. So that's been a really helpful process for us. Yeah, I really like that problem focused approach, because I mean, it's so much more kind of natural and easy in some ways to just brainstorm solutions, like, oh, it's not converting well. Let's make this prettier or let's add this testimonial or whatever. But I love that approach of like really deeply thinking through all the potential problems first and then brainstorming solutions. And then it, I mean, it just gives you such a clearer understanding of what you're testing, and then it helps you interpret the results better, because you had a hypothesis around exactly what the problem was, then you had a hypothesis around what would solve it, and then you can test both sides of that equation instead of just saying it was a complete failure. So I love that. Exactly. And just to add on that, so when you do find a winner, for example, one thing that's been really impactful for us is taking those learnings and applying what worked to other parts of the funnel. So say, for example, a messaging really worked on our quiz. Maybe we want to test applying that on somewhere in the app. So taking those learnings and applying it across all points within your product and your web. How granular do you get with testing? You know, this is kind of a controversial topic, you know, some people are like, oh yeah, I changed my, you know, CTA button from yellow to blue and conversion skyrocketed. Are you doing super granular experiments? Are you testing like kind of bigger swings? For us, we're definitely testing bigger swings. Again, it's about focus, and we'd rather just focus on what we think based on the data can move the biggest needle, and I don't think for the most part, it'll be with changing colors and changing a small piece of text. Yeah. I mean, it does work sometimes, and maybe some people have seen it work and then over index on that, but sometimes those kind of things can move the needle. Some little things can be dramatically, I mean, especially when it does come to like CTA text and things like that, like the subtle aspects of how people are thinking about what action happens after they tap a button, like I can see some of these like really high leverage points being able to test more granular stuff, but yeah, it's like, if you can test a big swing and try more dramatically new stuff, you just get clear results, too. Yeah. No, I agree. Not to say that a little text change won't make a difference. I think for us, it's just been when we have a problem that we're trying to solve and we have these different hypotheses for why we'll usually test it as a bash just because again, it also depends on how much traffic you have coming in. We won't have enough to do a bunch of mini tests, so we'd rather focus on a bigger swing if that isn't work. Okay, maybe we'll strip things down a little bit, but that's usually how we've been approaching the experimentation. Yeah. And then because you, as you said earlier, have kind of looked through the funnel and kind of estimated the likely impact. You're also like looking for those high leverage places anyways, like places where people drop off the funnel, places where you feel like conversion should be dramatically higher than it is and not testing as much the places where you're already seeing really good results. Exactly. Yeah, so it's like picking the test is leverage and then what you test is leverage or picking where to test and then what to test are both leverage points that it's easy to like, yeah, want to test a million different things, but realistically, even with AI and moving quicker, it's honestly more about traffic these days is like, can you, like, if it takes a week to get to statistical significance, that's a very different flow of experimentation than if it took an hour to get to statistical significance. You're not getting as many at bat, so you gotta be really careful about where you take the swings, right? Yeah. Hopefully it is much traffic as Netflix just Spotify, I think we would be definitely testing a lot more. Yeah. That makes sense. One thing you do that I really love is, I think what you called quiet launches, what is that and why do you do that? Yeah, so this is more on the product side, but what we've noticed is in the past, what we would do is anytime we had a new feature that we wanted to release, we release it on the app and then we would send an email to all our members, get them excited about it as a marketer. I'm sure you would want everybody to see this new thing that you're building, but what we've noticed is that this often inflates a lot of the engagement metrics that you obviously want to measure when you launch a new feature. So we've actually taken the opposite approach where we'll roll out a new feature, we won't say anything, and we'll just, we'll just see how the metrics do on its own. And if there's some signal that, okay, this product is working well, it's adding value, then we'll feel confident enough to launch that out to the public. So it's mainly a choice of how we can learn the best from some of these new releases. Do you also have a pretty good beta tester pool where you're even rolling out some of these experiments in beta before they even go out to the main app? Yep, so we have a group of about a hundred people who were actually our previous web base course members. So they've been with us for years, and we also have a slightly bigger pool with about, I think a thousand or so beta tester. So we have a pretty good base to test before we actually release it to the public. So with those quiet launches, how often are those quiet launches just to the beta pool? Or do you tell the beta pool, make sure the feature actually works and doesn't have bugs and stuff like that, and then the quiet launches actually public, or how do you, how do you do that? Yeah, so the beta pool is where we're telling them about the feature. We want them to test it. We want feedback, then we're iterating on that feedback, then we're quietly launching it to see if that, because the beta pool are the most engaged users. They probably use whatever we put out, but we, we just want to refine it from there. We'll really set to the public quietly. Well, and speaking of product launches, I know this year you launched something called Jeremy AI, what is that and how did it go? Yeah, so Jeremy AI has actually been one of our customers' favorite features. Essentially surprising, because a lot of people screw up their AI. I mean, during the whole, whole launch of the app and, and the growth of all our features, we've never really promoted or done much with AI, besides some of the back and stuff. But Jeremy AI is essentially an AI chat that's trained off of all of Jeremy's past YouTube videos, his blog history, and it's a source for users to get any fitness, nutrition-based questions answered, just like Jeremy would be talking to them on the phone. But I think what we've done to really increase the value of this is, not only is it trained off of Jeremy's research and content, but it's also able to read the users' workout, nutrition, and progress data, so it could actually give really, really personalized answers based off where they're at. That's really cool. I mean, if you don't mind digging a little bit into that, like what are you, what are you using on the backend for that, just one of the off-the-shelf AI provider, like big models? We actually have some incredible engineers, so they all built it in-house. I'm not exactly sure. I think it was one of our, I think you've met him, but his name is Kevin. He's one of our engineers, and I think he just spent a weekend, he had this idea, he spent a weekend, he rolled it out, and we just never looked back. That's incredible. We've talked about it a few times here, but I did want to dive into the research angles, so for a health and fitness app, that kind of trust and authority, and even the name built with science, and that may be part of why you're able to charge a premium is because you established that trust, but you are actually doing legit science and collaborating with researchers and things like that, so walk me through how all that works. Yes, so it's rooted in all of our videos as well, but a lot of the content is informed by the research that comes out, so Jeremy does a lot of investing in finding some of those new content and research that comes out. He has a team of dieticians and coaches and researchers who will look out for that sort of content, but he's also started to work a lot with releasing his own studies, so he'll work with this university where we live, and it's called UBC, but he's done self-partner studies, and we've actually started to do a few studies within the app with some of our customers. An example of this in the app is we had a three versus six day experiment, and we invited our users to join, where half of them would do three days a week working out, half would do six days a week working out and we use their data within the app too. measure some of those results. So that's just another tool for us to continue to get data and run it by with some of the researchers. So it's science and the research and being on the leading front of it has definitely been something that has been core for us. Yeah, and it seems like it does play across the entire mission of the company across the app, the content and everything else y'all do. I mean, one of my favorite videos recently was one that Jeremy did on body fat percentage and calorie intake. That was so fascinating. I told Ethan before that I didn't want to turn this into like a health and fitness podcast because I'm a health and fitness nerd and could easily have done that. We could we could be talking lifting this whole time and all the different. We actually were before we hit the record button talking about different like lifting techniques and stuff. Anyways, I love this video he did where he actually took like what six or seven like real people like measured their body fat, measured their like total caloric expenditure, measured how much they were actually eating and it was just it was fascinating. It was done in a very like scientific way and like a very informative way. I believe that's probably one of his better performing videos than the last year, right? With that kind of a video, it's more about the trust and the science and stuff but but did that one perform well? How do you think about the those kind of experiments and that kind of science and that kind of content actually impacting the app business? Yes, so I think for that piece of content, that was definitely a big bet that just Jeremy wanted to take. It definitely was one of our highest performing videos this year and it was honestly very fun to do. We rented out a full studio brought in all these participants but I think for us it was just we wanted to take a shot at trying something completely new. It did have a decent conversion to the app but it was more for just educational and also entertaining purposes. I imagine that was pretty expensive too. So I mean that's another thing I think a lot of people don't think about with these influencers is that some of these videos end up being very very expensive and very very time consuming to produce because it's not just you know you see a polished 20 minute final video but there's hours and hours of recordings and planning sessions and scripting and like so much goes into that. So I imagine that one specifically with all the like Dexas scans and everything else like really was a massive effort to pull it off. Yeah, that one was over months of work for sure. And this is what we're talking about earlier too. It's like you probably have what through your four big videos going at any one time and you're having a man it's like a massive like project management nightmare of these like long running multi month experiments and research and shots and all that kind of stuff. So you've got you've got your hands full. And that's why you know the the whole thing you were saying earlier about focus I think is is just so important. Yeah, I agree. I think in the past we've just tried to do too much and we've got to know where and now it's like okay what are the one to two things that are really going to move the needle and let's just a hyper focus on that. So that's what's worked the best for us. Nice. Well as you wrap up I did want to ask the three questions I ask every guest now. Over the past year what was your biggest win the biggest like experiment or change that you implemented? So as I mentioned earlier I think the biggest change was an experiment we ran on our pricing page where it heavily changed the shift from people choosing monthly to annual and all we did was change the layout so that it presented the annual upfront and reduced risks of actually starting the free trial. So we didn't change the price we didn't change the positioning it was simply a UI shift which had a massive impact for us. So for those listening on the podcast just walk us through verbally like some of the things that you think had the biggest impact on that change. Yeah so prior to the experiment we had a pricing page where monthly was shown on one side and annual was shown on the other and then after the experiment we rolled up the variant which essentially just showed only the annual and on top of the annual we had kind of a blincus style layout where we showed day one you'll get your personalized plan day seven your your plan this is how your plan will adjust and adopt with you and then day 12 right before the free trial ended we made it clear that we're going to send you a reminder and that you can end your trial whenever you like. So we made the risk of actually starting the trial a lot more clear but also gave them the annual as that upfront choice and then there is an option to you know look at the monthly plan as well but for the most part reducing the actual perceived risk and choices that they had to make had a big impact. Yeah so it was a monthly like just kind of a view all plans and they should do the monthly yeah yeah that makes sense and so you're throwing back to like the very early part of the podcast you were saying that something like 85% of people choose the annual plan that's since you rolled this experiment out right. Correct man such a huge win especially at $190 a year. Yeah I think it was around 60/40 before that. Okay yeah that's a big that's a big increase and dollar dollar wise and LTV wise that's a big shift. So what's your biggest fail of the past year? We recently added more context to one of our pricing pages as an experiment just because the problem we were trying to solve is increasing the making the value proposition a lot clear and what we noticed was that actually had an opposite effect adding more context onto this page seem to actually hurt the conversions a lot more than not having anything at all and I think our learning for that was it just reinforced how adding too much context especially at a state where a user might be already you know excited and motivated by might just overwhelm them then actually get them to start. Yeah that makes sense and then you had another one I really like this one so I don't want to leave it off around the no trial offer. Yeah so we're we're also testing a no trial offer compared to the 14 day free trial. We thought to be a big win just because a lot of our traffic right now is organic and they're already warm and trust the brand but we're seeing it's quite negative just because it does of course add a little more friction even with a 30 day money back guarantee. To be more specific the trial is actually a toggle right where they get 20% off for not doing a trial and paying up front. Exactly yeah so you could pay up front and get 20% off or you could toggle off and just take the 14 day free trial. Yeah that you know I mean yeah at 190 dollars maybe that's part of why it's not working for you but that's an experiment I think more people should should do. Had you seen that in another offer did you just come up with that idea? It was it was a mix of both. We always knew we wanted to test no trial and we took inspiration from some of the other in-app paywalls we haven't seen it anywhere on the web but what's funny enough is going back to my point about documenting everything so this experiment hasn't been winning so we went back to the drawing board and looked at you know why we think it's not winning and one of the core hypotheses for why I wasn't working is because the copy on the toggle the no trial option as being like way better and you're kind of losing out if you just start the free trial. So that was our hypothesis so we actually have been rolling out another arm it's been a few weeks now where we make both options seem pleasant so it's not one frame does good or bad it's more of like okay they're both good and that actually has been a very strong lift. Nice so yeah it made up winning and then back to our conversation earlier like you got to understand what you're testing why and then know it to change when it didn't work because it might not be that it was a bad idea maybe just didn't quite execute it the way that's gonna make it work. Exactly. Yeah all right last question and so film the blank growth would be easier if you build a product people genuinely love and tell their friends about. That's my favorite thing to talk about on the podcast you know at heart I've always been a product guy so I love it that we're wrapping up on that topic like I mean it really does just it makes everything easier like if you're struggling with conversions you're struggling with ads you're struggling to scale like having a product that people care about that at least it's just night and day and people talk about this like PMF and and you know once you hit product market fit that the market is pulling you for you're not pushing against the wall so yeah I love that I love that that was your answer for this. All right as we wrap up anything else you want to share with the audience any roles you wanted to shout out or anything our audience could do for you. Not that I could think of I mean if you want to reach out I'm on LinkedIn I'm recently on Twitter so we're always looking for new roles people who get obsessed and are passionate about solving hard problems we're always looking to bring people on the team. Also Ethan it was so much fun having on the podcast thanks for joining me. Likewise thank you. Thanks so much for listening. If you have a minute please leave a review in your favorite podcast player. You can also stop by chat.subplub.com to join our private community. (upbeat music)