How to Do AI-Powered Discovery (Step-by-Step with Live Demo) | Caitlin Sullivan

AI for user research is mess. It's unreliable, but what if I told you I know how to fix it? There's a real art to doing mess, right? But if you do it the right way, then AI actually isn't going to hallucinate on you, and you're going to cut your analysis time and half. The key is actually replicating the way that you would do things in a rigorous way as a human and just doing it like that with AI. This is Caitlin Sullivan, one of the world's leading experts in user research. And in today's episode, she's going to give you all the tools, the demos, the workflows, so that you can do amazing user discovery and customer research, enable the AI. These results genuinely show what does good AI customer research and analysis actually look like. So I think there's actually a really simple answer, which is it looks like replicating the way that we do rigorous analysis as humans. You've chosen Claude. If people have the option to choose between all the AI models, why have you chosen Claude? I keep coming back to Claude because by default, it does a more thorough, more nuanced analysis than the other two platforms. And then there's two big types of analysis that people tend to run surveys and interviews. What are we going to do today on the surveys from? I'm going to demo how you can use any of the large language models to run a few key steps that I just talked about to get really solid results. Wow, that's insane. Before we go any further, do me a favor and check that you are subscribed on YouTube and following on Apple and Spotify podcasts. And if you want to get access to amazing AI tools, check out my bundle where if you become an annual subscriber to my newsletter, you get a full year free of the paid plans of mobbing, arise, relay app, dovetail, linear, magic pattern, steep sky, reforge build, descript, and speechify. So be sure to check that out at bundled.acoustge.com and now into today's episode. Quick note for my audio listeners, there are some things that we showed, which you can see on Spotify or YouTube, but we've edited this audio so that it's a really good listening experience nonetheless. Caitlin, welcome to the podcast. Hey, I gosh, thanks for having me. What does good AI customer research and analysis actually look like? I'm not reinventing the wheel with pretty much anything I do. I have just taken what I know to be true, doing really solid research as an individual contributor as the head of user research and replicating those processes with AI. Amazing. So let's break this down for people in terms of what the actual tasks are and what the actual tools are. So analysis actually looks fairly similar, whether you're analyzing interview transcripts or survey responses or other kinds of short customer feedback. We're going to use a similar set of kind of phases of analysis across all of them. So good analysis, rigorous analysis with AI or as an individual as a human doing it is going to involve actual analysis where you dig through data and you pick it apart, you look at what's in there closely, put it back together in different ways and see what pops at you and then verification or stress testing, basically seeing how the findings you have hold up under pressure and then finally synthesizing. But what most people do both as humans and also especially with AI is jumping straight ahead to synthesis and that's exactly what we don't want to do. And then there's two big types of analysis that people tend to run surveys in interviews. I'm going to demo how you can use any of the large language models to run a few kind of key steps that I just talked about to get really solid results out of any analysis that you do with surveys or short customer feedback. And I'm going to do that in cloud in the normal web UI, but you can replicate this process in chat, you be T or Gemini, I do that all the time as well. Awesome. And then we're going to show you everything here in blue. So we're going to cover the cloud analysis process, but also show you the agent version with the chat GPT code X. You could use other tools like Zapier, NADN, and are people using those frequently in the fields? So I'm going to do the agent part in cloud code in my terminal actually, but yeah, people are using the other tools, the agentic, the more advanced agentic tools, I would say are not commonly used by most of my clients. But what a lot of people are already doing because it's kind of more a more familiar space is automating a lot of these processes rather than making them fully agentic. So you can use NADN Zapier, make.com, any of these automation tools to link up with your chosen large language model and sort of put some of these prompts in place to happen automatically for you. Awesome. And then the other side of course everybody knows our interviews. And so there are the human moderated side of interviews. What are we going to cover for people there? I'm going to basically replicate the same sort of process across survey analysis and interview analysis. What's worth knowing for interview analysis is that things can get really complicated, really fast. AI is built to understand structured information best. So with really long transcripts and sometimes messy transcripts where speaker labels aren't particularly good, we can run into a lot of problems with AI. So I want to show you how being systematic in terms of the sequence of prompts we use and the types of prompts we use can actually ensure that you get much better results even when we're dealing with a tougher space like interview transcripts. Awesome. So we're going to use transcripts for chat GPT and Clawed. If you had a video that is actually possible now with Gemini 3, we're not going to demo that but that is something to keep in mind. And then the agentic version will show you guys how to do that in Clawed code and do that in parallel with the survey analysis. Now a lot of people have been talking about AI moderated surveys. So can you just talk to us about what these are, what the tools are and how you might use these. So there are already a lot of AI moderation tools on the market. There's a huge list of them. They are tools that run sessions with customers for you. So the idea is sort of that you can scale your team's ability to talk to customers by having all of these sessions run when you're sleeping or making your coffee or spending time with your family instead of having to line up, you know, 15 people to talk to every day. So as we've talked about before the recording started, there has been a little bit of discussion this past week where Clawed, the team behind Clawed Anthropic launched their own study where they're using an AI moderation tool of their own that has surprised some people in our space in terms of the quality of the questions that are being asked. So this space is a little bit tricky. There's a huge range of quality between the different AI moderation tools to be honest. Some of them are asking really, really good questions and handling bias, not asking leading questions and also doing decent analysis after collecting data. Some of them are not quite there yet. And I've seen the questions that Clawed was asking in the Anthropic Team study and I also wasn't particularly impressed. I can't emphasize this for you guys listening enough. In my 16 years of product management, one of the things I learned is to respect the craft of user research. Respect the people who are working on it in your company. Try to get some time shadowing the best PMs who do user research and the best user researchers in your company because even Anthropics interviewer couldn't do a good job, right? And so you want to think about this as a skill that you're going to continue to improve throughout your career. We could cover it all, but we don't think the AI interviewers are quite the most important thing for you to learn. So we're going to cover what's in blue for you guys. How to analyze surveys, how to analyze interviews. So without further ado, can you get us started, Caitlin? Yeah. Okay. So as I said, we have to go through multiple steps in the analysis process to get something really rigorous and reliable out. And I mentioned that we have to start with kind of picking apart the pieces of the data before we synthesis before we do synthesis, which is unfortunately what most people don't do, most people jump straight to synthesis. Another piece of the puzzle here is of course giving Claude or any of these large language models enough context that it can do a really good job for us. And I know that this metaphor is really overused at this point of saying that it's like your junior assistant or something. I prefer to think of these models at this point not as juniors, but like someone who's new to your team. So if someone new joined your team, they don't necessarily have no skill in this space, but they don't really have the expertise in the domain or about your customers that you have. They might not even know exactly what kind of feature functionality you have built in if they haven't been fully onboarded. So the first step that I use in a lot of especially interview analysis is just loading context. And I make this an individual prompt rather than building it into anything else because large language models actually can be a little bit overloaded. If you try to paste a two page long prompt in one go, a lot of the models will still have trouble handling that and you'll see that they drop some of the instructions even if you said this is really important. So I'm just going to copy my first prompt here, which is only about context. If you're enjoying this masterclass and AI analysis, you will love Caitlin's cohort-based course. This runs for the month of March, where Caitlin will walk you through not just these first-level workflows that we've shown here, but the next layer deeper of complex workflows. The agent marked on files, everything you need, should help you get to the next level of craft using AI for discovery and user research. For my money, it's one of the most important skills for PMs, but not just PMs, designers, engineers, go to market people, and obviously user researchers. So if you enjoy what she's saying now, take her course. You will learn even more. She'll walk through all of this in detail. And the best thing is you can get a discount on the course using my code, link in the show notes. So check her course out and back to today's episode. Today's podcast is brought to you by Pendo, the leading software experience management platform. McKinsey found that 78% of companies are using Genai, but just as many have reported no bottom line improvements. So how do you know if your AI agents are actually working? Are they giving users the wrong answers, creating more work instead of less, improving retention, or hurting it? When your software data and AI data are disconnected, you can't answer these questions. But when you bring all your usage data together in one place, you can see what users do before, during, and after they use AI. Showing you when agents work, how they help you grow and went to prioritize on your roadmap. Pendo agent analytics is the only solution built to do this for product teams. Start measuring your AI's performance with agent analytics at pendo.io/acosh. That's p-e-n-d-o.io/a-k-a-s-h. And obviously you've chosen Claude. I've chosen Claude. It's my favorite. It has been for a long time. And I keep checking myself on that, doing tests across the three major ones, meaning Gemini Claude and Chattu-Bt. And I keep coming back to Claude because by default, it does a more thorough, more nuanced analysis than the other two platforms. I will say, though, it do like Gemini quite a lot because it seems to be fine-tuned a bit more for accuracy and focusing on frequency of findings. So you're more likely to get kind of a top five themes or patterns from Gemini really quickly that are solidly evidenced than you will in Claude or Chattu-Bt, just because that's kind of what it lifts up. But you'll have to push Gemini more than you have to push Claude to give you the complete picture and all of the smaller themes or the outliers and things like that. Okay. I'm excited to see how Claude does. So I'm with my context prompt here. I'm just going to get all of my files ready here. I'm just dragging them over. And I want to quickly point out, as I send this context prompt that I'm using, markdown files. And this is important because in Claude, if you don't have a max account, like I'm using right now, if you have pro even, you've probably experienced uploading a bunch of transcripts and hitting this token wall. Even though Claude's token limits have been expanding and expanding and you know, it can handle a lot more at once now, you can kind of circumvent that issue by transforming your interview transcripts into markdown files. But this is something that I teach in my course because you have to do it right, you have to do it in a way that really keeps maintains all of the most important information from those transcripts and carries them over into markdown files. But what you get out of a markdown file instead of just using the transcript is you've turned it into a format that's more structured than the original transcript is, which means that it's not just better for file handling. It actually helps these models do a more accurate job as well. Love it. And just the high level, like what is that transformation into markdown look like? What do people need to focus on? You're basically telling it to take certain types of information and it's not going to take everything from the transcript. There's a lot of bloat in your transcripts, not every conversation necessarily needs to be carried over. Some people will argue with that, but for especially for a lot of the sort of product research that I've done, you have a specific focus, you're not always doing exploratory research. So you can usually have a large language model with you comb through a transcript and extract the right kinds of information that are most relevant to the topic areas we're exploring or the hypotheses you have and turn that into a document that already has tagging or groupings of conversations under certain headers and things like that, putting in metadata a little bit of extra coding that makes it easier for the model to make sense of it. And then how would you break down this context prompt? What are the key elements here? So it looks like a lot at first glance, I realize because some people share a half of this or a third of this within a prompt with actual tasks, but I would call out what's most important here is I do have a bit of project context. I have a business goal, reduce churn. The percentage is kind of arguable here, though I would say in my experience saying 15% versus 50% like 50 will make a difference. And then I have product context. So just like I was saying about onboarding this new team member, I want to make sure that it fully understands what kind of functionality is involved in the service that we offer, what the differences are between the product tiers so that in its reading through of people's experiences or comments about the product, what they're using, it fully understands what kind of functionality would actually be present and what what they would be using when they have that experience. Otherwise, if we don't include something like this, I've seen huge, hugely wrong guesses, gross assumptions by large language models about what someone has as part of their account, kind of making things up or just trying to piece together bits of information it had that were entirely wrong. Point it in the right direction of what your product is actually doing in that case. Me exactly. Yeah. What's important here, though, is I said that for context prompt, we actually are just loading context. So at the end, I said internalize this only, do not run analysis yet. It's really important that you don't let it jump ahead and start doing tasks because I haven't given any instructions here for how, how to do analysis. So it hasn't done analysis yet. I'm just going to jump to the next prompt and send that now. I think this is a really cool lesson for people. I don't think many people have done this two-step prompting, right? Yeah, first step, context, second step analysis. Most people try to shove it into one prompt. Exactly. But this will get you much better results. It will basically help the large language model focus on one thing at a time, which means that each individual step will be done more thoroughly. So let me start working on it on this step so that I can talk through it at the same time because it will take a minute. What's really important here is I've started analysis, but I'm not saying, hey, go find me some themes and patterns. So what I said earlier about starting rigorous analysis means doing actual analysis and not synthesis. It actually means that we need to tell it to go look at every single file that I just uploaded. I'm basically asking it to pull out for each participant value anchors and I'm telling it that means what's keeping them subscribed. And then fragile points here, the formatting is a little wonky. What frustrations exist despite subscription. And I'm spelling things out in detail. So where I say I want it to give ratings, right? Like how strong, how big a deal is this? How how strong a fragile point is this? I'm explaining what a high fragile point means. What is a high rating? So I'm not letting it make any assumptions here. I'm trying to be pretty explicit in every case about how I would rate things. I've also started this prompt with quote selection rules, which seems a little funny because I think most people just assume large language models know what a quote looks like. But if you prompt in Gemini and then you prompt in Claude, you will see that they interpret quotes completely differently. So Gemini will give you much shorter snippet, Gemini will give you much shorter snippets of customer statements and Claude will give you a longer sentence or even a short passage. So I like to avoid making the assumption that that Claude in this case has the same understanding of quotes that I do and I spell out what a quote looks like to me. Very interesting. Yeah, I've also noticed that sometimes the AI will like you've had here take away some of the hedges and qualifiers. It will try to like clean up the quote as if it were cleaning it up for a news article. But in user research case, we don't want to clean up the code. So we're giving that instruction to the AI. Yeah, and in that case, you could even line at a line here that says something like leave ums as pauses, you know, fumbled language, things like that. I sometimes include that in here too to keep things exactly as they are with the customers voice. And can you define some of these these words for me because I'm not totally familiar with them or I can imagine a lot of the audience is trying to guess at exactly what they mean. Value anchors, fragile points, stability rating, what are those? This seems like cool terms of art and the user research discipline people can benefit from learning. Let me give you a background quickly about the the data I'm using here too. So I have eight interview transcripts from a synthetic user study that I create. based on a study that I actually ran. And it's all about improving retention and decreasing turn. So these are all people who had a premium tier account on a fictional app called Flow that is a meditation and anxiety management app that also has on demand therapy. And so what I'm doing here is I'm trying to figure out-- I'm doing a bit of a retention analysis where I'm trying to figure out what's keeping people subscribed and then what's likely to break their experience. So what frustrations exist despite subscription basically means the flip side of the things they're finding really valuable about the service, what could actually push them to cancel anyway. And I want to collect both sides of that because I'm not assuming that when someone is super happy, they're just happy. No one is so one-dimensional as that as a customer or as a human being. So I want to know if they're happy about using the service and they continue to pay. That's great. But what would cause them to turn anyway if it continued or increased over time? Makes sense. So we have some results here. And I've told it to put this into a table. So it followed my instructions really well. Basically, this looks like the value anchors are in a table here for participant 1. This is by participant. So this is the first participant here. Value anchors then fragile points. And then I also ask for a stability rating. So overall, how risky or how much risk is there that this person will charm. And then I have a few key themes that I asked it to call out to. And then we start participant 2. And it just looks like that for every participant. So what I said earlier about making sure we actually pick through the data and kind of look at it closely first is what I've forced the large language model to do here. But if you jump to asking for themes, patterns, pain points, whatever it is, you're actually forcing it to jump into like synthesis mode instead of real analysis picking through the data mode. So I usually start with something like this where I force it to look at every file and kind of look for specific pieces of information so that it actually comes through properly. And that's replicating the human process. That was the high-level advice you knew at the beginning, which is we're always going to analyze the specific interviews. So let's not have the AI jump that step. Let's have it analyze those first. Exactly. So as a very experienced user researcher, you always come through. We used to just sit with one interview transcript and look through it manually, every line copying, pasting into Miro or sticky notes or whatever. And so I'm basically replicating that process here. Awesome. So the kind of information that we get here is going to be much more granular, much more specific. It hasn't really other than this bottom section here. It's not mapping things into themes. It's really just picking up the foundational elements that a really good analysis will be built on. So we have things like this full passage about someone who is doing breathing exercises with this app every day, usually between meetings and so on, all of the things that she's using. And so the strength of this in terms of stickiness with the product is strong because there is daily integration into the work routine. Multiple features used regularly. And so on. Let me see what else is there. It's maybe more than what she expected because she thought it would just be meditation, but it's become this whole system for managing my new role. So dealing with stress at work is now dependent on this app. That's a good sign. And then the therapy sessions can be book spontaneously. I don't love that this means I won't always talk to the same person. So there's something interesting there. It's labeled this as moderate. I would almost say, OK, well, this is where my perspective comes in because they don't just have AI do analysis without thinking about it myself. But this seems very positive. And the rest is starting to get a little bit more into the fragile points. So this is the sort of thing that I want to be pulled out because if I were doing this as a researcher, I would be pulling out all of these quotes myself and then trying to categorize them, label them, give them ratings and so on. So this is really what it looks like to do good analysis. Whether you're doing it with AI or not, but it's doing it much faster for me. And there are a lot of tools that I think help with this. When I was at a firm, we used a tool called Dovetail. What has your take on some of those tools and which ones do you like? So I use a few that are less talked about. The two that I have continued to return to because I think they're doing a particularly good job. Arbreita.ai, I can send you that for those show notes. And reveal. They have completely different approaches. And I think there's so many different tools out there that really do things just very differently than each other. So it's hard to compare them all kind of apples to apples. But Brayda is getting to a pretty good nuanced level of analysis fast because they have really good prompting in the back end and reveal maps, the customer comments, in a bit of a grid and does a bit of hypothesis testing for you. So it lays things out in a way that you, you know, kind of like this. I'm almost mimicking what they do as well. Being able to see things yourself and make your own comparisons in reveal versus just having a finished analysis for you. Dovetail.ai, honestly, hear mixed reviews about from researchers, especially. And I have had mixed experiences with it. So I haven't tested it in about five months, so I wouldn't dare to kind of give it a final rating right now. But a tool like that is really meant to get you to insights as fast as possible. But you don't have control in any of these platforms over exactly what you're looking for and exactly the type of analysis that you want it to do. So I tend to stick to the large language models for the most part, unless the time to insight is so extreme that I feel better uploading things into a platform and having it do that for me, but it's rare. So the platform is going to take some of this control that we're getting from these prompts, from the step-by-step prompting process out of your hands. Yeah, it's the power of this workflow. We're kind of showing you guys the advanced-- this is what an advanced user researcher would do. Yeah. And obviously, the teams behind all of those tools have worked tirelessly on those prompts. They've also worked with the team at Maze to kind of develop the prompts behind their AI analysis and AI moderation. So I know what it looks like behind the scenes, and they're all working very hard to kind of figure out the right prompting sequences to do this for you. But I prefer, as someone who's used to doing research, especially I prefer to kind of have more control over the granularity in the analysis or focusing it on a retention analysis in a certain way and so on over using a tool to do it for me. Makes sense. So is there step three now? No. There is a step three. So I had said two steps before, but it's actually a three-step prompting process. It's three step here. And I would say it's four if synthesis is added. So the context is almost like a step zero, I would say. You kind of just load the background information. So I would call this three steps in this workflow, which is condensed compared to what I usually do. So for the most rigorous possible version of this, you have more steps because you need more feedback loops with AI. But you'll see one version of this here. And you just space this out so people can see this more easily. Just going to let it do its thing while I talk. So this is what I referred to earlier as a stress testing or verification step. We need this because even if you think everything looks really good, which most people go, oh, AI did this analysis for me. It looks really solid. It's usually not as solid as you think it is. And what happens down the line is then a product manager or a designer or researcher presents some findings in a meeting and then can't trace them back or realizes there are actually lots of holes in this insight or set of insights. But if you push your model through an additional step for verification, or I usually call this some audit step, then it very often catches its own mistakes. And I get questions all the time from people who are really rigorous or kind of know that these models have a lot of bias built in. They ask me, how is it going to critique its own work? It's bias toward its own work. The truth is they do find mistakes. They find mistakes all the time. And I've tried this kind of splitting the process between tools too. But there's actually using it in the same model that you're doing analysis and does find issues. So let's see if it found some things. So yeah. So in this particular prompt, actually, let me bring this back up again. This version of a verification check, I'm asking for contradictions in user statements. So it was very common that one person in a transcript will be telling a story one way like they say, well, first I got my shopping cart when I was going grocery shopping and then I looked for the apples or whatever in a certain order. The second time around in a different part of the conversation, they tell the story of the user journey in a completely different order. Or they contradict whatever they said earlier in some way. Well, I use this every day. The equivalent of like, I go to the gym every day or five times a week and then, oh, one was the last time you went to the gym. Oh, well, it was probably Saturday and it's Wednesday today, right? So very often, if you're doing a good interview, you find some contradictions. in what someone does and says or what someone said the first time around and then what they said the second time around. But AI is very likely to blur that kind of cherry pick the stories that it likes or that match what you asked for most most closely. And forget about or ignore the second thing they said that is actually the opposite. So I'm putting it through this pass where I'm checking for contradictions and that will make the full analysis in the end much more bulletproof kind of preventing the cherry picking of the story that best aligns and getting the full picture. So to spare you the details, I think people can look at this on the screen, stop the video if they want to. But I'm basically asking for for contradictions and I'm defining what contradictions look like to me. I've even spelled out what is not a contradiction and a few examples. So I'm consistently giving examples definitions and then what that looks like to me to be really, really clear. I see a few contradictions here. So something was re revised the rating was revised from let's say original no this is confirmed sorry this one is moderate risk in the first place like at a risk of churning and then it was revised to high risk so upgraded because it double checked what the person's you know with saying the stories they were telling and actually realize that there's more risk involved. So this is the sort of correction that I'm looking for. I want to make sure that it has combed through the data again made sure that the claims, the statements that it made were actually verified. You know what kills momentum spending 30 minutes on a brainstorm with Claude and then copy pasting it into a slide tool to create the slides that loop is dead gamma now works inside Claude through Claude connectors. So from thinking to a finished presentation without ever leaving the conversation here's what that actually looks like your brainstorming a product strategy in Claude mid conversation you say turn this into a deck gamma generates it right there you keep refining your thinking in the presentation updates with you no copying no pasting no starting over in another tool and it gets better connect Gmail notion or get up alongside gamma and Claude will use those connections to pull live data straight into your slides weekly business reviews feature PRD's all generated from the data. You're already using your deck finally keeps up with your ideas right now connect gamma to cloud at gamma dot at go from thinking to presenting in one flow. Claude helps you think gamma helps you show it. Today's episode is brought to you by the experimentation platform Camille. Nine out of 10 companies that see themselves as industry leaders and expect to grow this year's say experimentation is critical to their business. But most companies still fail at it why because most experiments require too much developer involvement. Camille handles experimentation differently it enables product and growth teams to create and test prototypes and minutes with prompt based experimentation. You describe what you want Camille and builds a variation of your web page lets you target a cohort of users choose KPIs and runs the experiment for you. Prompt based experimentation makes what used to dig days of developer time turn into minutes try prompt based experimentation on your own web apps visit Camille and dot com slash prompt to join the wait list. That's K M E L O O N dot com slash prompt today's episode is brought to you by amplitude replays of mobile user engagement are critical to building better products and experiences. But many session replay tools don't capture the full picture some tools take screenshots every second leading to choppy replays and high storage costs from enormous capture sizes others use wire frames but key moments go missing creating gaps in your understanding. Neither approach gives you a truly mobile experience amplitude does things differently their mobile replays capture the full experience every tap every scroll and every gesture with no lag and no performance. It's the most accurate way to understand mobile behavior see the full story with amplitude. So this so we increase the certainty that it hasn't hallucinated and it's giving us the right analysis based on the quotes exactly so it's a bit of an analysis check in the first place it's also an exaggeration check in some cases you can run an audit to make sure that you're staying as close to the data as possible. That it no statements were made by AI in the process that are a bit over or under exaggerated. So you can use it for lots of different things. This is just one version of it. Excited to see where we go from here. So I would end this process for this kind of very condensed workflow version with synthesis. I think if it's okay with you, I would just skip synthesis step because I think most people know how to do synthesis most people would just say summarize this or give me the key themes the top three action items that sort of thing. And I would hop into survey analysis. Does that work? Let's do it. Okay. So let me open a new window. The key takeaway here what we did is we didn't jump into synthesis we did this whatever you want to call it two three four step process to make sure that we increase the certainty in the results in the quality of the analysis and replicate the human process. Exactly. So this is what I'm dealing with here. This is a very basic turn survey, but pretty similar to what I've ended up working with in the past. We've got some automatically logged information. We've got a column like reason text where the customer chose a category from a drop down and these are a lot of other categories. So this isn't particularly well organized in the first place. Then we have a reason detailed text and this was an open text field where people could just say something about why they were canceling pretty basic. And then we have what tier of service they were on at the end. So that's the file that I will be uploading now. Like what all of you guys have. So let me get my file in here. Just dragging it over. And what most people would do here is similar to what I said before kind of toss in a CSV file say show me the biggest pain points from the column with the open open responses or count all response types in this other column where we have the type of cancellation logged. But that's actually not what I want to do here because a lot can go wrong if you start there similarly to the interviews just jumping into pattern finding a lot can go wrong. Lock and be missed and a lot can be miscategorized. So when we're working with survey responses or short customer feedback, we want to code first. And for people who don't know what does coding mean. It's not software engineering. So definitely not I'm not a particularly technical person. So definitely not that type of coding. We're talking about applying labels to things putting things in buckets that help us make sense of them. So normally if you're doing data analysis, you're looking at all of these responses. And there is an open text response field like, you know, customer feedback form or feature request form that sort of thing. And it's kind of makes sense of what people are talking about in those message fields in those message messages responses and label them with something before you can start grouping and saying well most people can play in about this or most people are requesting this. That coding needs to have an approach right you need a bit of a methodology for the coding in the first place. So you do inductive coding where you look at the data available and you try to make sense of it from what's there. Or you can do deductive coding when you already have a taxonomy. You already have parts of the product and you want to label all of your data with those those features or customer types or things like that. And you already have something that you like to use as labeling. But in this case, I'm using inductive open coding where I'm saying let themes emerge from the data rather than forcing preset categories. And what's important about this, they very often use inductive coding at least at first. And let AI try to see what I may not have noticed about the data and apply codes from a more open perspective. First, if you just apply your own tags and say here's the list of tags I want you to code this with what very often happens is it will start force fitting will take the quotes that it sees and try to make them make sense in the set of kind of required buckets that you gave it. And it doesn't lead to the most accurate coding and analysis. Makes sense. This goes back to the craftsmanship of research that you want to be applying. And so we have this first step here where we code. And I'm going to let it start working because it will take a couple of minutes on this. But I also have rules here. So I'm not just saying go ahead and code this however you want, you know, be open, find some interesting things for me. And also saying each response needs one primary code they need to be mutually exclusive not overlapping. I want really tidy little boxes. I want all of the quotes to be in one box. And I want to be able to categorize right in a way that I can make sense of later, not having too many kind of messy overlapping things. This is really up to the person doing the analysis with AI you could, you know, anyone watching could decide you would take a completely different approach and that's fine. But what's most important here is just that there are rules that there are criteria that there are kind of lines that we need AI to paint within. We don't just kind of let it run this however it feels like running it. And I'm saying while it's still working on this most likely. Yep. I finish with a line that is maybe becoming obsolete but not quite yet. I've always included this as a bit of a fail safe in the past. All of the models have made mistakes with math calculations and a lot of the time the calculations on a data set like this come from sometimes. running tasks with math in text instead of in code, which is a really goofy thing because we know that all of these tools have been coding for many many months at this point, but if you know they are originally text-based machines and so I've seen so many cases in my students projects that they've submitted and in my own work of saying you know calculate how many people said this and then seeing that the answer is different if you prompt it three times because it doesn't do it in an actual math equation. Yeah, it's they're notoriously bad at math unless you tell them code is coding for. Yeah, so I'm hoping I can remove that in my latest tests with all the models I you know got really consistent results without having that in as well, but I still am a little anxious about taking it out so I I just leave it in all of my prompts. It's better better safe and sorry. Good take away for anybody watching if you want to be safe on your math tell it to code it. What I love about Quad these days is how much of the thinking is exposed to and how much planning it's doing and how much step by step and how you can even click into some of those and just watch it work. I find myself just like getting fascinated with this and getting better at coding and stuff like that by doing that. Yeah, and you don't really need to be a Python expert here to just see you know how it's thinking through things like team-specific issues. It's already kind of providing an overarching code group here in voice reimbursement primary primarily a team issue and then you know logging if you find any of these terms then code it like that right. So this is how it's thinking through I found these topics and this is how I'm going to start coding them. Seems to have brought a lot of the Claude code functionality into the Claude user interface. So if people have just been stuck in Claude code the user interface version has gotten quite a bit better in the last couple months. People are curious it's almost always in an import pandas which is like a really common data science library. So as you watch this thing and learn you get to learn a little bit more deeply too. Yeah, exactly. And so if you're doing this as a human what would have been the process? More or less the same. I mean again what I'm showing you is a bit of a condensed process compared to what I would usually do. I have more steps involved. So one way I might have done this as a human or with AI as well is actually to break this prompt into multiple steps. So human wise, right, I would be reading all of the responses. If we didn't have a code book already, if we didn't have a taxonomy or a set of tags that we wanted to apply and we really were doing this from inductive open coding, I would read through all the data or you know a sample of it at least and start coming up with the labels that I think most apply from what I see. So that's this creating a code book. Like what are the labels that I'm identifying the codes? Why do I think that they should exist? What's the definition of them? How would we know if something else should fit into this group or not? Giving some examples and then applying that code to other responses? You could even say in between these two there's a step of testing it like taking another sample from the data set and checking do these codes that I just came up with actually apply to this new sample or am I missing something? Right. So you're doing this kind of chunk look at it, create codes, apply it to the next chunk, did that fit? Yes or no? Okay, iterate, keep going. This is really how I would have done things manually as well. So I've just laid out the steps that I would follow and I've given them to cloud. But here then is where things do get a little bit better if you have slightly more complex stretched out process where you separate these steps into individual prompts like first going through the data and thinking about what the code book might be and then I could give it another set of data and say do they still fit for this for example and so on? No, it's okay and looks like it's got a response for us. It's got a response. I'm going to close this version because it has things down here hopefully. Yeah, it's still generating. But so dominant turn driver is clear 23.5%. What I actually am more interested in is this. So there's some quantification. I want to see the code. I see usually it gets me a code book. This is already so much more than I would have done as a VM. This is cool. So you've got a bit of the code book in here and what's funny is it didn't give me a code book usually gives me a code book. But it has basically coded everything again. So I have a downloadable version of my CSV file with all of the codes applied to it. So every single line every row is now coded. And then I have my code book here. I think this is the code book. Usually from this sort of prompt it gives me a code book. It gives me the recoded CSV file. And then it also gives me it's kind of summary of the analysis. So that's what this is. What I actually don't need yet from my perspective is this full analysis. But I want the quantification. I want to see how it like first how it coded things the codes that it came up with. To see do I agree or do I feel like it coded things in a funny way that I wanted to iterate on. And then I want to see the how you know how many responses it coded as each each of those codes. So missing favorites or save functionality because this fictional app didn't have a way to kind of save your favorite meditation sessions. So you can return to them. This was a big chunk. So 23.5% and yeah, that's that's like a clear that rises to the top. And then we have some other things like switch to competitor content gaps didn't resonate. But to be clear, what's really important about the way that I structured this prompt is actually the order of things. I'm forcing it to go through the coding and the analysis in this order. I'm not saying, hey, provide frequency counts. I'm saying first look at the data, then create the codes, then apply them and then decide what the frequency counts are. Because if you jump to give me some patterns, it hasn't coded anything or made sense of it in a way that you know we would agree with or make sense to us before it starts counting things. And that doesn't make sense. I think this is a more generalized advice for anybody watching this step by step prompting is how you get the AI to follow your human process. And that's our north star here. Yeah. So let me jump into the next step. You can keep asking me questions about that if you want to, but just to keep things moving. Copypasting prompts. Yes, if the last step was coding. So this I'm actually adding a little bit of a step here for for the survey analysis because I want to compare this a little bit with, for example, sentiment analysis. A lot of people for a lot of people sentiment analysis is a natural thing to do on survey responses like NPS or turn. The problem with with sentiment analysis is and any even a prompt like this is that we're assuming that a large language model will understand sentiment and rate things in the way that we would. So first of all, with any rating task like does this fit into, you know, negative positive or neutral sentiment or, you know, which buckets to do customer statements fit into, we need to explain what we would rate things at or how we would label them. So that's what I'm doing here. But also with the turn survey, for example, there's a pretty strong likelihood that a large language model will decide that everything is negative and that doesn't really help us. So instead of saying, hey, categorize our group things and by sentiment, I'm having it add intensity ratings so that we can make more sense of which of the problems that are causing turn are actually most worth us working on as a product team. So if people are having a soft exit where they're circumstantial like things in my life just changed, like there's no hard feelings, I'll come back if things change back again. Well, this isn't necessarily something that we're probably going to want to fix. Whereas if something is really frustrated, if it's, you know, a group of people who very much fit in our ICP group or they are high value customers and they're frustrated or angry, then these are probably the things we want to focus on. And what's happening in this calibration example? So all of this is about how to rate things. What do we call a soft exit versus an angry exit versus, you know, a vocal detractor? What's the difference between frustrated and angry? This, it would be making a mistake if we assume that Claude or any of the large language models know the difference or interpret the difference the same way that we would. And this is where it gets a little bit subjective. But, you know, as someone doing a lot of research, I have beliefs about some of these things or I have an idea about what an angry customer looks like from my domain, right, like from this flow app or whatever I've been working on. I know the difference between angry and frustrated or you could kind of come up with your own definition. So I want it to follow my lead or the way that I've grouped people in the past so that I have comparable results, something like that. So it's really important to me that it doesn't just kind of come up with its own rating scales or ways of labeling things, but that it kind of follows, follows my lead and way of thinking. So when you're defining something, put some few shot examples so that it's very explicit and clear what that means for the element. That's when I'm taking away here. Is that right? Yeah. Yeah. And what's actually really important about few shot is is reasoning, not just showing what the example looks like, but why as well. Okay. So giving it that reasoning so it can replicate you. So here today, it's putting things into text T files, which is really funny because usually it outputs everything in the normal view, but here we have analysis here. But we see most people are clustered in these like level two, three and four here. There are far fewer angry exits than neutral and frustrated, which at least is a good thing. But we want to see how they line up how they stack up with these different codes as well, right? So this is kind of a stacking of prompts, getting more specific or more detailed in our analysis or adding a layer to the analysis so we did coding. And now we're determining the emotional strength of these different codes. So this is the point here. What I wanted to see was, okay, how frustrated are people who are missing favorites or save functionality, for example, pretty frustrated. I mean, it's in the middle level three, but there are more level three than anything else. So that means they're pretty frustrated. And that would say to me, hey, this is something we need to work on. Makes sense. But we have all of the codes ranked by intensity. We have a little bit of a visualization because Claude decided to go a little bit beyond the brief here. I didn't ask for any visualization, but hey, I'll take it. So overall, you're happy with these results? I am happy with these results because I can also check them. So because I had that exportable CSV that was recoded for me, I always ask for these kind of outputs at the end where I can double check, first of all, that from the interview analysis that the quotes are real. If it gives me, hey, this quote came from this participant on this page or it was quote number, whatever in my markdown file, then I can go back and I can make sure that that quote exists, that it came from the participant. It said it did. And in this case, when it quantifies things, but it's already given me the full set of coded, recoded responses, I can go do my own math there too if I want to. Awesome. Where do we go from here? So we still need an audit that verifications step. Just like we did with the interview analysis, we need to check or we need AI to check its own work here. So it's going to select gray number responses, check for code consistency. Yeah. Okay. What's really important here is simply, first of all, simply that you do a step like this, like I said before, we just need to force it to kind of do another pass through its own work. And it will usually find something that's a little off. Sometimes what it finds is some big calculation errors. It's rare if you have really good prompts and you push it through this step-by-step process, it's rare to find big calculation errors. Sometimes they're small though. Sometimes it goes, hey, I would change the coding a little bit here because I think there's another group that deserves another code that I kind of mixed up in some other groups. And then, you know, the quantification changes a little bit. So that's the sort of thing I want to see here because I want to stay as close to the data as possible. I don't want to exaggerate things. I don't want to blow up responses to super angry exits, you know, from the previous step. If they weren't super angry exits, I want to be as realistic as possible. So it's, for example, here, it's rechecking level 4 because I asked it to specifically check level here. For the intensity rating audit, recheck all responses rated 4 or 5, we don't want to go kind of running around saying, oh, people are super, super angry, and we have to fix this thing right away because they're all level 4 or 5. We want to know, is that true? Like, should this actually be priority number 1? Are they actually really angry? Or did we over exaggerate a little bit? I think this is a really interesting way to use AI. I'll call this the CYA. You wait to use AI. Cover your ass. What is anything that could come back to bite you when you're using AI in the project? Have AI check that out. I love this lesson. Yeah, CYA. Yeah, that's a good name for it. I've heard so many people, it's less common with researchers because I think they're, they're feeling more cautious with AI and a bit more protective of this work naturally. But I've heard a lot of product managers I've worked with and friends and designers as well, kind of say, like, yeah, I came up with all of these really nice sounding insights. And then when I went digging into the data myself, or I presented it and someone asked questions, I couldn't answer. And, you know, they didn't do a step like this. And if they had a bet you that a few of the issues, a few of the the holes that could be poked in the insights would have been highlighted here. So if we review the three prompts so far, the first prompt was coding. The second prompt was quantification and the third prompt is verification. Is that the right summary? So the first prompt was quoting and quantification in one. The second one was rating the emotional intensity, you could say. And the third is auditing the results of everything so far. Got it. So here you can see it's corrected things. It says survey final corrected at the top. So it's recoded potentially the primary codes and also the intensity levels here. Let me see. So corrections made the coding eight corrections. So some things were missing. Oh, missing favorites was the old code and the new code is actually lack of control over selection. So it's gotten more specific about the code that it feels should be applied there. And because it's lifting this up and because it had recoded the whole thing, originally, I can go through this myself and I would in a normal workflow scenario. I would go through when I would double check, do I agree with these? Like, based on my understanding of the data, do I agree with these and would I kind of label them the same way? And I think this is a really important step because it gave us a preview of the synthesis even though we didn't ask. And it's synthesis was like missing favorites is like driving all the churn. And I was like, wow, like if that's true, then you can really quickly solve a lot of your journey problems. But going back and coding it, it realized, oh, there's some nuance here, beyond just favorite. And here exactly as I expected, the intensity rating for at least two things was redone. So it kind of over exaggerated the frustration, right? It dropped things down from some from angry to frustrated, which could be meaningful, especially if it found more issues like that. So now we have got it to check its work. What is the next step? So from here, same as with interviews, I would just do a synthesis. Like I said, I would say wrap up the top the top issues that are worth fixing. I would maybe ask for things per tier, like per segment based on the service tiers that they were using, like the people on individual premium accounts versus team accounts and things like that. Because those might be a little bit different. We see things like billing account admin issues or technical issues, right? That might be across both of them, but billing an account would only be with team account users, for example, or proper invoice issues, things like that. So I might do a synthesis like that. But I think again, most people are decent enough at asking for a wrap up of the insights and not as good at pushing through this. So I would jump to the mega version of this, I would say, which is if it's okay with you that I skipped synthesis again, I would judge. Yeah, I know. I think people can imagine it on their own, right? And I'd say like how a PM and a user researcher do it might be different too. And so that's build it to your own use case. Like as a PM, I'd probably be saying like, you know, okay, draft up my first version of my product strategy document based on this. Like what are the key problems we need to solve across those? Please put one or two quotes from the specific survey and make it link back a bowl so that I can go trace it if needed. But a user researcher might have some different goals. And so build the synthesis prompt for your goals. Exactly. Yeah. And all of these prompts can be modified for your goals, right? I've done this in a particular way with, you know, retention and churn as the focus, but this can be used for any kind of analysis you're doing, any kind of data sets. It's really it's supposed to be a flexible model that's kind of why I developed it this way, but I developed it like I said, based on the way that people like me have been doing analysis for for decades anyway. So it's not not reinventing the wheel here. Awesome. So what does the agentic paralyzed mega version of this look is somewhat scary, but awesome cloud code is here. What you just install it with like a simple command, it's like NPM something something you might need to install NPM. In this case, it'll have you use home brew to install that. So it can feel a little bit intimidating to install this. But once you have it installed, you just type in the word cloud and you're here. Yeah. So I'm not using anything like a special code brewer like VS code or anything. I usually use the the terminal exactly as it is, especially for demo videos because most people who aren't used to using the terminal will download it and look at it like this. So I want to kind of keep it consistent. What's worth knowing in the first place is you have slash commands and agents is what I'm going to be working with right now. So if I just look at my agents, I have a bunch set up already. I kind of add new ones to leave them and reiterate them all the time. But what I'm going to be using right now are my survey analyzer light and my interview analyzer light. And this is exactly what you just saw. I've pretty much set up the same workflows or mostly, but just to work a little bit more on autopilot. So I set up those agents beforehand. The only difference between that or the biggest difference between what you've seen me do step by step in the web UI versus this is I have programmed those agents with system prompts that refer to and retrieve information from more files. So instead of being in the web UI and just uploading things every time and having an individual prompt for everything, I actually have a context document that it's going to retrieve from instead of me prompting, hey, here's my context. And I can change that context over time so I can kind of use this agent for different things and update the context when our team information project information customer information, whatever it is features that changes. I can keep that updated, but it's kind of doing things a bit more automatically than my step by step process. in the web UI. - So just to summarize for folks, I heard three inputs. Your context document to summarize the context prompt we saw, your agent markdown file, in this case, she's got two. She's got a survey and an interviewer, like agent markdown file, and then your actual data or responses. - Yeah. And for some reason, my pasted prompt wasn't showing for you. I really want you to see it. Any second. There we go. So what I'm saying here, this prompt, what you see is really simple, compared to the other things, right? So what I just said was, I'm basically loading all of the steps automatically in the system prompt. It built in a little bit of extra kind of feedback loop type stuff in there as well, and then it's pulling from documents. So I don't need to break this up into individual prompts. I just sent it one prompt to kick off using my pre-programmed agents. I'm calling the interview analyzer light here. It's not a slash command. I just write it in and the survey analyzer light. And I'm just basically telling it to refer to the markdown files here. And I can hit enter, and it will just in a moment tell me, okay, it's starting that. So it's reading the markdown files that I just referred to for each agent. So it's searching for them. This happens all the time if my paths, if I didn't give a file path here, and I just name the markdown file, and it has to go hunting for them. Sometimes I write the full path wrong too, and then it has to go hunting for them. But the great thing about Clawd is it will troubleshoot that on its own, so you don't normally have to go, oh wait, go look over here, it's there instead. It will just go find it in your files for you. So it found them. Sometimes it might ask you for like a permission or something, usually just hit like two I think, and then it'll just keep going forward. Yeah, I have something I don't necessarily recommend to everyone, but you can see at the bottom this red text bypass permissions on, so you can actually type in a command to remove that request, which means unfortunately, if I were to go beyond this prompt, it can kind of do anything at once on my computer and destroy all of my files. So I don't recommend that you do that every time, but for a demo like this, it's really helpful because it won't ask me every five seconds whether I really want it to do the next action or not. And I think it's called YOLO mode, right? You can just type in like slash YOLO. So you can see it's hunting through my files in my Dropbox projects, AI tests, live demos, Akash, right? I have these files set up here, so you can see the paths, and it's reading everything it needs to, based on what I told it to do in those system prompts that you didn't see. And it says, "Now I have the paths here for interviews, and it's reading all of the files with the names here, so I can make sure that it is read that the right interview files." What's really cool about this, I mean, the reason I'm showing this in the first place is you just saw, me do these analysis processes where, you know, if I weren't talking so much in between the survey analysis and the interview analysis processes individually for that amount of data and that number of prompts, took about 20 minutes each. But that means you would spend 40 minutes on that, which is still remarkably fast if you felt that that was complete with a short synthesis prompt afterward. And usually it takes me a little longer than that because I have an even more rigorous process with more steps and feedback loops built in, but that's still just really fast. This allows us to do both of those things in parallel. So because the interview analysis and the survey analysis here don't build on each other, they don't need to be sequential. So instead of saying, "I have one hour to analyze all of this stuff, will I have enough time?" Well, yes, you have enough time if you do it like this because you just let your agents with a pretty solid process do both things at once and then you can triangulate on your own at the end or you can put that into AI and triangulate for you, but you're cutting out half the time by parallelizing this. And how did we parallelize it? So simply because I have agents set up that run the processes I went through manually in Cloud in the WebUI, I can call them both here. So in this prompt where I say number one, use interview analyzer light, this has all my prompts that I just showed you in Cloud plus a few extra feedback loops. And so it's basically saying, "Do that what I did in Cloud's WebUI and then also do the survey version, but do them at the same time." Nice. Yeah, sometimes I did this like out open this in cursor and I'd have multiple terminal windows, but I think this is actually even cleaner way. It is really clean. You can just follow the whole thing here to there will be sections of the process where it says expand with command, "Oh" or something like that. So you can really look at the whole process it's going through just like you would in the WebUI and make sure that it's kind of following the instructions, doing things properly. And I've told it to output everything, like output the final analysis to markdown files as well. So I'll be able to see them in my files and kind of review them there. I also kind of like seeing how many tokens. Yeah, you always want to know me tokens are running. While this is running, could we look into one of those agent markdown files maybe and start to teach people about what those look like? Yeah, gonna stop sharing now. So this is, I mean, this should look familiar if anyone really stops and looks at the prompts that I've used. So this is the interview methodology markdown. And I've got the project context and the product context and everything that was already in those prompts before, right? So I'm just feeding it all of that background information in this file and then also the definitions of contradiction types, things like that. So all of those pieces of information where I'm trying to teach it to analyze or identify information or label it the way that I would. That's basically built in here as a sort of context load that it can retrieve from. And does an agent markdown file need any special formatting or instructions? I mean, it depends on what you're trying to do. There are ways of getting a little bit trickier about it. But generally, it's still all grounded in being specific, being clear and specific just the way that you would be in prompting. And for markdown files specifically, it's about structure, right? So the reason we use markdown files is really to help the large language model retrieve information in structured, organized ways that will make it easier for it to do the process as faster and more accurately as well. OK. And Claude can help you create these agent files. If you guys are nervous about them, you can put in some of these prompts and say, hey, create me a nice interview and analyze their light and it'll work with you on it. Exactly. So let's see how Claude is doing over here. Still working. You can see it's running interview analyzer light here. So it says both agents are still analyzing the data. Let me wait for them to complete. Generally, do you find yourself using Claude code more or the UI more for this these days? It really depends on what I'm trying to do. But if I have everything based in my files, then I find Claude code in the terminal to just be a sort of more efficient way of doing things. I'm like I said earlier, I don't consider myself a super technical person. I have a marketing and then design and then research background. So I'm not coming from coding. And once I got used to this, like the whole interface, not being particularly, you know, design-y. I started really appreciating just being able to use slash commands to kind of call things, start tasks with a couple of keystrokes. I think it's really interesting to automate these processes that I would normally do step-by-step manually, but just learn how to make it call the right information from different documents. Or the fact that it can find the right information within my documents without me having to go, "Oh, where did I put that?" Or even saying, "Do we have, you know, for repository cases?" Where someone's saying, "What do we already know about X?" Well, if you have Claude code and it's actually synced to all of your company files, your project files, then it can just go figure that out for you. You need to know how to prompt really well. You need to be clear and explicit and you need to know how to structure documents in a way that makes it easier for your large language model to call all of them. But, you know, if you figure that out, then a process like this goes really, really fast and much easier. So interview analysis is complete here. Survey analysis is complete. And it has a few quick findings, but what's more important is that it has output things to a file. So I'm gonna stop the screen share here and I'm gonna go over to where I hope it saved it. So here's our meditation app premium user retention analysis. There's an executive summary in here because of my instructions. So we have how many stable subscribers, moderate wrist subscribers there are, and so on from the interview analysis, right? So this is my interview analysis results, markdown file. Phase one data confirmation, phase two individual retention analysis. So there's a little bit more in here, but it's basically doing the same thing. We have the value anchors analysis and the results here. This particular therapy chat feature directly addresses a gap that traditional therapy couldn't fill. And so on, we've got quotes and timestamps from this participant. So it's done exactly what it did when I was doing this step-by-step in the browser UI per participant, political. out everything that we've got pp2, part has been 02, and all of the value anchors from this person. It's all of this the output you've already seen, but just now in a markdown file. So what I usually do from here is I kind of form out this differently. I put it in pages or Google Docs or something, and make it look a little bit nicer. And then I'll pluck out things that kind of stood out to me. It has that executive summary of the top. So most of the time that's pretty ready, I can kind of copy paste that and share it with someone. But it has all of the details that I would need if I want to review this on my own as well, like you saw in the browser and even some tables. Epic. I mean, this is a sick workflow. And I have another. Sorry. Oh, go ahead. I have another file like this for the survey analysis too, just to be clear. Cool. So if I'm to summarize for you guys what we covered today, we gave you a master class in how to do discovery with AI discovery is the most important skill for building great products. And we've covered examples for how to do that both for surveys and for interviews. And we've showed you how to do that with cloud step by step using multi step proffering or to use cloud code and build an agent to do it. Now you have all the tools. It's up to you to go apply this, see where you get stuck, drop in a comment below on this video or find Caitlin online on LinkedIn or somewhere else, maybe take her course to learn deeper and really embrace this new way because the best products are built with the best user understanding. And this is your roadmap. Caitlin, thank you so much for sharing all these valuable insights. For free. It was my pleasure. Thanks for having me. See you later, everyone. I hope you enjoyed that episode. If you could take a moment to double check that you have followed on Apple and Spotify podcasts, subscribe to YouTube, left a rating or review on Apple or Spotify and comment it on YouTube. All these things will help the algorithm distribute the show to more and more people. As we distribute the show to more people, we can grow the show, improve the quality of the content and the production to get you better insights to stay ahead in your career. Finally, do check out my bundle at bundle.akashg.com to get access to nine AI products for an entire year, four free. This includes dovetail, mobin, linear, reforge build, descript, and many other amazing tools that will help you as an AI product manager or builder succeed. I'll see you in the next episode.