Speaker 1Today we're talking with John Nerona, co-founder and chief product officer of Gamma, and we're going to talk about agent harnesses, all the opportunities around it, vertical agent harnesses. I can't wait to dive in and I want to give a special shout out and thank you to Anthropic, the team behind Claude, for sponsoring today's episode. John, thank you so much for coming on the podcast. By the end of this episode, what are people going to learn?
Speaker 2Thanks for having me, Greg. I'm really excited to be here. Today we're going to dive deep on what it means to actually build an agent in practice, including demystifying what is an agent harness and what that looked like for us at Gamma, which is a creative agent harness, an agent for visual communication.
Speaker 1And not just, you're being humble. Gamma is, I mean, it's grown extremely quick. I think it's hit a hundred million ARR. You've just relaunched the platform. Huge, huge release. And there's a lot you're learning. And what I hope to get out of this podcast is really just little insights around opportunities and agent harnesses, what you're learning in real time, and other opportunities that exist, startup ideas and stuff like that. So John, can you commit that by the end of this episode, I'm going to learn some of those things?
Speaker 2I'm in. Let's do it. Let's make it happen.
Speaker 1So I want to start off by, like back here, you can see what is an agent harness? Because I think a lot of people. You know, scroll X, they see agent harness, but they don't really understand what it is. So can you clearly explain what it is?
Speaker 2Absolutely. Let's break it down into the two words, agent and harness. And I'll use maybe Gamma's journey to illustrate both. So for everyone's context, Gamma is a tool primarily for making presentations, also documents, websites, social graphics, that sort of thing. We've actually been around for several years, long before agents were ever a thing. We started the company in 2020, really took off in 2023. And early on, we were just making these one-off LLM calls. So do things like write a single slide or make the outline for a deck. And we very manually strung them together into these preset workflows to do things like generate a presentation with 10 slides. The big revolution that's happened in the last year or two is this agent revolution, where the most powerful AI products now are not stringing together these individual LLM calls by hand, but they're actually having one higher level loop of an LLM run that will do some kind of work, inspect it, use tools, and then turn that into something over a whole series of executed calls. And so we actually went through this whole journey over the course of this year of rebuilding our product to be an agent. And building your product to be an agent involves building a harness. So the harness is all the tools that agent uses to execute things in its environment, inspect its own work, and deliver something finished to the user, including soliciting from the user what their actual preferences are. So probably the most famous agent harness is something like Cloud Code. Cloud Code is a harness for programming. Cloud Code teaches the agent how to go through all the files on your system and see what's there, teaches it how to run tests, and even prompts it with certain rules, like go into plan mode in certain cases, or run these tests after completing execution, or do this stuff on GitHub. In our case, Gamma is not a coding agent. It's a creative agent. So it needs to do things like understand your art style and creative preferences, help you craft a story. And we care a lot about visual design. We're doing things like generating images and laying them out. So we know how they look and make sure they look right. So our harness takes all of those things and turns them into tools that an agent can use. And I can talk through what that looks like.
Speaker 1Yeah, I mean, please do. And I think also, I'm curious if you agree or disagree with me. I've been saying agent harnesses are the new GPT wrappers or new wrappers, right? Is that just a good tweet? Or is there some truth to it?
Speaker 2I think there's truth to it. Although I always disputed this concept of the GPT wrapper originally, because it sort of implies that 90% of the work is the LLM. And then there's just this thin layer on top, which is the wrapping back and forth. I think maybe what harnesses illustrate is that actually the wrapping is most of the work. You actually have all these LLMs, which are quite similar in their core capabilities, not to say they don't have some variation. But generally, all the magic of turning them into a product is building the harness around them. And even the model companies like OpenAI and Anthropic are putting so much of their work into harness development as well. So they are themselves doing wrapping of these things to achieve different capabilities.
Speaker 1Could you walk me through? What happens in 30 seconds after someone types a prompt on Gamma? How many models does it touch realistically? Because I think a lot of people are not sure how that actually works.
Speaker 2Let me walk you through it. Let's go through a sample prompt. So a very typical use case for Gamma is someone will paste in a bunch of messy notes or a document or PDF and say, help me turn this into a 20-slide presentation. So the moment you put that prompt in, a whole bunch of things actually happen under the hood. The first thing is we actually spin up an entire computer in the cloud. This is what they call an agent sandbox. This is what the OpeningAI ones broke out of famously in the last few months. So we actually spin up this entire sandbox where the agent gets all these files representing the user's prompt, the file they brought in, also other context about them. We then prompt the agent to go look through all that file and their prompt and run certain tasks. And for us, actually, one of the first tasks is usually to figure out what other information we need from the user. And so typically, we will actually pop up a form and ask a few questions. Questions like, what's this actually for? Is this an educational presentation for school? Is this for adults? Is this for your work thing? We will actually call out often to different connectors. So we've built integrations into tools like Notion or Slack or Linear, where we can actually pull in all this relevant context. Just already in that first maybe 10 seconds, we've now made calls using probably three or four different models, including expensive and cheap ones. We'll talk about actually how we manage costs as well. Just to sort of source information and ask a few questions, we've rendered a UI around it with some generative UI to actually show you what those choices are. Then we're going to actually write an outline for your deck. That's usually several more LLM calls. And we will style a template for you. So we have a library of templates with different visual styles. We're going to use vision models, try to figure out which one's a good fit for you. We're going to do some cool search embedding things to sort of look at the space of thousands of templates and find the right one. And even customize it if you said, yeah, but I want it red or blue or whatever it is. Then we're going to, once again, show that to you and let you edit it and inspect it. We haven't even started making the deck yet. And already we've probably made something like 30 or 40 different LLM calls to maybe four or five different models doing different tasks. Now you're going to hit that generate button and we are going to build up a presentation for you before your eyes. And there's actually a lot of pieces involved in doing that. We have to write the words on the slide. We have to lay out all the things. In our case, we generate a lot of imagery as well, whether decorative or more substantive. We're often making diagrams or infographics or charts. So actually all of those are now more calls that are going out to different models, often in parallel. There's a huge performance consideration because if we did these one by one, it's going to be quite slow and you're going to basically leave before we ever finish. So we do a lot of work to make it all pretty speedy. And then in the space of about a minute, you're going to actually see that whole presentation streaming in front of you and generate. And now we've made probably close to a hundred different model calls. And I would say probably 10 to 15 minutes. And then we're going to do a lot of different AI models have been used somewhere along the way. And we're still not done because you probably won't stick with the first draft. In fact, we don't stick with the first draft. After we generate that first presentation, we're now going to have a AI vision model look at every single slide and look for problems that it can find. Misaligned text, the person's head got cut off in the image. There's weird contrast. Something's not padded right. It's actually amazing how many things can go wrong in something like a simple presentation that a human eye can detect right away. Or sometimes a human eye can detect a problem right away. And then we're going human eye can't detect because it takes a designer's eye. So not everyone can, but we can train models that can catch that. So we go through this whole round of visual QA. We actually will do multiple rounds of fixing things based on what we find in that QA and then redo them. And then after all those rounds of fixing, we will finally serve it up to you. And now at this point, we've probably used 20 different models making like maybe 200 different calls along the way to deliver what looks to you like one finished output.
Speaker 1So what you've just described to me is. Like a workflow in a particular niche. In this case, it's, you know, creative output. Totally. You started with presentations. You've now moved, you know, it's essentially a creative suite. Yep. You know, where do you see opportunities to create agent harnesses outside? You know, if you were starting a company tomorrow, where do you see an opportunity to create a vertical agent harness? What's an example of a niche that, you know, you see underexplored?
Speaker 2That's an interesting one. I mean, you know, just leaning on my experience, I think where we see a lot of value of making our own distinct agent harness is when that problem looks very different from just plain old coding. In our case, what makes it different is that it's very subjective. There's a lot of judgment calls to make along the way where two different people will disagree about what good looks like, which means you want to have the human in the loop. And also where the person using it actually really cares about the specifics of the output. And so I would think about maybe those two traits as something that go into like a QA. Wow, these are a good case for making your own agent harness. So just thinking loud. Some examples are like, I know a lot of people are making games with cloud code. And I'm really bullish on this idea of like the personal video game where you almost like vibe code it yourself. Coding agents are already really good at making video games, but I think they're often not good at actually sourcing the player's opinions on where it should go and where it should evolve. And if you think about, you know, things like Roblox or Minecraft, there's all these things where people are actually shaping the world for themselves. And usually it's like non-technical people who are not coders themselves, I wonder if there's a harness to be built there. Where, in fact, the whole harness runs in like a 3D world or something. I think people should really think broadly and try to, you know, in general, my philosophy of business is try to be different. Try to do something that looks nothing like what everyone else is making. And for us, that's been a challenge because we try to be different, but then others, you know, copy us or do the same thing. And we have to keep looking for more and more different.
Speaker 1I think that's, you know, it's not even just for you guys. It's anyone who's building anything in software today. As soon as you catch lightning in a bottle, people are copying you. But I think the thing that people, I think the reality is they're copying where you were three months ago, six months ago, right? So you as the founder have to be like, okay, where do I, what's my strong point of view? What's my thesis on where the world is going? And how do I essentially create like a product studio where, you know, every two weeks, you know, I'm building a feature zero to one, exploring a, you know, a new piece of this puzzle.
Speaker 2I think that's totally right. And I think for anyone thinking about building their own agent harness, the really important question to ask is what's my edge going to be? And there's a couple of different types of edge you can have. One type of edge is what you just said, which is you have a distinct vision of the world and you are just moving faster towards it. And it's not the one that others are focused on. I think another edge is you have some, some kind of proprietary data. You've got some information that other people don't. Maybe because you already have a successful platform or something. I think a totally different edge too is that it's not just software because software is getting very crowded and it's a scary market to be in. But, you know, there's hardware, there's like physical services, there's all sorts of real life businesses to be made that incorporate AI in interesting ways.
Speaker 1How do you, you know, physical services, like can you elaborate on what you mean?
Speaker 2Yeah, I mean, to be clear, I'm a software guy, so I don't know anything about this. If I were not a software guy, I'd be thinking a lot about how can I integrate AI into some kind of real life experience that people are having where there's actual, some kind of human exchange of value, whether it's like, you know, a person giving you a massage or a restaurant that's serving you a meal or whatever it is. I think there's so many times of like real businesses that are not quite reinvented with AI yet. I don't know that they all need their own harness, although perhaps some of them do. Like another category where I'm seeing a lot of harness, a lot of development going on is like legal tech, for example. I suspect medical tech is like a big one that has a lot of interesting like data lock and regulatory stuff going on.
Speaker 1Yeah, I think legal was one of those where it was like, obviously this makes sense. Yeah, yeah. Accounting obviously makes sense. I think the thing I would be thinking about is like, would I be building a legal tech AI agent harness today? Probably not. Yeah. Just because it feels so crowded. Totally. I like when, you know, when you're talking about service businesses, even boring businesses in general, like agent harnesses for HVAC companies. Absolutely. Or, you know, if someone wants to build an agent harness startup in a niche, how do they approach designing it? In the sense of like, you need the key insight, you need the niche.
Speaker 2One really simple answer about what an agent harness is, is an agent harness is just a bunch of tools that an agent uses to complete a task. And so I think the question to ask is, what are the unique tools that either I already have or I can build that are not already out there? And ideally, I think as a founder, you should always ask the question of, what am I uniquely suited to do based on my life experience, where I am in the world, the people I know, that sort of thing.
Speaker 1And how do you think about AI smell? Yeah, totally. You know, because I think a good agent harness is going to kill that smell as much as possible. And I know you have a strong opinion on that.
Speaker 2I do. I have been using this term AI smell all year. I've been talking about deodorizing, spraying it off, all that sort of thing. Maybe for people who just don't know the term, AI smell is what we use to refer to this fact that when people make work with AI, you can often tell they used AI. I think it shows up most in writing, where when someone writes you an email, maybe with a cloud, you're just like, you don't normally use that word. That's not what you sound like. And I notice it a lot with AI images now, where I just like walk around in San Francisco, and every cafe, every restaurant has a poster where I can just smell like, that's nano banana, that's GPT image. I do this stuff all day, so I can like spot them a mile away. And I think more and more people are starting to recognize that AI smell as it diffuses just through the economy and normal people. So for us at Gamma, it was actually a huge goal of this launch. We had kind of been a victim of our own success as one of the early AI tools, where when nobody used Gamma, if you used it, your stuff would look really cool and stand out. But once everybody started using it, everything started to look like Gamma, and people didn't want to be recognized as using AI. And so kind of our whole goal of this launch we just pulled off was, we need to rapidly expand our sort of creative landscape. The truth is there's still going to be AI smells, but there can be a thousand AI smells instead of one monolithic one. And we can help people jump to different little pockets. And we needed, this is another form of a harness, by the way, we needed to create our own automated system for constantly cycling through different creative looks. We actually use the analogy of fashion. And fashion has a smell too. You're like, I know that H&M shirt you're wearing, like I have the same one. And, you know, with fashion, there's kind of this dynamic where people are cycling through different looks. You know, right now we're into like really big wide jeans, but soon it's going to be skinny jeans again. And so we realized we had to actually ride that cycle and give people the tools to adapt to it.
Speaker 1So is there an eval for this looks generic?
Speaker 2You know, evals are such an interesting topic because I think evals work really well for things that are objective.
Speaker 1Actually, if you don't mind, break down what an eval is.
Speaker 2Yeah, totally. So an eval just means a, you could think of it almost like a software test. It's a hard-coded question that you ask in a way of judging it. So a simple example in our world of Gamma is we often want the agent to ask questions when you do something ambiguous. So we'll give it an eval like, when the user says, make a deck about tigers, the agent should ask something like, is this for kids or for adults? Because there's very different kinds of tiger products. And so we typically use evals to test out new prompts or new models. And usually you score them and you can put a bunch of evals together into a benchmark where you say, oh, it turns out model B scores 3% higher than model A. Every time the labs come out with a new model, they're like, oh, Opus 7 crushes like these 15 evals. The problem for us in a very creative domain is that evals have always been hard for very subjective taste-oriented things like, does this look cool? And even more so, very personal ones like, is this me? Does this sound like something I would say? There's no one objective eval for this. And so we've tried to create some evals that are proxies for this. Like, does it follow instructions? Does it, for example, if you provide your brand guideline, does it stick to it? That's an objective question that we do test ourselves on. But we've never limited ourselves to just evals. And I think this is actually interesting insight too of how you can compete in this world. Ultimately, what founders can bring to their domain is taste. They have their own sense of what looks good. They're actually talking to their users and those users have one. And the less verifiable a domain you're in, like, say, fashion, the more I think there's room for these unique perspectives. I think when it comes to clothing design, there should be 100 different harnesses. I think, you know, Nike should have a different one than, like, Adidas because they have a different perspective on taste. Also, for us, because we have a super diverse user base who are all different, we actually rely a lot more on A-B testing. We will actually run two models or two prompts at very big scale and measure across that wide user base what's actually performing better because that captures subjectivity.
Speaker 1And what's the metric that matters most there?
Speaker 2You know, we have a bunch of different ones. To give you a few examples, we look at things like after you generate, how much time do you spend editing what you made? Do you end up sharing it or exporting it? Are there signals that show you're investing time? We also have a freemium product, so people will use our product for free and then if they like it, they'll upgrade to paying. That's kind of my favorite signal because it's hard to fake. Generally, people only pay if they're really seeing value out of a tool. So we use that one as like a North Star. Sometimes we'll even look at like downstream retention. Like are people still coming back to the product a week later or a month later? It's obviously slower to measure, but it's also one of those hard to fake signals.
Speaker 1The chart that's going viral this week is I think an A16Z chart that says I think only 2% of people are spending on AI tools. What's your opinion on that?
Speaker 2Yeah, it doesn't surprise me. I think in general, subscriptions are always a bit of a niche thing towards people. They're doing some kind of valuable work. I don't think consumers like subscription. I, as consumers, don't like subscribing for things. I'm still always looking to jump on somebody's Netflix account or whatever it is.
Speaker 1No, I will not share my password with you.
Speaker 2Are you sure? What if I explain what an agent harness is? But I think it's different for work. I think for work, it's a totally different story where work subscriptions are what has driven essentially the entire SaaS boom of the last 10 to 15 years in the software industry. And those things will keep being successful. And I think that same chart says that when you actually zoom in to professionals, the numbers are rising like crazy in terms of who's paying for AI. So that's certainly our main focus. And I think most people's main focus in AI is how to find those prosumers, the professional consumers, and how to break into B2B where there's a totally different kind of spending going on.
Speaker 1If someone wanted to build the gamma for X, what would you pick? Like if someone listening to this is like, wow, like, you know, I'm inspired by what gamma is building. Yeah. But I think that Like, you know. basically what you just described but like for a different niche yeah anything coming to mind
Speaker 2it's such a personal thing because if you know if it was my thing i'd be building it already i think you need to do the thing that i don't know anything about whether it's like pokemon cards or fashion or sneakers or whatever it is like find your domain where you know what's going on
Speaker 1and build it and and you personally is there like a harness that you wish existed that doesn't exist that you'd use tomorrow if it came out you know
Speaker 2um i'm really intrigued by this whole sort of like personal agent space and i haven't gotten to use all the latest ones but what i feel uh nobody's totally cracked is like work collaboration i think if i could build a harness for anything it would be slack and i'm a little disappointed that slack themselves have not built the harness for slack
Speaker 1what would that look like in your
Speaker 2mind so there's just this absolute flood of communication for anybody whose company uses slack or microsoft teams of course they're so many messages going through um you're constantly trying to stay on top of like well which ones do i actually have to do something about um i've seen them nibbling around the edges of this of like you can summarize 30 slack messages into one but i want something that just like runs it all truly for me and it learns me it learns how i would reply before i reply i want to be able to open any slack thread someone's pinged me on and have the first draft drafted for me
Speaker 1and it learns you not by answering like a quiz or onboarding and by actually like your actions by observing me yeah by
Speaker 2observing i have sent approximately a billion slack messages that should be all you need to know how i'm going to reply right even in emails doesn't feel totally cracked i've seen gmail has the little things that are like oh you know like yeah so i'd love to go to the dentist on thursday which is not how i'd ever reply in my life but uh i think there is so much more to just like represent me in a professional setting authentically to me like maybe that's what i'm really craving and it's a little bit of what gamma does is i want an agent that can learn my voice and of course that's what i'm really craving and it's a little bit of what that's hard because it's personal and this is why the agents haven't done it it's really hard to make
Speaker 1agents personal so you know at gamma and you know you're you're building something on top of other people's models um what's your advice for founders building on other people's models yeah in general
Speaker 2i think there's no shame in building other people's models we you know we have done this from the very start often bouncing between anthropic open ai google models and so much of the discourse has always been like no you got to train your own stuff you got to fine tune it because you need the moat i have always felt that that's um unrealistic for early stage startups in particular i mean first of all the money involved in training your own models is really hard but also the models out there keep on getting better uh i rarely would want to be competing against the models that are out there i think though it does change as you get further along in the journey because you start to amass real usage and i mean let's be honest just amassing real usage is the main battle people should be focusing on by all means use the first place once you do though you can start to do really interesting stuff we've actually had great success fine-tuning models to bring our costs down and in some cases even to start to do more personalization for people we also have started to identify specialized tasks that you know the big models are bad at like i mentioned earlier this whole process of qa in a slide and telling if it's good we've actually not had great results using any of the models out there for it so we are actually doing some of our own research to build models for those specialized tasks i think people should absolutely aspire to that but by all means use the great models that are out
Speaker 1there what about what about uh thought you know current what are your current thoughts on frontier models or cloud models versus open
Speaker 2source local yeah um i think you know cloud models are underrated again i think open source gets a lot of hype and excitement but follow the money it's all going to the uh closed source models because they're really darn good so much work has been done for you and the great thing for all of us is there is intense competition in the world of closed models i think in a world where you know anthropic had just purely run away with it this would be actually a different story because we wouldn't begin to choose between like great open ai models great gemini models but i feel like all these companies are just a few months behind each other and for us in the application layer that's the best thing ever it keeps the heat on them to keep doing better and
Speaker 1better you've once said that uh you you were surprised with how shallow your ai stack was is that still true today you
Speaker 2know i would say that our ai stack is broad but shallow what i mean by that is we actually use a ton of different models but we often use them in fairly simple ways like we get so far with prompt engineering with providing and maybe your listeners have heard the term context engineering increasingly of just like providing the right context to the models i really view that as kind of the whole game here of just and it comes back to harness building can you get the right context from the user can you express the ai's output to the user in the right way and can you distill all that into things that make the lms job really easy you can use any off-the-shelf model and
Speaker 1get good results there's a big uh debate prompts or fine tuning yeah you have a strong opinion there yeah
Speaker 2i think for us it's 98 prompts is where we get the value out of because prompting is easy and it's portable so like and by the way nobody's writing prompts by hand anymore like claude's writing your prompt for you and so it's really easy yeah and so i think most problems can be solved with a prompt including breaking up a prompt into different pieces and fine tuning to me is more of an optimization on top when you want to do things like control costs or eke out that extra two or three percent of
Speaker 1quality when a new model drops how quickly are you implementing it or are you kind of like waiting and seeing or how do you
Speaker 2think about it oh no usually uh within the hour if it's uh if it's a model that i have reason to believe is going to be good so you know not every model is interesting but i would say for us like any model from anthropic or opening eyes interesting generally if they're dropping it and i think i mentioned we have all these different jobs in our harness some of those jobs we use a like you know fable or astral level model for some of those jobs we use a little haiku gemini flash level model for some we use a little baby gpt luna model for and so anytime any model comes in this class that can help us fill out that sort of pareto frontier i want to try it and see how it does and generally because our use case is so subjective yes we'll run some evals but i just want to run on a sample of users and we're very lucky to have a big user base where i can start a test in an hour and get data two hours later on how it's doing
Speaker 1um so one of the things i learned because i'm a co-founder of a design agency called lca and we actually worked on a project together at gamma and one of the things i learned working with you guys was you guys run a lot of experiments yeah i'm curious about um experiment you know maybe one experiment that the results surprised you um because you guys are just it feels like you guys are running a lot a lot at a time we
Speaker 2run a huge number of experiments at any given time we probably 10 or 20 running yeah um and we're cycling through many of the experiments last less than a day
Speaker 1so walk me through like the process of an experiment because i think it's really really
Speaker 2interesting well i'll walk you through one that i just started in the uber ride over to talk to you i just fired off eight different experiments because uh claude haiku 5.5 came out yesterday we were really excited to get to use it because you know i mentioned that we have a lot of these tasks that don't need that like super fable level model they're quick and cheap like picking a or sometimes like writing an outline based on something and you know we've been looking forward to new small models we could use to test this so uh this morning i fired off eight different tests for it and right after we got off this podcast i'm going to check the results and probably cancel four of them and see how they did um and we're testing it for things like making micro edits on a slide or you know like revising an outline whatever it is in terms of the kinds of things we learn from these tests um often the great thing that we learn is that we can use a cost of another model and get the exact same results but sometimes we actually get uh the opposite finding we find like wow actually the big beefy expensive model is necessary to achieve a certain task um sometimes we even discover things like good friction like we actually just ran an experiment a few days ago of a fast mode of our product um and fast mode just means skip through some of those steps that i told you about earlier like showing you an outline of your presentation or asking you questions up front and the hypothesis was of course if you remove friction in the process then people will get through it faster and get to that sort of magical value moment more quickly i was i was really rooting for this test i wanted to see it win but it did not win for us actually um it turns out those speed bumps we had along the way like seeing your outline and reviewing it you know choosing uh the kind of high level arc of your story answering those questions led to much higher satisfaction ratings even though it was slower the people that got through felt more invested in their output and did more and they were able to do it faster and they were able to do it faster and they were able to do it faster and they actually converted to paying at a higher rate um but i think what's also cool is a test is never just one answer or another you can like dig a little bit deeper so we actually shipped a version of fast mode for brand new users on mobile because you know on mobile you don't want to be typing a bunch of answers to
Speaker 1a question i just right before this i posted an x article about marketing engineers um my thesis is that uh marketing's changed yeah if you think about the mad men here mad men era it was like you know you did one big commercial a year and in the 50s and 60s and 70s then it became um in the 90s and 2000s the digital marketer maybe a few more uh experiments then we had um the growth hacker era a few more experiments per week um i think in 2022 i would say ai assisted marketer uh more experiments and you know i think we're now entering the age of the marketing engineer where you're you're you know as a marketer you're running dozens of experiments experiments per day. How are you able to do that? You're using AI agents to go and do that for you. So I'm curious, your POV on that. Do you think that marketing has changed? And like, where do you think, where do you see it going?
Speaker 2It's funny because before I worked at GAM, I worked at a company called Optimizely. And this is where my love of A-B testing came from. We were A-B testing as a service. And a lot of our vision was about personalization. This idea that, you know, as it gets easier to change your website, we are going to create a hundred little micro variations of it. And everyone will get served a slightly different version. And that will optimize conversions in all these interesting ways. Surprisingly to me, that vision, I would say, largely has not come true out on the internet. I would say websites are still mostly one size fits all. So are mobile apps by and large, except for these like specialized areas, which are like hyper-personalized and succeeding. When I say specialized areas, I mean things like your TikTok feed, your Instagram reels. We know that companies are spending billions of dollars to eke out every last dollar from you in those surfaces.
Speaker 1I heard that McDonald's is testing. Is that personalized pricing? No way. You hear about this?
Speaker 2Wow, that sounds like a backlash waiting to happen.
Speaker 1That's why I saw it.
Speaker 2Yeah, yeah. But I think maybe that McDonald's example illustrates why this isn't happening. First of all, I think there's a real societal allergy to this kind of thing. In the same way that if you found out that we both bought the same airline ticket, but they charged me five bucks more because I was in San Francisco.
Speaker 1Well, I think in general, like even surge pricing. I feel like when I had surge pricing last night, coming home from dinner, and I was like, oh, you know? Yeah. It just, it's not a good feeling.
Speaker 2Yeah, yeah. And, you know, I actually think in this age of hyper efficient, you know, like performance marketing, actually what's working is companies that really doggedly invest in their brand. This like Apple mindset of like, just protect the brand at all costs. And so I think that trend is going to counter some of what you're describing of everybody just going wild with their marketing agents. That said, I still buy the thesis that like the marketing engineer is rising. And I just love that term marketing engineer because it sounds like a contradiction, you know? Much like GTM engineer sounded like a contradiction. These are supposed to be different departments, but I've loved seeing how everybody's skill sets are just blending. We're seeing this with Gamma, actually, where people are more and more using Gamma, not to just make one presentation here or there, but creating these really cool automations where they're doing things like every time a new deal opportunity is created in Salesforce, let's automatically create a pitch deck based on the call notes that have happened.
Speaker 1How do you set that up?
Speaker 2We've actually been building kind of connectors. Where you can automate more of it. So you can just like plug into Salesforce. But I would say there's still a fair amount of plumbing going on. People are using maybe a tool like N8N or Zapier or something to trigger it all together and wire it. That is a good example of a thing where it's still the domain of specialists, I would say. There's like these really smart people or agencies that know how to help a business like wire things up. I think there is still a business opportunity in taking these practices that are done by the top one or two percent of companies who are really savvy about engineering. And just turning it into a super easy thing that everyone does. Another example of this, to go back to the personal agents, is like what's happening now with tools like Muse and Instinct. You could do some amazing stuff with OpenClaw or Hermes if you could get your Linux box set up in the right place and avoid getting hacked. Personally, I skipped out on that whole trend. I was like, I'm a busy dude. I don't have time to set up my OpenClaw. I'm going to wait a few months for them to just make it easy for me. And now I got Muse and Instinct blowing up my phone because they made it easy for me.
Speaker 1Right. So you don't have a Mac mini. I never got my Mac mini.
Speaker 2I'm ashamed to say as an AI founder.
Speaker 1Going back to marketing engineering, in your mind, what does a top one percent marketing engineer look like?
Speaker 2Well, one thing that comes to mind is they're really a marketing engineer data scientist. So they are like ultra data driven and grounded in what's performing. And this is something that's gotten easier than ever. Like absolutely Claude is my data scientist. And so like I would say now everybody on our team is constantly. Armed with like very fine grained analysis of like failure modes and opportunities. They're looking at traffic on our site. They're finding gaps there. You know, like I look at our, you know, the person we have on our team who's closest to a marketing engineer. And he is just constantly thinking of new ways to do SEO, ad creative at just an unbelievable scale for one person. You know, they're like they've gone from being like individual creatives to people who are running this army of agents. And they are just in this high level. Supervisory role. Yeah.
Speaker 1I mean, it's basically a department.
Speaker 2It's department. Yeah.
Speaker 1It's like we're getting to the place that if you're a top 1% marketing engineer, your surface area of what you're working on isn't just, uh, well, the service area is huge. You're looking at SEO, GEO, you're looking at, uh, social content, you're looking at paid ads, you're looking at, um, pretty much like all creative brand, you know, it's, I think that's what's interesting. I think that's what's interesting to me because I think that the people that figure out how to become a top 1% marketing engineer have an unfair advantage around getting their product seen. Well, and the
Speaker 2other 98% are quickly becoming dinosaurs. If you can't keep up with the people in the market that are doing this, I've kind of had this idea that all of us over the last, maybe like year or two, at least all of us who are using these tools kind of got this rapid series of promotions. We went from individual contributor to manager to director. To like VP. We now have these like layers and layers of agents under us in a pyramid. And it turns out that job's actually kind of hard. Like, um, uh, there's a reason that like few people do it because you are sort of constantly barraged by information. You're having to make so many decisions. Uh, you kind of are like overwhelmed and confused by all of it. You need tools to even make sense of it all. You also need people to help you make sense of it all. You kind of need your, your assistant and you need your chief of staff and you need all of these things. And so all of us are having to rapidly upskill to essentially like how to manage a thousand people and building that skill is one of the most important things we can all do. Also, I think building tools to help the average person become a like VP of their own life is still underexplored. Going back to my Slack idea.
Speaker 1I also think, uh, you know, uh, experiments in general, it feels like building a startup today is, or building a business in general is, it's like you have, is building a business that's more, it feels like almost a little more science than art now in the sense like you're the ones who are going to win are the ones that have the experimentation muscle and just trying a bunch of different things. That's why I like, I love talking to you because it feels like that experimentation muscle is, I mean, that's your, like your thing. Um, what's one thing at Optimizely that you learned that every AI founder should, should know? You know, would be good for them to know.
Speaker 2That's a great one. You know, one thing that comes to mind for me, I think from years of like helping other companies do experimentation was that so much of it just comes to like how many at-bats you have. Um, uh, it's all just sort of a numbers game, just like sales, just like dating, just like kind of anything else in life. You got to really try a lot of different stuff versus expecting that you'll run one big, you know, test that magically unlocks your business. Um, I like how you called it a muscle because, you know, it's like, you know, it's like I do think it is first of all, because it's trainable, you know? Yeah. Maybe some people are born with more muscle than others, but if you put in the work, that muscle is going to grow.
Speaker 1I also think like your first experiment is not going to work. Totally. Just like if you're trying to get fit and you're trying to build muscle, you have to just show up to the gym every day and you have to put in the work and then two months later, then, then you might see a six pack. Right. But you can't expect it on day one. I think, uh, that's the hard part about being. Uh, an experimenter is like, there's like a trough of sorrow that you have to go through. Um, and I think what a lot of people end up doing is they're like, you know what? I'm just going to go with my gut. Absolutely. Um, and by the way, the same
Speaker 2is true of things like posting on social media, writing a blog, sending cold email outbound, there's a trough of sorrow in all of it. And I think all success in life comes from swimming through some kind of trough of sorrow.
Speaker 1What's, uh, what's something that's underhyped? Right now that you're like, you know what? People are just not talking enough about this. I was talking with, um, a friend of mine about voice AI, um, that felt like underhyped. Yeah. Right now people are talking about personal agents, anything that comes to mind.
Speaker 2Maybe it's a lame answer, but I actually think AI image models are underhyped. It's a lame answer because we've been talking about image models for years. Ever since like stable diffusion came out in 2022, people have been making like, cool stuff. But I think, um, ironically, as they got good, people stopped talking about them as they stopped messing up in so many ways and just became sort of like part of the fabric. We sort of like forgot that we have this unbelievable technology that can make anything. And I think if you look at the way people are using like chat GPT images and a few other things, it's kind of just the same three or four use cases over and over, you know, they're, they're making like maybe the, the menu for their small business. They're making like a logo for their dad's side project. There's like a handful of things they're doing, but these models are capable of so much. And we have a really cool vantage point to this at gamma. We didn't talk as much about images, but we use probably like 10 different image models as well for different tasks. And the pace of innovation there has been so much and just this sort of like unexplored capabilities for us. It's things like making beautiful diagrams or infographics to go in a presentation. making cool little textures. that can be the background of a presentation, making custom icons. Like all these things are possible now. And most people just don't realize it's there. And I think you can pay attention to almost like any domain that's creative and find ways that you could be using these things. And especially the average person just has no idea. You know, even one that I think is like probably many AI native people are doing, but most people don't realize you can do is using AI image models for like clothing and like home decor. You know, like you found a new couch you want to buy on Amazon. You can actually just chat GPT to say, what would that look like in my house? And then say like, go find me a few others and visualize those too. Like so many of these things are just waiting to be more deeply productized.
Speaker 1Do you think there's an opportunity to build vertical products there? Like just focused on, you know, interior design, for example? Or do you think the big model companies are just going to own that use case so it's not even worth trying?
Speaker 2I think there is so much room for these vertical use cases. And it's all about understanding the specific needs of the people that do them. Like, let's just take this furniture one, for example. I actually just bought a couch recently. So I- Congratulations. Thank you. It's really comfy. And I went through this and it was so interesting, like trying to do this. And it turns out the AI image models are amazing at showing me what the couch will look like, but they're not actually showing me exactly how it's going to measure, right? And there's a huge difference between like, you know, let's say a 68 couch and a 58 inch couch if your wall's a certain size. So like, could you actually show me the units in the display? That's a great example of a harness, right? The models are actually capable of a lot of these things. One way they're capable is you could imagine doing a thing where you render the couch in my room, but then you use some good old fashioned programming to draw a line across in pixels and measure how long it is and see if it actually came out of the right size and prompt the model to tweak it to get exactly right. I don't see the labs doing that anytime soon.
Speaker 1Or prompt, hey, like this couch, isn't the right couch for you. Yeah, totally. But here's another couch from Restoration Hardware that I think would be perfect for these reasons.
Speaker 2Totally. And dude, what I learned about couch shopping is there's hundreds of websites that sell couches and you'll have to click through a lot of little buttons on all of them. I ended up on Google Shopping. It was not a pleasant experience. Maybe somebody's already built a great product for this and I missed it, but I haven't seen it yet.
Speaker 1I think there's been examples. I think Levels.io has like a.
Speaker 2Yeah, he has like an interior, I think it's called,
Speaker 1but I think there's still room. You know what I mean? I think, especially as the models are getting better, the image models are getting better, the concept of a harness is getting more and more mature. There's more here. I think it's safe to say that also the consumer experience today around couch shopping or furniture shopping isn't optimized. And what's cool about that specific use case is it's high ticket. Totally. So, and there's margin, right? Like if, for example, like, I don't think it's crazy to do a deal with Restoration Hardware where they're like, we'll give you 5% commission if you recommend. If you sell us more couches.
Speaker 2Yeah. Yeah, absolutely. I think the other version of this, when you think about like what will survive competition with model companies, and we're certainly facing this question constantly at Gamma, there is the pure consumer version, which to use the couch analogy is like help me buy a couch. But then there's the like prosumer version, which is, hey, I am an interior designer. Maybe I'm a part-time interior designer, but I also have like two kids at home and that's what I do. But I, you know, I do it by the hour. Those people need a tool and they will seek out the best. And if it helps them do their job better, they will happily pay a subscription for it. Then there's also the true pros. There's Restoration Hardware itself has people who work there. What tool are they using? I bet it's not just ChatGPT to do all of this. I think there is actually an opportunity to build companies for those. And I do suspect that highest tier of professional, the bigger companies, is where you can really build something durable.
Speaker 1Yeah, and I feel like it's underexplored.
Speaker 2Yeah, it is.
Speaker 1Because the narrative I think is there's not that much opportunity creating vertical SaaS or vertical AI products, which is unfortunate because I think that there is.
Speaker 2And it's a big world out there. You know, that's maybe one other thing we've learned, from the Gamma journey. Our user base is very broad. Because if you just think about how many people on earth make presentations, it's a lot of different kinds of people. For us, less than a quarter of our revenue is in the United States. It's spread across many different job roles, from consultants to teachers to marketers to salespeople to lawyers. I didn't know lawyers make presentations. Turns out they do. And the more you can tap into little niches, you know, like in many ways, a big company, is just a thousand little niches piled on top of each other. So if you can find something that cuts across, and you don't have to start broad. You can start with just one, but then look for the adjacencies you can tackle.
Speaker 1100%. And you actually don't know where it's going to lead, right? Totally. You didn't realize like, hey, lawyers are creating presentations and then you end up, who knows, you end up creating something specific for lawyers. Absolutely. And then all of a sudden, that becomes a huge part of your business. Yeah, totally. And that's why I'm always like, get started. Like, put something, put something out there. Put something out there.
Speaker 2You got to just find out. That goes back to experimentation. Experimentation doesn't just mean run an A-B test with like 10,000 users in each arm. It usually just means put up a landing page, get a product someone can try, which has never been easier.
Speaker 1If you, I know you're not leaving Gamma, but like, let's say you were to leave Gamma tomorrow. What's something, what's a company you would start to build?
Speaker 2Oh, that's a good question, man. You know, it's, I just love circling this space of like enabling people's creativity I think there's other kinds of like creativity that like intrigue me though. Like as crowded as the space is, I'm so intrigued by just like vibe coding as I practice and just the way I had this experience maybe as somebody who knows a lot of other tech people and lives in San Francisco earlier this year, where every time I would catch up with a friend I hadn't seen in three or four months, I'd be like, what are you up to? And they'd be like, check out my app I made. It sort of took the world by storm. I want to know maybe what's the next vibe coding or what's the next friend? What's the next frontier of vibe coding? As people can make anything, where will that take them? And so like within that, I'm certainly intrigued by video models finally getting good. I'm intrigued by voice models getting good. Like what's the version of some kind of thing you could call and it would do something for you? Games are so interesting to me. I just think like the video game industry is ripe for some kind of disruption, but I haven't cracked it yet. I don't know.
Speaker 1I think on the video side, it does feel like we're hitting like the vibe coding moment for video games. Yeah. Opus 5.5 is obviously.
Speaker 2It makes some cool stuff.
Speaker 1Makes some cool stuff. Yeah. So I think the question is, in a world where anyone can create video, what are the types of videos they can create? Yeah. And then I'm also interested, it's like as a founder, I'm like, okay, in a world where anyone can create video, what are some startup ideas that I can create? Totally. On top of that. And that goes back to agent harnesses. Yeah, absolutely. Everything goes back to agent.
Speaker 2Everything goes back to the harness, baby.
Speaker 1Dude, thank you so much for coming on the podcast. Is there anything you want to leave people with?
Speaker 2One thing? I would just always encourage people who are thinking of starting a business to know that it is a long and sometimes lonely journey. And because of that, you really have to love the thing you're working on. One of my kind of rules for myself before we started Gamma, and it would stick around if I ever did another startup, is it has to be something that I could dog food, that I could be a user of, and that I could believe in. And to be honest, that actually rules out a whole lot of boring businesses, but it was good for my soul. I just needed to have something that I wanted to spend hours every day in. And so I always encourage people to dig deep and think about what is your passion and the thing that only you know, the thing that you want to spend eight hours a day on, that when you look around, your friends don't want to spend eight hours a day on, and zero in on that as your North Star.
Speaker 1I think that's the important point too, around the thing that you want to spend eight hours a day on. That other people don't. Totally. Because you might be really into sports or something, but a lot of other people are into sports, and that might be very competitive. I think if you're choosing a startup idea or choosing a business, I always say marry the niche, date the product.
Speaker 2I love that.
Speaker 1Fall in love with who you're trying to serve. Yeah, yeah. Because we live in an age where you're going to be iterating on the product so many times, and it's going to look so different. But if you love the niche, and that's why I always encourage people to build media, build a following in that niche, because then you can sell them a bunch of different products.
Speaker 2Now you have a built-in audience. Jeff Bezos has a very similar quote where he says, be stubborn on the vision, flexible on the details.
Speaker 1Well, I'm glad I'm in good company with Jeff Bezos. You sure are. Thank you. I appreciate you. Thanks for having me. This was great. This was awesome.
PodcastTranscript AI