Go back

Inside the little-known expert network quietly training every frontier AI model | Garrett Lord (Handshake CEO)

0m 0s

Inside the little-known expert network quietly training every frontier AI model | Garrett Lord (Handshake CEO)

Garrett Lord, CEO of Handshake, discusses how his company transformed from a decade-old platform connecting college students with employers into a rapidly growing AI data labeling business. Handshake’s network of over 20 million students and alumni, including hundreds of thousands of PhDs and master's students, became invaluable to frontier AI labs. These labs need expert-level data for post-training models—a process that improves model capabilities after initial pre-training on internet data. As generalist data becomes less useful, experts in advanced domains like physics, biology, and education are needed to identify model weaknesses, provide correct answers, and create step-by-step reasoning data. This data, often in JSON format, includes trajectory information (screen recordings and problem-solving processes) to teach models how humans think. Handshake’s new business launched from zero in January and reached $50 million ARR in just four months, with projections to surpass $100 million ARR within a year—outpacing their original business’s revenue. The story highlights how AI disruption creates opportunities for companies that can leverage existing assets, like engaged expert communities, to meet the growing demand for high-quality training data.

Transcription

14179 Words, 77945 Characters

English
Introduction to Garrett Lord There will never be a time like this. I've never seen anything like it. I doubt I'll ever feel anything like this in business again where there's unlimited demand. How do you make sure that three months from now, six months, are all you have? Like no regrets? Get on the plane to go talk to a customer. Make the late night push. Check the data 6 times over again. Speaker 2 Your company creates new data to continue advancing the intelligence of models. This is a business that you built on top of a business you've already had. Speaker 1 We're the largest expert network in the world. We have this massive strategic advantage, which is like no customer acquisition cost. The only mode in human data is access to an audience. Speaker 2 You guys come in after the models trained to tweak the weights based on additional data that you create. Speaker 1 The models have gotten so good that the generalists are no longer needed. What they really need is. Speaker 2 Experts There's this tension between all these students training models to become smarter, and then there's that they will have harder time potentially finding jobs. Speaker 1 That's not what we're hearing from our employers. This is just enabling human beings to be even more productive. You used to put Google search on a skill on your estimate like grew up. With Google being like AI, native young people are at a huge advantage. Speaker 3 Today my guest is Garrett Lord. Garrett is the Co founder and CEO of Handshake, which is one of the most interesting and incredible AI success stories that you probably haven't heard of. Handshake has been around for over 10 years. They're essentially LinkedIn for college students. It's a place for students to connect with companies to find a job. They are the platform of choice for every single Fortune 500 company. Over 1500 colleges, over 20 million students and alumni, and over 1,000,000 companies use them to hire graduates. At the start of this year, Garrett and his team realized that their huge proprietary network of students, including 10s of thousands of PhDs and master students, is extremely valuable to AI labs to help them create and label high quality training data. So they launched a new business from zero to 1 in January. Four months later, they had 50 million ARR. They're now on pace to blow past 100 million ARR within just 12 months. They'll exceed the revenue that they're making with their decade old business in under 2 years. This is a truly incredible and rare story and one that I think a lot of teams can learn from because AI is creating a lot of opportunity but also a lot of potential disruption. And this is an amazing story where the company basically disrupted themselves. This episode is packed with insights, including a primer on what the heck are people actually doing when they're labeling and creating data to train models. A huge thank you to Garrett for making time for this. His wife just had a baby this week. He's also in the middle of scaling this insane new business. So thank you, Garrett. If you enjoy this podcast, don't forget to subscribe and follow it in your favorite podcasting app or YouTube. Also become an annual subscriber of my newsletter. You get a year free of a bunch of incredible products including Lovable, Replit, Bolt, N8, M, Linear, Superhuman, D Script, Whisper Flow, Gamma, Perplexity, Warp, Granola, Magic Patterns, Raycast Chat, PRD, and Mobbin. Check it out at lennysnewsletter.com and click bundle. With that I bring you Garrett Lord. This episode is brought to you by Coderabit, the AI code review platform transforming how engineering teams ship faster with AI without sacrificing code quality. Code reviews are critical but time consuming. Code Rabbit acts as your AI copilot, providing instant code review comments and potential impacts of every pull request. Beyond just flagging issues, Code Rabbit provides one click fix suggestions and lets you define custom code quality rules using AST graph patterns, catching subtle issues that traditional static analysis tools might miss. Code Rabbit also provides free AI code reviews directly in the IDE. It's available in VS Code Cursor and Windsurf. Code Rabbit has so far reviewed more than 10 million PRS, installed on 1,000,000 repositories, and is used by over 70,000 open source projects. Get Code Rabbit for free for an entire year at Code Rabbit dot AI using Code Lenny. That's Code Rabbit dot AI. This episode is brought to you by Orcas, the company behind Open Source Conductor, the orchestration platform powering modern enterprise apps and agentic workflows. Legacy automation tools can't keep pace. Siloed, low code platforms, outdated process management, and disconnected API tooling fall short in today's event driven AI powered agentic landscape. Orcas changes this. With Orcas Conductor, you gain an agentic orchestration layer that seamlessly connects humans, AI agents, API's, micro services and data pipelines in real time at enterprise scale. Visual and code force development, built in compliance, observability and rock solid reliability ensure workflows evolve dynamically with your needs. It's not just about automating tasks, it's orchestrating autonomous agents and complex workflows to deliver smarter outcomes, faster. Whether modernizing legacy systems or scaling next Gen. AI driven apps, Orcas accelerates your journey from idea to production. Learn more and start building at Orcas dot IO slash Lenny. That's ORKES IO slash Lenny Garrett, thank you so much for being here. Understanding data labeling and its importance Welcome to the podcast. Yeah. Speaker 1 Thanks for having me a long time subscriber. Speaker 2 I appreciate that. OK. So before we get into the insane trajectory that your data labeling business is on, which is just an amazing story that I think a lot of founders and product teams that are trying to navigate this AI disruption that's happening will have a lot to learn from. I want to first help people understand what the hell data labeling actually is. Just like, what are people actually doing? Why is this so valuable? Some of the most, I don't know, fastest growing companies in the world today, including you guys are just are, are this is what you do. Clearly there's something really important here. I sort of understand it. Probably not really. I think a lot of listeners feel the same way. So let me just ask you this. What is data labeling actually like? What are people actually doing? And then just why is this so valuable to Frontier AI Labs? Speaker 1 Yeah. So I, I think it's helpful to take, I guess step back of like what, what does training model look like? So there's really two primary functions. There's a pre training and a post training process in training a model. And for a long time these AI providers or LLMS or Frontier labs we're focused on basically sucking up more and more information on the pre training side of the house. And that's basically the entire corpus of like written human knowledge that's not just written, but like every YouTube video, every book, basically, you know, their pursuit of sucking up everything that was on the Internet. And that was the pre training side. And there's a lot of gains from pre training. Like models continue to get better. And about 18 months ago, 24 months ago, we started to really see like an asymptoting of gains coming from because they had essentially like sucked up all of the knowledge on the Internet. And so labs really shifted towards most of the gains now coming from the post trainings out of the house. And what post training is, is it's augmenting the and improving the data they have across every discipline or capability area that they care about. South take coding or mathematics or law or finance. You know, they are focused on collecting high quality data that really improves the state of art capabilities of their models. And you can see a lot of these popular benchmarks on what are called model parts. You know, when Llama force released, you'll see like the benchmarks across various domains. And each one of the research teams inside of the labs are have different use cases. Basically, they're running experiments, I almost think like the scientific process. They have like a hypothesis around how to improve the model. They're trying to collect small pieces of data to see if that hypothesis works out. If that hypothesis is proving true, then they expand the overall collection of the data in that effort. And it couldn't. It could look like reinforcement learning environments. It could look like trajectories. It could be audio and multimodal. It can be text based, like prompt response pairs. It can also be like reinforcement learning with human feedback, which is like, you know, preference ranking data. And so that's the, that's the state-of-the-art of models. And most of the gains that are happening for models right now are, are coming from the Bush training side of the house. And there's just an an incredible amount of demand to stay at the absolute frontier of where models are going. Speaker 2 So training pre training is feeding it say the entire Internet. Here's like all the data that the humans have ever created, figure out knowledge and facts and had a reason and all these things. Post training, is it correct to say there's essentially 2 buckets of things to do? There's reinforcement learning, human feedback, RLHF, and then there's kind of this bucket of fine tuning. Speaker 1 I mean, yes and no, because like what take for example, like trajectories or like you want to be able to do people use flight search like an accounting and end process or you want to be able to like conduct biological like experiments, like you need actual trajectory data like you, you need to there. There's still very much a lot of the labs are still that points of view on what data collect. It's evolving very quickly. But I think, you know, reinforcement learning is really like preference ranking, right? Like which, which question do you like more? Question a or question BSFT data is like a prompt and a response. And obviously the labs are very focused on these like thinking or reasoning models. So in order to improve a reasoning model, you'd actually have like the step by step instructions of which when you interact with a lot of these frontier models, they're that, you know, they struggle in very advanced domains. And so, you know, I think there's a variety of datas that that they're working, you know, working with to improve capabilities in their models. Speaker 2 What I'm hearing is there's other ways to post train. Which of these are you guys focused on? Where do you help models most of these 3:00-ish buckets? Speaker 1 Are like real unique proposition as a business is the fact that we like have an engaged audience. We have 18,000,000 professionals across, you know, we have 500,000 pH DS, we have 3,000,000 master students. We're a global platform and so you know, depending on kind of what you're looking for across any area academic knowledge, you know, what is the definition of a PhD? It's essentially to like be at the how do you get your how do you get your PhD? You defending your thesis, defending your thesis means generally speaking, like you have proven that you have extended the the world's knowledge in a particular domain. And so the ability to like hyper target this audience into chemistry, math, physics, biology, coding and really touch parts of human knowledge that have never before made it to the Internet is really where we we excel. And I would say that when you talk about the labeling market, something to to make it more abstract is like it used to be generalist to work like a lot of the market before the model started to get better was leveraging talented international lower cost labor to do basic generalist tasks. But really what's happened is the models have gotten so good that the generalists are no longer needed. Like what they really need is experts, experts across every area that the models are focused on. And, and really you could think about these model model builders as they're focused on like the most economically valuable capability areas in the economy, right? And so that generally speaking right now is focused on, you know, advanced STEM domains, advanced science in math domains, and then the kind of derivative functions of like accounting, law, medicine, finance, where they want to make the models more capable. And then the work that we're doing, I think to come full circle to your question, like we're doing work across across so many domains. I mean, we have, we have millions of bachelor students that are being used for work in like audio work in customizing a model depending on The Voice and tone where you are geographically in the country. What do women versus men prefer all the way to the most advanced PhD STEM domains out there? Speaker 2 OK. So is it fair to say essentially all the data that is available has been trained on and your company for creates new data, new knowledge to continue advancing the intelligence of models? Speaker 3 Yeah. Speaker 1 And I also say we help point out where the models are weak. So in order to break a model, you know, it's pretty tough for the average person to break a model and get an incorrect response. But if you're a PhD in physics, like you can go in in multiple kind of sub domains of physics and prove where the models actually breaking, either breaking in its reasoning steps or it's where it's broken and it's ground truth. Right answer. Or you start throwing tools in there or needing to, you know, follow some step by step process. And it's, it's it's, I wouldn't say it's easy for them, but the average person cannot break the models. And that's where we really come in. Speaker 2 So essentially it's just like catching mistakes that the model has made. OK, So what are these people actually doing? The role of experts in AI model training What is it? I know there's all kinds of different types. You describe all the ways that data is generated, what kind of data is useful. So maybe just like the most common examples like. Speaker 3 Let's say APHD. Speaker 2 Person is sitting there doing stuff. What are they actually doing? Speaker 1 A great example is a public paper called like GB QA. So for the engineers out there that want to read about it, like essentially the the crux of the paper is you break the model, you provide a ground truth, the right answer to the question, you provide the step by step reasoning steps. So, you know, you can, you might imagine like, because models are non deterministic, like the model can get the answer right once, but it might not get the answer right, you know, three out of five times. So you actually prove where the model's failing. You actually break down into like where is it failing? You know, maybe it can get the, it knows the question, but it can get the right answer. But the actual steps to get there are wrong. And they're really focused on like the steps to get there. So there's like 10 steps in a math problem, right? Like step six through 10 is wrong. And so like, how do you fix the actual steps? And, but what are they doing? So they're going in, we put them, you know, we really focused on calling this like a branding the experience and treating people like experts, like PhD students expect to be treated different than a lower cost international labor with a different work expectation. And so these PhDs come into a community. We have a instructional design team and an assessments team that's going through and basically iteratively helping them understand how to use the tools that we built and how to interact with the latest models. Then they go in and start actually creating data and that you know, that process is on our side. The model builders, they want to know that the data we're producing is high quality. So we have our own research team, our own post training team. I hired a a gentleman from Meta that went along along the post training over there. Speaker 2 And they hope you paid them well. Speaker 1 Yeah, so war for AI talent is very expensive. Super, super privileged and proud to be working with him. And so, you know, each unit of data, you, we have to build an environment for them to actually create the data. Then we have to understand at a, at a, in a unit level, we're trying to approximate the actual gain from that piece of data and whether it can improve in a particular capability area. And then we're also focused on, you know, evolving the use cases to also follow what the mono builders want, which is they want more they they want more real world tool use and trajectory based data as well. Speaker 2 OK, there's so much here and like we could go infinitely down here, but I think that this is really interesting cuz just like people hear so much about all of this and they barely understand what the hell it actually is. The future of AI and human collaboration So this is for me really interesting. I think it's gonna help a lot of people. So essentially APHD, say a biologist, biology PhD is just their job is find flaws in what say ChatGPT is producing and then come up with here's the correct answer and that is used to fine tune the model. Here's like here's something you're doing incorrectly. Here's the correct answer and that improves the model. Is that a simple way of thinking about it? Please correct anything I'm saying that is incorrect because I don't want people to misunderstand it. Speaker 1 I mean like a great example, let's take like a non verifiable domain like education. So there's like APHD student Rachel on the network. She got her PhD from the University of Miami, spent two decades as a teacher teaching students in the eighth grade, and she was an adjunct professor at a local Community College in the field of education. And so she is interacting with the state-of-the-art models in educational design. So actually trying to understand what is the best way to teach people and like, how do you frame the how do you how do you spot incorrect issues in a model in the way that they're like training people and help the models understand the forefront of educational design with the hands on experience of being an eighth grade teacher for 10 plus years and having a PhD in education. So that's an example of like, you know, you're going to have that all the way down to like a verifiable engineering problem that you're seeing the latest, you know, you know, seeing the latest models fail on. So you have, yeah, I think that gives you a, you know, the big gamut. You also have, you know, we talked about professional domains like these reinforcement learning environments. Like, you know, there's a bunch of papers out there that's basically speak to like people narrating over their step by So as they go to solve a problem from start to finish, interact with multiple different service areas, interact with multiple different tools, you know, they're like, you know, there's papers that talk about this, like, you know, talking over what they're doing, actually following and screen recording where their mouse is going, how they're problem solving. When they run into a roadblock, what do they do? They really want to understand how humans think. Speaker 2 You mentioned this term trajectory. Can you just explain what that actually means? Because it feels like you've mentioned that a few times and that feels important to all this. Speaker 1 A trajectory is basically just like the entire environment that is collecting what you're doing. So it's your screen, it's your mouse. Speaker 2 Yep, including this voice over. OK and then this might be too technical, but what is the output of all this work this say teacher? Is it just like a Jason file? An XML file like a text file? Speaker 1 Yep, you think about it, Jason Data. Speaker 2 Jason, data, OK. Speaker 1 And then you also have like multimodal work, like audio, like classifying music and understanding. We're engaging like thousands or not thousands, like probably hundreds of top music students at, you know, the wheat music schools in the country who are improving models, understanding of music. And you also have the thing called, which we haven't talked about here, like a rubrics and a rubric like models are you can you can put a model in as a judge like you, you can, if you what is a good, what is a good educational design or what, what's a good MRI results? And instead of having some of these in some of these domains, you actually don't have a, a guaranteed correct right answer. And so models can sit in the middle as a judge and actually understand, you know, what is, you know, kind of like think back on your school days, like what it, how do you get an A on your 5000 word paper? Well, there's like a great introductory statement and they're scientific proof, you know, like, so you can build a rubric that allows a model to sit in the middle and actually auto evaluate responses. We're seeing a lot of rubrics work as well. Speaker 2 And you would think like, why would you trust this 1 teacher's opinion that this is the right way to do it? But that's cool is the market speaks for itself. If these models are being used more and more and people love them and value them. I imagine there's steps in between to verify this is good and other people think this is a good idea. It feels like the market dynamics will tell you if the data you're providing is correct and what people want. Is there something more there? Speaker 1 You know, I didn't get a PhD in in AI or math or physics and I haven't trained myself via front of your mouse. But you know, there is a lot to each unit of data, whether it's improving, if you know, there's a ton of science and research out right now around like how do you make sure that the data that you're producing is improving the model? And it's very hard for a model to understand. You know, they, they can really care about to, to zoom out. They care about three things they care about like quality first and foremost, like you have to have high quality data. And if you, you imagine you're training a model, like teaching a student and you're giving it the wrong data, it's extremely, you know, challenging to overcome that. So quality is first and foremost. And then the other huge problems you have is like a volume, like how, how do you generate thousands of of pieces of data in the most advanced domains of chemistry and mathematics and physics? And how do you ensure that high quality? Well, for us, we say in physics, we just reach out to students at Stanford and Berkeley and MIT and like they're the top GPA the at the best physics schools in the country. And so our ability to get to scale or volumes of data with that, it's a very high quality data is, is something they care deeply about. And then the other thing I'd say model builders care about is speed because they have all these hypotheses and they're constantly testing different pipelines. And so you might have like 3 or 4 bats going at once. And then as soon as one is actually showing a, a game, imagine you're a researcher or you know, you're scientific prostitutes once running again. Then you're trying to grow that pipeline and grow that piece of data that's actually improving it. And you're maybe ditching two or three other projects you had that weren't showing improvement. So your ability to quickly turn around for them and then in a period of days and then get to high volumes of data that are high quality is the number one thing they care about. And so there's quite a bit of technology we built on our side to assess each unit of data. We have our own post training teams, we're renting our own GPUs and we're trying to make sure that we can sit directly with these researchers and help share like what we're seeing with the data that we're creating and how it how it could improve the model, how they could best train with it. So hopefully that helps. Speaker 2 Going back to the types of post training, just because I think this might be helpful, at least for me, the mental model of there's pre training, there's post training within post training, there's reinforcement learning, human feedback, there's kind of this concept of fine tuning. There's also evals and stuff. Speaker 1 Like SFT? Speaker 2 SFT, which is supervised fine tuning, is that so? The stuff you've been describing is that. Would you mostly describe that as supervised fine tuning? Speaker 1 Yes, I'm working in the conference ring kind of doing all of the above. We don't do the auto eval. We we produce rubrics which are used in auto evals. Yeah. Speaker 2 OK, awesome. So essentially there's a model strained on all this amazing data. You guys come in after the models trained to tweak the weights based on additional data that you create. What's interesting is that this is a scalable system. I, I want to talk about just like the supply of amazing people that you have producing this, but it's amazing that humans can do this. Like you would think it needs to be this infinitely scalable thing, but like humans sitting there, adding creating data is working in improving the intelligence of models significantly. Speaker 1 Oh, yeah. I mean, I think like maybe a funny joke. It's like all the MBAs think this is also like going to go away. It's like. And I think for as long as models are improving, humans will be needed in this process. And when you talk to the lead scientists and researchers at these labs, it's like the data types will evolve and what they're trying to capture and collect. But, you know, there will be there'll be humans needed in the space for the next decade until we reach like full ASI. So yeah, it's I mean, you think about like, you know, a lot of them have struggle to do basic trajectories right now. So, you know, right now people are very focused on academic domains and I think they'll continue to be focused on academic domains, but they'll also be, you know, far, far more demand for professional domains as well across basically every every trajectory or step by step kind of problem that a knowledge worker solves in the workplace. You know, it's the pursuit of these labs to make sure that they're trying to collect the data to help add as much value in that process for humans as possible. Why AI won’t eliminate entry-level jobs So let me ask you about this. There's this tension I imagine people might feel between all these students training models to become smarter and smarter and smarter. And then there's that they will have harder time potentially finding jobs if models are so smart that people at entry level aren't being hired as much. How do you think about just that tension? Do you think this is a real problem or not? Or do you think this? Speaker 1 Goes, I'm brought in the camp of like GDP growth over like universal basic income. Like I, I like very much like believe that this is going to improve and accelerate every human's ability to like create an impact in the economy in the world. And that, you know, we're hearing from there's like a million companies that use handshake. Like we have one hundred, 100% of the Fortune 500 uses handshake. So we, we basically power the vast majority of how young people find jobs. And a lot of people are kind of hyperbolic at saying that all young people won't have jobs. And like, that's not what we're hearing from our employers. What we're hearing is like like social media marketing, like before you needed like somebody that could do Photoshop and take pictures and have created videos. They needed somebody that understood like marketing analytics platforms to track, you know, you're posting on different social media forms. It's like now one person, one like young talented AI native Iron Man suit enabled young person can get on and like they can build their own videos, produce their own creative assets, post across multiple social media platforms, run all of their own analytics. They don't need a data science degree to be able to do that. And that's an or like take an intern in our in our company. Like he had his first PR up. Like I think like the afternoon he started right, like you were APM, like you realized how how challenged that would have been historically your dev environment set up and like figure out where to add value. He just took a bug and and squashed it. And so I'm really a believer this is just like enabling human beings to be even more productive and create more impact. And yeah, like, of course, like, like hundreds of millions of jobs will become, you know, the jobs will evolve. Like, people would come displaced. They'll have to upskill and rescale. And I think Handshake has a huge role to play in in in helping knowledge workers evolve. Speaker 2 This has come up a couple times, this point that I think is really good, that younger people coming out of school are actually going to be much more likely to be successful because they're kind of growing up with these tools and are much more native to all these advanced tools. And so they just come in as beasts, just doing so much more. Speaker 1 Do you remember, do you remember like I mean, I, I, I used to a little predates me, but like you used to put a Google soak on as like a skill on your resume, right? Like you were in person, you were like good at Google like right, 'cause you like grew up with Google. It's like, I think being like AI native and having your Iron Man suit on and understanding how to watch these tools is like young people are at a huge advantage. Speaker 3 Especially if they're involved in training. Speaker 2 These models imagine there's some other cool advantage there I. Speaker 1 Mean just to hit on that, like what we're hearing from like our thousands of fellows is like they're in the classroom, they're actually producing research. Like we're talking about, you know, PhDs of the top institutions in the country. And like they, they can make like 100 hundred and $5200 an hour in their area in their field of expertise. It's pretty sweet. Like you can make like 25 bucks an hour being a teacher's assistant, or you can actually make $150.00 an hour breaking the latest models. And like you're learning what we're hearing from our, our fellows. It's like they're bringing a lot of those insights into the classroom to help them be more effective at teaching. More importantly, they're they're starting to learn how to leverage these tools actually advance the area of research. So they believe that these tools can help them advance their area of research by helping them be more effective with their time. And so it is quite cool to get kind of paid to learn a skill. The continuous improvement of AI models Before I get to the story of how this all emerged, because that is an incredible story, is there anything else about this whole field of labeling of reinforcement learning that you think people just kind of don't fully understand or you think that is really important? There's just like so much happening. Like I said, some of the fastest growing companies in the world are in the space scale was just like acquired for, you know, 30, like sort of acquired for $30 billion. Just like what else is there, if there's anything that you think people need to understand. Speaker 1 Generally speaking, like anytime that you're interacting with a model and you're asking it to do really advanced things and it's not performing your expectations. Like somewhere there's probably an expert that is, you know, the top mind in that domain working directly for the best researchers in the world at the Frontier labs trying to understand and go to the scientific iteration process of how to make that better. And that the assumption there is that like they already have the entirety of human knowledge that's written and recorded. And so, you know, for as long as there are problems in solving any problem with AI, you know, any human problem, there will need to be humans in the loop helping advance that. And like models don't generalize. I mean, there obviously the field will advance a lot and the type of data they'll collect a lot will will evolve a lot. But it's it's pretty exciting at the frontier. Speaker 2 Kevin Wheel is on the podcast the CPO at Open AI and he he made this point that really stuck with me that the model of today is the worst model you will ever use. Speaker 1 I love that line. Speaker 2 Will only get better this just boggles the mind and now we know why these are getting better because all the work you guys are doing just one quick question on this whole scale thing. I guess they were like, I don't know the main company doing this now they're swallowed up and Alex is running super intelligence and meta. Are they still like a big player in this labeling space or are they kind of out of it And and that's. Speaker 1 The whole scale team out of respect and for what what they built just many great companies operating the space. I think to the core of your question, it's like, I think if you were building the most, if you viewed your research team and your model building team and the experiments they're running to be, you know, really the cornerstone of how you're improving, you probably wouldn't want the latest research of what you're trying to work on being, you know, being invested in by a peer. I mean, this is generally what we hear in this space. And so we have seen a, an incredible search in demand and are, I think extraordinarily well positioned. We, we like to say like the only, the only Moat in human data is access to an audience. Basically there are, you know, many, many small players in the space, so mid sized players in the space and they're basically, you know, running TikTok ads, running Instagram ads, paying money for Google search, display ads, YouTube ads. And they will be like, can you get me 200 physics PhDs? They what do they do? They only can do one thing. They like, you know, they have 100 recruiters on staff. They all get on LinkedIn, they all send messages. They spend a couple 1,000,000 bucks on performance advertising campaigns. Somebody's scrolling on Instagram feed. That's a physics PhD of which you can't target them that well. And they like see, now come train a model. It's like, I've never heard of this brand before. The huge advantage for that we've had and why we've resonated so fast. The marketplace is like, we built a decade of trust with, you know, 18 million people and they trust us and, and we built a ton of brand affinity and they use Handshake, they have an active profile and we have a ton of information on their academic performance and what they've done in school. And so we're able to really target people really effectively and get to scale and volume of high qualitative faster than anyone else. And I think that competitive advantage of access to an audience is really resonating in the marketplace. Speaker 3 Today's episode is brought to you by Anthropic, the team behind Claude. I use Claude at least 10 times a day. I use it for researching my podcast guests, for brainstorming title ideas for both my podcast and my newsletter, for getting feedback on my writing and all kinds of stuff. Just last week, I was preparing for an interview with a very fancy guest and I had Claude tell me what are all the questions that other podcast hosts have asked this guest so that I don't ask them these questions? How much time do you spend every week trying to synthesize all of your research, insights, support tickets, sales calls, experiment results, and competitive Intel? Claude can handle incredibly complex multi step work. You can throw 100 page strategy document at it and ask it for insights, or you can dump all your user research and ask it to find patterns. With Claude Four and the new integrations, including Claude 4 Opus, the world's best coding model, you get voice conversations, Advanced Research capabilities, direct Google Workspace integration, and now MCP connections to your custom tools and data sources. Claude just becomes part of your workflow. If you want to try it out, get started at Claude dot AI Lenny and using this link, you get an incredible 50% off your first three months of the pro plan. That's Claude dot AI Lenny. The emergence of Handshake’s new business model OK, This is an awesome segue to where I wanted to go, which is just how how this business emerge. This is a business that you built on top of a business you've already had. From what I understand, you were at like $150 million in revenue. You've been at this for a long time. You found this opportunity. And now that I, you know, looking back, it's like, obviously this is an amazing idea. Labs need data. You guys have the supply of incredible experts. What an opportunity. Talk about just how you first realized this was something that you could be doing and should be doing, and then how you started to kind of execute down this path. Speaker 1 Yeah, I, I think it's been a pretty natural extension from like helping people jumpstart, restart or start their career, like, you know, monetizing your skills and in this new employment ecosystem is going to look very different in the future. And we wanted, you know, to, to, to zoom into like how we discovered it's like we because we have such a large access to this audience. And as the world shifted from generalists to experts, we're the largest expert network in the world. We have, you know, more pH DS 500,000 of them using Handshake than any other platform. We have 3,000,000 master students who are, you know, in school alumni. And so we started to see all the what I would call like middleman companies reaching out to us saying, can we recruit your pH, DS and master students? Then like any great marketplace, you know, we started sending them to these different platforms and started to really realize that, you know, from hearing from our users that like the experience was really frustrating. Like training was very transactional. The payments were, you know, there is very amorphous how you can get paid. Like there's immense amount of drop off in the process to actual project like completion and these other platforms. So we started to, we started to think the company was, you know, making 10s of millions of dollars from helping these other platforms. And we started to realize like what really kicked it off was like hearing also from the Frontier labs, like they started to reach out to us and started to go direct and trying to like almost kind of cut out the middle man. And we started to realize, well, we could really serve our fellows, our PhDs, our experts, we could treat them. We, we just believe there's like there will need to be a platform, an experts first platform in the pursuit of ASI and advancing AI. And there will need to be a place that everyone in the world could go to to monetize their skills and their knowledge as these labs are focused on improving in these, you know, in all these multidisciplinary outcomes. And yeah, we, we entered the business in, I really like, I started doing it over like Christmas and New Year's. Like that's when I sort of like flying around my family kind of thought it was a little wild that I was like on, on planes trying to chase different leaders. But we, we built an incredible team of people that came from the human data world and really started building out our platform in January and then started really monetizing the relationships about 5 months ago. Fast forward to today, we're working with seven of the Frontier Labs, basically every lab that's doing that's doing work and building the best large language models. And the team has exploded and revenues exploded and it's been, it's been really a incredible ride, kind of like running back, new company inside of a company for the second time over again. Speaker 2 And just to share some numbers, tell me if this is correct or if you're sharing these, but I heard that you hit $50 million in revenue just four months into this. Today, we're at 8 months in and you're on track to hit $100 million in revenue in the first year. Speaker 1 I think we'll blow through that number, but yeah, OK. Speaker 2 Incredible and I didn't even know there were 7 Frontier laps. That's. Speaker 1 You're at 50 is pretty good in four months I think. Speaker 2 Zero to 50 million in four months, That's something. It's like the bar has been shifting constantly. Like, you know, a year ago that'd be legendary. Now it's like, all right, well, another one of these, I think. Speaker 1 It's. Speaker 2 50 million in four months, no big deal. It's truly insane just to zoom out one second for people to that don't know a ton about Handshake, the original business. Incubating new ideas in established companies What was that like? What was actually this network that you have that you sat on top? Speaker 1 Of yeah, that that network does about 200 million and this will do about 200 million. Yeah. So that's, we have like 600 ish like superb passionate teammates that work on, on the core business, which is, you know, I would separate those like these aren't 2 businesses. I think it's like it's a one business, but that what is that business? It's the, if you're a young person in America that's graduated in the last 5678 years, you probably have a handshake on your phone. You like definitely know what handshake is. It's like a, it's a verb with young people in America. It's a verb with people that like are in College in their PhD or masters, you know, program. And it is, I call it an unconnected graph, meaning like you don't need to, you know, LinkedIn is very focused on like who you know and like what your experience is. The first question on LinkedIn is like, what's your job? And a lot of young people start off like they've never had a job before, right? They don't, they don't have like 500 connections to add to their to their to their craft. Whereas on handshake, you start off like trying to discover and explore and figure out how to navigate through a school and figure out, oh, I'm an engineer. Maybe I want to be a PM. Maybe I want to work at a start up. Maybe I want to look at a larger company. Like what are the pros and cons you want to learn from near peers and young alumni? And so handshakes this I, I call like a very like social platform with like groups and messaging and profiles and short form video and feed all focus on your interests and helping really like build your confidence in your early career to find your first job, your second job and to manage, you know, kind of 18 to 30, I would say. Speaker 2 And how long have that has that business been around? Speaker 1 It's been around 10 years, 10 years, OK. Speaker 2 So it's just like, again, it just feels like such holy shit, you guys are in the right place in the right time with the right network. That is extremely valuable. Now. What an interesting story. I feel like. I feel like it's just another interesting example of you've been doing something for a long time and then all of a sudden AI is just like opens up a whole new way of leveraging something that you have been doing for a long time. It makes me think a little bit about Bolt and Stack Blitz, which was building for seven years. This like browser based OS where you could run an OS in the browser and they're like, I don't know, no one needs this. Why are we, what are we doing? And then all of a sudden AI and they're like, oh, what if we build AI apps in the browser and just generate products for you with AI? And now it's, I don't know, one of the fastest growing companies in the. Speaker 1 World, Yeah. Speaker 2 So interesting. And so I think this is just an interesting time for our people to think about what have we done that may give us a new opportunity to build something huge based on this unfair advantage that we have. Speaker 1 I think also like as your company grows in size and headcount and maturity, it's also like hard to like incubate something new inside of a business. Like it's hard to, you know, it's hard in so many ways, right? Like the way that you build zero to 1 and find product market fit and scale team very quickly and is very different than the way that you run a a more mature business that has been around for 10 years with hundreds and hundreds and hundreds of people. So I've really. We've had a ton of fun and and been spent a ton of passion in like running it back again for the second time inside the business. And then, yeah, we have this massive strategic advantage, which is like no customer acquisition costs and we have like much higher conversion rates and retention than like any of the other platforms by a large margin because we have such consumer affinity. Handshake's competitive advantage There's actually 2 threads here I want to follow. I'm going to follow the second one first. This idea of where this data labeling work can come from. This isn't a really clear, simple, understandable one which is just experts sitting there creating data. Another one that I know a lot of other companies in the space use. Scale I know especially is just like low cost labor internationally. Are there other methods for doing this that isn't one of those two? How are other companies doing this? I think. Speaker 1 If you like, care about building a really high quality business and having like good gross margin and like high quality growth, like, you know, the, the, the ecosystem here is like one of the leading players has like they have like 200 recruiters. It's like unsustainable. There are like 200 people on LinkedIn sending individual messages to acquire these people because there's no brand, there's no trust. They spend, you know, they're spending 10s of millions of dollars a month on performance advertising Google ads. Speaker 2 To find experts and to find folks and and it's experts mostly at this point. Speaker 1 And then they put them onto an experience that like is treating them like they're drawing like boundary boxes around stop signs in the Philippines. Like, you know, the, the frontier tax accountants don't want to be treated like low cost international labor, right? And I, I don't, I mean, I don't think anyone enjoys that process. And so, you know, the ability to build a experience that's rooted in community, that's rooted in like high quality training. Like if you're getting your PhD at MIT, chances are you're just not being taught well enough on how to use the tools. Now you can't break the models. It's just like, you know, the other platforms, you know, they're spending thousands of hours to acquire an individual user and they're put right into a project with no training. So we just started from day one at building like this expert. We believe there'd be a deep network effect here that's very connected to our core business of starting, jump starting or restarting your career. And like, you know, you come in, you build a profile, you see the community, there's, you know, groups and a feed of here's how people are learning. Like you come into actual individual cohort with like peers that that look like you and have your similar background. You're being taught on how to interact and there's like a trial and error. And it's we have an instructional design key. So you can't do it. Then you're put on the projects we're building like, you know, there's certain swim lanes where we're actually pre building data and selling that data to all the labs. So we can do this thing where, you know, we produce 1 unit of data ourselves. We pay for it almost like a movie production. We pay for a unit of data and then we, you know, we make sure it's very high quality. We, we run our own post training on it. And then we produce a bunch of specifications of the data and we actually sell that individual package of data to like many different labs. And so that you get put on a project like that. Once you're doing a really, really good job on our projects, often times they will put you on customer projects where, you know, we they only want the best of the best people in, you know, machine learning, right? And then they go from our projects to their projects. And so, you know, there's a huge customer acquisition. I mean, it's a basic, you know, you all going deep on your podcast. So just to talk about it, it's like, you know, you really have a couple things that matter. You have a cost cost to customer acquisition at your CAC and you have your LTV, like the lifetime value of a user and an LTV is calculated pretty simply in this business, like it is based on the retention of a person and how many projects they can participate in. So if you treat people really well, you train them really well, right? Like, well, a, we have no customer acquisition cost because we partner with 1600 universities, power 92% of the top 500 schools in the country. We power almost every institution and Community College in the country. We have no customer acquisition cost to acquire the people. We'd have a ton of brand and trust with them built up. So they convert at, you know, really, really high rates. And then if you treat them really well and because that's what they expect from us, like they know, handshake their school pies, handshake. Like we, we need to treat, we, we care about treating these people well. But like the universities would not tolerate our partnership with these, with these fellows unless we treated them well. See, you put them into this process where our Lt. VS and repeat engagement rate and retention rate on different projects is, is really high. And so these structural advantages are quite significant when you contrast like a leading provider that has like 200 individual contributing recruiters and are spending 10s of millions of hours a month on performance marketing, you know, so that's I think why we've seen so much success. Speaker 2 So extremely interesting. And it feels like as you said there, we used to be a big focus on generalists, which is people anywhere in the world for low cost can do the work like draw bounding boxes around things and IT. And essentially the market has shifted from low cost generalist to experts. And a lot of these companies like Scale were optimizing for general work model training data. And you guys are set up to be extremely good at expert based data. And so you're in the right place at the right time with the right supply. What a business. Scaling up and meeting market demand Yeah, nice. Speaker 1 Work. I would say it's not been easy building business 2 inside a business 1. Speaker 2 But so let me actually, yeah. So let me follow that thread. That's where I want it to go. What was just that? Like, so you started noticing that model companies were coming to your people, that people are having hard times with some of these other companies in this space, and you're like, oh, maybe we should be doing this sort of thing. How did that just like initial inception start and how did you start to explore that idea and to see if it was a real thing? Speaker 1 Tactically, you know, we were working with many of the middleman companies doing work. We started to see the demand, as I talked about earlier, we we started to see direct outreach from the Frontier labs reaching out to us, trying to cut out the middle man in their pursuit of getting higher quality data. When we started to put together the dots on we, we could build a way better experience for our fellows. We could serve them directly to the labs and build a direct customer relationship with the labs and basically cut out the middle man and provide a better experience to the labs, brought a better experience to our fellows and provided a better experience long term to our like our million companies in the network. And you know, and you might, you might think about just like upscaling and rescaling what's going to happen there. So we want them to the space we started in, you know, really December exploring and learning more about it on like expert calls and hammering down. You know, I hired like 3 expert firms alpha in the alpha sites and like GLG and started doing a bunch of calls with the latest researchers because we had resources like one of the cool things about being or other companies like we, we have financial, you know, our core businesses, $200 million. Err, so it's like, you know, we, we, we had resources to be able to like accelerate the learning curve here. And then we started working with the arguably like the number one lab about 5 months ago. I wonder who that is? Yeah, Yeah. I wonder who it is depending with Benfmark key different answers working with the number one lab and and I've just you know, now we're working with Devin on the Frontier labs and the number one thing we're trying to do is just focus on like scaling up and we've gone from four or five people working on this to 75 plus people working on it. We're trying to I think we had like 12 people start last Monday. It's like we're, you know, we are so bottlenecked on just meeting this opportunity because in this market there's, there's essentially like unlimited demand. Like if you can produce high quality volumes of data, you most likely will be able to sell whatever you produce. And so on our side, it's like we're really focused on making sure that we pick the right longer term strategy, making sure that we don't grow too fast as to erode the trust that we built up with these Frontier labs. Yeah, but it, you know, it's, it's been, it's been fun. Overcoming challenges and adapting You said it's also been really hard to start this business within an existing business. What it's been, what's been hard, what's been hardest. You touched on a couple of these elements already, but what else? Speaker 1 I think I just kind of followed a lot more of my intuition around this doing this. I mean, the story of Handshake was we had to sign up 1600 universities. So I had to learn how to be like the best. We are the fastest growing higher education company in like history. So we to sign up to 1600 schools, then we had to build an employer business or you know, we had to figure out how to sell the 100%, you know, 70, you know, all these 4 private art companies use it and like 70% of it pay for it. So I had to learn about upmarket sales to like Goldman Sachs and General Motors and Google and the biggest companies in the world, which is totally different than selling universities. And then we had to learn how to build like an incredible student like kind of social network, like what is the, what is the best feed look like? What is group messaging look like? You know, so we had, I felt a little bit of familiarity in this like kind of 0 to ones of times like marketplaces are like many zero to ones. Sometimes I dream that we just like, I actually don't dream, but I make a joke that like, I just wish we were like a cybersecurity company. We had like one buyer and just like 1 product And it was just like, you know, we had to in a marketplace, you have to serve three different sides, you know, from your time at Airbnb. And so one of my warnings in spinning up these three different businesses in starting handshake was like, you know, you, I was pretty hands on. So like, you know, everyone reported directly to me. I really did not try to be like I, I really said in a lot of music, I'm not trying to be the boss. I'm just trying to be like another smart guy in the room. Like I hired, I was just, we've hired an incredible team of people that have had spent a lot of time in the space and have been big leaders at a lot of the human data companies in the space. And so everyone saw very clearly the structural advantages that we had and a lot of the focus was on making sure that we could deliver high quality data to 1 customer before we expand to anyone else. Like we just, you had to say no to a lot of things. And then you also had a lot of people in the core part of the business that rightfully SOAP like there's just checks and balances that. So a lot of people like try to get involved, right? Like everyone wants to say, not everyone misses a stretch, but you know, it's easy to say no, right? It's easy to be like, I I can't prioritize that this week or this month. I have an existing set of priorities. So you know, I essentially, with the exception of a few things, like everyone just came straight into this new orc that I built. Everyone did not have any responsibilities in the existing part of the business. It was extremely clear who was like the directly responsible individual across each area of the new Co. And now we've got deeper coupling and integration points across the rest of the business. But like we sat in a separate part of the office, you know, we are, we, you know, everyone's in the office five days a week, a lot of weekends. There's a totally different expectation in hiring talent too, where it's like, hey, this is a this is a 24/7 job, right? Like this is an early stage company with the compensation was also different too and based on like hurdles in this new business. So people felt like owners creating the new Co. And yeah, it's like it's still extremely nimble, very, very flat. You know, just because you want run one function doesn't mean you're directly responsible individual on a project. We picked the best person who's most capable of driving an initiative forward regardless of the function to be the DRI. We're a lot more metrics oriented. You know, when I, when I built Handshake, we, we, we resisted this like operating cadence for a long time, like this weekly, monthly, quarterly operating cadence with Handshake AI. We, we've been a way more focused on like operating with data and metrics and rigor from an early stage. This is a gentleman named Sahil on our team who's been doing an incredible job with that. Shout out Sahil. Shout out Young, shout out Paco. Yeah. Speaker 2 OK. This is incredible. So a few kind of elements of what allowed us to succeed within a decade old company. And by the way, so you're at 200 million a year in revenue with the traditional business. You're going to, as you said, blow past 100 million in the first year of this new business. So it's wild that in the first couple years, if things continue to go this way, you'll exceed the sizes, the run rate of a business that took you 10 years to build. The importance of separate teams and ownership Incredible to make this successful. A few of the things I noted as you were talking, One is clearly you were just like in founder mode. You are the CEO of this, you're like the lead of this new business you were taking. You weren't delegating to someone. Hey, go start this thing. You dedicated people here, we're going to pick people. You have nothing else going on. This is your new job. You're going to work on this. This stuff you worked in different part of the office, there's a different, there's a metric space cadence. So it's just like, let's stay really diligent about here's how it's going, here's where we're going, here's our track, here's our KPI's, things like that. Anything else there that you felt really important to making this work? Because a lot of companies are going to try to do this, I imagine. And so I'm curious what else you found important to make this work. Speaker 1 Yeah. I mean, I just really believe it's separate everything like separate engineering team, separate design team, separate accounts and operations team, separate finance team. Like early on everything was separate. People only had one job and one job only, and that was making Ng guy successful. We had a couple integration points more and I have it. I have an incredible executive team in the core part of business. And now there's becoming more and more involvement. But like, you know, I that our executives that have built handshake for a long time, like ran the core business and I focused 80 plus percent of my time and attention on just this. And you know, we hired an incredible engineer leader like Avery, who, you know, we, we focus on hiring a lot of we have a lot of entrepreneurs, people that have started companies inside the company or pardon me, people that started companies before like that was huge. A lot of familiarity with hiring talent that have like only worked at early stage companies before that feels super comfortable with ambiguity. We were also like way more up front around this is going to be chaotic, like just like owning that narrative, like in front of all hands at the core company, owning it directly with the team. We have a separate all hands. We have separate onboarding, We have a separate recruiting team. Like, you know, everyone was essentially, you know, I had some connection points, but mostly separate. And I think that was like absolutely critical. We took some of the top people and we've got great people in the core business. We took some great people from the core business and, and basically said, sorry, like I know you love your old team, I know you love what you're doing. Like will you join us in Hinchiki eye? And they like completely forgoed their historical response to voters and came over. That became really critical with engineering when things started to scale and topple. And like, you know, we're growing so quickly. We took some of our top senior engineers who are very entrepreneurial and principal engineers, staff level engineers, like parachute them in. And you know, that that's been like, it's been awesome to be able to like we have, it's been awesome to like ask some of the most talented people in the core business like, Hey, do you want to come over here and do this? And sometimes they say no, like they're like, I don't want to work. You know, most of the weekends, I don't want to be on the number of two AM, 3:00 AM nights we've done in this business. It's, it's, it's that, I mean, it's quite regular. Like people sometimes don't want to commit to that. But we've been up front, like here, here are the expectations for this team. It's a it's a you know, it's an insane pace. If you want to be a part of one of the fastest growing, you know, businesses in Silicon Valley, you can join it owner. The ownership too has also been huge, like owning this outcome and like we have we have this model like leave nothing to chance. Like I always for a while there, we like drew the number of days in the year on the white board and it was like, there will never be a time like this. I've never seen anything like it. I doubt I'll ever feel anything like this in business again where there's unlimited demand and it's just our ability to execute against it. And so we had this motto like leave nothing at a chance. Like how do you how do you make sure that three months from now, six months, you have like no regrets, Like get on the plane to go talk to a customer, like make the late night push check the data 6 times over again, like ship the extra feature that helps and really a huge celebratory culture too, like calling people out across. It's very flat right. So there's there really isn't this principle of, you know, the there's so many people putting up points, like directly calling up the people that are putting up points and creating a really fun environment around impact. I think it's been, it's been awesome. Speaker 2 Believe nothing to chance piece I imagine speaks partly to the value of trust in what you're doing. People are going to like you win if they can trust that your data is awesome and great and consistent and I could see why that ends up being such an important part of what you're building and like just listening to you describe this. The future of job matching with AI I understand like it's they're so it's obviously a massive opportunity. Obviously a massive advantage you guys have and just like the stress that comes with that burden also imagine is very high of just like this is, we can't screw this up. Speaker 1 Dude, cannot, cannot. Yeah, it's handshake should be a business does billions of dollars revenue, it's a public company like you should, we should be able to continue to. I mean, and it also helps our core business, like the longer term opportunity that we see is it's connecting, it's building the best job matching marketplace on the Internet. It's like, you know, it's probably one of the largest problems in the world, like labor supply matching. Like it's where people spend most of their time and energy, just hours of their life, they spend it at work. The process of like searching for a job, applying to a job is going to be completely reinvented with AI. We've been leading the charge there like, you know, an AI interviewer that's collecting the skills and actually asking about your experiences doing work, simulation experiences that, like help employers find the best candidate. To me, I don't know the last time you've done this, but like the hiring manager process of like reviewing 200 resumes. Are you kidding me? Like I'm going to sit there and review 200 resumes like not a chance five years from now, right? Like students manually making cover, like, not a chance, right? So there will need to be a marketplace that wins in connecting, you know, supply and demand and, you know, town with opportunity. And we think and get psyched about like the opportunity for impact here. Like that's my story. Like I went to Community College, a pavement with your school. I went to a no name school in the Upper Peninsula, Michigan. I worked at Palantir as an intern. It totally changed my life. And like I started handshake because I wanted to make it easier for like anyone, regardless of who you knew, what your parents did, what school you went to, to find a great opportunity. And I think AI what totally step function improvement in matching. And I think that our human data business is really serving as like the foundation for improving matching. Like a lot of things that we're doing in the human data business are being integrated to our core business. I think that's going to improve outcomes for employers, save them, you know, in the aggregate, like billions of dollars over time. And I think it makes the experience way better for students. So it's, it's just like we have to meet the moment, like, you know, I, we still have the stamina and the excitement and the passion internally in our core and in the new business to like go charge after this. And that's a lot of the messages we've been sharing internally. It's like it's, it's time to amp it up. It's time to like, this is a once in a life time opportunity to be positioned as well. And like we're we, we are going to meet the moment as a team. Speaker 2 It really is. This is very much feels like a once in a lifetime opportunity. Let me ask a few other questions along these lines that are something I've been thinking about, something that a lot of people think about. Just while I have you, there's always this question of will we run out of data? The biggest bottlenecks to advancing models further Will models stop advancing? Are we going to hit some plateau and there's not actually going to be some AGI moment, SGI moment. So we're first of all, you think we'll run out of data. There's a point at which we just can't produce more knowledge and data to feed these models. And kind of along those lines, what do you think is the biggest bottleneck to advancing models faster and further? Speaker 1 Yeah. I mean like it's just the type of data we're going to need is going to evolve. It's going to be CAD files. It's going to be, you know, scientific tool use data as they are trying to automate scientific discoveries and drug discovery. It's going to, you know, it's going to be esoteric, you know, operating systems that exists on, you know, scientific tools. It's going to be, you know, so I, I love this like trajectory and like stitching together step by step instruction following like, you know, there will need the type of data we're going to need is going to evolve a lot. And we haven't even talked about like multimodal and video and text and audio like audio is, is, is this huge demand for audio data right now. So the type of data is going to evolve. Speaker 2 Yeah, I use voice mode all the time. That's on my default chat DBT experience, just talking to it's. Speaker 1 Amazing. It's amazing. I just had a baby on we or my wife had a baby on Sunday and voice mode has been incredible. I mean, every night at, you know, every two hours it's like I have more questions. Voice mode, it's been huge. So I shout out voice mode And yes, the type of data is going to collect a lot or change a lot. I think synthetic data has a role to play in like in verifiable domains. But like what would consistently hear from companies is like, you know, they're synthetic data is not going to dominate. Like it's not going to be like the there, there's, there's, there's billions and billions and billions of dollars of value to extract as a company over the next decade and following the frontier of AI development. Speaker 2 Let me first say just huge kudos to you for just having a kid, your wife just having a kid a few days ago and building this business that is growing bananas and doing this podcast conversation. I really appreciate you. Speaker 1 Of course. Lightning round and final thoughts Is there anything else that we haven't covered that you think might be helpful for folks to hear or part of your story that you think might be helpful for folks to learn from or something you may want to just double down on that we've talked about before we get to a very exciting lightning round. Speaker 1 I mean, the thing I always love like talking, I'm really passionate about like people starting companies and helping them do so. And like, I just think in this moment right now with AI, like for young entrepreneurs that listen that read this podcast since I've been a reader since 2020, we lucked. Speaker 2 Yeah, we did check. That's incredible. Speaker 1 A long term reader, I'm just like so curious and love sucking up your interviews, but it's like you just focus on doing something like a meaning like that really helps people. And I think with AI there's like going to be so many opportunities to improve the way people learn. Like just, you know, I just really passionate about trying to make Handshake a platform that is not only an incredible business, but it's also something that like really helps solve a societal problem that matters. And yeah, it's maybe my one one shot out here. If anyone wants advice on how to do that or wants to reach out I'm like happy to chat. Speaker 2 OK, So this is an offer to share advice on starting companies within AI. Is that is that the offer here just? Speaker 1 Folks, it'd be great. Speaker 2 OK. I don't know how much time you have for the hundreds of thousands of people coming your way, but but I appreciate the offer. That's very cool. Anything else before we get to a very exciting lightning round? No. Well with that, Garrett, we reached our very exciting lightning round. I've got 5 questions for you. Are you ready? Ready. What are two or three books that you find yourself recommending most to other people? Speaker 1 I'm a I'm a sucker for Peter Thiel's zero to 1. I read it when I started the company and watched Peter Thiel's like start up school class at Stanford. He taught back in the days where there wasn't everything written on the Internet of how to start companies and like just think he's was the coolest love, love shoe dog, like think it, you know, so it pitted me of like starting a company hard things about hard things, obviously, but these are these are all quite common books. Speaker 2 But also classics. Ben Horowitz is coming on the podcast talking about hard things. About hard things. Super cool. The hard thing about hard things. Yeah. OK What? Have you seen a recent movie or TV show? You really enjoy it? I imagine you don't have much time for this, but I'm. Speaker 1 Going to get blasted for this, but I did start Game of Thrones to my wife and I. Speaker 2 For the first time, yeah, OK. Speaker 1 So I got a lot of hatching up too. Speaker 2 Why would you get no, this is great. That's like people that have watched it. You've loved it so far. OK. It's quite, quite gruesome. That's the only downside of that show. Don't watch it before you go to bed. I don't know how many gruesome scenes you've seen already. Do you have a favorite product you recently discovered that you really love? Speaker 1 The snoo, the baby automated snoo it's like has really helped us a lot. So love the shout out Snoo team. Speaker 2 Amazing at us new as well. We never actually turned it on, we just ended up using it as a best. Speaker 1 And yeah, mostly it's not turned on, but a couple cries, it's been turned on. It's been very helpful. Speaker 2 Your favorite life motto that you find yourself coming back to sharing with other people. Speaker 1 I love that like leave nothing to chance. Like leave it all out on the fields, you know, grow up and you know, like a really hard working family and dad worked really hard to provide make it make it happen for us. And it's like just give it your off, leave nothing to chance. Speaker 2 OK, so last question, I've been, I was researching you in prep for this podcast and there's a story that I love about your hustle early on is when you were you were going from campus to campus pitching schools to join Handshake. And there's a story where you had to shower in the the Princeton's pool to save money because you just didn't have a place to stay. Is there something there? Is there a story there you could share? Speaker 1 Yeah, So it was a tough one. I I mean, I almost got arrested at Princeton because, I mean, I guess for entrepreneurs that are traveling around all the time, you, you, we're sleeping out of our car. We had this like Ford Focus put 2030 thousand miles on it, sleeping the back of like McDonald's parking lots because they're well lit and had good Wi-Fi back in the day. And instead of staying in a hotel way to freshen up ahead of your meeting is like every university has a pool and the pool's almost always it is always open. We never had a situation where it's always open for people to swim in the morning, like fitness faculty, students and every pool, what do they have? They have a shower. So you could go to any pool, any university in the country and you can get a free shower and freshen up. So the Princeton campus security did not appreciate me showering as a non student, but I think it meaningfully helped us because the Princeton campus security like called the career service Center director was selling to being like, who's Garrett Lord? Like, is he really here to like pitch you software for your Career Center? And it made the start of the meeting with the Career Center, like really stimulating and exciting because they're like, you showered in our pool when you drove here. Yeah, we drove here from Michigan. You know, we like. And so I think that showed a level of commitment that was exciting for them. Speaker 2 Fast forward to all these founders now starting to use this growth lever, getting in trouble with the campus police to get better meetings with the school, school leaders. Incredible. Garrett, this is such a insane, amazing, inspiring story. Just like what you're building and the opportunity here and just how it's fast it's going and all the advantages have like if I was an investor in Handshake, I'd be like, all right, 10 years, it's going great. And that's like, well, holy shit. Where? Speaker 1 Did this come from? Speaker 2 Incredible. And it's just also really meaningful. So I'm really happy that you made time for this in spite of the madness you are in right now. Two final questions. Where can folks find you if they want to maybe reach out or maybe if you're hiring, like, let us know And then how can listeners be useful to you? Speaker 1 I mean, sign up for Handshake if you want to message me on there. It's the easiest way to to reach me. So you just find me Garrett Lord at Handshake and you find me on Twitter. Love or love Axe, huge Hijax guy. You can e-mail me at [email protected] and double R, double T. And how can you be helpful? Like we are trying to hire so many people. We have offices in New York and in San Francisco and London and Berlin. If you have friends that are maybe passionate about this, you want to know or you're interested in learning more, like please reach out. We'd love to talk to you. Hiring. Hiring is like the number one problem we have right now to meet the demands. So if you're talented and interested in learning more about Handshake, if you want to work on our consumer product, if you want to work on our employer product, cool PLG issues or the state-of-the-art consumer social experience like reach out or you want to work on the AI business, we'd love to talk to you. Speaker 2 To make it even more clear for folks, what roles are you most hiring for? Is it every role? Is it engineering? Engineering. All right, if you're an engineer and want to join one of the fastest growing AI companies in the world right now, here we go. We'll link to your careers page in the show notes. Thank you. Yeah, of course. Garrett, thank you so much for being here. This was incredible. Speaker 1 Of course. Speaker 2 Bye everyone. Speaker 3 Thank you so much for listening If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennyspodcast.com. See you in the next episode.

Podcast Summary

Key Points:

  1. Handshake, originally a LinkedIn-like platform for college students, launched a new data labeling business in January, reaching $50 million ARR in four months and on track to exceed $100 million ARR within 12 months.
  2. The business leverages Handshake's existing network of 18 million professionals, including 500,000 PhDs and 3 million master's students, providing a zero customer acquisition cost advantage.
  3. Post-training data (improving models after pre-training on internet data) is now the main source of AI model gains, focusing on expert-level data across domains like STEM, law, medicine, and finance.
  4. Experts (e.g., PhDs) identify model weaknesses, provide correct answers with step-by-step reasoning, and create high-quality data (e.g., JSON files) to improve model capabilities.
  5. Data types include prompt-response pairs, reinforcement learning with human feedback (preference ranking), and trajectory data (screen recordings, mouse movements, voiceovers of problem-solving processes).

Summary:

Garrett Lord, CEO of Handshake, discusses how his company transformed from a decade-old platform connecting college students with employers into a rapidly growing AI data labeling business. Handshake’s network of over 20 million students and alumni, including hundreds of thousands of PhDs and master's students, became invaluable to frontier AI labs. These labs need expert-level data for post-training models—a process that improves model capabilities after initial pre-training on internet data.

As generalist data becomes less useful, experts in advanced domains like physics, biology, and education are needed to identify model weaknesses, provide correct answers, and create step-by-step reasoning data. This data, often in JSON format, includes trajectory information (screen recordings and problem-solving processes) to teach models how humans think. Handshake’s new business launched from zero in January and reached $50 million ARR in just four months, with projections to surpass $100 million ARR within a year—outpacing their original business’s revenue.

The story highlights how AI disruption creates opportunities for companies that can leverage existing assets, like engaged expert communities, to meet the growing demand for high-quality training data.

FAQs

Traditional data labeling uses generalist labor for basic tasks, but Handshake leverages a network of 500,000 PhDs and 3 million master’s students to provide expert-level data. This allows them to break models in advanced domains where generalists cannot, creating unique data that improves frontier AI models.

Experts interact with state-of-the-art models to identify flaws in reasoning or answers, especially in advanced domains like physics or education. They provide correct ground-truth responses and step-by-step reasoning, often through tasks like breaking down a math problem to fix incorrect steps.

Handshake has a dedicated research and post-training team, including a hire from Meta, to assess each unit of data for its potential to improve model capabilities. They also use instructional design and assessments to train experts on tools and model interactions, ensuring high-quality output.

A trajectory is a complete record of a human’s problem-solving process, including screen recordings, mouse movements, and voiceovers. It captures how experts navigate tools, overcome roadblocks, and think step-by-step, providing rich data for training reasoning and tool-use models.

The rapid growth was driven by massive, unlimited demand from frontier AI labs needing expert data to improve models. Handshake’s existing platform gave them a strategic advantage with no customer acquisition cost, as they already had a large, engaged audience of PhDs and master’s students.

In education, a PhD with classroom experience, like a former eighth-grade teacher, interacts with models to spot flaws in teaching methods or curriculum design. They provide data on the best educational practices, helping models improve in subjective areas where there’s no single correct answer.

Chat with AI

Loading...

Pro features

Go deeper with this episode

Unlock creator-grade tools that turn any transcript into show notes and subtitle files.