The world of voice AI, with Mati Staniszewski of ElevenLabs

[MUSIC PLAYING] Maddie Stanishevsky co-founded 11 Labs in 2022, and has since scaled it to the $11 billion leader in AI audio. He's credited with capturing the humanness of speech to realistic emotional inflection, and they're now expanding into everything from agentic workflows to music. Actually, I'm this. Thanks, very much. Let me go play the stars is describe to me how-- I know how an LLM works at a high level, describe to me how an audio model works. Like if we were Carpathi style looking to build a toy one from scratch, how does it work? In early days, you try to replicate it exactly I go, replicate it with the human body. So you'll try to completely try to reproduce and machine, analog machine, that will create a vocal tract effectively. Then that progressed into trying to create effectively a digital signal for speech. Bell Labs was one of the first to try to create a structured set of signal that will represent the speech. And that is the first precursor to what we would do today. Then you would try to stitch in phonemes, effectively different sounds of how you'd speak humans, and then try to concatenate them together. It's another important part in that equation where you would, based on the most probabilistic approach of the next word, you would effectively try to bring the phonemes from your library of phonemes and bring them together. And then down to the modern history, where now we effectively do similar neural nets in other domains. So you predict the next sound, based on, of course, the context of the previous sounds, if it's a streaming speech. If it's a context of audio, you will use combination of predicting of the phonemes, but you also use the contextual text element of that work. And here, credit to my co-founder, Piotr, who effectively came with that new idea of how you can now create voice models, which are both reliable, high quality, quick, where you would bring a lot of the ideas from transformer models, from the fusion models, into the speech space. So that prediction of the next token on the phonem space wasn't something that was possible. You might be-- you always talk briefly about this, of how you can operate on the text, on the waveform space. There's also a mouse spectrogram space. So usually you do text, mouse spectrogram waveform. So what's the spectrogram space? It's like a visual representation of how the speech sounds, like other pitch, of course, energy, and then you transform that into waveform. So like, and wave net came along and tacitron models, they would effectively use text to mouse spectrogram so that visual representation. And then how you decode and encode that into the waveform to bring it across. And Piotr figured out how to abstract some of those steps. And the code and encode them a lot, a lot better. So that predicting of the next phonem was one of the big piece. And second big piece was, how do you bring that context into the equation? So what I mean by context is, the voice actor was reading a textual copy. You would know that, OK, this is a dialect sequence. I need to produce a dialect. If it's a happy sentence, I might need to pronounce it as a happy sentence. But what happens before and after comes into the equation and you need to bring that across. And then there's a last big piece. So voice model has the sound of how you intonate the given fragment. But the second big part is the voice itself of the characteristics of accents, of style, of prosody across that voice. So when you actually try to vocalize something, when you create that voice model, you turn text into audio, you need the text. You also need the voice reference of how you wanted to be spoken. So here was kind of the second big innovation. So part of the context is how you decode and encode those features. So when bell ups came with their initial representation of speech, the big piece there was, you would have effectively hard-coded parameters for that speech. With 11-lubs models. Hard-coded parameters for enthusiastic speaker, British accent. Exactly. Exactly. That kind of stuff. Why could the set of pitch elements that you can select, set of energies, spectrums you can select from? And in our approach, effectively, you would give the model open-ended ability to select what those parameters should be. So it's not going to be British, Polish, Spanish, English speaker. But the model will deduce them themselves the same for other set of parameters that are not hard-coded with it. The enthusiasm, whether it's the sadness, et cetera. Yeah, you're saying, kind of, Britishness is an emergent property. Exactly. In your voice models. Exactly. OK. Yeah, and those kind of those two big parts. So it's encoding and decoding of how you create a voice. Super hard problem before and figured out too how you then construct it in the sand. It's how you get the context across it and predict the next volumes. So how you bring them them together in a reliable, stable way while doing it quick. And these were kind of the two first big innovations in the voice models that continue to today. But OK, so if LLM's reason about text and words and parts tokens as the way they think about the world, what is the equivalent of a token in the voice model? You mentioned phonemes a bunch. Like, what is that representation? So we store the voice embedding effectively for the speaker. So you need that reference when you produce and create a speech. Yes. Of course, in the input to the voice model, you still get the text. And you bring the speaker and coding. And then when you produce speech, you do operate on the waveform or effectively on the phoneme level of that speech. And then when we kind of go the opposite, so of course, what is a phoneme? Fill in my understanding. It's like a syllable that constructed even to smaller elements. OK. And these are effectively the human sounds you can produce. Got it. So these would be like the most close to the representation. But of course, in our models, now it's going to be a combination of not only a gradient of phoneme level, you also create on the text level, you operate in both in sync because when you are predicting the context, you need to understand how that sentence will get constructed. And especially if it's more of a streaming real-time use case and like a voice agent setting, you need both parts to work across. So it's similar to how you operate on the token level on the text side. We operate on the token level on the audio side. It feels like a big part of the magic of 11 was your voices were much more human sounding. How did you accomplish that? So kind of give you a quick synopsis of what-- how we think about the models on the text to speech side today. In any model, you need the architecture. You need compute. You need data. So architecture innovations were one thing. The data part was second to the second big thing. With audio, you will have a lot of audio data available, but frequently you will not have it annotated in the right way. You won't have which speaker is speaking when some of the what is annotated, but how isn't. So like I said, speaking now, what's the emotions that we use? What are the actions that we use? So we would invest a lot internally on effectively creating our own data labor layers, our own team, to be able to create those data sets that will be better. And that was combination of, of course, like semi-automatic techniques and then manual techniques. And actually, a lot of the models that we did afterwards actually spun out from a lot of that research too. So speech to text model. Initially, was a model we did for ourselves because the models on the market just were good to annotate that data. And then another brilliant researcher on our team was kind of being able to construct it so we could span it out as a model that we brought to the customers. You've just been doing useful stuff in voice, and that has emerged with a whole bunch of products that you might not have expected because you find your building useful stuff. Exactly, exactly. And that's kind of combination of data of being able to do it automatically create a team that's coached on voice on how to describe it. Because most of the labor layers out there just aren't as well versed on understanding the audience voice. Help us a lot to bring that back. And then of course deploying those models in production, seeing how customers interact with them, having them annotate all of the data helped us refine those models over time. A very interesting thing on the side. So we spoke about kind of the speech representation. The first guy who created the speech representation is a guy called Kemple Land, Ron Kemple Land. So he created those analog machine that would represent effectively a human vocal truck and try to produce that sound. He'd spend decades on that and that kind of start producing balls. But that's the same person that created a chess machine, the first viral, let's say, chess machine, that would kind of simulate playing chess. This is a chemical Turk. It was called Turk. But exactly, but there kind of crazy thing behind it was operated by a human. And it was all a fluke. Yes, that's where the mechanical Turk, from which actually we use it out to kind of data labeling production to make that work there. Yeah, yeah. And sorry, we kind of jump right in. But if you describe the 11 business today, people think of you as the speech company. How should they actually think of your business to the extent you can describe the big areas? Text speech, speech to text, voice agents, just like break down the business for us. Cool. So in the nutshell, describe 11 laps as a research and product deployment company. We built foundational audio and voice models and then we built a platform for businesses to transform how they communicate with their customers, with their employees. And that will apply through AI agents from customer support, sales, hiring training, all the way through to marketing and storytelling for our creative tools. And in that set, we've created all types of foundational audio models. So text to speech models for producing speech, speech to text models that work over 100 languages and happily beat others on benchmarks, all the way through to conversational models of how you loop them together to music, to outer domain self audio. And then, of course, beyond the model, when you actually bring them to production, that's where the second level of the platform comes in, where that meets the businesses on the specific use case. So on the agent-specific example, it would be how you now connect those models to the knowledge base, to telephony, to the integrations that you need to perform the actions, how you evaluate and monitor the agent that it behaves in the right way, how you build the right sabre. On the creative side, on the marketing side, it's how do you create a good ad so you can create a good video voiceover for one of the campaigns, how you create an article that's narrated with a specific voice that represents the brand in a good way. So that's where we combine the models and understanding of the customers we work with into one policy platform. Every platform company has this question about how far they go into applications. So how do you think about where you go horizontal and power the whole ecosystem versus where you develop applications? Because like you can imagine there being a whole ecosystem of close captioning tools that grow up that again are built on the 11 labs tech. Like it's not necessarily a space that you would have to go after yourself. I think the big difference between a new kind of question. Today, we see ourselves as a platform where if you're building a horizontal use case in your business, a great place to come. If you have a lot of domain specificity, that's where I see a lot of kind of the application companies forming over time, where they will, where that's the specifically know the spaces we will go into. And I think it also is interesting when the tech is moving as quickly as it is here. It's one thing with SAS where you get these vertical specific providers. But I would imagine one of the biggest risks for you guys in being intermediated is if there's like in this example a close captioning service that is on a two versions old version of 11 labs and hasn't upgraded. That's a problem because you want people to be using the latest and greatest model that you've developed. And you'll be deploying new capabilities every week. And I presume that's part of your thinking is that just when it's moving vast quickly, you need to go direct in one of the cases. That's right. In the close captioning, here already now we know that our services is going to be able to tackle like 99.9% of the cases that customers have. And then there's like added benefit of we work with healthcare customers where we will create custom models for those customers where we'll get that transcription perfectly. The context is the tricky thing in close captions where we talk a lot about a lot of technical stuff on this. Yeah, for sure. And that's where you need effectively a dictionary of words that you've done before. Beforehand, which as we work with the businesses, we know we need to embed in that creation process. We're talking about that kind of products here. And one thing I know is that it loves our amazing. And you have the usage stats of check GPC in Gemini and all the popular elements where they're working on people using a ton. It feels like there's a big product overhang when it comes to voice, where the leading-edge voice models are incredibly capable. And yet I was driving home the other day and I needed to read a PDF that I was driving. And so I said, okay, I'll just have my phone read the PDF to me. And you can kind of try and hack it with like iOS green reader, but it doesn't really work with this scrolling. And then in theory, you can upload a Gemini, but you're trying to get it to not summarize it. And I actually just hung when I tried to press the, like, read this to me button. And so there's no way I could get my phone to read me something, which seemed like a fairly basic feature. And all cars advertise voice control. And yes, it sucks in separately. If you want to input something to the navigation, just no car has a good version of that. Yes, maybe Tesla does. And an end. And so why does it seem like with LLM's and Cloud Code and everything, we are using all the capabilities of the intelligence. Whereas with voice, we're like living 10 years ago somehow. Well, I'm thinking what I agree with the premise that we are 10 years behind in the lived experience of people today, like they're using series transcription, which has gotten better. And it's still way behind the leading edge. Yeah, like there's definitely a piece of like, I think the technology in many of those cases ready, there's a deployment gap towards your saying. It's like an automatic or some of the big companies are not adopting that quickly enough or bringing that into the production. But like plenty of different problems that you need to fix in a way. I mean, the quality of voice models for them to actually sound good. Like this is only like last three years thing. It was three years. It's a three years thing. Cars have over the years after updates now. So that's three years for the first voice model that can narrate text they think. Two years ago, we can start seeing the real-time version of that. And not really. I think the real break was like a year ago, where you can start seeing that in production. And then I think over 2025, the big piece that hasn't been possible is how you connect now the real-time voice interaction with something, which I think you're referring to. It has context of what you want to do, what is the material that you want to read, how does it connect to set of your preferences from the past and gets that across. I think that's like only recently became possible. And where we've seen like kind of the big adoption across the enterprises leading on the technical side. I think this year it should be in the automotive side too. Or the some of the applications. Okay, so you think we'll start seeing kind of great voice models in cars this year. This year for the on-cloud use case. It's like on-on-car in-car so like without connectivity, not yet. 20 tenets is deployment of course gap of like how you bring that into the gaps. But I think like the next two years, three years, how about the PDF reading use case? Like you should. That should work. I think, yeah. Well, how should I have done this? We've, so back in the day, we I'll preem this with a story to Q11 reader. But we had this problem where we have so many audiobook authors come into 11 apps. So 20 only for you, we released for a software. We had a lot of creators and then a lot of audiobook authors, or book authors that tried to put an afford professional relation and wanted to create an audiobook. However, none of the companies accepted AI audiobooks. And you can't sell an audiobook content. Exactly. It's audible with like block AI content. So we had no choice. Like we need to create an avenue for them to add because it was new distribution. Exactly. For AI audiobooks. Exactly. So we created a 11 reader and that kind of came with functionality where you can upload your PDF. You can upload it your text and have it read out loud with a number of incredible voices. So whether it's a "Sare Michael Cain" all the way through to a state and working together with "Sare Dijar Faeiman" where you had that. So this you are working with the "Sare Michael Cain"s of the world? Exactly. And then you can actually read it on that. And that kind of works extremely well. So that works. Yeah. Now, how can you do it? I think I do want everything read to me by Michael Cain. It's a great voice. Yeah. Shouldn't you guys have a consumer app where I can just do the common voice things? Like I want to be able to have an 11 app on my phone. And then if I upload a PDF to us, it can do the common things that I would like, such as have it read it to me. Yeah. That's exactly your own reader. So that works. Okay. The phone makers allow third-party keyboards. Do you think they do they allow third-party transcription engines? Will they, do you think? The phone makers, you said, right? It's like Apple and Google. Yeah. They have not-- So they have-- OS makers. Yeah, not all of them. Android-- With Android, you can work through it. There's variations that I got. Nothing.Tag and others. I feel like if you're the popular 11 app that allowed for transcription, people would use it a bunch and maybe eventually Apple would say, oh, we should allow third-party transcription engines if that's what people want. I mean, it seems like there might be going in that direction. Recently, they announced that we'll open up the elemental system. Hopefully, they will do the same with voice system, which is kind of similar. I think rational to do when it's moving so quickly. Yeah. The voice assistant paradigm is one of the oldest paradigm, UI paradigms in confusing. Like, the Open-the-Pod Bay doors hell from 1969. Yeah. I will claim it's not working yet. So Siri doesn't have the intelligence. And then on Gemini and Chat GBT and those apps, I mean, I want to use the voice mode. But I know that you just doesn't work. And so sometimes I'll be using my phone and I'll use the iOS keyboard transcription to type in the field. And then say a bunch of stuff and then send it off. But this suggests to me that consumers really want voice mode that works. And yes, it's just not working yet for the major LM apps or for anyone. Why isn't it working yet? It is pretty hard to do because you want two things. You want to be able to say things that you want, but you want sometimes for it to execute and sometimes to wait for you to finish and add something in the sentence. Sometimes you want it to be interactive. So it asks you questions back to clarify and get some of the additional details. And all of that is actually pretty hard. That's where the magical ideal version of a voice agent for us comes through, where you need the speech to text element. You need the transcription side. You need the turn-taking mechanism. So when do you finish sentence? When is it likely based on silence, based on the context? And then sometimes you want it to speak back and clarify or at least give you the text back to clarify and then maybe execute certain instructions. So that problem is still very hard research. So I agree with the claim that this orchestration side has not passed a true conversational agent hearing test where it behaves as you would expect from another person where you can say, like, the simple way of saying what I'm saying is that we have passed a hearing test with text elements a long time ago. I actually know her near that on voice elements. It's going to be interesting how that's a final frontier. Yeah, I feel like it's going to work in specific domains. Like in customer support call. Yes. Passes device to test works well. And like let's take another spectrum of that and interactive. gaming experience, like a truly interactive as you would have with another human in that game. It's so hard and further out there, we haven't passed it yet there. >> Yes. Yes. >> Yeah. But I think that's a combination of like even like a simpler version of within that, like sometimes you might give a response immediately back, sometimes you need a tool call to get additional information from the database, how you orchestrate that. So that's probably the most common thing we see as we work with some of the companies out there. You want those systems to orchestrate extremely well, where if it's a conversational use case, pretty simple, you can root the agent to speak with, but if you need to authenticate, if you need to pull additional information from the database, what do you do, how do you handle that graciously? >> That's right. >> To that extent, I would grade that. That's just getting there. Hopefully, our goal is to pass the voice-tuning test in all those cases, or the Turing test for all conversational agents outside of voice-tune. I hope we will all be there in the next year or so. >> Are you guys working on personalized voice transcription, where it feels like part of the way we're making it hard for ourselves is when I speak to Siri a bit of an accent, and so it sometimes has a hard time understanding me, but my accent doesn't change. It could just get good at listening to John, but my understanding is it's not. It's just running the global voice recognition model. I'm guessing it's the same for 11 labs where you're running the global voice recognition model, but again, you have an accent. If someone's understanding, if you watch out for someone in a coffee shop and said two words, they might have a hard time understanding it because they're not putting it through their matte Polish accent filter. Where is this going with actually interpreting the person that you know to exist on the other side? >> I have a very tricky one to detect. My voice is frequently used in the test. >> I work for the test. >> Yeah. >> For text-to-speech, for speech-to-text, for everything. >> Yeah. >> Yeah, it's a speech-to-text. >> But again, trying to parse your voice in a global model is just making life hard. It's like having a matte specific model. >> Yeah, so on the speech-to-text, I expect transcription exactly. The big part now that we are bringing in is you have two parts. One, effectively a person or a voice-specific detection. Which is true for the accent side, but it's also true for a crowded room. So that's where we have incredible research team that's like able to continually do both the accuracy high but also add things like speaker detection, of course, a noise reduction. But the second part is also keyword detection. So there are specific words that you would want to say in those settings that you want to effectively monitor for. So we spoke about, you know, like let's say I'm going to the coffee shop and order things. The set of actions, like the coffee shop would expect me to do. >> This is a very scary theory. It's like they can just listen out for the coffee words. >> Exactly. And then try to match it to the closet's proximity. So that both things will help. In a setup where you have my voice perfect, you can decode it and code it on that. If you don't have my voice or even if you want to double amplify it, we already support effectively a keyword detection. Which is useful for like real-time setting and async setting. So back to like cheeky pine transcription, you're going to effectively pre-generate that from the previous podcast and like look for a set of words that you would use traditionally in that. >> And so how hard, okay, so you do the keyword detection already. But how hard are the, I want to get superhuman transcription performance by feeding it an hour of Massey audio before it listens to Massey. And then it should be able to do a much better job transcribing. Is that just a really hard research problem? >> No, it's all the ball. >> And so we think we can roll it out in one of the next versions, which is like hopefully in the next months. >> So you think this year you're doing? >> Yeah, for sure. >> Person-specific transcription. Like we can already dire speakers extremely well. Like if we are speaking and of course, dissimulate who is speaking when? >> Yes. >> Which is like in transcription side, or part of accuracy, the realization is one of the harder problems. And we do that extremely well. >> And now it's going to be like effectively what you're saying. Like fine-tuning based on the speaker that I want to listen to. Which we know will be important. I mean like in healthcare setup, such an important part. You're in operating room, you're a doctor, you want to say a command, then you want to really be able to listen to that one person specific piece. >> Yes. >> You have a hardware device on home, let's say it's a pilot that helps you control the TV. Here too, you will want that to listen to you versus let's say the family roaming around. Or maybe you want it to everyone. So like you could do to decide that. But in many cases, you want to be able to specify that. >> Okay, that's really exciting. >> It's great because there's still so many unsolved research problems. >> Yeah, they just break through after breakthrough coming in the domain of voice models. How about on the flip side when it comes to speech generation? You know, the Zoom touch up my appearance feature. I've always thought about that in the context of voice, where should you offer a deaccenting filter for voices, or even there's one podcast that I'd actually listen to. But the voice is a little mumbly and I always thought they should put it through a de-mumbling filter. Just like making it see. >> Or slow down, slow down. >> Yeah, I make the initiation a little better. But all these things, again, like photoshopping an image, there's no reason that the, like, have you talked about voice to voice, basically, rather than voice to text or text to voice? >> Yeah, so there are kind of two big parts. One on the speech generation side, similar. So many, so many evations still there. There's like a wider piece and that's like, we released a V3 model that we're solving that for the first time, is like, can you control speech? So you can have the text to speech, you generate something that sounds emotionally great. Previously, until, until end of last year, effectively, we'd rely on model to decide what's the best performance. You could regenerate it, but that ultimately model decides the best performance. So that's where the controllability came in, where we can finally give it cues of say it in a slower way, or change how you deliver the dramatic pause, or kind of any cues that you give. And to be able to do that, you need the architectural changes, and the data that we kind of created over time, where you annotated what was said and how it was said, so we can actually train the model to do that. So today, finally, you can have both speech generation or entire voice agent experience, with what we call expressive mode, where the agent knows the emotions on the other side. So if the person is stressed, it can react and be reassuring. And that's generating a limb response on the reassuring side, and response in that a set of emotions, too. And that breakthrough was super hard to do. And that, of course, stretches to a lot of what you said. It could be some version of speech enhancement, in their real time, or in a post, a set up to change how that's delivered. And that's relatively recent innovation, and it's like, we know we can still be so much better. Like, the edge cases of how you want to describe it is pretty large. So that's one. And then the second part of the question, which is a huge question, that speech-to-speech models. So as you said, our approach, as you think about voice agent, conversational agent, is effectively cascaded approach. We use transcription and speech detects, a lamb text to speech, and orchestrates all of that together. And then you have a speech to speech, which again, goes directly from speech and there's a speech response on the other side. And we say speech to speech is that the idea that it doesn't go through text, doesn't include in the intermediate. Exactly. For performance reasons, for accuracy reasons. Usually for latency. For latency. So like faster to run a model that does not have to transcribe and then generate. Exactly. It's quicker, but on the flip side, you lose reliability. Yes. You look like all visibility into the parts of the pipeline. And in motionality, we think you can deliver both on both sides extremely well, and maybe you can make it more controllable too. So today we are optimizing heavily on a cascaded approach. That's right. A cascaded approach. Is the speech to text, going for the text layer. Okay. Going for the text layer. And as we work with like all of the businesses and enterprises, they will need that visibility into what happens. They will want to execute certain tasks. On top of that, they want a good visibility into each of the steps and great accuracy of all the models. But beyond that, they can abstract away what's the alarm layer, what's the intelligence layer, the integrations are easier in that system. So that's like where we are betting a lot of the research work of how you can make that great. And we think we can make that great. And speech to speech as you think about maybe more of like a companion version of the of the applications. That's where that will will flourish because maybe the hallucinations are not as important. But the latency is a little bit more and maybe hallucinations are even a feature. And maybe in the future, future is to finish that part. You will have like some version of combination of the models. That for like low complexity, easy models, you will have speech to speech. And for like higher complexity, you will have the cascaded. Okay. So we're going to ask about this. You know the way there is research on how the invention of writing changed humans' brains and just like change the neural pathways in ways beyond kind of the actual written language. Do you observe that speech to speech models think differently than cascaded models? Like it sounds like there's dumber. - They are definitely them. You need smaller model, you cannot, you-- - But that's interesting, right? That like forcing models to reason about texts, I mean, I know they just have much more in there as well, but they're smarter. - Yeah, but it's like, you know, like, if you are getting speech speech usually, you will use smaller model so it's still quick. So, you're gonna like, - I see, so it's also just a model-sized thing. - Yeah, it's also that way. - Okay, but are there interesting differences beyond like correlates like size? - When I can say it's like slightly different to your question, the people interacting for voice and the performance we see for like how they interact with the business changes just by nature of interacting with voice. - A good example. You can contact 11 laps and register for your interest, you go through the form, and at the end of that, we have supplemented that instead of going through the form process, you can speak with our agent and leave more details. And what happened are two things. One, people were actually much more keen to leave the forms through speaking with the agent, so we would go through the form a lot easier. But second, there would be a lot more open-ended in terms of what the use case are. So they would start giving us information about the wider set of use cases, the complexity of the use case. So like the writing out was tedious and tricky. - This is like an open-ended adventure. - Open-ended, you could ask follow-up questions, you could clarify, but people were just more at ease, and could like trust the system while doing that, that it's working, and that kind of helped us a lot. And then free, which maybe is more of a technological buyer, it also works across all languages. So now we have leads from all parts of the world coming in and leaving their details. So we did that use case, and now we have a few different companies building their HDR versions of that too, to help them capture the leads, coming in from banks all the way to, actually one of the automotive companies that leaves that where people are just more keen to speak through voice. - Sure. - Yeah. - And I want to ask about this kind of the second order effect. You have, you know, you talked in the past about how growing up in Poland, I guess the dubbing of TV shows, they were cheap and so they only have one voice actor for a TV show. So no matter all the parts, mail and feedback, I love you. I love you too. - You know, there's like one voice actor doing all of them. And now, you know, thanks to better voice models, you'll be able to just have like really good voices, AI generated for all the dubbing. 'Cause again, it's not like, it's taking jobs from great dubbing, those happy previously, it's like awful dubbing. You know, happening in Poland previously. So that's like one example of the, in the second order effects. What are the other second order effects you're seeing of ubiquitous goods, text-to-speech, speech-to-text, it seems like across a broad array of languages, 'cause whatever in it about in English, just this didn't exist in Polish or Irish or, you know, pick your language. - One, the breaking down the language barrier, we, you know, the kind of the inspiration came from, from the movie side, but it also applies in any communication setup, like, could in the future, could I travel to another country and speak Polish or speak English, and that language is being understood in the local native language. Like from Hitchhikey's guy to Galaxy, this version of the bubble first. So exactly, then you can like actually understand the world. And voice, of course, will be an interaction layer, but similarly, all of us will have our own kind of extension and voice agents that can help on our behalf. And there's like very clear and great examples of that of people that lost their voice and can get it for the first time, for the first time back. We see that everywhere, whether that's people that lost it due to ALS or fraud cancer that can get it back just recently. There was an example of a patient that had a neural link to work with them to bring the voice, that that person could speak with their own voice back to their, back with the family around. We worked with the lady that lost her voice before she got married. And then finally, technology became possible. We were able to recreate that voice. And for the first time, she could replicate the marriage ceremony and speak the vowels together, which was like such a hard-salt moment. Probably the most important from all the work that we do. When you guys talk about voice agents, is a voice agent just the idea that you have some long running or persistent agent that is going out and interacting with the world through voice? And so customer service would be one example of it. In the other direction, your claw going and making a restaurant reservation, actually calling up the restaurant, is that kind of how I should think about voice agents? That's right. It's exactly whether it's the reactive side of being able to talk to the customer or the product of to call it back. We recently had a very interesting one, typical because it was a Guinness-related one, where there was a developer developing a Gindex effectively. Oh, what's all that? They were calling all the pubs in Ireland, checking the price of a pint. Yeah, you could ask that or report information. Or that was called the Gindex, which was built with 11 labs, which was built with 11 labs too. So people could actually do both sides. Could you proactively reach out, reactively reach out, all was captured for voice. And then kind of 3,000, 3,000 different entities could report their prices and get that across. Have you, by the way, hooked up your open claw to 11 labs? Is the open claw 11 labs combo something there's a lot of people as 11 are doing? So, yeah, as you know, the open claw will kind of look for the most popular tools frequently where it tries to cook up. So 11 labs is one of the recommended to all of the top option for voice. Can you tell me a bit about the business of voice models, where I think people have an intuition around big LLMs, where there are these very expensive training runs. And yes, they kind of appreciate quickly, but there's so much usage that all of the models trained today have paid off their training runs and then some. And then there's this kind of ever larger capex going into-- I mean, a lot of it is inference these days, but also training. And so we have some intuitions from the LLM world. I'm curious just how I should think about voice models for one, how expensive is it training the voice models, is the expense in the researchers, is the expense in the training runs. And I mean, the economics is presumably kind of simple, or it's just per usage, but yeah, just talk us through the business. Yeah, definitely cheaper than LLM and MWD models. So you mean smaller models? OK, so models are smaller. Smaller. What's a parameter count for a leading edge voice model? A few billion to a lot, tens of billion per meter models. And for context, I think the-- I mean, kind of like CPUs moved away eventually from gigahertz as like the metric as they moved to more cores. I think we've mostly moved away from just raw parameter count, but I think the leading edge of LLM's are in the hundreds of billions of parameters. I think the leading ones, yes, but of course, you know you have the variations that you will use at lower scale. So capex is still pretty high. We've, of course, raised recently half a billion-- 11 billion valuation to like-- To make sense. To make sense, to continue being able to build the best models in the world. Researchers, of course, you want the best people in the world. I think we have those people working in audio and in Michael Fander, who is leading that work. So that's definitely a big piece of like-- not financially, but even like how you keep them vicious. The deployment, so you kind of continue building leading models helps you attract more talent and building that. And then on how we serve it, of course, inference is correlated with how the models are used. And for us, like we've seen an incredible growth across the work, mostly this is charged per-- if it's input text, so text to speech, it's usually per text token. If it's voice agent or transcription, then it's per minute. And we see that kind of being the bigger part. But usually, broadly, it's per token basis. And of course, as we work with businesses, it's like an annual agreement. The bigger to spend, the bigger to comment, the bigger to discount to go get across. The way we usually do is when we have a new model, we try to give it up cost to a lot of the customers so they can experience the best. It's still usually not as reliable. I just imagine the newest thing is often the most expensive. Whereas you make the newest thing the most economically attractive one. We try to make it attractive so the customers are-- it's more expensive for us than any previous generation. We don't like-- The quality is higher, so we try to keep the price still competitive to that. I used subsidizer, but it's inherently more expensive. Exactly. Exactly. Exactly. And over time, we might do some tricks to optimize it. But we want the customers to experience-- because of research, the big thing that we've seen is the reliability of the model in the early days might not be there. And then, too, people don't even know what's possible with that model. So you kind of want the wider set of distributions so people can show the world what's possible. So you can have it, of course, as the distribution mechanism learning yourself what to improve, what to change, and then get it out there. Are the voice models just getting bigger and bigger? Like, will we have voice models in the hundreds of billions of parameters, or have we found-- Like, it seems like for certain types of model architectures, there's like an upper limit on the natural size. Have we found that upper limit for voice models? It feels like for specific use cases, like say, audiobook narration. You probably found that size. You probably don't need to stretch it too much bigger to make the quality as much higher. But for certain use cases, that will probably grow. The thing that's, you know, like I hesitated on the question is, in a cascaded approach, you probably will not see dramatic size changes. You inherently want the models to be quick. In a reliable, you want to orchestrate them in a smart way. In a fused approach, probably that will get into like 10, 100 billion parameter models because you combine, of course, the L-L-M side and the voice side. So that will get bigger. But on the just voice, I think it will keep being small. >> Okay. But there are certain domains where we'll see big amounts. >> Yeah. Yeah. >> It is amazing how it does seem fun from a research point of view. How there are still these various unsolved aspects and how you guys are just making technical breakthroughs and then releasing them down the product pipeline. That's like a really fun stage of a company's lifecycle. For sure. It's fun because it feels like we can do innovations on both sides. There's so much on research side, so much on product side. And then the kind of ultimately the biggest paths that we deployed to the customers who are like SMB will have very different dynamic than the enterprise. It's not vendor-sas relationship where you just give the product out there for the biggest companies out there. But you are more of a part in there in their AI transformation part. So you want the resources to work alongside them to work on the frequently very new use cases that were impossible to help create and bring those voice agents to production. >> Yes. >> So that's like a big shift. >> Yes. >> But the biggest focus is how we bring the conversational agents out there to the businesses around the world. So when you say can bring conversational agents is the biggest priority? Is this for customer service type use cases like what are the most popular use cases for conversational agents? >> Yeah, like we want to be a partner for like full interactions between businesses and their customers or their audience. I'm saying that they're audience because that will apply in support. Support is the easiest one because that's where it's most ready. But like and that's maybe the big difference to how we see ourselves to some of the other companies in the spaces. This can also apply to sales. You can have the proactive side of reaching back. You can have AI is the R versions of that. And then you can have all the way to the marketing use cases where we are your partner for for working on on on even outside of like the conversational agents pays of how you create a great marketing campaign. So how does break down between you know we had Death Trainer from Intercom on here and they have finned their agent and it's a thing in the website that you can go talk to. And he described a very similar phenomenon that you described which is you start maybe thinking oh this will help me answer customer support queries. But it becomes like a generic UI for the you know for the website where it's a box you can type in to go do things and understand things. And so why wouldn't you read the docs in design your integration that way or whatever. And so will I have like one for text and then one for voice will you guys do text to will just how does that because it seems like this is also succeeding at the text level with financier and all these things. The places where we know we will be able to provide a biggest value is like where ultimately today you will have more either a big portion or most of the interactions coming for voice. So if that kind of intersection is there that's where we can provide higher value. And of course like if you if you need a text chat about there that's like if you fix the voice agent you'll have fixed text piece or like inherently as well. And by the place where we do optimize today is going to be like how do you select the right voice for the right customer interaction. How you pull that in the in the pretty complex case or what you mentioned earlier of like how you orchestrate that to pause or look for something deeper into the docs. How it can be extension of entirety of the business. So not only in support but across entire of the user journey. But bottom line is like we want to be able to provide you across entirety of the interactions. Voice is usually a big part of those interactions. And yes we need to we need to solve the integrations we need to solve the knowledge resource of the self text as part of that. But like we wouldn't for example go into what I think will happen in a lot of those cases like very deeply into reasoning version of those use cases where you maybe need to like the multi touch. A lot of complex acts. A lot of like financial analysis of like is like that would be not not something we optimize for. Can we talk about your revenue ramp? You're just one of the fastest growing startups period of the past few years. What's your most recently announced revenue figure? Most recently announced was end of 2025. So most recently announced was 350 at the end of 2025. Yeah. But the best proof of the technology working. So recently we are in as our work with Deutsche Telecom and team mobile with Revolute with Clark now with Meta and IBM. And why the set of use cases and this quarter was kind of one of the the best for enterprise growth where we had the first quarter hit 100 million in an additional ARR growth. I see. Which is great. In net new ARR. In net new AR. Okay. So if you're saying this quarter was 100 million in new ARR and 350 million at the end of the year. I'm no mathematician but it's up in the 450 million range. So and that's versus this time last year that's a several fold increase. Just what's working like from the outside I would assume that there is really strong cohort growth within accounts and then you seem to have self-serve and enterprise businesses that both contribute a lot. I know how big self-service but as a user I like to be able to fiddle with 11 labs and not have to you know go talk to sales but maybe you can just talk about what worked to reach 450 million plus of ARR so quickly. Yeah so exactly so we are over 50% is now sales as an enterprise. Yeah. And I think largely that technology that powers out of their agentic interactions just became reliable at the same time as high quality over the last year year and a half. So that's frequently you know this extremely well you will start the account and then of course it continues expanding and we see there's even land and expand motion on this level. What does that expand look like? Is it like new departments? Is it just the usage start to take off when a customer expands? Both but usually the first part too it's like we try to make it very easy for our customers. Maybe that kind of against ourselves where we give the technology at a pretty attractive economics because we so much believe in the technology providing value so you can actually try it and test it and then we tend to find the part of usage. Exactly that usage the kind of comment continues increasing because you know it's providing value and and then it's so much easier to make that a choice. Yes. And then of course cross department pollination is there too. And it is like you know our work of digital comes sort of marketing size so we did magenta work and Pock does generation and then it kind of expanded to customer support and then it expanded to us working on the in agent across the entire tier of the network so people can call in and have the agent so that you could see those step changes step changes across but we are now 470 people as a company so we keep keep on growing but some of the things that stay consistent is small teams so we have less than 10 people teams for each of the product or research initiatives or even as you think about sharding some of our go-to-market strategy those will be smaller teams understanding the industry and that understanding in market and that from going independently and going quickly so that definitely contributed largely to that. To especially on the biggest enterprises what we found works is and it's like we have the full spectrum self-serve PLG motion that helps drive distribution drive kind of awareness of 11 labs and on the completely other spectrum we have the high tide for the ployed engineering working side by side with the customers to to to to customize the entirety of their work together why did you guys do self-serve because I presume you have a lot of competitors where they have tech and it's behind the contact sales forum and you have to go talk to an SDR and then talk to an A.I. blah blah blah and you guys just offer the tech available on the side and I'm a huge believer in this I mean a huge part of Stripes growth has been driven by the fact that we just made Stripe available to anyone and built a lot of product around the adoption pattern but so many companies seem to skip it so I'm curious how you guys can. So many reasons so many reasons I think you know the quick ones that come to come to mind is feedback loop just you can immediately understand of how good your technology is too which is extension of that we stand behind our tech we believe it's is the best in the world for models for voice agents for deployment so we want people to experience that and I think you did the same in Stripe where the best version of the technology is available to everyone which is so attractive to like actually try it out we like always try to make everything we built for the highest in use cases bring it back to the ecosystem free frequently the newest of the use cases you know the for enterprise you will need reliability you need compliance you need the scale which we deliver so frequently as you develop new technology it might not not be ready for a lot of those parameters but it's definitely ready for developers and SMBs and we love what they are doing because they are showing us the future and effectively helping us find a trajectory of very loud and up should go I'm totally convinced how many just always amazed that more companies don't pursue us or it feels like they're really shooting themselves in the first by not like did you guys self serve on Stripe or did you we sell their own Stripe yeah for example you know 11 is a huge company and yes 100 percent you start on Stripe on a self-sripe you kind of like initially and it's like you know we were two of us at the at the beginning you try to see what's working in the industry but you try to think from first principle so you want to try it out you wonder understand how it works so the more friction elements before you're trying it out the less your trust whether it's available whether it'll be additional payment that's hidden behind some of those steps so you don't want to go through them so it's so much so big a Stripe do you have any Stripe feedback for us anything you want us to fix here my most - First common feedback until recently is like, why don't you give us pay us, you go and use it based billing type version. But one of our finance needs magic, I know I was speaking with your team, and that was day before, because he was like thinking about it for a long time. He was great. - Was it like you said, you guys should buy a metronome. - You should buy a metronome. And then the next day, metronome acquisition was announced. So now you have it. So that's my most common feedback, and it will be launching. That's a good announcement for this, for this, for this podcast, we'll be launching user based billing to everyone. - But I'm shocked. - Oh, as in previously. - So how has you got, pay us you got. - Okay, pay us you got. - You had it on enterprise basis, but everything on the self-serve basis, for some time. - So we had some subscriptions, yeah, subscription plans, you could go over them. But now we are launching a full pay us you go experience. So you can just try out voice engine, which is effectively this all orchestration loop, all the way food to any of the models, directly. - Going back to the self-service. - The new thing in AI is that all self-serve products should have pay as you go as an option. Maybe you want to have a subscription with some unlimited tiers, but I don't know if you had the experience of using Claude, and your typing away your queries, and eventually you hit some rate limit, and it's like, "Sorry, you've hit your usage limit, and you want to be able to do the thing that you can do a Claude code, which is just pay per a, it's like, "Fight out, pay for us." And it's kind of very funny as a consumer to not have the option to pay more, to use the product more. And so yeah, I think every AI product will need, they probably want to have some all you can need, most of what you can use, subscription with limits, and then the ability to pay for over just, so it sounds like that's what you're doing. - Yeah, exactly, that's what we're doing. - The other thing I wanted to ask you about is, I feel like all CEOs of larger companies today are trying to figure out how do all these AI advancements change the nature of the organization, and how do you redesign your organization a bit around all this new intelligence. And so that could be about what the scaling factor is of like the number of people you need to do the work, but it also should be like, do you need more senior people because they're better able to direct the AI's and the AI's are maybe kind of doing the work of what previously would have been junior people. Do you need more junior people because they're gonna be more AI native in how they work, do one smaller teams, do one bigger teams, how do you actually go to the process engineering of your finance team should be using cloud extensively, but like finance teams do not historically, they have a lot of home built software. And so there's all these questions that are floating around, and you have very rapidly built a much more AI native company, and so I'm curious what lessons we should all be learning from 11 labs as a large business recently built and so without the baggage of decades of how we've all been in this journey. - Yeah, we started this way between two, we're just a year when the two topics of the day were crypto and metaverse. So just before, and then I, of course, AI flow started. - Yes, you did scale. - Exactly, exactly. But we could like had a privilege of like scaling through the world when it was all happening for us what works and we really believe in that being the big part of the future. The first is small teams, like keeping the teams small and super flat. So like, can you have both me and my co-founder who will have over 15 direct reports each that will work with and most of those people will have that same scale of direct reports and-- - Okay, so your space control is way larger than the traditional company. - Normal, exactly. - You have double bass and obviously that's an exponential. - Exactly. And of course, there are some teams which in the short term might not do that, but ultimately that's where we think it's going to be headed. It's like roughly 10 teams size within each of those work items. - And start up pretty close. - No, thanks, but like start ups often have pretty wacko management ideas, like there's a funny tweez or grant me the confidence of early stage starter founder blogging about their management theories. But like, you think this is not a start effect? This is an AI effect, we're basically-- - No, it's not really a little start up effect. - Okay, yeah. - It's a-- - I think it's like-- - Okay, hindsight, hindsight, - And canceling our strike changes. - Yeah. - No, no, it's like, I need to pre-end it. I'm not going to, you know, it's the hindsight of this may be working. We'll see in like five to 10 years. - And there are places-- - Much flatter org. - Much flatter org. So it works for us, may not work for all the companies. And there are some parts where like go to market, we still are trying to figure out what's the best way. But smaller James, flatter org. And I think there are two paradigms, but like generally people being more technical. Or if not technical, even in non technical teams, having a technical resource. So, you know, we will have a person in ops or in talent that will, we have effectively a tech lead for that team. That helps them automate a lot of that work and helps up level the rest of the team too. - Yes. - So there are kind of two parts that are helping. - Okay, so talking through this in talent or something like that. Is it that you are building your own software or other companies might have bought software like a work day or a greenhouse or something? Is it thus they are using the existing software. You have better is the process that will be spreadsheet and a traditional company are built with software. How do you kind of use the software in these sorts of organizations? - Yeah, like sometimes, but we still use a lot of like the traditional vendors. Like one pattern is of course, allowing them to find everything, like making the data explorable for you to be able to interact with it. - Yes. - Like who is in the pipeline, what worked, who does the best reference, is like all of that work. So you can double down on that. But two, it's frequently things that you manually do that a lot of the car, like there's a gap between where the agents are today versus what you could do if you have the technical skill set. And a good example is like, how do you scrape all the right profiles to be able to reach out to the right candidates? So you're like analyzed whether it's, you know, how much I should want to say, but I try to detect specific things that we know worked. So you'll bring that across to the people. On go to market side, like there's just so many things you can do with additional amplifiers. You know, it goes from understanding what case studies are relevant and creating a good pre-read for you before you go to the meeting, through creating the AISDR experience that we spoke about to creating an entire deck experience. So you have like pre-populated deck with the right numbers that is customized to that customer, which you want still the person to go through and develop, but ultimately is in there. So there's plenty of those additional things that you know will amplify the work of the people around. Potentially replace some of those easier tasks that are done. And then there's like, you know, we wanted for people to explore the culture at 11 labs. So we created a voice agent that people can speak with and see what's the culture, but also get prepped for the interviews. They've been across many of those teams, like additional, additional benefit of what they can do. Interesting piece. So of course in Ukraine, we're done going work. They need to rethink a lot of how their development or systems, their support works for the citizens across the country. And people are in the war zone. They don't have the same access to the information that can't rely on us in phone lines. They cannot rely on the same physical services around the country. So they've developed effectively as central-- - You're a lawyer in the Ukraine. - We had a few, but they reached out because they were developing their central map called DIA. They developed it over the years, but now with war they were double-doning of how this can doubling down on how this can be a way of supporting the citizens. And of course there's an easy part of how you create the first-agenda government where you have a help with the benefits and what's happening on the front line or education. So that's delivered to everyone or health curse. You can book your check-up or appointment. So how you create all of that. And of course we traveled to Kiev. We worked with them on bringing that and making that available for voice or everybody can access it. But the thing we've learned while being there was that the model of what we speak about where you have technical resources in each of the teams. They actually have the same in every of the ministries. So every ministry had technical resources working on creating that agent like version of their work. And then it was like a central digital transformation team that would like assemble this altogether to deliver that through the central citizen support, which a photo is brilliant. - That's very kind of forward by Ukraine. - So tech forward. Like the most advanced set of work we've seen. So we got a little validated like, okay maybe technical resources in each of the teams is a good idea. And that works, I have for us. And you mentioned some of the other parts like the senior or younger. Like main thing we tried to filter for, of course the culture piece is so important. You can scale people, but it's scaling culture is much harder. So like you want to optimize for that being right. And in our case it's for its principles taking ownership, striving for excellence, but staying humble. And the main thing that's kind of in that ownership part that I think works well for the AI world is agency. Like if you are, if you have that agency to explore regardless of where you are in the experience cycle, it's going to be a tremendous amplifier to your work. - My biggest takeaway from all this has been that around agency where I feel like high agency people are the winners of the advances in AI and within organizations, low agency people will lose out. - Yeah, completely agree. Probably the most proud thing that Piotr and I are is as we scale the level of apps, the people that are at 11 apps, it's been like just the culture and seeing the expansion of the culture where culture builds the company now rather than any single person or any single product builds the company. That was probably the biggest validation and happiness. And there is the kind of the other angle of that where I think people are striving to be incredible in their craft and their work, but at the same time have fun and a lot of their work and that kind of combination of agency and just enjoying what you do is is probably the best thing we've been able to do today at 11.00. Well it sounds like a really fun stage like we're saying interesting research breakthroughs really fast growing business so I'm sure you're enjoying it. Adi, thank you. John, thank you so much.