Retrieval After RAG: Hybrid Search, Agents, and Database Design — Simon Hørup Eskildsen of Turbopuffer
60m 32s
In this conversation, Simon Eskildsen, founder of TurboPuffer, details the motivation and vision behind his company. The idea originated while he was consulting for Readwise in early 2023, where building an AI-powered article recommendation feature proved technically feasible but cost-prohibitive due to expensive vector database infrastructure. This sparked his insight into a latent market need for an affordable solution. TurboPuffer is designed as a search engine for unstructured data, combining full-text and vector search to serve as the external knowledge base for AI systems. Eskildsen argues that building a transformative database company requires three key conditions: a new universal workload (every company connecting its data to AI), a new underlying storage architecture, and the ability to support increasingly diverse queries. TurboPuffer's architecture is built entirely for the cloud era, eschewing traditional database designs by going all-in on object storage for data durability and leveraging NVMe SSDs for high performance, which was not feasible 10-15 years ago. This design aims to drastically reduce costs while maintaining capability, addressing the gap he identified during his prior infrastructure scaling work at Shopify.
I don't think I've said this publicly before, but I just call Lockheem, I was like, "Lock Lockheem, like if this doesn't have PMF by the end of the year, like we'll just like return all the money to you, but it's just like, I don't really, just need that I don't want to work on this unless it's really working." So we want to give it the best shot this year, and like, we're really gonna go for it, we're gonna hire a bunch of people, and we're just gonna be honest with everyone. Like, when I don't know how to play a game, I just play with open cards. Lockheem was the only person that didn't, that didn't freak out. He was like, "I've never heard anyone say that before." (upbeat music) Hey everyone, welcome to the Latent Space Park. This is Celestial, I'm the Recurnal Lats, and I'm joined by Swix, Editor of Latent Space. Hello, hello, we're so, we're recording in the Colonel Studio for the first time. Very excited. And today we're joined by Simon Eskildsett, of TurboFarfer, welcome. Thank you so much for having me. TurboFarfer has like really gone on a huge tear, and I do have to mention that, like you're one of, you're not my newest member, the Danish Akus Mafia, where like there's a lot of legendary programmers have come out of it, like Bjorn Stostrup, Rasmus Lerdov, Anders Halsberg, and the V8 team and Google Maps team. You're mostly a Canadian now, but isn't that interesting? There's so much like strong Danish presence. Yeah, I was writing a post not that long ago about sort of the influences, so I grew up in Denmark, right? I left, I left when I was 18 to go to Canada to work at Shopify, and so I would still say that I feel more Danish than Canadian. This is also the weird accent, I can't say TH, because this is like, I don't, you know, my wife is also Canadian. And I think like one of the things in Denmark is just like there's just such a ruthless pragmatism, and there's also a big focus on just aesthetics. Like there are like very people really care about like where, what things look like, and Canada has a lot of attributes, you as has a lot of attributes, but I think there's been lots of the great things to carry. I don't know what's in the water in Alhustow, and I don't know that I could be considered part of the Alhust Mafia quite yet, compared to the C-Cuts. The phenomenal individuals we just mentioned, Baratis Lodov is also a Danish Canadian. Yeah, I don't know where he lives now, but he's the PHP. Yeah, and obviously it'll be German, the Multicannitist, like this is like import that is an interesting talent move. I think I would love to get from you the definition of TurboPuffer, because I think you could be a vector DB, which is maybe a bad word now, and some circles you could be a third-changin, it's like let's just start there, and then we'll maybe run through the history of how you got to this point. For sure, yeah, so TurboPuffer is at this point in time, as third-changin, right? We do full-text search, and we do vector search, and that's really what we're specialized in. If you're trying to do much more than that, then this might not be the right place yet, but TurboPuffer is all about search. The other way that I think about it is that we can take all of the world's knowledge, all of the exabytes and exabytes of data that there is, and we can use those tokens to train a model, but we can't compress all of that into a few terabytes of weights, right? We can compress into a few terabytes of weights, how to reason with the world, how to make sense of the knowledge, but we have to somehow connect it as something externally that actually holds that, like in full fidelity and truth, and that's the thing that we intend to become, right? That's like a very holier than now, kind of phrasing, right? But being the search engine for unstructured data, is the focus of TurboPuffer at this point in time. - And let's break down so PEO may say, well, didn't Elasticsearch already do this, and then some other PEO may say, is this search on my data? Is this like closer to Ragn, then to like an XR, like a public search thing? Like how do you segment like the different types of search? - The way that I generally think about this is like, there's a lot of database companies, and I think if you want to build a really big database company, sort of you need a couple of ingredients to be in the air, which only happens roughly every 15 years. You need a new workload. You basically need the ambition that every single company on earth is gonna have data in your database multiple times. You look at a company like Oracle, right? You will, I don't think you can find a company on earth with a digital presence that it not doesn't somehow have some data in an Oracle database, right? And I think at this point, that's also true for Snowflake and Databricks, right? 15 years later, it's, or even more than that, there's not a company on earth that doesn't indirectly or directly is consuming Snowflake or Databricks or any of the big analytics databases. And I think we're in that kind of moment now, right? I don't think you're gonna find a company over the next few years that doesn't directly or indirectly have all their data available for search and connected to AI. So you need that new workload. Like you need something to be happening where there's a new workload that causes that to happen. And that new workload is connecting very large amounts of data to AI. The second thing you need, the second condition to build a big database company is that you need some new underlying change in the storage architecture that is not possible from the databases that have come before you. If you look at Snowflake and Databricks, right, commoditize like a massive fleet of HDDs. Like that was not possible in, it just wasn't in the air in the 90s, right? So you just didn't, we just didn't build these systems. S3 and, and so on was not around. And I think the architecture is now possible that wasn't possible 15 years ago, is to go all in on NVMe SSDs. It requires a particular type of architecture for the database that is difficult to ratchet fit onto the databases that are already there, including the ones you just mentioned. The second thing is to go all in on Obick Storage, more so that we could have done 15 years ago. Like we don't have a consensus layer, we don't really have anything. In fact, you could turn off all the servers that TurboPuffer has and we would not lose any data because we are completely all in on Obick Storage. And this means that our architecture is just so simple. So that's the second condition, right? First being a new workload that means that every company on Earth, either indirectly or directly, is using your database, second being there's some new storage architecture. That means that the companies that have come before you can't do what you're doing. I think the third thing you need to do to build a big database company is that over time, you have to implement more or less every query plan on the data. What that means is that you, you can't just get stuck in like, this is the one thing that a database does. It has to be ever evolving because when someone has data in the database, they over time expect to be able to ask it more or less every question. So you have to do that to get the storage architecture to the limit of what it's capable of. Those are the three conditions. I just wanted to get a little bit of the motivation, right? So you left Shopify, your principal engineer, InfraGuy, you also hit a kernel, labs, inside the Shopify, and then you can solve it for read-wise and that kind of gave you that idea. I just wanted you to tell that story, maybe you've told it before, but just introduce the people to the new workload that are a moment for TurboPuffer. For sure. So yeah, I spent almost a decade at Shopify. I was on the infrastructure team from the fairly early days around 2013. At the time, it felt like it was growing so quickly and everything, all the metrics were doubling here on year. Compared to what companies are contending with today, it's very cute in growth. I feel like my luck some companies were seeing that month over month. Of course, Shopify has been the compounding for a very long time now. But I spent a decade doing that. And the majority of that was just make sure the site is up today and make sure it's up a year from now. And a lot of that was really just the, you know, the cadascans would drive very, very large amounts of data to Shopify as they were rotating through all the merch and building out their businesses. And we just needed to make sure we could handle that, right? And sometimes these were events, a million requests per second. And so, you know, we had our own data centers back in the day and we were moving to the cloud and there's so much starting work and all of that that we were doing. So, I spent a decade just scaling databases 'cause that fundamentally was the most difficult thing to scale about these sites. The database that was the most difficult for me to scale doing that time and that was the most aggravating to be on call for was Elasticsearch. It was very, very difficult to deal with. And I saw a lot of projects that were just being held back in their ambition by using it. And I mean, self-hosted. Self-hosted, because yeah. And it's almost just like 2015, right? So, it's like a very particular vintage, right? It's probably better at a lot of these things now. It was difficult to contend with. And I'm just like, I just think about it. It's an inverted index. It should be good at these kinds of queries and do all of this and it was, we often couldn't get it to do exactly what we needed to do or basically get loose scene to do, like exposed loose scene raw to what we needed to do. So, that was like just something that we did on the side and just panics scaled when we needed to but not a particular focus of mine. So, I left and when I left, I wasn't sure exactly what I wanted to do. I mean, it spent like a decade inside of the same company. I'd like grown up there. I started working there when I was 18. You only do rails? Yeah, I mean, yeah, rails and knees are rails, guys. Love rails. So good. We all wish we could still work in rails. I know, no, I know. But I try learning Ruby. It's just too much like, too many options to do the same thing. It's my, I know there's a way to do it. I love it. I don't know that I would use it now, like given Cloud Code and cursor and everything. But still, if I'm just sitting down and riding a T-Sonal Code, that's how I think. But anyway, I left and I wasn't, I talked to a couple companies and I was like, I don't, I need to see a little bit more of the world here to know what I'm gonna like focus on next. And so what I decided is like, I was gonna, I called it like Angel Engineering where I just hopped around in my friends' companies in three months' increments and just helped them out with something, right? And just vested a bit of equity and solved some interesting infrastructure problem. So I worked with a bunch of companies at the time. Readwise was one of them, replicate was one of them. CAUSEL, I don't know if you've tried this. It's like a spreadsheet engine, yeah, where you can do distribution. They sold recently, we used out an FBNA at TurboPuffer. So a bunch of companies like this, and it was super fun. And so we're in the chat GBT moment,
I was with with Readwise for a stint. We were preparing for the reader launch, right, which is where you queue articles and read them later. And I was just getting their pulse grows up just enough, like which basically boils down to tuning auto vacuum. So I was doing that and then this happened and we were like, oh, maybe you should build a little recommendation engine and some features to try to hook in the LLM. They were not that good yet, but it was clear to us something there. And so I built a small recommendation engine, just okay, let's take the articles that you've recently read, right? Like embed all the articles and then do recommendations. It was good enough that when I ran it on one of the co-founders of Readwise, like I found out that I got articles about having a child. I'm like, oh my god, I didn't know that they were having a child. I wasn't sure what to do with that information, but the recommendation engine was good enough that it was suggesting articles about that. And so there was recommendations and it actually worked really well. But this was a company that was spending maybe five grand a month in total on all their infrastructure. And when I did the napkin math on running the embeddings of all the articles, putting them into a vector index, putting it in prod, it's going to be like 30 grand a month. That just wasn't tenable, right? Like Readwise is a proudly bootstrapped company and it's paying 30 grand for infrastructure for one feature versus five, it just wasn't tenable. So sort of in the bucket of, this is useful, it's pretty good. But let us, let's return to it when the cost comes out. Did you say it grows by feature? So for five to 30 is by the number of, like what's the scaling factor? It scales by the number of articles that you embed. It does, but what I meant by that is like five grand for like all of the other, like the Herokodino's, Postgres, like all the other, and this is the story of just 30. Yeah, and then like 30 grand for one feature, right? Which is like what other articles are related to this one? So it was just too much, right? To power everything. Their budget would have been maybe a few thousand dollars which still would have been a lot. And so we put it in the bucket of, okay, we're going to do that later, but we'll wait for the cost to come down. And that haunted me. I couldn't stop thinking about it. I was like, okay, there's clearly some latent demand here. If the cost had been a tenth, we would have shipped it. And this was really the only data point that I had, right? I didn't go out and talk to anyone else. It was just, so I started reading, right? I couldn't help myself. Like I didn't know what like a vector index is. I generally barely do about how to generate the vectors. There is a lot of hype about, this is early 2023. There's a lot of hype about vector databases. They're raising a lot of money. And so I didn't really know anything about it. It's like, you know, trying these little models, fine tuning them, like I was just trying to get sort of a lay of the land. So I just sat down. I have this get up repository called NAPKINMATH. And on NAPKINMATH, there's just rows of like, oh, this is how much bandwidth, like this is how many, you know, you can do 25 gigabytes per second on average, DRAM, you can do, you know, five gigabytes per second of rights to an SSD, blah, blah, all of these numbers, right? And S3, how many you could do, how much bandwidth can you drive per connection? I was just sitting down, I was like, why hasn't anyone built the database, where you just put everything on Obics Storage. And then you puff it into NVMe when you use the data and you puff it into DRAM if you're querying it a lot. So it's like, this seems fairly obvious. And you, the only real downside to that is that if you go all in on Obics Storage, every right will take a couple hundred milliseconds of latency. But from there, it's really all upside, right? You do the first query, it takes half a second. And it sort of occurred to me, it's like, well, the architecture is really good for that. It's really good for Obics Storage. It's really good for NVMe SSD. It's, well, you just couldn't have done that 10 years ago back to what we were talking about before. You really have to build the database where you have as few round trips as possible, right? This is how CPUs work today. It's how NVMe SSDs work. It's how, as three works that you want to have a very large amount of outstanding requests, right? Like basically, go to S3, do like that thousand requests are asked for data in one round trip. Wait for that, get that, like, make a new decision, do it again, and try to do that maybe a maximum of three times. But no databases were designed that way. With NVMe SSDs, you can drive, like, within, you know, within a very low multiple of DRAM bandwidth if you use it that way. And same with S3, right? You can fully max out the network card, which generally is not maxed out. You get very, like, very, very good bandwidth. And but no one had built a database like that. So it's like, okay, well, can you just, you know, take all the vectors, right? And plot them in the proverbial coordinate system. Get the clusters, put a file on S3 called clusters.json. And then put another file for every cluster, you know, cluster 1.json. Cluster 2.json. You know, that, like, it's two round trips, right? So you get the clusters, you find the closest clusters, and then you download the cluster files, like the closest end. And you can do this in two round trips. You end near as neighbors locally. Yes. Yes. And then you would build this file. I just like ultra simplistic, but it's not a far shot from what the first version of TurboPover was. Why hasn't anyone done that? In that moment from a workload perspective, you're thinking, this is going to be like a read heavy thing because you're doing recommended. Like, is the fact that like, rights are so expensive now? Oh, with AI, you're actually not writing that much. At that point, I hadn't really thought too much about, well, no, actually, it was always clear to me that there was going to be a lot of rights because at Shopify, the search clusters were doing, you know, I don't know, tens or hundreds of QPS, right? They used to have to have a human sit and type in. But we did, you know, I don't know how many updates there were per second. I'm sure it was in the millions, right? Into the cluster. So I always knew there was like a 10 to 100 ratio on the read right. In the read wise use case, it's even in the read wise use case, there probably be a lot fewer reads than rights, right? There's just a lot of churn on the amount of stuff that was going through versus the amount of queries. I wasn't thinking too much about that. I was mostly just thinking about what's the fundamentally cheapest way to build a database in the cloud today using the primitives that you have available. And this is it, right? You just, now you have one machine. And, you know, let's say you have a terabyte of data in S3, you paid a $200 a month for that. And then maybe five to 10% of that data needs to be an NVMe SSDs and less than that in DRAM. Well, you're just, you're paying very, very little to inflate the data. But when you say no one else has done it, would you consider Neon to be on a similar path in terms of being sort of S3 first and separating the compute and storage? Yeah, I think what I meant with that is just build a completely new database. I don't know if we were the first. Like, it was very, it was, I mean, I hadn't, I just looked at the napkin math and was like, this seems really obvious. So I'm sure like 100 people came up with at the same time. Like the light bulb and every invention ever, right? It was just in the air. I think Neon was first to, and they're trying, they're retrofitted onto Postgres, right? And then they built this whole architecture where you have, you have an in memory and then you sort of like, you know, M-map back to S3. And I think that was very novel at the time to do it for, for all Tp. But I hadn't seen a database that was truly all in, right? Not retrofitting it. The database felt built purely for this. No consensus layer, even using compare and swap on object storage to do consensus. I hadn't seen anyone go that all in. And I mean, I'm sure there's someone that did that before us. I don't know. I was just looking at the napkin math. When you say consensus layer, are you strongly relying on S3's strong consistency? You are. So that is your consensus layer. It is the consistency layer. And I think also, like, this is something that most people don't realize. But S3 only became consistent in December of 2020. I remember this coming up during COVID. And like, people were like, oh, like, it was like, it was just like a free upgrade. Yeah, it was just, it was just announced that we saw consistency, guys. And like, okay, cool. And I'm sure that they just, they probably had it in prod for a while. They're just like, it's done, right? And people are like, okay, cool. But that's a big moment, right? Like NVMe SSDs were also not in the cloud until around 2017, right? So you just sort of had like 2017 NVMe SSDs. And people were like, okay, cool. There's like one skew to dust is whatever, right? Takes a few years. And then the second thing is like, S3 becomes consistent in 2020. So now it means you don't have to have this like big foundation DB or like Zookeeper or whatever sitting there contending with the keys, which is how, you know, that's what Snowflake and others have to do. So they do that. They're gone. Exactly. Just gone, right? And so just pushed to the, you know, whatever, how many hundreds of people they have working on S3 solved. And then Compar and Swap was not in S3 at this point in time. By the way, I don't know what that is. So maybe you want to explain this. Yes, yes. So what Compar and Swap is is basically, you can imagine that if you have a database, it might be really nice to have a file called metadata.json. And metadata.json could say things like, hey, these keys are here and this file means that. And there's lots of metadata that you have to operate in the database, right? But that's the simplest way to do it. So now you have might, you might have a lot of servers that want to change the metadata. They might have written a file and want the metadata to contain that file. But you have 100 nodes that are trying to contend with this metadata.json. Well, what Compar and Swap allows you to do is basically just do download the file, you make the modifications, and then you write it only if it hasn't changed while you did the modification. And if not, you retry, right? Just you just have this retry loops. Now you can imagine if you have 100 nodes doing that, it's going to be really slow, but it will converge over time. That primitive was not available in S3. It wasn't available in S3 until late 2024. But it was available in GCP. The real story of this is certainly not that I sat down and like, big brained it and it's like, okay, we're going to start on GCS. At S3 is going to get it later. Like, it was really not that. We started, we got really lucky. Like we started on GCP and we started on GCP because to Shopify ran on GCP. And so that was the platform I was most available with, right? And I knew the Canadian team there because I'd worked with them at Shopify. And so it was natural for us to start there. And so when we started building the database, we're like, oh yeah, we have to build a, we really thought we had to build a consensus layer. Like, have a zookeeper or something to do this. But then we discovered the compare and swap. It was like, oh, we can kick the can. I think it will just do metadata on JSON and just it's fine. It's probably fine. And we just kept kicking the can until we had the
very, very strong conviction in the idea. And then we kind of just hinged the company on the fact that S3 probably was going to get this. It started getting really painful in like mid 2024 because we were closing deals with notion actually that was running at AWS and we're like, trust us. You really want us to run this in GCP and they're like, no, I don't know about that. Like we're running everything in AWS and the latency across the cloud was so big and we had so much conviction that we bought like, you know, dark fiber between the AWS regions in Oregon, like in the inter exchange and GCP is like, we've never seen a startup like what's going on here. And we're just like, no, we don't want to do this. We're tuning like TCP, Windows, like everything to get the latency down because we had so high conviction in not doing like a metadata layer on S3. So those were the three conditions, right? Comparance swap to do metadata, which wasn't in S3 until late 2024. S3 being consistent, which didn't happen until December 2020. And then NVMe SSDs which didn't end in the cloud until 2017. I mean, in some ways like a very big cloud success story that like you were able to like, put this all together but also doing things like doing, buying dark fiber that actually is, is something I've never heard. (laughing) I mean, it's very common when you're a big company or I do like connecting your own like data center or whatever, but it's like, it was uniquely just a pain with notion because the, or like most of the, like if you buying an Ashburn, Virginia, right? Like US East, the Google, like the GCP and AWS data centers like within a millisecond on each other on the public exchanges. But in Oregon uniquely, the GCP data center sits like a couple hundred kilometers like East of Portland and the AWS region sits in Portland. But the network exchange they go through is through Seattle. So it's like a full like 14 milliseconds or something like that. And so anyway, yeah, it's, it's, so we were like, okay, we have to go through an exchange in Portland. Yeah, and you'd rather do this than like, run your zookeeper and like, yes, way rather, just to have state. I don't want state into systems. Anything, all that is just informed by just me and my co-founder and I, I just been on call for so long. And the worst outages are the ones where you have state in multiple places that's not syncing up. So it really came from a, like just a very pure source of pain of just imagining what we would be okay being woken up at 3 a.m. about and having something in zookeeper was not one of them. We're talking to a connoisseur and there's something, do they care or do they just, they just hear about latency. The latency cost, that's it. They just cared about latency, right? And we just absorb the cost. We're just like, we have high connection in this. At some point, we can move into AWS, right? And so we just, we'll buy the fiber. It doesn't matter, right? And it's like five times and we, usually when you buy fiber, you buy like multiple lines and we're like, we can only afford one. But we will just test it that when it goes over the public internet, it's like super smooth. And so we did a lot. Anyway, it's, yeah, it was, that's cool. - You can imagine talking to GCP rap and it's like, no, we're gonna buy because we know we're gonna turn. We're gonna turn from you guys and go to AWS in like six months. - I mean, the meantime, we'll do this. It's a, I mean, like they, you know, this workload still runs on GCP for what it's worth, right? 'Cause it's so, it was just, it was so reliable. So it was never about moving off GCP. It was just about, honestly, it was just about, giving notion to latency that they deserved, right? And we didn't want them to have to care about any of this. We also, they were like, oh, Egress is gonna be bad. And was like, okay, screw it. Like we're just gonna, like VPC peer with you in AWS. We'll eat the cost. Yeah, whatever needs to be done. - What were the actual workloads? Because I think when you think about AI, it's like 14 milliseconds, it's like really, doesn't really matter in the scheme of like a model generation. - Yeah. - We were told to latency, right? That we had to beat. - All right. So, so we're just looking at the traces, right? And then sort of like hand, like, you know, kind of like looking at the trace and then thinking what are the other extensions of the trace, right? And there's a lot more to it because it's also, when you have, if you have 14 versus seven milliseconds, right? You can fit another roundtrip. So we had to tune TCP to try to send as much data in every roundtrip, pre-warm all the connections. And there is, there's a lot of things that compound from having these kinds of roundtrips. But in the grand scheme, it was just like, well, we have to beat the latency of whatever we're up against. Which is like, I mean, notion is a database company. They could have done this themselves. They do lots of database engineering themselves. How do you even get in the door? Like, yeah, just like talk through that kind of. - Last time I was in San Francisco, I was talking to one of the engineers actually who was one of our champions at notion. And they were just trying to make sure that the, you know, per user cost matched the economics that they needed, you know? Like it's like, the way I think about it is like, I have to earn a return on whatever the cloud charge me. And then my customers have to earn a return on that. And it's like very simple, right? And so there has to be gross margin all the way up. And that's how you build the product. And so then our customers have to make the right set of trade-offs that TurboPuffer makes. And if they're happy with that, that's great. - Do you feel like you're competing with build internally versus buy or buy versus buy? - Yeah, so sorry, this was all to build up to your question. So one of the notion engineers told me that they'd sat and probably on a napkin like drawn out, like why hasn't anyone built this? And then they saw TurboPuffer and was like, well, it's literally that. So, and I think AI has also changed the buy versus build equation in terms of, it's not really about can we build it? It's about do we have time to build it? And I think they, like I think they felt like, okay, if this is a team that can do that, and they feel enough of like an extension of our team, well then we can go a lot faster, which would be very, very good for them. And I mean, they put us through the tests, right? Like we've had some very, very long nights to do that POC, and they were really our biggest, our second big customer after cursor, which also was a lot of late nights, right? - Yeah, I mean, should we go into that story? The sort of Chris' story, like a lot, they credit you a lot for working very closely with them. So I just wanna hear, I've heard this story from Swally, it's one of you, but I'm curious of what it looks like for your said. - I actually haven't heard it from Swally's point of view. So maybe you can now cross reference it. The way that I remember it was that the day after we launched, which was just, you know, I'd worked the whole summer on the first version, justine wasn't part of it yet, 'cause I didn't tell anyone that summer that I was working on this, I was just locked in on building it, because it's very easy otherwise to confuse talking about something to actually doing it, and so it's like, I'm not gonna do that, I'm just gonna do the thing. I launched it, and at this point, TurboPuffer is like, a rust binary running on a single eight core machine in a team of instance. And me deploying it was like looking at the request log, and then like command seeing it, or like control seeing it, to just like, okay, there's no request, let's upgrade the binary. Like it was like literally the scrappiest thing you could imagine. It was on purpose, because it's just like at Shopify, we did that all the time. Like we like, we ran things in Teamix all the time to begin with before something had, like, at least the inkling of PMF. So it's like, okay, is anyone gonna care about this? And one of the cursor co-founders, Arvid, reached out, and he just, you know, the cursor team on like all IY, IMO, like, contenders, right? So they just speak in bullet points, and facts. Just like this amazing email exchange, just of, this is how many QPS we have. This is what we're paying. This is where we're going, blah, blah, blah. And so we're just conversing in bullet points. And I try to get a call with them a few times, but they were so, they were like, really riding the PMF ball here, it was like late 2023. And one time, Swally emails me, I'd like five, no, what was it, like 4 AM, Pacific time? Saying like, hey, are you open for a call now? And I'm on the East Coast, and it was like 7 AM, like, yeah, great, sure, whatever. And we just started talking, and something then, I didn't know anything about sales. It was something that just compelled me. I have to go see this team. Like, there's something here. So I went to San Francisco, and I went to their office. And the way that I remember it is that Polskruss was down, when I showed up at the office, did Swally tell you this? No, okay, Polskruss was down, and so it's like, they were distracting with that. And I was trying my best to see if I could help in any way. Like, I knew a little bit about databases, back to tuning auto vacuum. It's like, I think you have to tune auto vacuum Swally. And so we talked about that, and then that evening, just talked about what would it look like, what would it look like to work with us? And I just said, look, we're all in. Like, we would just do whatever you tell us, right? They migrated everything over the next week or two, and we reduced our cost by 95%, which I think kind of fixed their per user economics. And it solved a lot of other things. And we were just, just, this is also when I asked just to come on as my co-founder. She was the best engineer that I ever work with at Shopify. She lived two blocks away, and we were just, okay, we're just gonna get this done. And we did, and so we helped them migrate, and we just worked like hell over the next like month or two, to make sure that we were never an issue. And that was the cursor story, yeah. And it's code a different workload than normal text. I don't know, is it just text? Is it the same thing? - Yeah, so cursors workload is basically, they will embed the entire code base, right? So they will chunk it up in whatever they would, they do, they have their own embedding model, which they've been public about. And they find that on their e-vows, there's one of their e-vows where it's like a 25% improvement on a very particular workload. They have a bunch of blog posts about it. I think it works best on larger code bases, but they've trained their own embedding model to do this. And so you'll see it, if you use the cursor agent, they will do searches. And they've also been public around how they, I think they post-trained their model to be very good at semantic search as well. And that's how they use it. And so it's very good, like, can you find me on the code that's similar to this or code that does this in just, in just queries? They also use Grap to supplement it. - Yeah. - Of course. - It's been a big topic of discussion, is raggeded because Grap, you know. - And I mean, like I just, we see lots of demand from the coding company. - You do the semantic search, right? - Every part, yes. We see the results.
And so I mean, I like case studies. I don't like just doing like thought pieces on this is where it's going and like trying to be all macroeconomic about AI that's as turned out to be a giant waste of time because no one can really predict any of this. So I just collect case studies and I mean, cursor has done a great job talking about what they're doing and I hope some of the other coding labs that use TurboPover will do the same. But it does seem to make a difference for particular queries. I mean, we can also do text, we can also do rejects. But I should also say that cursors like security posture in the TurboPover is exceptional, right? They have their own embedding model, which makes it very difficult to reverse engineer. They obfuscate the file paths. They ought like, it's very difficult to learn anything about a code base by looking at it. And the other thing they do too is that for their customers, they encrypt it with their encryption keys in TurboPover's bucket. So it's really, really well designed. And so this is like extra stuff they did to work with you because you are not part of cursor. And this is just best practice when working in any database, not just you guys. Yeah, I think for me, the learning is kind of like, all workloads are hybrid. You want the semantic, you want the text, you want the reg X, you want SQL, I don't know. But it's silly to be all in on one particularly query pattern. I think I really like the way that Swally at cursor talks about it, which is, I'm going to butcher it here. And I'm a database scalability person. I don't know anything about training models other than what the internet tells me. And the way he describes it, this is just like cache compute. It's like, you have a pointed time where you're looking at some particular context and focused on some chunk. And you say, this is the layer of the neural net at this point in time. That seems fundamentally really useful to do cache compute like that. And how the value of that will change over time, I'm not sure, but there seems to be a lot of value in that. Maybe talking a bit about the evolution of the workload, because even like search, maybe two years ago, it was like one search at the start of like an LLM query to build the context. Now you have a gentex search however you want to call it, where the model is both writing and changing the code. And it's searching it again later. Yeah, what are maybe some of the new types of workloads for like changes you've had to make to your architecture? I think you're right. When I think of RIG, I think of, hey, there is an 8,000 token context window and you better make it count. And search was a way to do that. Now, yeah, everything is moving towards-- just let the agent do its thing. And so back to the thing before, the LLM is very good at reasoning with the data. And so we're just a tool call. And that's increasingly what we see our customers doing. What we're seeing more demand from our customers now is to do a lot of concurrency. Like, notion does a ridiculous amount of queries in every round trip just because they can't. And I'm also now, when I use the cursor agent, I also see them doing more concurrency than I ever seen before. So a bit similar to how we design the database to drive as much concurrency in every round trip as possible, that's also what the agents are doing. So that's new. It means just the enormous amount of queries all at once to the data set while it's warm and it's few turns as possible. Can I clarify one thing on that? Yes. Are they batching multiple users or one user is driving multiple groups? One user driving multi-reg. One agent driving parallel searching a bunch of things. Exactly. So the conditionals that did this for the fast context things, like eight parallel at once. Yes. And an interesting problem is, well, how do you make sure you have enough diversity so you're not making the same request eight times? And I think that's probably also where the hybrid comes in, where that's another way to diversify. It's a completely different way to do the search. That's a big change. So before it was really just like one call, and then the LLM took however many seconds to return. But now we just see an enormous amount of queries. So we just see more queries. So we've tried to reduce query. We've reduced query pricing. This is probably the first time actually I'm saying that. But the query pricing is being reduced like 5X. And we've probably tried to reduce it even more to accommodate some of these workloads of just doing very large amounts of queries. That's one thing that's changed. I think the right ratio is still very high. Like there's still an enormous amount of rights per read. But we're starting probably to see that change if people really lean into this pattern. Can we talk a little bit about the pricing? I'm curious. Because traditionally a database with charge of storage. But now you have the token generation that is so expensive where the actual value of a good search query is much higher because they're saving inference time down the line. How do you structure that? Is what are people receptive to on the other side too? Yeah. The turbo buffer pricing in the beginning was just very simple. The pricing for search engines before turbo buffer was very server full. It was like, here's the VM. Here's the per hour cost. Great. And I just sat down with a piece of paper and said, if turbo buffer was really good, this is probably what it would cost with a little bit of margin. And that was the first pricing of turbo buffer. And I just sat down and was like, OK, this is probably the storage amp or whatever on a piece of paper. Vibracing. It was very by price. And I got it wrong. Oh. Well, I didn't get it wrong. But like turbo buffer wasn't at the first principle pricing. So when cursor came on turbo buffer, it was like, I didn't know any VCs. I didn't know anything like that. I didn't know anything about raising money or anything like that. I just saw that my GCP bill was a lot higher than the cursor bill. So just in it, I was like, well, we have to optimize it. And I mean, to the chagrin now of the VCs, it now means that we're profitable because we had so much pricing pressure in the beginning because it was running on my credit card. And just in it, I had spent like tens of thousands of dollars on compute bills and spending off the company and very bad Canadian lawyers and things to get all of this done because we just like, we didn't know. If you're steeped in San Francisco, you're just like, you just know, okay, like you go out, raise a pre-seed round. I never heard of a pre-seed at this point in time. - When you hit cursor, you had no funding. - With cursor, we had no funding. Yeah. By the time we had no, "Locky was here." Yeah. So it was really just, we've vipriced it 100% from first principles, but it wasn't, it was not performing at first principles. So we just did everything we could to optimize it in the beginning for that. So that at least we could have like a 5% margin or something. So I wasn't freaking out because cursor's bill was also going like this. Because they were growing. And so my liability and my credit limit was like actively, calling my bank is like, I need a bigger credit, it was, yeah, anyway. That was the beginning. Yeah. But the pricing was, yeah, like storage, rights and query. Right? And the pricing we have today is basically just that pricing with duct tape and spit to try to approach like, you know, like a margin on the physical underlying hardware. And we're doing, this year, we're going to see more and more pricing changes from us. Yeah. And like, it's how much does stuff like VPC peering matter because you were working in AWS land where egress is charged and all that, you know? We probably don't, like, we have like an enterprise plan that just has like a base fee because we haven't had time to figure out skew pricing for all of this. But I mean, yeah, you can run TurboPuffer either in SaaS, right? That's what cursor does. You can run it in a single tenant cluster. So it's just you. That's what notion does. And then you can run it in BYOC where everything is inside the customers VPC. That's what, for example, in Thropic does. What I'm hearing is that this is probably the best CRO job for somebody who can come in and help you with this. Like TurboPuffer hired, like, I don't know what number this was, but we had a full time CFO is like the 12 hire or something at TurboPuffer. I think I hear a lot of company. I don't know how they do it. Like they have 100 employees and not a CFO. It's like having a CFO is like a run out of business, man. Like, you know, it's so good. Yeah. Like money might like he just, you know, just handles the money and a lot of the business stuff. And so he came in and just hopped a lot of the operational side of the business. So like CFO, CFO, like somewhere in between. Just as quick mention of Lucky, just to some curious, I've met Lucky and like he's obviously a very good investor in our physical intelligence. I call it a generalist super angel, right? He invests in everything. And I always wonder like, you know, is there something appealing about focusing on developer tooling, focusing on databases, going like, I've invested for 20 years in databases being like a Lucky where he can maybe like connect you to all the customers that you need. This is an excellent question. No one's asked me this. Why Lucky? Because there's a couple of people that we were talking to at the time. And when we were raising, we were almost a little, we were like a bit distressed because one of our peers had just launched something that was very similar to TurboPuffer. And someone just gave me the advice at the time of just choose the person where you just feel like you can just pick up the phone and not prepare anything and just be completely honest. And I don't think I've said this publicly before, but I just call Lucky. I was like, "Lock Lucky." Like, if this doesn't have PMF by the end of the year, like we'll just like return all the money to you, but it's just like, I don't really, just think that I don't want to work on this unless it's really working. So we want to give it the best shot this year. And like, we're really going to go for it. We're going to hire a bunch of people. We're just going to be honest with everyone. And Lucky was the only person that didn't, that didn't freak out. He was like, "I've never heard anyone say that before." As I said, I didn't even know what a Cedar pre-Ced round was like probably even at this time. So it's just like very honest with him. And I asked him, like, "Lock, have you ever invested in database companies?" He was just like, "No." And at the time, I was like, "Am I dumb?" But I think there was something that was just like really drew me to Locky. He is so authentic, so honest. Like,
And there's something just like, I just felt like I could just play, like, just say everything openly. And that was, I think, that was like a perfect match at the time. And honestly, still is. He was just like, "Okay, that's great. This is like the most honest, ridiculous thing I've ever heard anyone say to me." But like, that, that- Why is this a competitor launched this mean I work out? It was more like, if this doesn't work out, I'm going to close up shop by the end of the year, right? Like, it was, I don't know. Maybe it's common. I don't know. He told me it was uncommon. I don't know. That's why we chose him. And he'd been phenomenal. The other people were talking at the time where database experts like, they, you know, knew a lot about databases and Lockheed didn't. This turned out to be a phenomenal asset, right? I, like, just saying that I know a lot about databases, the people that rehired know a lot about databases. What we needed was just someone who didn't know a lot about databases, didn't pretend to know a lot about databases, and just wanted to help us with candidates and customers. And he did. And I have a list, right, of the investors that I have a relationship with. And Lockhees just performed excellent in the number of sub-bullets of what we can attribute back to him. Just absolutely incredible. And when people talk about like, no, ego, and just the best thing for the founder, I, like, I don't think that anyone, like, even my lawyer is like, yeah, Lockhees, like the most friendly person you will find. Okay, this is my most glowy recommendation I've ever heard. He deserves it. He's very special. Yeah, yeah. Okay. Amazing. Since you mentioned candidates, maybe we can talk about team building, you know, like, especially in a set, it feels like it just easier to start a company than to join a company. I'm curious to your experience, especially not being in a set full time and doing something that is maybe, you know, a very low level of detail and technical detail. Yeah, so joining versus starting, I never thought that I would be a founder. I would start with like turbo puffer started this a blog post and then it became a project and then sort of almost accidentally became a company and now it feels like it's, it's like becoming a bigger company. That was never the intention. The intentions were very pure. It's just like, why hasn't anyone done this? And it's like, I want to be the like, I want to be the first person to do it. I think some founders have this like, I could never work for anyone else. I really don't feel that way. Like, it's just like, I want to see this happen. And I want to see it happen with some people that I really enjoy working with. And I want to have fun doing it. And this, this, this is all felt very natural on that, on that sense. So it was never like join versus versus, versus found. It was just just found me at the right moment. Well, I think there's an argument for you should join Cursor. Right. So I'm curious like how you evaluate it. Okay, I should actually go raise money and make this a company versus like, this is like a company that is like growing like crazy. It's like an interesting technical problem. I should just build it within Cursor. And then they don't have to encrypt all this stuff. They don't have to skate things like, was that on your mind at all? Or before taking the small check from Locky, I did have like a hard like, look at myself in the mirror of like, okay, do I really want to do this? And because if I take the money, I really have to do it. Right. And so the way I almost think about it is like, you kind of need to, like you kind of need to be like fucked up enough to want to go all the way. And that was the conversation where I was like, okay, this is going to be part of my life journey to build this company and do it in the best way that I possibly can. Because if I ask people to join me, ask people to get on the cap table, then I have an ultimate responsibility to give it everything. And I know, I think some people, it doesn't occur to me that everyone takes it that seriously. And maybe I take it too seriously. I don't know. But that was like a very intentional moment. And so then it was very clear, like, okay, I'm going to do this and I'm going to give it everything. A lot of people don't tell you this year's link. What? Let's talk about, you have this concept of the P99 engineer. People are attending, saying everyone's saying, you know, maybe engineers are out of their job. I don't know. But you definitely see a P99 engineer and I was just wanting to talk about it. Yeah, so the P99 engineer was just a term that we started using internally to talk about candidates and talk about how we wanted to build the company. And, you know, like everyone else is like, we want a talent dense company. And I think that's almost become tight at this point. What I credit the cursor founders a lot with is that they just arrived there from first principles. I'm like, we just need a talent dense team. And I think I've seen some teams that weren't talent dense and like, seem to counterfactual run, which if you've run it, been in a large company, you will just see that. Like, it's just logically will happen at a large company. And so that was super important to me and Justine. And it's very difficult to maintain. And so we just needed, we needed wording for it. And so I have a document called traits of the P99 engineer. And it's a bullet point list. And I look at that list after every single interview that I do. And in every single recap that we do. And every recap we end with, I end with some version of, I'm going to reject this candidate completely irregardless of what the discourse was. Because I want to see people fight for this person. Because the default should not be, we're going to hire this person. The default should be, we're definitely not hiring this person. And if everyone was like, oh, maybe throw a punch, then this is not the right. Do you agree like, if there's one, there must have at least one champion who's like, yes, I will put my career on the line for this, you know? I see the career on the line. Maybe it's like, yeah, you know, I would say someone needs to like have both fist up and be like, I'd fight. Right. And if one person said them, okay, let's do it. Right. And it doesn't have to be absolutely everyone. Right. And like the interviews are always designed that you're checking for different attributes. And if someone is like knocking it out of the park in every single attribute, that's fairly rare. But that's really important. And so the traits of the P99 engineer, there's lots of them. There's also the traits of the P, like triple nine engineer and the quadruple nine engineer. Right. This is like, it's a long list. That's right. I'll give you some samples of what we look for. I think that the P99 engineer has some history of having bent like their trajectory or something to their will. Right. Some moment where it was just they just, you know, made the computer do what it needed to do, there's something like that. And it will occur to them at some point in their career. And hopefully multiple times, right? Give me an example. One of the engineers that like, I'll give it an inch. So we launched this thing called ANNV3. We're also, we're working on V4 and V5 right now. But ANNV3 can search 100 billion vectors with a P50 of around 40 milliseconds and a P99 of 200 milliseconds. Maybe other people have done this. I'm sure Google and others have done this. But we haven't seen anyone, at least not in like a public consumable sass that can do this. And that was an engineer, the chief architect of TurboPuffer, Nathan. Who moralized just bent the software was not capable of this. And he just made it capable for a very particular workload in like a, you know, six to eight week period. With the help of a lot of the team, right? It's been, there's numerous examples of that at TurboPuffer. But that's like really bending the software and X86 to your will. It was incredible to watch. You want to see some moments like that. Isn't that a triple name? I think. What's it called? That was only a tiny like, I feel like this is too high. Nathan is like, yeah, there's a lot of nines of the that P. So I think that's one trait. I think another trait is that the P99 spends a lot of time looking at maps. Generally, it's their preferred UX. They just love looking at maps. You ever seen someone who just like sits on their phone and just like swirls around in a map? Or did you not look at maps a lot? You guys don't look at maps. I guess I'm not here in there. I don't know. You just disqual, what are your trains? Do you like trains? I mean, they're not enough. This is just like, with a nice autism is what I call it. I love looking at maps. Like it's like my preferred UX and just like, you know, I like lots of like random places. So like, you know, yes, okay, there you go. So instead of like random places, like how do you explore the maps? No, it's just a joke. But it's a laugh. It's like, you are just obsessed by something and you like studying a thing. The origin of this was that at some point I read an interview with some I/OI gold medalist. And it's like, what are you doing your spare time? It's just like, I like looking at maps. And I was like, I feel so seen. Like I just like love like swirling out. It's like, oh, Canada is so big. Where's Baffin Island? I don't know. I love it. Anyway, so those trains of P99 is obsessive, right? Like there's just like, you'll find traits of that. We do an interview at a turbo buffer or like multiple interviews that just try to screen for some of these things. So there's lots of others, but these are the kinds of traits that we look for. I'll tell you some people listen for like some of my devs. I do think about devs as maps. You draw a map for people. Maps show you the what is commonly agreed to be the geographical features of what a boundary is. And it also shows you what is not doing. And I think a lot of like developer tools companies try to tell you they can do everything. But like, let's be real. Like you're three landmarks of here. Everyone comes here and here and here and you draw a map and then you draw a journey through the map. And like that to me, that's what developer relations looks like. So I do think a lot of things that way. I think the P99 thinks in trade-offs, right? The P99 is very clear about, you know, hey, turbo buffer, you can't run a high transaction workload on turbo buffer, right? It's like the right latency is 100 milliseconds. That's a clear trade-off. I think the P99 is very good at articulating the trade-offs in every decision, which is exactly what the map is in your case, right? Yeah, my world's. How do you reconcile some of these things when you're saying
you bend the wheel, the computer versus like the trade-offs. I think sometimes it's like, well, these are the trade-offs, but the three-nines is actually, it's not a real trade-off because we can make something that nobody has ever made before and actually make it work. The way I think about the bending trajectory to your wheel is, if you sit down and do the napkin math, where you're just like, okay, like, if I have 100 machines, they have this many terabytes of this, they have this bandwidth, whatever, right? And you sit down and you just do the, like, high school napkin math on, this is how many QPS we should be able to drive to it, similar to how I did the vi-pricing, right? If you can sit down and do that, and then you observe the real system, and you see a were off by like 10X, bending trajectory to your wheel is like, just making the software get closer and closer to that first principle line. The B99 might even be able to cross the line, right, by finding even more optimizations than from first principle. So bending the software to your wheel is about that, right? Like, a hundred milliseconds P99 to S3, I mean, now you're talking like someone really high agency that like goes to Seattle, finds the S3 team, and it's like, how are we gonna make this 10? You know, like, it's not quite what we talk about, right? But yeah, let's just teach a turbo puffer. Turbo puffer started out, act one of turbo puffer was vector search. That's what's all we did to begin with. Act two of turbo puffer is, and was full text search. Turbo puffer today has a fairly start of the state of the art full text search engine. We beat Lucene on some queries, in particular, very long queries that we've optimized for, because those are the text search queries we see today. They're generated by LLMs, we're augmented by LLMs, and we see them on web scale data sets, right? Like someone searching for a very long text string on all of common crawl. We beat Lucene on some of those benchmarks, and we expect to continue to beat Lucene on more and more queries. That's the performance and scale. Turbo puffer does phenomenally now at full text search performance and scale. What we work on now is more and more features for full text search. People expect a lot of features with full text search. And full text search is still very valuable, right? If you go in and you press Command-K and you search for SI, an embedding based search might be like, "Oh, this is something agreeable "because that's seed that's yes in Spanish, right?" But I said it's halving too. But in full-naked search, that's the prefix of maybe a document of like, "You know, these are all the reasons I hate Simon," right? Like this is like, that's completely different. So that augmentation to like how the human brain works and mapping like data to user is very important, but it's a lot of features. That feature drawing is what we're firmly on. And you will see us just adding to the change log every month, just more and more full text search features. So we're like fully compatible. And we see we're seeing people move from some of the traditional search engine onto TurboPuffer for that. That's a big focus of TurboPuffer this year. The other focus of TurboPuffer this year is just on scale. We're seeing more and more companies that want to search basically common-crawl level types of data sets, both internally or companies and externally at a time, like Corey, like 100 billion vectors or 100 billion documents at once. This is tricky. And we want to make it cheaper and we want to make it faster. That's a big focus for TurboPuffer this year. That's-- we just released ANNV3, which we talked about before. We were working on ANNV4, and we're also at plan what we're going to do with ANNV5. And then on full text search, we're working on a lot of these features. We'll be like FTSV3, but it will all roll out incrementally. Those are some of the really big features. And then the other thing is our dashboard. Have any of you ever locked into the TurboPuffer dashboard? There's not very much there. It almost looks like if a founder two years ago just sat down and wrote enough dashboard that there was at least something there. And then other people just sort of added stuff on for the next two-- the following two years. And then at some point, as a so and other things, to just catch up. And it may or may not be what happened. But adding-- I want PHP, my admin back. Do you guys remember? And it was so good. And I think that that software hardware integration between the dashboard of the console with the database and the database itself, I'm really excited for that. There's lots of other things that are going to come out in the next two-- we talked a bit about some pricing and things like that. But those would be some of the big hitters right now. You talk about errors of the TurboPuffer. I just have to ask, yes, there's the stuff that you work on this year. But I'm sure in your mind, you already have the next phase that you're already thinking about. Ax3, yes. Act4, yeah. Act5. What else are you talking about? Nobody can do this. You don't have to decide. Yeah. But I'll just say that if you want to build a big database company, the database over time has to implement more or less every query plan. Because when you have your data in a database, you expect it to over time, not just search, but also, hey, I want to aggregate this column. I want to join this data, all of that. But when you start up, your only mode is really just focus. So you have to lay out the ax and you have to not get over-eager. And I think we've seen some of our peers get very over-eager and overextend themselves. And what I keep telling the team was just having breakfast this morning with our CTO and Chief Architect that we were talking about, well, we're most likely to regret at the end of the year is having tried to do too much. And so Act3 candidates could be a bunch of simpler, old app queries. It could be lending ourselves a little bit more and to be see some people who want to do traces and logging and things like that. Some very simple use cases could be that. It could be maybe some time series. Some people are trying to do that. There's lots of different things that you can do with TurboPuffer. But for now, if you're trying to do not search on TurboPuffer as the primary use case, you probably shouldn't. But we've seen some customers that are like, oh, I get some point. Curse are moved like 20 terabytes of Postgres data into TurboPuffer. Because it's there. It works. And these particular query plans we know work well. And so they just move it all to defer sharding. So we look for patterns like that in what future acts of TurboPuffer are going to be before firmly doubling down on them. But we wouldn't-- today, if you're using TurboPuffer, it should be because search is very important to you. And then we might do a lot of auxiliary queries to that. But that should not be the main reason to go to TurboPuffer at this point in time. Yeah. You didn't mention one thing I was looking for was graph type queries, graph database, graph queries. Can you basically trivially replicate this with what you already have? We see some people doing that. Because you have parallel queries. And it's the same thing. Exactly. So we see some people doing that. At the under, TurboPuffer is just a KV. And then we expose things on top of it. So we are seeing people do that. And I think our roadmap is very much just the database that connects AI to a very large amount of data is what the path is to do that in the right order, which is what a good startup is around. What is the order to do things in? Our customers are P99. And they will tell us what they care most about next. And so some of them are doing graphs now. And if they need more graph database features, they'll be banking our door and will prioritize accordingly. T. OK. Give us the T. This-- you kindly gifted us your favorite T. This is Yabukita Kamayura Richa from the Green Tea Shop. That's right. So what you love about T. Yeah. We were just talking beforehand about caffeine, I think. And especially when I'm on a trip like this to San Francisco, I consume a lot of caffeine. But this is my preferred caffeine. It's this green tea. I have an air table with 200 teas that I've tried over time over the past 15 years. And this one is my favorite. Now, when you drink a tea, there's different-- there's like six different types of tea. I like green tea. In particular, I generally prefer Chinese green tea. And I don't really like Japanese green tea. But this little prefecture somewhere in Japan has specialized in-- they're like Japanese, but doing it the Chinese way. And it's just phenomenal. But then the interesting thing about the tea world is that all of the different-- you can find this particular tea. There's probably hundreds of places that sell it. But they all go to a different family, on whatever mountain that they have these, like, Camelia Sinensis bushes on. And this woman, Japanese woman, in Toronto from the green tea shop-- I don't know. She just like has found a really good family, because that's the best one. The best time of year to get this is in a few months when they do the spring harvest. Now it's kind of old. It's just like, I love the spring for the fresh tea. So I hope you enjoy it, but it's not the right time of year. It's out of season. Yeah. I actually didn't even know tea has seasons. This is unsusphysically. But I think it ties in with loving maps and being obsessed and being keen on everything that you do. Yeah, but that's great. Awesome. Well, as we were saying, we have instant hot water at Colonel, so M.E.T. lover and combined. I have a little teakip where I bring a-- where I bring a little thermometer to, like a little thermo-works thermometer. Last Friday, when we do demos, I have this thing where if there's not enough demos, then I fill the remaining time talking about something completely ridiculous as an incentive for people to actually demo. And last night, time, I spent 20 minutes walking through my air table and going through my entire tea travel kit, including the temperature monitor. Because like, yeah, you'll show up. There's only a boiler. You can't get it to the right-- You need this at 80 degrees. Anyway, yeah, sorry. We have a lecture at Kettle with the Unembroker thing at home. I would watch this. You should start a company, YouTube, but it doesn't have anything about search. It just has tea and like other brands. I don't think I could talk, but something that I started doing, do you two know Sam Lambert of-- Of course, Lenett's skill, of course. Very else-working guy. I love the guy. And we just last week, we just went on X-Live and just sat and shot the shit for like an hour. And I think we'll probably do that again. Yeah, so probably come up there. Well, I don't know what we'll call maybe P-99.0.0.
or the P99 pod or something like that. - T pod. - P pod. - P pod. (laughing) Cool, well thank you so much for your time here. I know you have to go, but this is a blast and you're clearly very passionate and charismatic. So I bet you'll get some T99 engineers under this podcast. - Yeah. - Thank you so much for having me. It was a pleasure. (upbeat music) (upbeat music)
Podcast Summary
Key Points:
Simon Eskildsen, founder of TurboPuffer, explains the company's origin from a cost-prohibitive AI recommendation feature at Readwise, highlighting the need for an affordable vector/search database.
TurboPuffer is positioned as a search engine for unstructured data, specializing in full-text and vector search, designed to connect large datasets to AI applications.
The database is built on a novel architecture that fully leverages modern cloud primitives: it is all-in on object storage (like S3) for durability and uses NVMe SSDs/DRAM for performance, avoiding traditional consensus layers.
Three conditions are outlined for building a major database company
The founder's background includes scaling infrastructure at Shopify and experiencing the limitations of systems like Elasticsearch, which informed TurboPuffer's development.
Summary:
In this conversation, Simon Eskildsen, founder of TurboPuffer, details the motivation and vision behind his company. The idea originated while he was consulting for Readwise in early 2023, where building an AI-powered article recommendation feature proved technically feasible but cost-prohibitive due to expensive vector database infrastructure. This sparked his insight into a latent market need for an affordable solution.
TurboPuffer is designed as a search engine for unstructured data, combining full-text and vector search to serve as the external knowledge base for AI systems. Eskildsen argues that building a transformative database company requires three key conditions: a new universal workload (every company connecting its data to AI), a new underlying storage architecture, and the ability to support increasingly diverse queries. TurboPuffer's architecture is built entirely for the cloud era, eschewing traditional database designs by going all-in on object storage for data durability and leveraging NVMe SSDs for high performance, which was not feasible 10-15 years ago.
This design aims to drastically reduce costs while maintaining capability, addressing the gap he identified during his prior infrastructure scaling work at Shopify.
FAQs
TurboPuffer is a search engine for unstructured data, specializing in full-text search and vector search, designed to connect large amounts of data to AI.
TurboPuffer is built entirely on object storage (like S3) and NVMe SSDs, with no consensus layer, allowing for simplicity and cost-efficiency by inflating data only when needed.
It targets a new workload of connecting data to AI, leverages modern storage architectures like NVMe SSDs and object storage, and plans to expand query capabilities over time to meet diverse user needs.
The idea came from a cost challenge at Readwise, where embedding articles for AI recommendations was too expensive, leading to a design focused on minimizing infrastructure costs using cloud primitives.
It relies on object storage's strong consistency for consensus, storing all data in S3 and puffing it into NVMe SSDs or DRAM only when queried, ensuring data durability and low operational overhead.
TurboPuffer is built from the ground up for cloud-native architectures, using object storage and NVMe SSDs, whereas older databases often retrofit these features and face scalability challenges.
Chat with AI
Loading...
Pro features
Go deeper with this episode
Unlock creator-grade tools that turn any transcript into show notes and subtitle files.