Parallel’s Parag Agrawal: Building a New Web for AI Agents

Our view at Parallel is that human click data is a bug and agent doing work with search should rely on agent feedback, not human feedback. We believe that these models are really good at compressing information and we can benefit from a lot of the research that have gone into building models and apply it to search indexing and ranking. And so you can now make many, many arguments and that's the arguments we made back then that actually now it's way more tractable as a problem because of the existence of agents, not just as a technology, but as a distinct customer. Parag, thank you so much for joining us today. We're delighted to have you on the show. For those who don't know, Parag of Twitter CEO fame was the CEO of Twitter before selling it to Elon and is now back on the founder arc. You founded a company called Parallel Web Systems, which is scaling up agentic search for the agentic web. We're very excited to have you here today to talk about the future of search and the future of the internet. So let's get started. What's Parallel? Thank you, Sonia, for having me. Thanks, Andrew, for joining us. I'm really excited about this conversation. At Parallel, we're building a bunch of technology in order to allow agents to search and use the web. So just like humans forever have figured out how to use browsers and search engines and clicking and browsing on the web to access information, agents need to do the same things. We started Parallel with the bet that agents would do it a thousand times more than humans ever have. And as a result, we need to reinvent the technology that can power search for agents, but also the business models that go alongside it. And that's what we're doing at Parallel. Okay, I want to go back into what makes human and agent search so different. But before we get there, you've told us that, you know, you're unlearning a lot of the lessons that you learned from running Twitter as you build Parallel. Why is that? Listen, when I was at Twitter in leadership, I was like, oh, I'm going to do this. Twitter was a post-product market fit, extraordinarily scaled business where your feedback loops were from the hundreds of millions of customers using the product for 30 plus minutes every day, right? In that world, you operate differently than a pre-product market fit company based on the premise that in a few years, you're going to be able to do this. In a few years, a new customer is going to show up on the internet and let's go build technology for the not yet here customer that we are all learning every day and every week. Fantastic. Okay, let's talk about Parallel. I have a question, actually. Can I jump in? No, you cannot, Andrew. I'm Andrew. I didn't get an introduction, but I'm also happy to be an inaugural guest on the Training Data podcast. Andrew is making his podcast debut on Training Data. We are honored to have you finally here. Thank you. I feel like all of us in Silicon Valley have a very surface level idea of search. What is search? The job to be done is getting the answer makes sense, but we all know there's crawling and there's an index and there's ranking, but maybe let's take a big step back and explain what is the problem of web search, be it for a human or an agent, and then we can dive into the differences. The problem of web search, and we all know it and experience it, when we want to find something, and we do not know where it is on the web, we go to a search engine like Google, and the search engine then hopefully surfaces the answer to us in the most convenient of locations. That's the base problem. Now in order to do this, what is the search engine doing? The search engine is going and crawling the web, which is finding every URL that's out there, trying to read it, trying to organize that information in what might be called an index. And by doing that, by having all of this stuff in memory in one location, you don't have to, once the query comes in, you don't have to spend time loading up pages because you already have them, you've already done pre-processing to organize them. And then when the query does come in, you quickly understand the query, you find the most relevant results, and then there are many, many, because you're essentially taking hundreds of billions of pages and narrowing it down to, what, five or ten or ideally one in terms of what someone is looking for. So you go through these multiple stages of retrieval and ranking in order to surface the most relevant result. So that's the broad problem. One way to think about it is it's a billion-to-billion matching problem, right? So you have hundreds of billions of pages and hundreds of billions of queries over time, and you need to figure out how to match-make across these two. That sounds like an enormously expensive infrastructure challenge, and I think for the longest time, basically only Google and Bing had, you know, done full web-scale crawls and indices. Why did you think about, you know, you as a young company could go off and tackle that problem? It seems like a problem of the giants. So it is expensive in the long term. So what's actually interesting is that when I was starting the company three years ago, you could imagine a world where, one, some of the reasons it was difficult for others to compete. One is not having access to great feedback in terms of, "Is this a better search result than this?" To collect that feedback at scale, there was a problem around human ratings. There was a problem around click data. You need to access those. Now, it is, of course, expensive to crawl the entire web and index it, but as you think about what agents and the large models that we now have access to have enabled, it's the ratings data. It can be now created by experts way more cheaply. Our view at Parallel is that human click data is a bug, and agent doing work with search should rely on agent feedback, not human feedback. We believe that these models are really good at compressing information, and we can benefit from a lot of the research that have gone into building models and apply it to search indexing. And so, you can now make many, many arguments, and that's the arguments we made back then, that actually now it's way more tractable as a problem because of the existence of agents, not just as in technology, but as a distinct customer. And then we figured out a way to go about building this business, which did not require us to go spend all of the money on infrastructure upfront before we could service a customer. Right? So, if you can paint a path of incrementally being able to build an increasingly larger and more sophisticated index over time, as you solve problems for more and more customers, that's the insight that actually convinced me that this was a viable problem for us to take on. And how do you make that happen? Because I imagine this is one of those things where customers want full coverage day one. So, how do you go about making that happen? You launch the product, right? We launched in, we did not launch a search product first. We launched a search agent product first. Our search agent could go essentially crawl the web after a query arrived. So, if you're doing deep research, you have patience to the extent of a minute. And we've had products which sometimes take, do 10 minutes of research. That's a lot of time to be able to crawl a lot of pages, if only you have enough of a map to know. So, you can make up for shortcomings like indexes, oftentimes you can think of it as a latency optimization. So, if you give up on that dimension, if you're competing with humans, that's why our search agents were competing with the alternative being outsourcing to humans to curate amazing data, right? So, we said, it seems like humans sitting on search engines are far way easier to compete with than a search engine on day zero. So, by building a product that was a search agent to do real work on top of web data, we were able to incrementally go build our index. Well, there's some examples of what people built with your search agents. In the very early days, there were search agents being built with us for doing some kinds of insurance underwriting workflows and claims processing workflows. People in sales were doing all kinds of sales data. People in finance, who would previously and historically go to build a model, collect data by sending it overnight to a set of humans who would curate data that would feed into a modeling exercise they would do, would start using us to do that instead. And so, we were going initially after replacing where there was outsourced data. And so, we were going essentially after replacing where there was outsourced data. to run agents to figure out what search for agents should look like in the first place with empirical use cases instead of theoretical evils. I see. You're trailing off the crawl for inference time compute effectively. Yeah. You guys were started before the term like Neo Lab came into existence and you have, you know, web systems product, but you also have like a bunch of AI people. From the outside, before we met for the first time, like it wasn't obvious to me how much of a web systems company or an AI company that you've built. The more we spent together, the more time we spent together, the more clear it's like, by some definitions, this is a Neo Lab. Do you want to talk about like where the research side of what you're doing comes in, not just the like infrastructure and system side of what you're doing? So we don't categorize ourselves as a Neo Lab. Well, of course you would not. No, because I think, I don't think. Our output is a model like I somehow, maybe my definition is broken. My definition of a Neo Lab is an output is a model. I think our output is a compliment to a model. What we build is something that multiplies on top of a model in order to give either, you can call it the model gets better or the agent built with the model gets better and has superpowers. Right. So we've always want to be in a place where whenever someone ships a better model, a new lab, for example, if somebody else ships a better model, they have a higher hill to climb for us. Somebody ships a better model. They have now unlocked four more use cases where we can be valuable. Now, whether we need to do work that might be framed as research that remains. So in that sense, we do have to do research. Right. But we're also not pre-training extraordinary large models. In fact, our job is to compress things down into tiny ranking models. Right. Like, I think if you go back to my framing around like this hundred billion, 200 billion matching problem, every query is essentially give me a thousand tokens from a trillion web pages on the web and make sure that the right thousand tokens, like that's the query that we are getting effectively from to ourselves. Right. And so now what do you have to do? And until last week, we would give ourselves three seconds to throw a bunch of compute at read time to do it this week. Last week, we should be a product which now does it in 20 milliseconds. Right. So you have now that much time to go figure out how to pick those. Right. So you will do. We're trying to organize information in memory across the memory hierarchy in a way that we can access it fast. We're trying to figure out how to train what model. Uh, To select the best thousand tokens. And so there's a large amount of research that goes into figuring out how to take a pre-trained model of some kind, adapt its architecture for this new problem and allocate effectively compute in a way that produces the best output with a limited compute slash latency budget. So a lot of these use cases seem like the deep research kind of shaped use case. And when I think deep research, at least in the earlier. Incarnations, it was effectively like an agentic loop with the model reasoning, and then basically just calling a search engine, calling, calling Google or some proxy to Google. Right. Why is that insufficient in your eyes versus what I'm hearing from you as agentic search is kind of a, a net new capability. So calling Google for every query in a deep research thing, it gets you somewhere, right? Okay. If you use. Parallel search, you will, for the most part use under half the tokens in your agent, it will become more accurate and to be faster into it. And so you can think of it as, but every time, if you use only half the tokens, if your model is context limited or memory limited, you can now do more problems. You can do the same problems. Change. It becomes cheaper or faster. So everything to me, when you have a infinite appetite for information and relevant information for all kinds of work, it's at its core and optimization problem around quality cost and latency. Every model advancement is about how do you squeeze out more intelligence and then how do you distill it down into keep most of it at a 10th of the cost. Right. And that. Applies to. Search as well. And every time you can produce all of the signal with less noise in your search results to give to a model, you now let give the model the ability to do more. And so a lot of the bet here is intelligent compute allocation across the model layer and the agent layer down to the search layer. The other interesting thing to observe is the interface actually changes. Huh? When you think about serving. Agents versus humans, humans rely on keyword search with the agents. We've had to innovate quite a lot on the, what does an agent tell our search engine? The more we know, the better we can do. Interesting. So humans can only hold a few words in memory basically versus an agent. No, I think we can. We're just lazy. Yeah. Right. Like we can, we can have a conversation. We can hold a lot of words, but when you start typing, we, we want to tell Google like two. Incomplete words with a typo in there and hope for the best or rely on some form of a dropdown autocomplete thing to avoid typing those three words in the first place. So we're fundamentally lazy. Turns out like models less. So fewer typos, better specified queries, perhaps longer queries, less for the search engine to guess agent might want. Uh, so you get to solve a different. That's problems. One other thing about humans is I feel like we know how to decipher the, you know, pre AI slop that populates many of the top ranking Google results, you know, for best product for X, Y, and Z. And you get all the affiliate advertising things. Um, and yet obviously they, they rank highly. I'm very susceptible to it. Are you? Yeah. I'm like, I feel like I have this incredible, I get to see coming a mile away. How do agents deal with things like that? Right. Where these are. You know, well-trafficked pages, they have seemingly good answers, but you just know they're not real. Perhaps with parallel agents have to deal a little bit less with that. Let me tell you why those pages exist in the first place, right? Let's work through a simple example. If you ask for a public company's most late, most recent financials, like just the headline revenue number, we can sit here and know that there exists an authoritative filing with the SEC. Which has that number, perhaps on page 73 of a PDF, but that is the authoritative number right now, imagine Google decided like I care about authority and whenever you ask that query, that's the first result you see, right? And then you click that this PDF takes what, like three and a half seconds to load instead of the second that another page does, you're already frustrated. And then you see a. Earnings page, that's the alternative, but here you have to now grab your way to page 77 to find your answer. We're lazy. We're not going to do the work, right? So now there exists an entire class of content on the web, which is like, okay, this information is needed by a lot of people. It is worth putting it on a page that loads fast, where this information is above the fold. All of you can go there. It's 99.99% right. So you're not like so skeptical when you go there that this is a hundred percent wrong. And it's added real value in the process because it identified out of the 300 page earnings report to NT bits of information that should be above the fold. - Yeah. - Right? And so you can call it slob, pre-AI human slob. Or you can call it catering to a lazy human and being successful at SEO, right? The good news is with agents, we're not making the agent click around and fumble around and grab a PDF, right? We're taking a excerpt from the most authoritative place on the web and trying to bring it to the agent's context window. And so we aren't forced into this weird trade-off and this trade-off existed with humans in the first place. Like with standard browsers and protocols and everything. We just didn't figure out how to have us point exactly consistently across everywhere on the web to the exact right highlighted tiny paragraph, right? With agents, we get to bring that to the agent's context and then let it figure out what's next. - Can you walk us through what actually happens when one of your developers sends a query off to your agentic search API? - It's in some sense, pretty standard. So we run some models to figure out what this query is and enrich it to figure out how it will flow into the rest of our system. We have a bunch of indexes which organize different subsets of the web. in different ways and so the first layer will essentially craft queries for each of these different systems each one of these systems perhaps is our sort of big index one of these is perhaps our fresh index one of these is perhaps like you can some people will describe it like a knowledge graph some people will describe it like a structured index like there are a bunch of these right so you're now deciding which ones this query needs to go to you're figuring out what is the query rewrite for each of these then each of these has like a big retrieval layer and then a ranking layer and then more ranking layers so you're going to try to boil down tens hundreds of billions of urls or documents down into thousands tens of thousands down into specific excerpts and paragraphs in those tens of thousands with more and more bigger models running at each stage with different architectures pulling more features to ultimately get down to here are the thousand tokens i want to bring back to this ai which has the highest signal right and now if you look at our various versions of our search api they just throw for different latency and cost constraints different amounts of compute at various points in this journey to hit those limits so in the abstract it's simple right what's interesting is what each of these models that i described how you curate and collect the training data for them right how you build those models and optimize them the index itself how you use the the memory hierarchy to store it to be able to hit certain cost quality latency thresholds what's the north star from a quality perspective like you know in google there's that you know did you get this result you wanted in the first three answers or something like what is the equivalent north star for you so i think of i don't know if i'm right on this but like my take is that a billion to billion matching problem is a forever problem and so the real question is at what point incremental optimization isn't worth the squeeze right so i don't think there is a thing as like okay we're done on improving this thing the question is at some point it's going to get harder and harder to improve this thing and it just won't be worth it right but i'm hoping that we don't get there actually because like if you think of what we're doing with ai we will have more intelligence that gets cheaper every few months and as a result of it it will come down to having great models which are cheap having great information matchmaking across a need and all of the information available to you whether it's your own or on the web and doing something unique and differentiated with it to produce more knowledge right and anytime you can do something 20 better than somebody else that might give you an edge so why wouldn't you right you can do something 20 better than somebody else that might give you an edge so why wouldn't you so in the super agi build worldview it feels like if you can push on quality across web search and in the model layer why wouldn't you i've heard a point of view that this is so fundamental to the companies that they're just going to own it and in part because as they're collecting data for pre-training that's a very very expensive infrastructure exercise that kind of is your source of truth for the for the crawl um um what do you think of that i don't see empiric data on the ground to support that view to build a fresh web index i don't think that crawl is particularly useful and let me frame why so if you think of what we are building we're building a complement to models we like to crawl things that people don't like to crawl for pre-training because like if the model already was trained on it like it's not useful model companies for the training aren't patient enough to go in a completionist way try to wait for really slow random javascript to load because the number of tokens you get per amount of compute you throw at it is like one order of magnitude so two order of magnitude too bad and so like is it worth the extra effort to get these tokens for us it is because we don't have the time to do that we don't have the time to do that we don't have the completionist right for them it's like i'll take extra trillion tokens so that's one now i do think this is a core part of every agent right my worldview is that if you're buying llms for doing work for nine out of ten use cases you will want them to have access to the web and great search infra optimizing for agents so it is a real adjacency for all kinds of llm inference and that supports your view that model companies could should have the best in class infra for it so now they can build it or they can buy it and that's the conversation and the we will see who builds and who buys and who partners and how things evolve are you partnering with any of the model companies that you can share we want to we will i can't share anything on that we did announce and i don't know how your definitions of model companies uh we announced today actually that we are working with uh google cloud to be a search and grounding provider for their enterprise agent apis so if you think of grounding gemini models or other models of service that are available on gcp when you build agents on gcp or chat apps on gcp or do any other inference with llms on gcp when you attach web search to it uh your options are google search or parallel search and parallel search is product integrated the integration is optimized we've spent time with technical teams and training teams and product teams and commercial teams to make sure people use gemini models with parallel like they get exceptional and great results so yes there are these partnerships now emerging uh i bet that there'll be several of these they will all look somewhat unique that's a big deal from the search king congratulations google's the original neo lab i i don't know how to frame google as a model lab versus a hyperscaler and i don't know what precise lessons to learn from this one whether it applies to other labs or to hyperscalers and so we will see the way that i use voice agents and i need to get reservation for dinner at night what restaurant should i go to and then that request gets fulfilled by the agent when we talk to a lot of the parallel customers there's this you know background agent whether the monitor product these agents that sort of are always watching the world or watching the world and they're not watching the web and when something happens then they go off and take actions and do something with it um it might be worth because if we think about what does a thousand x more mean there's like the depth of research and then there's just like the the you know what is actually initiating the tasks is it a human initiating the search or as an agent itself you want to talk about that dynamic a little bit yeah so there's a bunch of dimensions here let's go back to for a moment on search agents right if you run a typical search agent and even without doing deep research it'll it'll do somewhere between 5 to 20 searches even if it answers within a few seconds which is why not right so already if you transition from using a chat gpt medium or i don't know all the categories but like somewhere not on the high tier like instant low medium every time you write a prompt to it in chat gpt it will do 5 to 10 searches as you dial it up it'll do hundreds and thousands of searches so one interesting thing to observe is like a human action to a multiplier on number of searches that happened right so just by using a ai app you're kind of multiplying your way to perhaps one order of magnitude more searches now a lot of our initial takes on the product and the market were to go off after bigger multipliers than even that right so we were much more interested when you said i have a portfolio of 10 000 small businesses where i have given out credit to for all of them every month i have this human process that runs to feel out how my risk is going up or down now can we and it relies on a bunch of web data we're trying to use agents for doing this so here a developer is effectively the multiplier there is hundreds of thousands or a million in terms of the number of web searches that happen because a human goes into programs that instead of now doing this process every month we can do it every week right so doing a lot of searches when this agent runs every week to create a dashboard on the portfolio and a collection of action items that somebody needs to look at right um now you can go to another example which is even more interesting i know do you use something to do uh meeting prep documents for you all that's an agent there's a sequoia one there's a sequoia agent yeah yeah i was gonna say james flynn but he's one of our great young guys so i use um notions uh agent and you can build custom agents which look into all the internal data that we have at parallel plus all of the web data using parallels apis to create meeting prep docs one time i went and created one prompt to build this custom agent now it does tens and hundreds of web searches for every meeting i have every time i build a new agent for a new use case that keeps multiplying so i think the path to these background agents doing more and more and more work all the time for us it's only going to be bounded by vast value versus spend you know like i don't think it's rational right now like i'm probably spending more on it than i should be but it's not too much so i don't care yet so there'll be some rationalization in all of these agents um but i think we will deploy background agents to the extent that there is incremental value in doing that compute and the same thing applies to searches all of these background agents would do a bunch of searches and so our first set of products were really obsessively focused on these in part because we were building growing the index and we decided that our company was based on three dimensions quality cost latency and for the first couple of years we said let's focus let's ignore latency and let's just nail the other two because optimizing systems distilling to smaller models is much more known art than unknown research right so i think we're going to be able to do a lot of work on this and i think we're going to be able to do a lot of work on this so once we achieved the best quality search and search agent products at every price point we started working on latency and that's what we shipped with a product we call turbo now it is the fastest highest quality agentic web search on the market by a lot do you think there are more agentic queries than human queries on the web now i don't think yet i don't think just given some of the multipliers you mentioned i don't think there are any more agentic queries and given the background agents it seems yeah i think i don't have to remind you but we are early like we are very early in agent adoption like you go step outside of our bubble people haven't heard the word fable like so we are very very early i think there are now people like me who are probably operating at the thousand x i don't know what do you think how many google searches a day did you do three years ago before chargivity 20 or 30 yeah i would have guessed something like 20 30 i think today if you just look across at all of my agents like i i bet they're doing thousand x more than that maybe 100 to a thousand um if you count some of the things that happen at my company which isn't like assigned to a human yeah it might easily be more than a thousand x but i think we are the outliers rather than the norm so i think we're very very early on this journey i i do think i recently saw i think it was cloudflare that said that in their monitoring of web traffic the ai traffic is about the same as human traffic in terms of page reads which is slightly different from searches because it includes perhaps all crawlers that are out there and a bunch of other stuff but i think it's it's going to happen so maybe that's a good segue to talk about a topic that i know you are passionate about which is the economics of the internet as we know them some of the kind of fundamental assumptions there you know human eyeballs scarcity of attention seem to be falling right in front of us right now are the economics of the internet broken now and what's going to happen yeah no i think this was perhaps part of the thing that totally convinced me that this needed to be done by me and build parallel i know most people hate ads i used to do ads and build systems for ads i love ads oh wow i love shopping okay yeah so you get good ads i don't know if i love ads but i intellectually love ads because ads is the reason that so much amazing content and technology is a part of our lives and i think that's one of the things that i think is very important and one of the things that i want to emphasize is that ads are available for free to all of us right like google search wouldn't be free without ads twitter wouldn't be free without ads and these are truly useful pieces of technology a lot of content on the out on the web out there that we can access for free wouldn't be free if not for ads um and so ads is a very efficient monetization scheme it's a very efficient monetization scheme because it is exceptional at differential pricing right so most queries google loses money some of them make it up and it's an extraordinary business with extraordinary margins same with twitter like most users exactly you're welcome i'm subsidizing the free the free information you guys are getting you are subsidizing all of us thank you um but i think ads is extraordinarily efficient yeah at differential pricing and monetization on the web which is why it has been a dominant business model now the core assumptions as you noted going into it around limited human attention to translate into outcomes if humans don't show up any their agents show up on the web like what does this mean how does the business work um and yeah i think this was the if we don't figure out a new business model we're seeing it already right like people are going to say okay i don't want my content to be accessed by an agent because i have a business model for humans so i actually want to go pay someone to seo i'll optimize myself so more humans show up but then if their agent shows up i'm gonna cut it off right which seems confusing and discontinent like okay ultimately this agent is acting on behalf of a human but we haven't found business model alignment yeah in a way you can't monetize you can't monetize that visit yeah so you can't monetize that visit yeah so you can't monetize that visit yeah so let's say you are in the business subscriptions right so you get a bunch of you get a thousand humans and you convert 20 of them into monthly subscriptions right you don't know how that loop works yet with agents you can't distinguish you don't have statistics you don't know if these will lead to subscriptions that they'll just keep stealing your content as nameless agents and so there are real challenges around the old business models breaking and us not figuring out real scalable new business models right and so there are real challenges around the old business models okay so if you own high quality content and you currently put it on the web your one available business model pre-parallel was to be in the head and be able to transact with the lab on some fixed fee contract that's literally which includes some amount of training and liability and then inference time access right that was your one option that option is not available to you right so if you own high quality content and you currently put it on the most content on the web it's a very head phenomena and even for the head it is a broken business model because when ai's or inference grows let's say 7x this year and another another 7x the next year on this 50x their deal size is not growing 50x like none of them after signing a two-year deal believes that their share is going to be more than 50x and that's why they're not going to decline materially at renewal and so these are like fixed price constructs in the world of ai inference and so which doesn't drive for sustainability for for all of these businesses now our solution is trying to learn all the lessons from my work at ads on having transacted like i sold twitter data to open ai having transacted on that side to figure out what actually might work and be incentive aligned so what might work efficient differential pricing paying for differentially a lot for extraordinarily high value content paying a lot for high value work accessing the same content so it's differential in both dimensions quality and value of work being done with it uh and a way of doing this scalably and not just with business spoke deals so those are the properties needed for any reasonable solution and the biggest property of it all is Incentive alignment, right? Like at what point do people want to collaborate into this enterprise? Why do the model companies need to pay anything at all? You should ask them. My understanding is, one, you want fresh data during inference time to be able to display it in products like ChatGPT or Cloud. Two, you want training data. And three, you want some liability protection for training that you already did. And so the payments are some combination of these three things. And I can't be sure of how they value each of these three. You said even a very simple query will go do 10 searches. And so how do you do attribution between all the different sources that boil down to one paragraph response? Yeah, at Parallel, we like to build models. I heard. So, no, I think it goes back to my point around incentive alignment. So what are you, let's, before we try to build a model, let's try to figure out like if you were going to try to do this intellectually, theoretically, how do you go do it? So you have to ask the question of, okay, how much incremental value did somebody's content add, right? So you can run all of these simulation exercises. You take, you can take one piece of content out of the corpus. And then you say, let's run the agent. Let's see if the quality of our output declined. How much? To claw back that quality. Perhaps if I threw a little bit more compute or a better model in some way, could I claw back that quality? Oh, I could. It cost me a cent. Huh. I could get the quality I lost by not having this source. My alternative was to throw it away. Throw more compute. A cent worth of compute to get that same quality. It might be a different result, but as far as the end customer value is concerned, it's the same quality. And so you're like, okay, this source is worth close to a cent feels like, right? That's intuitive. Now the formalization of this kind of an intuition is the, the, the core framework we use. It's called Shapley values. Yes, let's go. What's a Shapley value? It's a music's my ears. It's a game theoretic. You're, you're a game theorist. A bunch of game theory. Yeah. Yes. Shapley value is this very theoretical mathematical concept. It is used in Shap values in feature importances for those who are ML folks here. Let's say the three of us collaborate on something and the whole is bigger than the sum of parts. In that moment, the theoretical question is, okay, how do I divide up this sort of bigger pie that we created by collaboration? So all three of us have incentive to collaborate, right? And Shapley values is a mathematical way of effectively answering this question. Right now that sounds amazing, right? Like you could, if you're co-founders, you could figure out how to divide equity. Unfortunately, it's not that useful. It's not that useful because in order to compute Shapley value, you need to simulate all words where some subset of the two of us collaborated, but the third one doesn't and play out those realities to then impute back to today in terms of how we should divide the pie, which in practice you can't do most places, right? In ML models, when you do feature importances, you kind of can, you can hold a feature back, run your model and see how well it did. In web search, we can run simulations of if we did not have access to this URL or this domain or this collection of them, how would the agent perform? We can, if you're, if you're good at evals, if you're good at assessing, we can run simulations of if we did not have access to this URL or this domain or this collection of them, how would the agent perform? We can, if you're, if you're good at evals, if you're good at assessing, quality, you can build that data by running various scenarios, collect a bunch of this data, and then you can train models to your favorite thing. So the challenge is Shapley values is computing Shapley values in our context is way more expensive than the amount of dollars we spend on the agent. Forget the amount of dollars we want to pay a publisher, right? So to compute a content owner, to compute that a content owner gets a dollar. If I decide to do the full Shapley value computation, that might take several dollars, right? So it doesn't make any sense, except you can really well estimate them if you build the right kind of data and models around it. But we have confidence that our estimations are good and that this is sound theoretically that if there was perfect information symmetry, people would want to collaborate. So the same way in the old ads days, people did second-price auctions and believed for better or worse, that people would reveal their true bids in an ad auction and end up paying less than that. Once feedback loops get established in the market, like today, everything is like an auto bid, right? Like most people are measuring ROI when they're doing advertising and running on auto bid instead of making up bids. I think a solid foundation. Waved on incentive alignment that Shapley Matt drives will ultimately maximize participation of content owners into this as well as content seekers via AI in an optimized system. And we have some positive evidence to support it. Like we've had some interesting partnerships that we've been able to do and announce and yeah, like imagine sitting with content owners and explaining Shapley math takes a moment. But ultimately once, once you pull out the properties, right, that you participate in the value, if you have unique differentiated data, you get paid more. If a banker in an expensive job reads your data versus my retired dad reads your data, the banker ends up paying more for that read, right? Because it's a part of high value work. And the, the macro math also seems to work. Like if you're going to spend a lot of money on inference on LLMs for knowledge work, if we allocated like what, two to 10% of it to data on the web, that's way bigger than all web data business models today outside of like Waldorf. So the macro math supports it. It is a scalable approach. And as agents on the web grow an order of magnitude year on year, it's like by my calculations, like 12 to 24 months from this math, being able to give meaningful dollars for a very wide range of content owners. That's very exciting. Especially at a time when, you know, you see the traffic data, the stack overflow plummeting, you see a lot of the human internet as we know it going away because of incentives. That's very exciting to see how you're thinking about incentive alignment for people to keep publishing. Yeah, it's, it's, it's why we started the company. Why is the company called Parallel? The company's original name was Shapley Inc. Really? It was. Do you know this? I knew that, yeah. When I incorporated, I'm telling you, I was obsessing about everything to do with the problem space. So while knowing that the first set of, also one, Shapley Inc is a terrible name for a B2B product. It was not going to be the long-term name. So we, we, but I incorporated as Shapley Inc for lack of a better word and Shapley.ai happened to be available. Shapley.com is a parked porn domain. So it was not going to be a name. I went around not talking about my company at Shapley and my badges at all events used to call it Nuco or Stealth Company. So, and it took us almost six, eight months to figure out what the real name of the company will be. We ended up at Parallel in part because it's like one, at the time we were doing a lot more in parallel. And two, we started visualizing this sort of a parallel web built for AIs and how its properties are different. And this metaphor that when you publish, you're now thinking of, as we all are now, of two audiences. Okay, I'm going to create a page. I know humans will read it. What should it look like to them? And then how should I make sure that agents can read it too. So it feels like you're dual publishing to two audiences. And so we had this like this parallel web for agents will emerge. And so that's why we started liking parallel. Love it. I was just thinking about this recently with the, you know, things like earning transcripts, right? Like I feel very confident that more people are consuming earnings transcripts through agents than are actually certainly listening to the audio. And honestly probably have been reading the transcript itself. So there are these like very high value portions of the web that have flipped already to being agent first. And obviously the communication. has not yet flipped um but i think if i were a public company ceo today and i was doing an earnings report i would make very clear that the thing that i'm saying will be transcribed and interpreted correctly by the agents not just by people listening theoretically yeah same for us when we're publishing docs for our apis like yeah it's like our customers are building ai solutions they are using ai to do it it's their agents reading our docs and code in our fdks it's not humans fumbling around docs pages for the most part so in fact for us the primary audience is a agent and that's how we test our docs a parallel web for agents that's very cool um maybe close us out tell us just give us a snapshot of where where parallel is today and if everything is fine and we can do it in the next couple of weeks anything that you hope to build comes true what does parallel look like what does the world look like what's your role in it i think of three levels in this journey level one is people are building simple agents that use the web more as like a tool like a web search tool because it's familiar because the first set of agents we built to your point was just like model and give it the same tools because these models have been trained to use all the tools human have been humans have been used to and let them do work by and large if you think of like most work it's there today of our influence it's there today we have now started seeing some subset of customers who are in this world where they're building more sophisticated multi-agent systems which use sub-agents and have agents wake each other up or orchestrate uh in interesting ways right people are seeing that as sub-agents most familiar one is sub-agents within coding agent harnesses but we see a lot more of that like if you're building an ai scientist uh some of those systems are very interesting very sophisticated very long running and like just throw large amounts of compute and data at a really hard problem uh and i think the third layer for the web specifically is the web goes from pull to push so today by and large across the first two modalities i described either an agent isn't calling a tool or a sub-agent but it is telling it a request saying go find this for me right now and i think where we will end up in a couple of years is a variety of use cases will be the web parallel call me if this happens so that my agent can do some work or a human can do some work so i use this line with my team all the time which is like imagine like agents are everywhere and they can do a lot of things right if there is something you can do today and it's worth doing and we still have gpus available we'll just go do it right like we won't like we won't be like oh let's just do this tomorrow for the sake of it if it can be done today do it so what will we do tomorrow we will do tomorrow work and we will do it tomorrow and we will do it tomorrow in response to something that circles through either another agent's work or something changes in the world as visible in satellite imagery or some customer commentary that happens some agent finishing some compute a human having a new insight to trigger work but they're going to be a few feeds like this which will drive new agentic work tomorrow and one of those feeds is going to be everything that changed on the web which is really exciting for me because then you're framing not a point in time need but a long term here is what is actionable for me if something like this happens as evidence by all of the information on the web that is changing all the time call me right and then i'll run my agent on it and so we get to then allocate compute onto the entire web all the time on behalf of all the customers and that's really exciting that's awesome really ambitious vision you're clearly extremely passionate about about this and exciting to see you building thank you so much for joining us today parag and thank you for joining us andrew you're very welcome sunya thanks thank you andrew for your debut thank you so much for joining us and thank you for joining us thank you for joining us thank you for joining us