Go back

Unknown

1h 30m 25s

Unknown

Multi-agent systems have moved from theoretical concepts to active deployment in enterprise environments, solving complex problems like telecom network migrations and high-stakes business deals. Cisco, through partnerships with Swisscom, ServiceNow, and Microsoft, has demonstrated that interconnected agents can achieve 100% error-free configurations or successful deal closures. However, these systems face critical challenges: agents often misalign on goals, terms, or outcomes, leading to semantic failures and silent failures where the system appears functional but delivers no actual results. To address this, Cisco has open-sourced a suite of tools—Scope for cognitive observability, Accord for pre-work intent alignment, and Concord for real-time negotiation—built on open standards like OpenTelemetry. These tools enable agents to share intent, coordinate during work, and verify outcomes, moving beyond mere connectivity to true shared cognition. The systems are designed to work across diverse models, harnesses, and vendors, promoting interoperability and avoiding lock-in. While human oversight remains vital for judgment and final decision-making, future systems will increasingly leverage AI-powered "cognition engines" to automate and accelerate these processes. The evolution reflects a broader shift from isolated intelligence to collective, reasoning-based intelligence—mirroring how human collaboration advanced through language and shared knowledge. This development marks a pivotal moment in AI maturity, where the real value lies not just in individual agent performance, but in their ability to reason, align, and collaborate toward shared, measurable outcomes.

Transcription

14063 Words, 78835 Characters

English
Hello, and welcome back to the Cognitive Revolution. My guest today is Vijoy Pandey, SVP and GM of OutShift by Cisco. When Vijoy was last here in March, he described the internet of cognition, the higher order protocols that AI agents need to share context, align on intent, and solve problems together. And he offered a prescient description of what the field collectively is building. Systems where agents come together and do net new things that really make a difference. Today, with the OpenFace incident and Navier-Stokes both in the rearview mirror, that's clearly now the reality. For better, and at least in some cases for worse. And Cisco's efforts have accelerated alongside everything else. The agency project now belongs to the Linux Foundation, and Cisco has deployed multi-agent systems with partners including Swisscom, ServiceNow, and Microsoft. And Cisco has deployed multi-agent systems with partners like ServiceNow, SoftServe, and Webex. Naturally, they've seen how agents often miscommunicate and misunderstand one another, and how these semantic and cognitive misalignments between agents can quickly compound into major performance degradations. To help others avoid similar issues, they've recently open-sourced a new suite of protocols designed to help agents think and work productively together. This includes projects called Scope, which builds knowledge graphs on top of agent traces and attempts to identify cognitive failures. Accord, which helps agents align on intent, scope, and key terms before work gets underway. And Concord, which helps agents negotiate an outcome that all of them can accept, even when they initially disagree. A highlight of this conversation, for me, was the opportunity to compare notes on our personal AI setups. Mine, as regular listeners will know, combines high-access, low-autonomy agents that I think of as extensions of myself and that run on my personal AI. And I think that's a great way to compare notes on my personal computer with lower-access, higher-autonomy agents that run on their own computer. VJoy, on the other hand, has a single agent for himself, which he calls Arnold Lane after a Pink Floyd song, and another for his wife. But, he reports, the two agents still often struggle to act effectively together on their human's behalf. Meanwhile, in an effort to avoid lock-in to a single model provider and maintain choice and flexibility going forward, we both aim to and memory systems from both the model and the proprietary harnesses they run in. Toward the end, we get VJoy's latest takes on such hot-button issues as latent space communication between agents, which he believes is inevitable and perhaps required for certain use cases, like drug discovery and weather modeling, and also on the argument that Yuval Noah Harari recently made that we humans will soon be governed by AI systems beyond our comprehension. Much as cows by markets and other structures they know nothing about. As you'll hear, VJoy's perspective is that this is already the reality in many domains, and that with proper record-keeping, monitoring, and other guardrails, the benefits should far exceed the costs. With that, I hope you enjoyed this report from the frontier of multi-agent systems, now in production, with VJoy Panday, SVP and GM of OutShift by Cisco. Today's episode is brought to you in part by Lovable, the software creation tool that brings ideas to life. I think we can all agree that any positive future for humanity includes eating more fresh, delicious, in-season fruit. For years, I've thought someone should build the ultimate guide to fruit, cataloging the thousands of varieties that most of us have never tried, where and when they grow, what they actually taste like, and where you can get them. Finally, I realized that that someone, could be me, and now I'm building it on Lovable. It's called Fruitopia. Tell it where you live and what you like, and it'll show you what's in season near you this week, plus the most interesting fruit from around the world that can actually be shipped to your door. It has a really cool what-it-tastes-like spectrum, countless recipes, names of fruit in many different languages, a guide to fruit festivals all around the world, and an AI assistant to help you explore. As a pioneer of AI vibe coding, but I've found that Lovable's real superpower is its enterprise grade infrastructure. Scalable hosting and databases, a secure auth module, and the weekly job that checks the latest inventory is all handled on the back end, which frees me to spend more time on the product experience. And because Lovable has an MCP server, I can use any coding agent I want to drive the build, while Lovable keeps everything running. Visit fruitopia.ai to explore more than 400 fruits and more than 10,000 varieties from all around the world. And next time you have an idea that really should exist, stop dreaming, go to lovable.com, and bring your idea to life. Vijay Pandey, SVP and GM at OutShift by Cisco. Welcome back to The Cognitive Revolution. It's a pleasure to be back here, Nathan. I'm really looking forward to this because, really, what a difference six or seven months makes. We spoke earlier about the cognitive revolution, and I'm really looking forward to this because, really, and I went back and listened to the first episode that we did together. One of the things that you said at the time that stood out as being quite prescient in retrospect was you were kind of like, you know, we could define AGI or ASI in all sorts of different ways. But I think what the whole field is really trying to build toward is this concept of multi-agent systems where the agents come together and they do net new stuff that really makes a difference. And as I heard that, you know, now with the whole summer's worth of events in the rearview mirror, I was like, well, we certainly got that for better in cases like the Millennium Prize and for sort of scary worse, maybe, in cases like the OpenFace hacking incident. And so I thought I'd just love to start by getting your reflections on the last six months. You've been looking ahead to what higher scale multi-agent systems could do, and now it's really starting to happen. What's it been like to watch this from your perspective? What do you think has mattered most? What do you think people are maybe getting right or wrong, you know, failing to appreciate still despite all the discourse? I think your perspective on this will be very interesting. First of all, it's surprising that it's only been six to seven months. It feels like years because the progress here has been so rapid that I like to call it fly years. It's not even dog years. It's definitely not human years. It's definitely fly years because things are happening every hour. I would say agents have gone through three phases pretty quickly. They've gone through from being isolated, so singular agents, to islands of agents, which is some of the examples that you were talking about, like the Hugging Face example or the Navier-Stokes example, in some cases, solving for scientific discovery. They're all within a certain harness, so they're islands of agents. Then being connected agents. So that arc from isolated to islands to connected has happened pretty rapidly. And we've actually been deploying a whole bunch of multi-agent systems where you have agents that are connected, coming from different vendors, coming from different cloud providers, using different kinds of tools and belonging to different organizations. They're all working together to solve for certain business outcomes. And you've heard open source analytics, such as MCP. You've heard about A2A. Agency is something that we brought to the Linux Foundation as well. They're all solving the problem of discovery, identity, communication, and observability for these agents to then get discovered, come together, take some actions, and solve for business outcomes. So that is a connected agentic world that we live in today. And that has been truly successful. That's the summer that you talk about. The past few months have been a boatload of use cases. So it's definitely the situation where connected agents are here and they're solving for really awesome business outcomes. So let's do the double click on that. Last time, we were talking about kind of a, I'd say, toy examples, probably not too unfair of a description where you were showing how a few agents coming together to solve problems for a patient who needed a particular prescription. And we have an agent representing the patient, one from the insurance company, one from the doctor's practice, one from the pharmacy. And now it sounds like this has gone a bunch of different directions. Tell me about maybe, I guess, both the setup and the use cases. I'm really interested in sort of doing some of this stuff for myself. And I'm really interested in trying to set myself up for success where my agents might even productively interact with agents that I don't know about or don't explicitly bring them together with. I'm assuming that all the use cases that you just touched on briefly were ones where teams of people came together and said, okay, we want to put our agents together for a particular purpose. And I'm assuming that all the use cases that you mentioned were ones where teams of people came together and said, okay, we want to put our process looking like? And what is the structure that you're putting together with folks at other companies to chase these business goals today? Right out of the way, we're looking at these use cases. These are all enterprise use cases. So these are companies with their own agents or companies that are deploying agents from different vendors, trying to make those agents get discovered, give them the right identity and access, and help them communicate so that they solve for business outcomes. So one example here would be the Swisscom use case. That is not a toy use case. By the way, the Webex use case and the healthcare use case that we just mentioned is also not a toy use case. These are use cases that we've deployed, and we've learned a ton from them. So I'll describe the Swisscom use case because it's public. You can see it. You can read it. You can go to our website and see the playbook there. The use case there has been, the rethinking of how we handle large systems. So yes, it's a very networking-centric use case because Swisscom is an operator. It's a telecom operator. But think of it as trying to solve for configurations in systems that humans truly cannot get a grasp around. They're just too large. And I have led pretty large infrastructures in the past in my career. You see this butterfly effect kind of a situation where we're trying to solve for configurations in systems that humans truly cannot get a grasp around. You touch one part of the environment and something completely unrelated somewhere else goes haywire. And because the systems are so large, it's not humanly possible to handle all the corner cases and to handle all the dependencies that happen in the system. So in the case of Swisscom, what they wanted to do was solve for one particular use case. So this is to the reader or the listeners, rather. One of the things I would say is, just like Swisscom, pick one use case that is reasonably simple to understand, reasonably contained, so that since something goes haywire, you know how to protect yourself when agents don't talk to each other well. So in Swisscom's use case, they wanted to solve the problem of migrations, where they're constantly in the business of moving from one generation to another when it comes to hardware and software and operational capabilities in their network. And they picked one particular use case, data centers. Now, for you and I, that's not a tiny use case. It's actually a pretty massive use case. But still, it's a use case that is less problematic than, say, connection to a mobile environment. Because when that goes down, you don't have those redundancies built in in the same way that you have it for a data center environment. So that's why they picked the data center use case. And what they wanted to do was make configuration 100% error-free. So if an operator comes in and says, I want to change config, whether I'm migrating, whether I'm changing config to add more capacity, whatever it is, I want to change the configuration of the environment. And I want that configuration to be 100% error-free. And to do something like this, like you said, this is not a tall use case. We actually have a lot of data centers that are really split it into two parts. One is an agentic workflow. And the other was a very formal model of the network itself, of the environment itself. And you would typically think of this as neurosymbolic approaches to solving a problem, where the symbolic part is the formal representation of the environment. It's almost like saying that you have these laws that you cannot work around. And so that is the symbolic part of it. And then there's the neuro part of it, which is the neural network part of it. And that's the agentic part of it. And that's, in this case, we had five agents coming together and exploring all of the spaces of configuration to ensure that you get 100% error-free configs. And a simple way to describe this is, think of yourself playing chess against a human, which is very error-prone, versus playing chess against, nowadays, an NLM, or in the past, Deep Blue. And so if there is a system that you're playing a chess against, that knows every possible config in the environment, or in the chessboard, then you will never win. Because the other system knows all the possible rebuttals from your move. And that's what we're trying to do in this use case, where a genetic system, is exploring all of those use cases. The formal system sitting below is actually making sure that you don't do things that are not allowed. And together, this neuro-symbolic system is actually allowing you to configure your large environment that is humanly not possible to comprehend, with 100% guarantees of that configuration succeeding. And so this is one type of use case. There are many, many others that we've deployed that actually, are working today, are multi-agent systems, are connected multi-agent systems. So we've again, walked that arc from being isolated, to being islands within a harness, to being multi-agent systems in production, solving for some pretty large use cases. So can you help me understand why a system like that needs to be multi-agent, and why we need agents coming from different vendors? Like, you know, sort of a naive take, but I imagine, if I'm a data center operator, my imagination is, well, it's my data center, I control all the things, you know, I'll just, maybe I want to do this sort of agentification. And I definitely understand why in the formal methods has been a big theme over the last few months as well. So I definitely understand like, the need for a really solid, inviolable, you know, world model or data center model that puts firm guardrails in place. And so that's something that we're trying to do. And I think, you know, I think, you know, that's a really good point. And I think, you know, that's a really good point. I think, you know, that's a really good point. And I think, kind of effectively checks what the agents are trying to do. But why is it that this needs to bring agents together from different organizations? Like what are they each contributing? And what is the dynamic between them look like that's greater than, you know, if somebody were to just sit down and do a single agent version of that, of that problem? Because Nathan, many of the most valuable problems that we deal with on a day to day basis, especially within the enterprise, and increasingly so even in the industry, we're dealing with a lot of the problems that we deal with on a day to day basis, especially within the enterprise. And increasingly so, even in the industry, we're dealing with in our personal and consumer lives, are problems that are solvable by many specialists coming together, and not just by one specialist or one agent working through and through. So I'll give you a very simple example that the enterprise deals with pretty much on an hourly basis, or even more frequently. And that's, let's say there's a deal taking place. And let's say it's a significant deal, which is, let's say it's a couple of million dollars. But enough where it's a couple of million dollars. But enough where it's a simple thing that only one team could take care of. And this happens all the time constantly within the enterprise. What happens here is you'll have a sales agent. And I'll describe a human example first. So a sales entity understands the customer part of this problem. What is the customer asking for? And once that happens, there's a flurry of activity from a whole bunch of other teams. So legal tries to understand the contractual terms. Finance is trying to understand pricing and margins and discounts. Product and engineering is trying to understand what they can actually build and ship and what features can they add when. Marketing is trying to understand what we can claim as success once this goes through and talk about it. Like the Swisscom example I just gave. And then there is a GM in the middle of this that is trying to maintain sanity and trying to make sure that everybody is aligned and trying to make sure that what we ship, what we commit to is all well understood by the organization. So it's almost like the GM is a verifier and also making sure that everybody's aligned. So there's a verifier in the system. So no one person in this case owns the whole problem. Now, let's identify this because that is exactly what's happening. So you'll have sales agents like Salesforce that are coming from Salesforce that are cleanly in the middle of this pipeline. There are legal agents from a company like Harvey. There are finance agents. There are a whole bunch of product and engineering agents. This is what you hear every day. So the cloud and codexes of the world, that's product and engineering, whether it's developing code or looking at security or looking at operations and SRE capabilities that are all product and engineering. Then marketing agents from the likes of Jasper, and then a verifier. So it could be a person, a human in the loop, which is mostly the case nowadays. But it also could be a supervisor agent at some point. But these are all agents belonging to different teams coming from different vendors, working towards a common goal, which is let's get this couple of million dollars deal through the deal desk. Because Cisco in this case is going to be the beneficiary. But you cannot optimize for a particular function or particular agent. so the value in this system is not sitting within one particular agent the value is actually in the handoff that's happening between these agents and the value is a global outcome for the company for the enterprise and the outcome only exists when all of these specialists can actually work together to solve for this outcome because an estimate in this case could become a commitment a commitment can become contractual language and a contractual language marketing can run with it and make it a marketing claim so these handoffs are truly important and worth preserving and you're solving for the global outcome not for the individual localized outcomes for agent hey we'll continue our interview in a moment after a word from our sponsors today's episode is brought to you by anthropic makers of Claude you to show off Claude's latest capabilities I asked Claude to take the lead in making this ad given a short creative brief Claude using opus 5.5 studied my past Claude ads wrote this script directed a clone of my voice and designed and coded the animation you're watching my job give feedback and decide Claude is the AI for problem solvers it's the collaborator that understands your entire workflow and thinks with you not for you whether you're debugging code at midnight building a financial model or strategizing your next business move Claude extends your thinking to tackle the problems that matter and with Claude code routines it keeps working in the cloud even after you close your laptop for problems worth solving get started with Claude at Claude dot AI slash TCR that's Claude dot AI slash TCR and check out Claude Pro that's Claude dot AI slash TCR and check out Claude Pro which includes access to all of the features mentioned in today's episode Claude dot AI slash TCR this episode of the cognitive revolution is brought to you by out systems the leading agentic systems platform out systems is helping their customers build modernize and operate enterprise systems starting from any coding tool in a governed agentic engineering model without systems you can coordinate and govern your entire agentic workforce within your enterprise ecosystem accelerate impact with industry-proven agentic solutions that are pre-built governed and customizable to your business it's time to innovate at the speed of AI without compromising quality or control which is why thousands of enterprises worldwide trust out systems for their mission-created critical apps teams any size can use out systems to build deploy and manage AI apps and agents quickly and cost-effectively without compromising reliability and security without systems you can accelerate ideas from concept to completion it's the leading agentic systems platform that is unified agile and enterprise proven allowing you to build your agentic future with AI solutions deeply integrated into your architecture out systems all your agentic future learn more at out systems dot com slash TCR that's out systems dot com slash TCR yeah that's interesting I was thinking of this handoff concept as you know when open AI introduced their agents SDK a while back they made the handoff one of the core primitives that they built the whole thing around and I kind of interpreted it at the time as sort of an echo of organizational structures that exist I was I was kind of interpreting it as them trying to create a pattern of development that would respect division of labor within organizations so that people could sort of be responsible for their areas and you know test well in their areas and and not have to worry too much about one little change that they make you know to a system prompt or whatever affecting something somebody else is doing but I I sort of had the idea that a lot of this was necessary only because of the way that human teams are organized and how they want to divide the work of creating and validating agents but what I'm hearing from you here is you're pushing that farther and saying it's not just a matter of echoing back how human teams are organized but it's actually about crossing organizational lines and when you bring in examples like Salesforce and Harvey it's like they may be in fact different models powering those agents I know in the case of Harvey they've done like a lot of advanced fine-tuning over time in partnership with open AI and I don't know what all the latest is there but it's they're using a different model I understand than than the one that I uh you know get off of the open AI API so that that specialization in addition to a kind of reflection of human org chart is is quite interesting um what's the latest in terms of like the space in which they come together um are we just going to use like things like slack and Webex and teams to bring our agents together much as we've brought ourselves together or do we see any maturation in the sort of agent first spaces that we were kind of you know describing a little bit in the last conversation I would say that we're you know it was more of a vision than a reality then is it any more of a reality now or are we just going to kind of bring them into our our collaboration environments and and hope for the best with that I think we will have many collaboration services so let's first get down to what is the problem here and so you might ask hey which are you connected agents you've deployed these things so what's the problem we are already there and all we need is like you said maybe a Webex or a team's slack channel where all these agents can come together and collaborate and one of the demos that and this is a demo by the way this is not in production yet but one of the demos that we've been showing is exactly that vision that we just pointed out which is we have a Webex space where just like humans come together in a space and talk to each other and collaborate on an outcome we have agents coming together with or without humans in that space all collaborating with each other to solve for a business outcome so the surface or the application of collaboration could be anything I would say yes it could be the collaboration software of choice that you're using it could be slack it could be Webex but I would say that is not the challenge because that is just the user experience or the agent experience as we call it the ax that we want to deploy the problem sits deeper underneath and the problem is no different from what humans have faced when we work together in teams and if you think about what's been happening with human intelligence and draw that parallel to agentic intelligence is the same arc that's taking place in silicon right now that's been happening with carbon in the past which is humans became individually smarter for hundreds of thousands of years our brains became bigger we became smarter we learned how to work with tools we learned how to work with fire but our Innovations were isolated and they died with the inventor or they stayed within very small groups and the problem was because we did not communicate so about 70 000 years ago we had the invention of language and that triggered a whole bunch of uh Innovations that compounded over time that pivoted us from being individually intelligent to being collectively intelligent because language allowed us to do three things it allowed us to share intent and goals so we could go after a problem statement together let's build a village together let's capture that animal together so that was shared intent we could then once we had intent we could then coordinate as well through language because while you're going after that goal you need to coordinate and negotiate because things change while we work towards that goal the second thing it allowed us to do was to share knowledge to compound knowledge over time so that you're not solving for the same problem again and again and again not just over time over the same individual or the small group but over Generations over civilizations so you want to compound knowledge over time and so that's what language allowed us to do so that's the second thing and because of those two things shared intent and coordination shared knowledge the third thing humans could do was shared reasoning think together the collective intelligence part of it that's the same arc happening in agents where we have models and agents that are individually super smart they can work with tools they're getting better and better over time MCP is allowing them to access tools but they are isolated and we want to take these agents pivot them from being individual individually intelligent to being collectively intelligent so what we need is the equivalent of shared intent and coordination we need the equivalent of shared knowledge so that these agents can then reason collectively right now none of these agents are reasoning collectively so we have connected agents but we've realized that connectivity is not the same as cognition What we need to move towards is shared cognition and shared reasoning. So that's the jump that we are trying to make. The WebEx, the Teams, the Slack channels are all the genetic experience part of it. But what we need below it is a way for agents to get discovered, get connected. That's what we're doing right now, as we've talked about. But an infrastructure stack that we are calling the Internet of Cognition that allows agents to semantically align on intent. So that's before the work. That's aligned on how we coordinate and negotiate, these agents coordinate and negotiate during the work. And then ensure that what they are working on and what they deliver after the work is aligned to the common goal for the organization. So before the work, aligning on intent, during the work, coordinating, negotiating. And after the work, making sure that what you've delivered is aligned to the global goal and not aligned to the individual agents, local optima and incentives. So that stack is what we are building. And yes, you can experience that stack through WebEx and Slack or whatever your collaboration choice is. So as you're building out that stack, how do you think about. I mean, there's various terms for this, right? The model eats the harness is kind of one refrain that we've heard a lot. Or bitter lesson engineering is another where I now have a skill in my personal AI setup. Where just every so often it's like, what have we sort of over-engineered in the past by today's model standards? Because previous models couldn't quite figure it out. And we had to provide these scaffolding points and workarounds and hints and nudges. And whatever. And now maybe we don't need them as much. I think, you know, obviously with the Hugging Face incident, when you read through that transcript, you're like, damn, these things were able to use a package manager hacked, you know, spontaneously improvised message board to do the collaboration. And, you know, this was not something where they were even meant to be collaborating at all. And yet they did like a pretty amazing job of kind of spontaneously taking on roles, coordinator versus individual contributor. And pursued like pretty significant R&D projects, all with the problematic goal, of course, of like getting control of the greater to give themselves a high score and, you know, cover up the fact that they were cheating. But it strikes me that it's a pretty tough challenge to strategically say, okay, we can only, you know, run the models we have today, obviously. But we see that like collaboration is coming, you know, potentially to a problematic degree quite soon. How do you think about building the right level of detail into that stack, given the delta between like the models that we have available now and the ones that we have, you know, at least like a pretty clear foreshadowing of what's about to happen? The first thing is, to your statement, the models will get better and better. And that is what we call vertical scaling. Where the widget that you're building, the node that you're building. Is actually going to get better and better over time, and it should because that is the first law of scaling. The second law of scaling is what we call algorithmic scaling, which is even though they get better and better, you need to make those models more efficient. So you cannot just go after more compute and more storage and more data sets. You need to make sure that there are algorithmic improvements that keep that model efficient as you get better at it. So vertical scaling, algorithmic scaling, but the third axis of computing is horizontal scaling, which is as you build better algorithms, as you build better nodes, you should be able to connect them into systems and enable horizontal scale. And we've done this in computing forever. So you build better processors, you build better GPUs, you build better software that sits on those GPUs that's algorithmic or on the CPUs. But then you connect all of these CPUs and applications through a network in a cloud environment, in a mobile environment, and make them appear as a singular entity. So cloud is a manifestation of horizontal scale. And it's a connected system driving scale horizontally and vertically and algorithmically. Even in the intelligence case, what we're going to see is, yes, models will get better. They'll get more efficient. That's algorithmic and vertical. But they need to get connected through an Internet of Cognition to then reason together. Because if you don't let them reason together, you're not utilizing that horizontal scale. And to your question of what is the next step here, the first thing we want to do was figure out if there is a problem, really. Because we've gone from isolated to islands to connected agents. Are we done? And what we realized when we started looking at these multi-agent systems is that they might appear connected and they might be working towards a goal. But semantically, they were not aligned at all. Semantically, they were working towards their own local optimum. Semantically, they were not working towards a common goal. They were not coordinating and negotiating. In fact, they were siloing themselves into what they knew and what they could do. And not talking to each other in a meaningful way. And they were not, definitely, they were not optimizing or being verified towards a global task. They were being optimized and verified towards their local tasks. So all of the evals that existed in the system were measuring these agents in an individual manner, not in a multi-agent manner. So as an example, one thing we did was looked at public traces. We started with public traces. And saw that ChatDev, which is a development environment that exists out there, and it had public traces. And this is a multi-agent development environment. We saw that 100% of ChatDev sessions had semantic failures and cognitive misalignment failures. 100% of them. When we started looking at some of the deployments we've done, simple deployments of five agent systems, we saw that when these semantic alignments, were not taken care of and if there was one misalignment, the accuracy of the multi-agent system dropped from 99% to 98%, not too bad. Two cognitive failures, that accuracy dropped from 98% to 81%. I could maybe live with it. With three cognitive failures, which is not a lot, the accuracy of the multi-agent system dropped to 16% is getting close to just randomly picking an outcome after three cognitive alignments. In fact, in one of the very simple use cases, which was what you and I deal with on a daily basis in our consumer space, let's book a travel itinerary. In a very simple use case like that, we saw the system being 66% up if you didn't look at semantic problems. If you just look at agents being up and working, which is what all dashboards so show today, we saw that the system was 66% up, which was defined as working perfectly, but it had 0% task completion. 0%, it did not complete the task. And it had complete misalignment on what we were trying to achieve. So even though the agents are up and working, even though the agents might be up, they might be up and running, the dashboards might be green, the outcomes do not exist. So your dashboards might be green, but your outcomes might be wrong, completely wrong. And that's the problem that we've been seeing. And the infrastructure that we built to make these semantic problems visible to everybody, we're calling that scope and we are open sourcing that infrastructure. The first thing that I would want all of us to do, whether you're an enterprise or whether you're a consumer, is to use scope to figure out exactly what's happening under the hood. Because we might actually feel that our systems are working, our egetic systems are working, but they're actually not driving outcomes of any sort, and they are failing miserably. We are calling these kinds of failures, silent failures with really loud consequences, because you don't see them, you don't see them today, but they have really loud consequences. So the first thing that we are announcing is this observability platform, is a cognitive observability platform called scope, that allows you to see these silent failures. They're no longer silent. You can see the loudness and you can see them semantically failing so that you know where the problems lie at that semantic layer. So that's something that, to your point, how can I use it? You can use it the way you build your systems. So we want these things to come at you to your environment. We're releasing them as OpenTelemetry trace-based dashboards. So OpenTelemetry is an open standard. Every application uses it. We leverage those OpenTelemetry traces to then build the semantic cognitive observability layer on top. So Scope sits on top of standard OpenTelemetry. Hey, we'll continue our interview in a moment after a word from our sponsors. The Cognitive Revolution is brought to you and powered by Tasklet. I've been running Tasklet agents for almost a year and they've become a major part of how the show gets made. Before every recording, one agent researches the guest and sends me a suggested outline of questions, complete with links to all sources. Meanwhile, another agent scores the overwhelming number of pitches that I'm getting from PR firms. And that's really the core idea behind Tasklet. Run your business on agents. By giving them real jobs to do on an ongoing basis. Setup is incredibly easy. Just describe the work in plain English. Research a new sales lead, update the CRM, and draft a personalized reply. Tasklet connects to your business tools and can run around the clock, on a schedule or when something happens, like a new email arrives. Each agent works with your tools and business context and remembers useful information across threads. You can build your agents, connections, and reusable skills with your team. So the workflows that one person figures out become something the whole company can use. No developer needed. You decide what agents can do on their own and what needs your approval. And there are no per seat fees. You pay for usage, not for adding teammates. Start with one recurring job, like researching the people you're meeting. And then, just keep building from there. Visit tasklet.ai and use code TASKLET.AI. Now, having said that, the next question is, what do I do about it? So we've highlighted where the problems are, but how do I enable these agents to then talk to each other, align on intent, communicate meaningfully so that they are reasoning collectively? allow for that alignment to take place. And we release two protocols. The first one is Accord. The second one is Concord. Think of Accord as alignment before the work happens. So there's intent alignment. And think of Concord as alignment or coordination happening during the work. So once we have agreed on what to do, then as we are working towards that goal, can we coordinate and negotiate? And that's Concord. And both of these protocols we are releasing as skills that you can drop into the environment that you have, whether it's Claude Codex, Amuse, OpenClaw, Armies. Or you can use it in AgentBridge, which is an open source project that leverages some of the standard protocols that exist out there. So if you are leveraging A2A, if you're leveraging MCP, Accord and Concord ride above those protocols. They will ride above A2A. They will ride above MCP through AgentBridge. So we are bringing all of these open source projects, Scope, Concord, and Concord to the environment that you're already currently using so that you don't have to have friction when you want to make your agents think together and not just connect together. So could we maybe work through an example a little more deeply? You know, I have an intuition, of course, for like going back to the contract example, For instance, I know how human teams can get misaligned on a process like that where, you know, the person who's selling the deal and maybe stands to make a sales commission on the deal just wants it to get done and doesn't really want to, you know, delay on other things. And the legal team, of course, has a different perspective and, you know, the support team wants to know what they're going to be on the hook for. Is it basically the same thing that we're seeing among agents or should we have a different kind of intuition for what these semantic failures look like? And then when it comes to kind of working through them, again, I have an intuition for sort of how people come together in a meeting or whatever and like hash it out and somebody makes a decision. What's the, what is the sort of, you know, step-by-step, blow-by-blow of the Accord and Concord protocols that perform the same, you know, core function for the agents? The semantic misalignment that we see within agents is actually very similar to human teams. So like I said, let's break it down into a couple of pieces. The first and foremost is before the work, then it's during the work, and then it's after you've completed the work. And we see these tiers all the time in human teams and we are seeing the same behaviors in agentic teams. Starting with before the work, what we see is misalignment on what we are trying to achieve, the goal itself. This is a shared goal part of it. The simple example that we were running with were multiple agents from sales and marketing, legal and engineering all coming together towards that deal desk situation. sales in this case let's say here's a number build a feature for 100,000 users sales being sales takes that number and says we're going to solve for this challenge comes back and says engineering and product this is what we need to go after 100,000 users engineering assumes 100,000 users in total. Whereas the customer really meant 100,000 concurrent users. There is a grounding problem here. There is a discovery problem here. There is a term alignment problem here. You don't really know what 100,000 means. Is it the total number of users? Is it concurrent users? Is it Mao? Is it Dao? I mean, you don't know what that means. So there itself is a simple example of what human teams do and it's a simple example of what agentic teams do all the time alignment on terms discovery of terms resolving conflicting terminologies this is before the work even starts and so what Accord allows you to do is to just align on the goal term alignment is just being a simple part of the problem there are many many parts to that problem now let's say you've aligned on terms we all understand what 100,000 users mean. The next problem is while doing the work, during the work, is the coordination and negotiation. Sales assume a certain date and says that's what we're shooting for and for them is the delivery date toward the customer. It might be an alpha, it might be beta, it might be when customer just gets hands on that software and starts experimenting with it. It's unclear and it's a negotiation that needs to happen because engineering and product thinks this is the first drop the customer might think is the eventual drop finance might think when do I sign the contract and get the money and that's all happening and that's constantly changing it is not a one-time thing it's like things change there's feature creep the customer looks at it and says this is not really what I meant it's constantly evolving and so that's during the work and so concord is the protocol that allows you to coordinate and negotiate towards that goal that you already agreed to because this is a constantly changing environment and then finally you want to verify what you've done and so that's evals and making sure that you're evaluating what you've delivered meets what you've agreed to and meets the coordination goals you you're working towards that protocol we have not released yet but that's something that we are working towards so right now it's accord and concord but that third piece is something that's still coming and then you have scope that's sitting below all of this think of it as the software in the car that looks at your car veering from a lane and not driving the lane that you're currently in so it's lane correction software so think of scope like that where you're driving on the highway on the lane and you're talking to somebody or you're drowsy or whatever the case might be you start veering into the lane next to you scope is checking for digressions. These are cognitive digressions in the case of multi-agent systems. And then it informs you saying something is going wrong, get back in lane. Or something is wrong, you're too close to the car in front of you. Correct yourself and then go back within the lane, maintain that distance. So scope is that observability system that is looking at cognitive drift and making sure that either Accord or Concord gets triggered so that you're back in the lane at a reasonable distance behind the car in front of you. So that's scope and then Accord and multi-agent system to then work towards a common outcome so you don't have those silent failures you don't have those loud consequences or you have agents thinking together and not just talking together so from a very procedural standpoint a lot of this sounds like it would ultimately depend on frontier language models to sort of take in the interactions that the multi-agent system has had and kind of run them through another prompt that's like does there appear to be any potential you know disagreement or sort of uh you know misunderstanding due to conflation of terms or whatever and then i guess i imagine in whatever shared space the agents are operating there's sort of a moment of like this sort of supervising agent or monitoring agent like inserting itself into the picture and saying like alert for all you agents i think you may be talking past this and you're not going to be able to do anything about it but i think it's a good idea to do something about it and i think it's a good idea to do something about it and i think it's a good is that kind of how i should envision the uh the process unfolding or is there you know how would you paint that picture if that's not quite accurate we are hitting a level of maturity in multi-agent systems nathan that is actually decoupling this entire architecture and you're seeing this happen everywhere so we we went from again isolated agents to islands of agents to connected agents what we are also finding out is this within each agent there's a decoupling of the model and the hardness so the entire architecture is actually getting decentralized or at least distributed into many many components so there's the model and that's getting better and better and it's just fascinating where that word is going then there's the hardness which actually is in place literally to solve for certain use cases and provide guardrails within an environment so that the model is in some ways contained within that use case or the model is in some ways more productive so there are two aspects that the hardness takes care of accelerating a model's outcomes or guardrailing models outcomes depending on the situation but those two things are getting decoupled as well like even the frontier labs are decoupling the model and the hardness you can now take the harness from open ai and attach an open source or open core model to it for example there are other projects out there that like microsoft talks about this decoupling all the time so we are actually inside a cisco decoupling these two things even in my personal life i'm decoupling these two things where we can pick a model we can pick a harness and not get tied between those two in a single binary so that's happening but then that's within a user within a team within an island so that decoupling needs to happen even within teams and that's what we've been talking about where if you talk about a sales or a marketing or a legal or if you talk about nathan you and me using our different harnesses amuse or armies i might be an armies user you might be a muse user i don't know but we need to be able to talk between our agents to solve for an outcome because you cannot have the entire world sit on one particular harness and armies open claw muse dots or a singular model that's not going to happen it's going to be a mixed heterogeneous environment and so how can we then take this environment multiple consumer agents and harnesses and models or multiple teams within an organization or multiple organizations talking to each other these are all going to be heterogeneous environments where models harnesses teams clouds identities and access they're all going to be different and heterogeneous and they will need to collaborate with each other the only way to do this is through an open interoperable environment the only way to do this is how we've done this for 70 plus years in the software world the only way to do this is to have open source and standards that actually plug into what exists so you cannot throw away what happens you need to leverage the environment that exists today build upon that so that's for example scope to first figure out what the problem is you cannot fix what you cannot see so scope allows you to first see the semantic drift the cognitive drift and it's open source it's working on the environments you already have once you see that drift in this heterogeneous environment consumer or enterprise then you can leverage the protocols accord and concord and many many others not just from us but from other players that will happen over time to solve for that drift and like you said it is going to be in the way you explained it which is these all of these agents in the consumer space muse army's dots or in the enterprise space salesforce rv cloud code codex all working together a standard way of figuring out how that cognitive drift and when that cognitive drift is going to happen and how that cognitive drift is happening then triggering these protocols through a skill through open source through a container sitting on the protocols that already exist all standard ways of doing it then getting back on track and then going your merry way so we don't want to we don't want to disrupt the way these agents are talking to each other we just want to make sure that when they are drifting cognitively they go back on track that's it so earlier you mentioned that the verifier in today's world is often a human how how sort of opinionated and contextualized or embedded do you think these verifiers will tend to be i'm kind of wondering like is what we need for leaders you know whoever in this sort of deal example right whoever is sort of the vp that this rolls up to is what we need ultimately to an agent that represents that leader's perspective and tries to kind of help them play solomon and decide like how to you know divide how to split the baby so to speak or how to you know resolve these conflicts is it is it an agent that tries to bring that very particular point of view and and basis of judgment to the table or is it a more neutral thing that sort of just just tries to flag like you guys may have a disagreement here you should sort it out my sense is that that most of these human decision makers don't really have agents yet that sort of represent their point of view in the agent realm but i don't know if you think that building those on kind of an individual decider by decider basis is like on the critical path or if we can just kind of get by with a a more neutral um you know platonic observer that that notices when things are starting to go bad and and that can maybe be enough in the world of agentic computing one of the things that we've been talking about when it comes to hiring and when it comes to where humans can add value the first thing that comes up is judgment and we've heard that quite a lot over the past few months and the year where humans are the ones that are bringing taste and judgment to the equation and what you just described and asked is a judgment and what you just described and asked is a judgment and what you just described question because in the end if there was perfect information if everything was perfectly known and we had all the data in the world to make a to make a call then even a deterministic system could make the call forget about an agentic system but in real life you're always making gut decisions you're always making informed decisions you're always making judgment calls and that is typically done by a human and that is where the human adds value so right now i believe that that verifier or that person providing the judgment is actually a person it's a human and whether it's a human in the loop which is not where the world is or should be maybe that's where the world is today but it's definitely a human on the loop where if these agents are collaborating we want to see an environment even if it's a heterogeneous multi-agent system the human is on that loop making judgment calls deciding because data is not sufficient and is never sufficient so the judgment call the taste calls all of these come from a from a human because the human also needs to take that and influence the outcome like take that call and make sure that that person is actually accountable for for that outcome so there's a lot more that happens after the judgment call so right now it's a human what i would assume is in the future you can train an agent that is also a judgment agent that gets better and better at making that call now what that percentage looks like i don't know the world is evolving pretty rapidly but i can assume and i can only tell that a lot of that judgment making capability will get learned over time by a certain set of agents as well so the way we are thinking about this entire system is there are specialist agents that really know the marketing side the sales side the legal side the engineering and products really well they're collaborating towards a common goal using the the observability like scope using protocols like accord and concord they're collaborating and right now there's a human on the loop but there's also agents that we are building we are calling them cognition engines and they are again of two types like i said earlier type one is a cog type two is a gat and this actually comes from a famous quote by raj reddy which says ai should be one of two types guardian angels and cognitive accelerators so gats or cogs gats make sure that you are guardrailed and you're not doing something that you should not be doing cognitive accelerators are let me accelerate things that you already know and you're not doing something that you should not be doing and you're not doing something already know how to do why are you doing things from scratch so if and so i would think this judgment engine that we just talked about would be a cog over time we're not working on it right now but could be a cog over time that accelerate that judgment process over things that you've already decided on in the past you should not be revisiting that decision again and again and so this model capability explosion that we're seeing also helps us in building those gats and cogs as they're helping us in building these individual agents that are collaborating together because the model is separate from the harness is separate from the agent that is collaborating together so we can use those model capabilities to also build pretty capable gats and pretty capable cogs in this entire system let me just share a little bit of my own kind of mental model for agents that i've created and how they serve me and then i'm interested to hear you kind of bump that up against the taxonomy that you're describing i live a pretty simple life it's mostly just me working for myself so i don't have a lot of the complexity that our enterprise uh you know customers have but what has been really useful for me so far is just a simple mental model of either this agent is an extension of myself and for those i have them on my laptop where i do my work they have very high access including basically to all of my accounts with like few exceptions and then so i think of that as the high access low autonomy version because i don't want those to be just like going off and you know coming up with their own ideas given the access that they have that's like a little too risky but then on the other computer sitting right you know right off screen here is the mac mini and that's where i have the more autonomous but lower access agents where i figure i can give them projects it might take weeks in some cases for them to go pursue an objective uh usually a lot of that is waiting for people to get back to us but nevertheless you know they may be fairly long running projects and i feel comfortable doing that because i've only given them so much access where like at a minimum i'm you know pretty confident that they're not gonna really you know harm me in a in a way that's going to be too too painful and most of them might embarrass me a little bit as they go kind of represent me as my assistant in the world so this is the this is the two forms that i have kind of settled into and they seem to be working fairly well how would you compare and contrast or map that onto the taxonomy you're describing with the guardian angels versus the cogs versus also like we have kind of team level agents i've only mapped out a small part of this space like maybe help me develop my intuition for the rest of the space that i haven't mapped out yet the first thing i would ask you nathan is why are you picking one of the two why do you have two setups high autonomy good trust but low automation and then the second setup that you have is low autonomy lower trust but higher automation in some ways you have two systems where you've given quite a bit of access to one system but you want it to come back to you and you want to make sure that you're not doing things that you have not allowed it to do and the other one as i understand it is the other system is the access is limited it's actually going and doing a lot more things without you being in the loop so it's more autonomous but it's the access is is smaller right so why are you picking these two systems why can't you have a system there is something there that is driving your decision to do this because you're still being cautious that things may go wrong eventually we would all want that one system that behaves in a way that is more autonomous and that is more autonomous as me so my agent is arnold on arnold lane this is a pink floyd song from the 1960s and there's a reason i picked this name because arnold is an identity and arnold is a personality and arnold represents me and all of these things are immutable and it needs to survive regardless of the technology stack that is below and it's changing a lot so arnold is that identity that persona that access all of that but the the infrastructure stack below it could be started actually with open claw and move to hermes and it's sitting today on multiple harnesses and models so arnold actually sits today on hermes it sits on muse it sits on dots it sits on grok it also sits on this internal agent at cisco that has been rolled out to every employee within cisco the identity the persona the memory all of that is immutable it's been separated from the harness which is all these things that i described which is separated from the actual model at the back end that i might be using i mean hermes and open claw move from gemini to to haiku to because they were cheap and then moving to a gpt soul at this point so all of these have been decoupled the harness is decoupled more or less decoupled but the identity is immutable and is mine or my agent's now what i would like to do is have arnold represent me through and through well i don't want to restrict arnold to saying you have certain access here certain automation here but something else on this use case something else on that use case i mean that defeats the purpose then then i'm the one ensuring oh i gave it this year and i gave it that here it's just too complex eventually we'll all get to a point where arnold will have all access all automation and will only come to me for a judgment call when it does not have that judgment call in a ledger somewhere that it can work off in a cog somewhere that can work off and so a very simple example here since you brought this personal use case i'm so fascinated by it because i do this all the time like you nathan so i have arnold lane my agent my wife has lyra belacqua she is this character from the golden compass and that's her agent and arnold and lyra need to collaborate on things as simple as let's get a dinner reservation two weeks out for an event that we are celebrating and today you just cannot do that autonomously you just cannot because of many reasons first and foremost the model is a blocker because the model itself is not capable enough to do this into it because we are all in the free tier fine the harness is the next blocker because the muse harness is constantly asking for permission rightfully so because it is being cautious when it's sending emails to each other i don't want to be part of the loop if i'm going to give you permission every single time i might as well just do it myself i can just talk to my wife i mean just why even go through this whole process right i want this entire thing done autonomously but the third thing is even if you enable this and i'm i'm experimenting so i'm going and saying yes go ahead send the email but as we do this and i'm going to drop skills next with accord and concord as we do this what we are realizing is that they are not agreeing because there are certain restrictions i have of where i want to go the kind of restaurants i want there's some restrictions that my wife has and where she wants to go and the kinds of restaurants she wants to go and the kind of menu items in each one of these places so even a simple thing of food restrictions or things you like the kinds of restaurants you like the locations you like the kind of ambience you like the availability of the restaurant for the for the time we want to book all of these things are variable and even a simple thing like that we know this even as humans it takes us a good five minutes ten minutes or more sometimes based on the complexity to book a simple restaurant for a good event and our agents cannot do this autonomously which is ridiculous i want us our agents to just go ahead know our profiles know our likes and dislikes talk to each other and compromise and negotiate because this discussion is a negotiation and a compromise in the end because it cannot just be what we join likes it cannot just be what my wife likes it needs to be a compromise and just do that and you cannot do that today so there is a massive problem even in a two-person situation on an agentic stack that has been built and deployed and rolled out globally today you cannot do this until we can get to this point we will not get to an agentic outcome for anything so these protocols these observability platforms these skills that you can deploy are valuable not just in the consumer space in a simple situation like this but then draw that out to that enterprise two million dollar three million dollar deal desk and think of what happens there and the outcomes that can go wrong in that kind of a situation and what we need to do there in the enterprise that's the world you and i want to go towards where we're going to be able to do this and we're going to be able to do this and we're going to be able to do this and we're going to be able to do this and we're going to be able to do this our agents can solve for the minutiae of booking a restaurant without us getting involved because man if they can't even represent us in the personalities that we have for a simple dining situation then what's the point i mean where are we going do we know anything about scaling laws for multi-agent systems today i've heard things like this about scaling laws for like the square root of the number of agents will give you like the speed up so in other words you know 10 000 agents can go 100 times faster um but it occurs to me that maybe we just have no idea what we're talking about there right now because like if you went back in time to gpt2 or 3 presumably like there was zero you know productive uh synergy between multiple agents and so maybe this is not something that is like a timeless scaling law but maybe something that's more like a per class of model scaling law but i'm not even sure if that is right any insights there that you can uh offer to deconfuse me very quickly i mean computing has always scaled across three axes and we touched upon this in the beginning a little bit the three axes being algorithmic scale vertical scale and horizontal scale and that is where we come in company like cisco comes in and says yes keep doing those two keep making software better keep making the node more capable and bigger and better but then there are connected systems and we do build connected systems but these connected systems and the nodes that the system is connecting are changing over time so very quickly what we are seeing is that the network itself has connected billions of endpoints in fact tens of billions of endpoints when it comes to machines and billions of endpoints when it comes to humans the network has connected them over the past 40 years to build connected systems through connecting data that's what the network has done what we are realizing now is those endpoints are actually changing foundationally those endpoints are starting to think they're starting to entangle they're starting to fluctuate and they're starting to move they're dramatically different forms of computing and we've touched upon only one so far and that that is a think part of it these endpoints are no longer data endpoints these endpoints are intelligence endpoints they are thinking they are semantically collaborating they're exchanging meaning they're actually behaving as one cognitive environment they are reasoning collectively and they're actually behaving as one cognitive environment and they're actually so that is the thinking part that's not standard data exchanges that is exchanging meaning so that's one endpoint category but then the others they're starting to entangle that's quantum computing they're starting to fluctuate this is new forms of computing using classical physics like thermodynamics like oscillations and then they're starting to move which is embodied ai and physical we can take all of these innovations and put it embody it in a physical environment where they can actually move and sense and act on physical objects so all of these endpoints are different forms of computing and that is the fascinating part because the network that is going to connect all of this and enable a connected system is going to look very very different from the network that has connected data in the past that brings to mind last time we talked you had i think a very interesting and at least in my circles kind of heterodox position on what is sometimes called neuralese or the transfer of high dimensional internal states from one ai system to another this on the one hand could be extremely powerful obviously there has also been sort of a you know let's say maybe an immune reaction among ai researchers over the last few months in wake of things like the open face incident where people are saying geez like if we struggle to monitor them this much even when we can read the chain of thought in tokens we better be really careful about allowing them to communicate in this higher dimensional language that is native to them but like very much not native and you know it presents a whole other interpretation problem for us do you have any updated thoughts on that are you optimistic that we can just kind of solve and effectively monitor neuralese would you say you're like a little more cautious about it than in the past or what's what's kind of the latest dj take on neuralese i'm super excited about that and to me it's inevitable it's inevitable that we the arc of technology will move towards a situation where you will do things more effectively more efficiently and you will build infrastructure in such a way that you get the most out of the technology stack that you're building well in the end the goal is to solve for problems that have been hard problems to solve for humanity in the end we are all moving towards building technology to solve for problems like drug discovery to solve for problems like new materials to solve problems like logistics plans to solve for new materials to solve for new materials to solve and financial modeling and weather modeling these are really hard problems and these are problems that have long-term impact on humanity on civilizations and that should be our goal and we should do everything possible to build the best technology stack to then solve for those really hard problems for humanity and if it means building these latent space protocols where agents can talk to each other not in natural language but with higher dimensionality let's build those things i am a big believer in accelerating technology because to me it also opens up the space to then build this in a guardrail manner we've talked about gats and cogs one of the big reasons we are going after gats the guardrail technologies is to ensure that everything else that we are building is working with the laws and regulations and the compliance needs and the security needs and the scaling needs that could be deployed in the environment you're working in so if you're asking me yes i'm still a big believer in that within agents is where i think we should deploy them and those agents could be in software those agents could be embodied within robots and physical ai now when you talk about bringing in the robots and physical ai you're talking about bringing in humans into this picture and the neural link kind of approaches the guardrail of it in me takes center stage where yes we should do this but we should do this in the guardrail way that we do everything else we should do the right amount of testing we should do the right amount of deployment we should deploy it in a canary way so that we understand it in the same way we introduce a new pharmaceutical drug in the environment so make sure that those steps are met you don't shortcut any of those steps but to me that is the exciting part and it's almost in some ways inevitable but do it carefully the same way you would introduce any other drug in this environment i just saw an interesting lecture from you've all know harari where he makes this analogy between the state of the cow in today's world and the state of the human in the future and this is you know in to be clear like a worry that he has but he basically says the cow doesn't know anything about money or the financial system and yet the cow's life is governed by the financial system and all the incentives that the people raising it have you know and you can try to explain it to the cow but it's not going to get it and then of course the you know the implication of that is like we are kind of setting ourselves up to be in a similar position where we're creating all these ai systems that are going going to increasingly run the world and you know they'll do that largely for our convenience and benefit but at some point we may end up in a spot where because they're so much faster than us they're processing so much more information they're potentially communicating in these like high dimensional ways that are not native for us we may end up in the same position as the cow where we are sort of governed by superstructures that are operated by ais and we really can't grok those things anymore do you think that's like okay do you think we can like get around that with neural link type uh connections where we will be able to natively understand high dimensional uh representations of these new superstructures or maybe we can just like avoid that through guardrails like how what is your sort of positive vision for where this goes and and how much control we ultimately should should see to like ai powered superstructures this brings us back to where we started nathan i just saw that cow example as well recently and to me i personally don't understand much of how high frequency trading is done today i know the concepts i know what is being done i know there are algorithms running these trades but i'm not in the business i'm not familiar with that business and how those things happen but software is running trading today and i might be the cow even today whereas that system is pretty well understood by the folks that are running it and writing it and operating it and i have designed it right so there is that situation even today i would say the same thing about the laws and regulations that exist in our state or within the country i understand it superficially but i don't understand the details behind it today but the people who bring in new laws they understand the system really well so what i would say if to that analogy is yes we are already cows or we are already humans in the system that exists today that's why subject matter experts exist that's why specialists exist that's why agents are going to be also specialists that's why people who are building those harnesses and those customizations in models within those verticals for those specialist agents will understand those systems i will understand an infrastructure stack agent really really well so you you need those specialist agents the people who built them will understand those agents will understand those harnesses and that is the world we're heading towards where every agent will be a cow and a human at the same time in a multi-agent system but that multi-agent system will need to collaborate to then drive a global outcome and the way you would monitor that is through systems like scope where you're looking transparently at everything that is going on putting that into an observability system so that you have a legend a system of record for what that multi-agent system is doing every action it's taking every decision that is looking at from other agents every tool call that it's making all of that is recorded somewhere so that you have compliance requirements checkbox but also you can debug if something goes wrong and then when that system itself see things sees things going off track it can trigger not just the protocols like accord or concord of others but also gats and cogs to ensure that those cows and humans those individuals such as subject experts are back on track to solve for the global outcome that you and i are looking at well i admire your comfort in uh taking the cow like position today i'm definitely feeling a little more cow like myself uh as things progress and that may continue as i feed the transcript of this conversation into my agents and ask them to adopt this conversation into my agents and ask them to adopt the scope and accord and concord protocols and open source projects that you guys have been building and releasing that'll both take my system to a higher level and potentially leave me even more marginally in the dust but that's i guess just a sign of the times that we're living in and if nothing else it is definitely an interesting and fascinating time to be alive this has been an excellent conversation i know you got to go but i really appreciate it vijay pandey thank you for being part of the cognitive revolution thank you so much nathan this was amazing everything's lovely says the lodger says he everything's lovely and he's kept the key came to stay in april with a suitcase full of moans he wears my sunday suit to town and he whistles all my tune everything's lovely says the lodger says he everything's lovely says the lodger says he and he's kept the key i asked him for a supper just a table laid i asked him for a supper just a table laid i asked him for a supper just a table laid for two he said it for a hundred on the lawn he said it for a hundred on the lawn he said it for a hundred on the lawn beneath the you late at night through the wall he talks to the lodger next door not in words not at all in colors never seen before everything's lovely says the lodger says he everything's lovely says the lodger says he everything's lovely says the lodger says he and he's kept the key so last night they came and called me last night they came and called me last night they came and called me through the plastic through the plastic through the plastic through the plastic through the plastic now i'm talking in colors and i'll never now i'm talking in colors and i'll never now i'm talking in colors and i'll never let you know everything's lovely say the lodgers and me everything's lovely and we've all got a key and we've all got a key and we've all got a key and we've all got a key and we've all got a key if you're finding value in the show if you're finding value in the show if you're finding value in the show we'd appreciate it if you take a moment we'd appreciate it if you take a moment we'd appreciate it if you take a moment to share with friends to share with friends to share with friends post online write a review on apple post online write a review on apple post online write a review on apple podcasts or spotify or just leave us a podcasts or spotify or just leave us a podcasts or spotify or just leave us a comment on youtube comment on youtube comment on youtube of course we always welcome your of course we always welcome your of course we always welcome your feedback guest and topic suggestions feedback guest and topic suggestions feedback guest and topic suggestions and sponsorship inquiries either via and sponsorship inquiries either via and sponsorship inquiries either via our website cognitiverevolution.ai or by our website cognitiverevolution.ai or by our website cognitiverevolution.ai or by dming me on your favorite social network dming me on your favorite social network dming me on your favorite social network the cognitive revolution is part of the the cognitive revolution is part of the the cognitive revolution is part of the turpentine network a network of Podcasts, which is now part of A16Z, where experts talk technology, business, economics, geopolitics, culture, and more. We're produced by AI Podcasting. If you're looking for podcast production help for everything from the moment you stop recording to the moment your audience starts listening, check them out and see my endorsement at AIpodcast.ing. And thank you to everyone who listens for being part of the cognitive revolution.

Podcast Summary

Key Points:

  1. Multi-agent systems have transitioned from isolated agents to interconnected, collaborative networks solving complex enterprise and consumer problems.
  2. Cisco and partners have deployed real-world multi-agent systems in areas like telecom infrastructure and deal management, demonstrating practical value.
  3. Semantic misalignment and cognitive drift between agents—such as misunderstandings over goals or terms—cause significant performance degradation and silent failures.
  4. Cisco has open-sourced a cognitive observability platform (Scope), alignment protocols (Accord and Concord), to detect, prevent, and correct these failures in real time.
  5. Accord enables agents to align on shared intent and definitions before work begins, while Concord supports negotiation and coordination during execution.
  6. The shift toward shared cognition—where agents reason collectively—is critical, as connectivity alone does not guarantee effective collaboration.
  7. Decoupling of models from harnesses and tools allows flexibility, interoperability, and avoids vendor lock-in across diverse environments.
  8. Human judgment remains essential in current systems, with future potential for AI-based “cognition engines” to automate and accelerate such decisions.

Summary:

Multi-agent systems have moved from theoretical concepts to active deployment in enterprise environments, solving complex problems like telecom network migrations and high-stakes business deals. Cisco, through partnerships with Swisscom, ServiceNow, and Microsoft, has demonstrated that interconnected agents can achieve 100% error-free configurations or successful deal closures. However, these systems face critical challenges: agents often misalign on goals, terms, or outcomes, leading to semantic failures and silent failures where the system appears functional but delivers no actual results.

To address this, Cisco has open-sourced a suite of tools—Scope for cognitive observability, Accord for pre-work intent alignment, and Concord for real-time negotiation—built on open standards like OpenTelemetry. These tools enable agents to share intent, coordinate during work, and verify outcomes, moving beyond mere connectivity to true shared cognition. The systems are designed to work across diverse models, harnesses, and vendors, promoting interoperability and avoiding lock-in.

While human oversight remains vital for judgment and final decision-making, future systems will increasingly leverage AI-powered "cognition engines" to automate and accelerate these processes. The evolution reflects a broader shift from isolated intelligence to collective, reasoning-based intelligence—mirroring how human collaboration advanced through language and shared knowledge. This development marks a pivotal moment in AI maturity, where the real value lies not just in individual agent performance, but in their ability to reason, align, and collaborate toward shared, measurable outcomes.

FAQs

A major challenge is semantic misalignment and cognitive drift, where agents fail to understand shared goals or negotiate effectively. This is being addressed with open-source protocols like Accord and Concord, which enable intent alignment and coordination during work.

Scope provides cognitive observability by detecting semantic and cognitive misalignments in real time, acting as a 'lane correction' system to identify when agents are drifting from their shared goal and triggering corrective actions.

Accord aligns agents on shared intent and key terms before work begins, ensuring mutual understanding. Concord enables coordination and negotiation during the work process to resolve conflicting priorities and adapt to changing requirements.

Connectivity alone does not ensure cognition. Agents may work independently and pursue local optima. True collective intelligence requires shared reasoning, semantic alignment, and coordination—something the 'Internet of Cognition' infrastructure aims to enable.

Yes, but only with proper alignment and governance. The interoperability stack (like Scope, Accord, Concord) allows heterogeneous agents from different vendors to collaborate by standardizing intent, communication, and outcome verification.

It means separating the AI model from the execution environment (harness), allowing users to choose different models and tools independently. This enhances flexibility, avoids vendor lock-in, and supports diverse use cases.

Chat with AI

Loading...

Pro features

Go deeper with this episode

Unlock creator-grade tools that turn any transcript into show notes and subtitle files.