Hello, welcome to the AI Engineering Podcast, your guide to the fast-moving world of building
scalable and maintainable AI systems.
When ML teams try to run complex workflows through traditional orchestration tools they
hit walls.
Cashapp discovered this with their fraud detection models.
When they needed flexible compute, isolated environments and seamless data exchange between
workflows, but their existing tools couldn't deliver.
That's why Cashapp relies on Prefect.
Now their ML workflows run on whatever infrastructure each model needs across Google Cloud, AWS,
and Databricks.
Custom packages stay isolated, model outputs flow seamlessly between workflows, companies
like Woop and OnePassword also trust Prefect for their critical workflows, but Prefect didn't
stop there.
They just launched fast MCP, production ready infrastructure for AI tools.
You get Prefect's orchestration plus instant OAuth serverless scaling and blazing fast
Python execution.
Deploy your AI tools once, connect to cloud, cursor, or any MCP client, no more building
off-flows or managing servers.
Prefect orchestrates your ML pipeline, fast MCP handles your AI tool infrastructure.
See what Prefect and Fast MCP can do for your AI workflows at www.aiengineeringpodcast.com/channel
Prefect today.
Unlock the full potential of your AI workloads with a seamless and composable data infrastructure.
Brewin is an open-source framework that streamlines integration from the command line, allowing
you to focus on what matters most, building intelligent systems.
Write Python code for your business logic and let Brewin handle the heavy lifting of
data movement, lineage tracking, data quality monitoring, and governance enforcement.
With native support for ML and AI workloads, Brewin empowers data teams to deliver faster,
more reliable, and scalable AI solutions.
Harness Bruins connectors for hundreds of platforms, including popular machine learning frameworks
like TensorFlow and PyTorch.
Build end-to-end AI workflows that integrate seamlessly with your existing tech stack.
Join the ranks of forward-thinking organizations that are revolutionizing their data engineering
with Brewin.
Get started today at AIengineeringpodcast.com/brewin and for DBT cloud customers, enjoy a $1,000
credit to migrate to Brewin Cloud.
Your host is Tobias Macy, and today I'm interviewing Craig McLucky about improving the security
of your AI agents through curated and optimized MCP deployments.
So Craig, can you start by introducing yourself?
Yeah, hey, thanks for having me on.
I'm Craig, I'm the founder and CEO of Stacklock.
Previously, my background is entirely in the distributed systems and enterprise software
domain.
I worked at Google where I was responsible for bootstrapping and shipping Google Compute
Engine. I then had the opportunity to look beyond and came up with the idea for Kubernetes
with a couple of other co-founders of the project.
That obviously worked out pretty well.
I started the Cloud Native Computing Foundation as a spiritual home for Kubernetes and other
open source projects.
And then I'm currently in my second company.
So I worked a lot in the kind of relatively traditional distributed system space.
And at Stacklock, we became really interested in AI as a disruptive moment, sort of this
epoch boundary.
And so we started really focusing on what we knew, which is the integration and the interfaces
between existing systems and these new assistive and agentic technologies.
And it's great to be on your show.
And do you remember, hey, first got started working in the ML and AI space?
Yeah, I mean, it's interesting.
And for me, it was the sort of journey of fascination and increasing kind of interest.
Right?
I mean, I think we all had this moment when Chachi PT was released and was like, wow, this
is just a wholly different class of system, it was fascinating and sort of playing with
it as a curiosity.
And I think it was an AI denialist for quite some time, right?
I think a lot of people probably were, where I think we sort of, a lot of folks, certainly
me, I'm an old kind of school distributed systems guy.
I imagine that it was going to be asymptotically convergent on a set of capabilities that would
make it interesting in a curiosity in certain domains.
And then we'd start to see it emerge as this kind of new class of system that you could
use to augment what you were already building.
And I think, as my own use of the technology started to emerge as just, you know, in day
to day use, initially, we're on just research-based functions, but then increasingly as I wanted
to do things that I didn't want to bother my engineers with, that it would enable me
to actually start running code, it became clear that I was deeply undervaluing the impact
that this really was this, you know, kind of epoch-defining moment.
I think every epoch of humanity is kind of defined by its dominant technology, right?
The Stone Age, the Iron Age, the digital age.
And I do think, you know, it became clear to me that we were entering into the space.
And for me, it was interesting because, you know, we started this company around a certain
supposition, this idea that 80% of the world's software, you know, historically has been
written by random people on the internet, you know, open-source libraries are this staple.
And I started stack-lock as a way to start, you know, just helping build this bridge between,
you know, the way that people are consuming software in the communities of producing it.
And I think the journey for me as a sort of AI-maximalist and building an AI-maximist
company really started when I have asked myself, are my assumptions for this company still
going to be true in five years, right?
Like, and are they going to be true in two ways?
Are they going to be true in terms of are people still building software the same way,
right?
Like, are they still using the same SDLC that they've always used or is the role of the
developer going to change?
And then the second piece was, you know, is the open-source ecosystem that powers this
going to be consumed via package managers, or is it increasingly going to be consumed by
a lot of that knowledge being parameterized and then being used to actually render up entirely
optimal expressions of what you want?
And it was clear that the disruption that this represented is profound.
And I think a lot of people that are, you know, like I was pretty involved in what we
call the kind of cloud-native disruption, like the sort of transition from, you know,
building, you know, relatively simplest, you know, relatively complex, but, you know,
simply describe vertically integrated apps to, you know, kind of using things like Kubernetes.
And this felt every bit as, you know, significant, in fact, far more significant than that.
And so that moment of realization really drove me to start looking at where to create value
in this ecosystem.
And I would say that that that that moment was, you know, defined by my own experiences
with AI, but then also this recognition of the, just the profundity of disruption that
represented.
And one of the standout aspects of that knowledge curation and knowledge delivery aspect
of the, I guess we can call it, "agentic revolution" for lack of a better term right
now is the model context protocol.
And that has become broadly implemented.
And in some cases, broadly adopted, the adoption is maybe a bit unequal, and I'm wondering
if you could just talk to some of the ways that you're seeing that as an exemplar of this
new way of thinking about system design, application delivery, and the ramifications that
it has on the development and delivery of software.
Yeah, I think, you know, kind of MCP is interesting.
And, you know, the best analog I can come up with, you know, like I always think back to
my own lived experiences.
And I remember the first time I saw Docker, which was, you know, I don't know how familiar
this audience is with Docker, but it was a technology that enabled you to kind of package
up, you know, an application into a reusable entity and ship it and run it in any environment.
And it was this kind of moment, right, like, wow, we now have the ability to unlock portability
and we've created this lightning in a bottle moment, which redefines the developer experience,
you know, it just changed the way that people worked.
And I think MCP is every, it's very similar to that, right, like it's, you know, the beauty
of what was produced was this incredibly simple specification that described a way whereby
you could translate, you know, this ocean of API and data and, you know, everything that
existed in the world's distributed systems into a natural language format, that was sort
of optimally consumable for models. It was just this very, very simple and elegant way
to start describing it. And so, you know, as I've looked at this, you know, obviously the
protocol is constantly evolving. But I think if you kind of cut the chase, like what does
this actually mean in its most fundamental terms is this is a gateway to building AI native
apps and AI native experiences, right? Like, you know, we've looked at a lot, you know,
I speak to a lot of enterprise organizations and, you know, a lot of folks are working
on this idea of like, Hey, can we both, we really do two things at the same time, right?
Like people are trying to do two things concurrently. One is, can we use AI to enable us to go faster?
Can we enable AI to let us, you know, do the work that we're doing today, you know, in
a sort of, in a much better space? And then the second thing they're trying to do is, can
we build out these competencies that allow us to be future proof, right? That allow us
to be able to start, you know, kind of delivering, you know, outcomes of time and our computer
competition. And it's interesting to see the tension there because, you know, the first
thing often says, like, Hey, just go bi-verically integrated AI application. And then the second
thing says, Hey, you actually have to start reusing about this from a platform-builder's
perspective and think about what you need to do. And those two, those two things are
intention. And I think MCP is, is this beautiful technology that enables you to start spanning
that. And what it leads to is the ability to build AI native applications. Like, a lot
of these applications that are being built today that are sort of, you know, wildly successful
and be.
being consumed at a furious rate, because they bring value engineering.
They actually solved the problem of rendering context in a way that models can understand
to create business value.
Feel a little bit like the Hossess Carriages.
I'm not the first person to say this.
I think VCs have been saying this for a while, but when the automotive industry started,
when the internal combustion engine emerged, people just put that onto a carriage.
It was a Hossess Carriage, right?
It was not a modern automobile value stretch of the imagination.
And I think AI native apps mean that people are interacting with a model in a multimodal
form factor.
And right now, when we say multimodality, people often think, "Hey, it's voice, it's
image, it's video, it's whatever."
But multimodality could also be rendered UI, like there's some things when you want to
be able to see a calendar.
You don't want something in plain text describing a calendar.
You just want to see a calendar, right?
Maybe you want to click the calendar, maybe you want to move things around.
And so, as we look to where app development and app architectures are going, the model
becomes the presentation and view model for modern apps, which then raises the question
like, "Well, if you're building an application today and you're building a web application,
the API is the interface between your presentation layer and the kind of backend systems where
you kind of organize logic and you do all of that."
And so, I really think that MCP is the API for AI native applications.
And it's going to herald a whole new class of experience where you no longer have to
have a bunch of vertically integrated apps where you constantly contact shifting between
them.
And it will offer up two things.
One is the ability to establish and share context if you do need different experiences.
But more importantly, to be able to start to establish the model as you're working area.
This is the place where you do your work.
This thing's up in a variety of different form factors.
You could call that the browser for kind of AI-enabled applications.
And it means that you will see developers starting to think not about how do I actually create
this kind of vertically integrated story.
It's really, how do I start to think about organizing workflows?
What are the nouns and verbs associated with the things that I need to do?
How do I express those in a way that something that's already powered on natural language
and increasingly some of these other concepts that will start to show up in the protocol
can consume it in a meaningful way and create a great experience.
And then oh, by the way, I also have to do all of the details around authentication,
authorization, observability, and all the other things that come with building applications.
But I think that is the most fundamental transition that's going to happen.
And in that context of MCP, there are a variety of different frameworks and implementations
around it.
The protocol itself has gone through a few revisions most notably going from not having
any concept of authentication or authorization to it actually now being well integrated into
the specification.
And I'm just curious if you can talk to some of the ways that you're seeing people approach
the overall exploration and adoption of MCP and then going from, hey, this seems like
a useful little tool to, oh, this is actually going to be a substantial benefit to me, my
team, my organization.
And now I need to actually get it into production and just some of the, I guess, misconceptions
or stumbling blocks that folks run into along that evaluation path.
Yeah, I mean, there's a lot of ground to cover there, but I would say, the starting point
for a lot of people is holy crow, this thing demos really well, right?
Like there's an upstream, like the way I see a lot of folks, maybe I can just describe
the canonical journey for a lot of folks, right?
The starting point's often like, hey, I'm using cursor and I want to integrate something
else.
There's probably a tool for that, right?
And there's this sort of incredible burgeoning numbers of tools that can be used.
And the starting point for a lot of people was like, hey, I'm using an assistant technology,
I want to integrate it to something else.
I can just run this thing locally.
I can just go fetch this thing from a package repo up there.
It's effectively allow cursor, copilot, whatever to endpx run this package.
And then I just provide that package a set of credentials because now authors are part
of the flow.
And that package that I'm running is now able to access a lot of these services and it
will do the mapping for me.
And that's great, right?
Like it's really exciting.
It's also absolutely terrifying.
Right?
If you actually think about what, like just take a moment to think about that, right?
Like, don't get me wrong.
I'm a huge proponent of this technology.
And I think that there's so much potential with it.
But I think it's also important to recognize this one truth, right?
I can't remember the exact statistics, but there are, I think at the moment, something
like many, many thousand malicious NCP packages out there.
And you're basically kind of giving a lot of your kind of, you know, like we're giving
up a lot of common sense in terms of how we start to run these things.
For real world enterprise organizations that are assigned to look to deploy these technologies,
the sad truth is that the rate of MCP-based exploits is growing at a pace that is outpacing
AI spent, right?
And almost 20% of organizations that have embraced MCP have had some kind of, of exploit
from it, right?
So, starting with this idea that yes, you can NPX run something, people do that.
And it creates a tremendous amount of value, but it also is exposing you to a lot of risks.
So people will be like, okay, let's not do that.
So the next logical point is, hey, this is great.
We need some kind of hard and surface so that we know this thing can be trusted.
So the next consumption pattern tends to be what I think of as a SaaS-based integrations,
right?
We've increasingly seen a lot of this where, hey, you have a SaaS-based system that you
want to use, and now I need to be able to connect that SaaS-based system to this thing.
And hey, conveniently, a lot of them are now starting to expose MCP-based endpoints.
They're actually doing the work on the SaaS side to start reasoning about how to transform
their API into these basic primitives that can be operationalized.
And that addresses some of the problems, right?
It certainly solves for the problem of the malicious MCP package and getting hurt and
having to reason about it.
And what it does do is it starts to give you a very nicely normalized set of interfaces
that you can do.
And you can get a certain kind of path down the journey.
The thing that I see a lot of organizations and developers stumbling with is like, okay,
well, that's great.
But how do I start to expose my data, right?
That's that transition from, yeah, there's all these kind of SaaS integrations that are
available, they're consumable.
How do I expose my data specifically?
And how do I start mixing my data into the tool set that people are using?
And that's kind of problem one.
And I think problem two is what people are starting to realize is, tool pollution is a
real thing, right?
Like it's, you know, context one does a finite input tokens cost money.
If you go and turn on one of these servers, you are going to burn a lot of tokens, right?
Like the GitHub server, we sort of saw this evolution where it kind of grew to about 110
tools.
And then GitHub really started thinking like hard and seriously about how to kind of optimize
the set of tool interfaces, you know, for the specific workflows that developers bring.
So the tool kind of came down.
You know, we've seen, we've set some division, but you're still probably adding something
like 10,000 input tokens by just turning on that server.
And so, you know, once you start to get to a point where you have a lot of tools, the
selectivity becomes a problem, right?
Like it's like, hey, I have to actually start manually turning them on and off after
reason about when I want them.
And so that problem around, you know, kind of adding, you know, semantic search and sort
of semantic awareness.
And we're seeing, you know, some, some systems start to implement this like, you know, anthropic
recently added their own semantic search, you know, search to tools so you could start
to aggregate them and not have to, you know, have all of those input tokens in every, in
every window.
But yeah, like the next point that people start to hit on that journey is, how do I expose
my data and then how do I actually make this halfway sustainable?
So not just, you know, burning a tremendous number of tokens.
And then, you know, I think the sort of step that happens after that and I'm happy to
provide an example of this is how do I start mapping this to my actual workflow?
Like what are the nouns and verbs that I care about, right?
Like, you know, like, I'll give you an example, a recruiter, like, you know, I manage a recruiter.
And he lives in, you know, basically four tools.
I mean, there's a couple others he uses, but for simplicity sake, we just describe it as
four tools.
He uses the Google productivity suite.
So Gmail and docs, he uses greenhouse for candidate management, he uses LinkedIn and he
uses a calendar to schedule interviews.
Now he can get to a certain point by just integrating those systems into his, his work, right?
So he can, you know, integrate LinkedIn and integrate this and integrate that and integrate
these, these pieces.
But there's still some amount of contact shifting.
Like, candidate does not mean anything in a normalized way in that system, right?
Job description is a relatively open ended thing.
But by looking at his workflow and starting to describe the things that he does, like,
hey, I need to source a candidate, you know, what does that mean?
You know, what, like, you know, what is our process here?
How do I start to describe that as part of the tool itself?
How do I start moving from, you know, like, I want to write a job description?
Well, that's great.
There's already a template.
But do I have to go and describe, you use the template to do this?
Or can I start to describe, write a job description as a tool that basically, you know, like, here's
the base, you know, here's the base thing, you know, here's the transcript of the interviews
that I'm using.
Let's kind of, you know, start to kind of organize the structure of that around the work
that the person's actually doing.
And so I think of that as the sort of that sort of the logical destination where we need
to get to where it's not just about a set of, you know, kind of, you know, villainous interfaces
where you're putting a lot of cognitive load on the model.
And, you know, all of the payload that's being kind of expressed there is is relatively
abstract.
It's relatively generic, but actually starting to become much more precise through the use
of tune tool descriptions, through the units of specific tool based workflows to actually
start fitting the work to the person that's driving it.
From that point of the selection and evaluation of the twins.
tools that are exposed in these various servers using GitHub as an example.
It can be very manual to understand which tool does the thing that you actually care about
for a given workflow and to your point.
I don't want to have to deal with toggling them on and off every time I want to do something
different.
And I'm curious how you are in particular in the project that you're building with
ToolHive thinking about that workflow of being able to define the tasks that you want to
do, map that to the tool selection within these various servers.
And I know that in ToolHive, you for instance, have a way of building a composite server that
exposes all the tools for multiple backing servers and just how that tool selection and
context optimization workflow looks for somebody who is starting to figure out what are the
things that I want to be able to do and what are the pieces of information that I need
for those different workflows.
I mean, I tend to take a somewhat, I don't know, I'm sure the audience has a lot of folks
have read the paper, the better lesson, right?
And I take a somewhat reductionist view on what really works out there right now.
And the answer to what really works well right now is frontier models plus great tools
create great results, right?
But you have to start really thinking about what a great tool is.
And so I think there's a number of different steps in this regard.
Tool selection is hard, right?
And there is an art to describing a tool so that is actually invoked in a specific context.
You know, there are sort of things you can do like right out of the gate.
So you know, for instance, what we started with ToolHive right out of the gate was just
adding semantic search into the call path, right?
So you know, and we did a bunch of analysis around this.
So we basically picked a bunch of the most popular tools out there.
We described, you know, we generated a big baseline of synthetic kind of data.
And then we just started running evils, you know, against, you know, query should map
to this tool, you know, like is, you know, are the various models being effective in tool
selection?
And it was kind of interesting because, you know, in, you know, when you look at the
corpus of servers that are published, these are just generic servers out there.
We see frontier models like the best of the best right now, you know, based on our
statistics, achieving about a 93 to 94% selection rate.
So 93% of the time when, you know, you have a query which maps to a tool, those models
will select the tool.
Interestingly enough, if you're in the business of, of, of using smaller models, like less
heroic models that, you know, because inferencing cost is expensive or you really care about
the latency of these queries, if you, if you try to do something in more real time.
The performance of the systems degrades very quickly, right?
So, you know, when you move from a frontier model to like a, sort of, you know, a cheap
and cheerful SLM, you know, tool selectivity falls off to something like 34%, you know, based
on, on our heuristics.
And so as a starting point, just putting a semantic search capability in front of these.
So instead of basically saying, here's a flat set of tools, pick one, saying fine tool
and basically being very careful and spending a lot of time, you know, building a very accurate
tool description, which is, this is the endpoint that you call every time to got it out, you
know, whatever the, whatever the, the, the sort of description of the tool that generated
models hitting this point.
And then, you know, delivering top K, you know, like the top six tools, you know, allowed
us to achieve very high rates of tool selectivity across every model class.
So we were able to get to a point where a much more modest model could achieve the same
level of tool selection for giving tasks.
But even in that case, you're still sitting at like, you know, at best case, you know,
about 93% of the time, the actual tool will be, will be mapped to the, to the task that
you're handling.
And so this is where Eval's come in, right, like this is one of the most, I think, significant
points about operating with these class of systems, right?
Like when you go back to the world of old school distributed systems, proof of correctness
was the unit test.
And the unit test, you know, you get the thing to run.
And if you have enough test coverage, you can be pretty confident that you built the thing
that works.
These are stochastic systems, like you're dealing with probability intervals and boundaries,
right?
And so you have to, you have to start with an Eval, you have to start with an Eval framework.
So if you're, you know, if you're a sort of generous to user, just using these for convenience
sakes, like, yeah, 93% tool selectively, that's fine.
Like, you know, I just redo an occasion, it falls down, it irritates me, I burn a few tokens,
but no worries.
But if you're serious, and you're trying to build an agentic system that is performing
a task, you have to see the whole loop, and it has to start with Eval's.
And so, you know, one of the things that we always encourage people to do is like, hey,
you want to start building tools, like what does your Eval framework, like have you got
that set up, like what are the set of queries that should actually result in just starting
with tool selection?
Look like.
And then, you know, can you start to set up an evaluation cycle so you can start to tune
your tool description to match the task so that the tool is selected.
And then once the tool is selected, can you start to tune the payload so that the task
is completed, right?
So you have to start, you know, doing this, this kind of deconstructions way of looking
at it.
So the second thing that we sort of, you know, tend to recommend when people are walking
on this journey is, you know, run an Eval framework.
We're actually working on one more ship at momentarily so that people have a nicely integrated
Eval system as part of the tool-hive platform.
And then, you know, the question becomes one of, like, how do you start to actually structure
workflows, right?
You know, you both want to do two things.
You want to kind of optimize the context, and I think of it as context real estate management.
You want to load that context window up with, you know, precisely the right information
so that it can start to perform well.
And it's interesting because that's not always obvious, right?
So like, let me give you an example, like, if you're working in the sort of geospatial
space, if you're building a GIS-enabled application, and you say feature, that means something
very specific, right?
It's a collection of vectors that describe an entity that actually exists in physical
reality on the terrain.
If you're a developer and you say feature, you mean, hey, it's an atom of work that I'm
using on, that I'm using, right?
So there has to be this kind of contextual awareness where the tools that you're building
have enough additional metadata to start describing the thing that they're presenting,
so that it's actually, you know, when you're influencing against that thing, it's actually
generating the right results.
And so that process of building tools, you know, becomes relatively complex.
And I think there's really kind of three things.
Like one is, you're getting the right nouns and verbs and optimally describing them.
Two, it's starting to reason about workflow, right?
Like it makes sense to start to think about, you know, workflow associated with something,
and being able to, you know, represent an orchestrated set of actions that generates and
optimally, you know, rendered view, because, you know, by doing that in a sort of one-shot
with structured workflow, it's avoiding a lot of round trips.
So you don't get that kind of cascading entropy that you get, you know, when you're running
a lot of sequential operations.
And then the third piece of it is transactionality, right?
Like, you know, as you're starting to move from, hey, I'm just trying to build MCP servers
that reason about, you know, getting context into the context when it's, I can answer questions,
to actually performing work.
Sometimes that work needs to be committed in a sort of transactional way.
So you need to start thinking about like, hey, if I'm updating, you know, three things
across two databases, how do I create a shared transactional semantic so that, you know,
if the operation fails, everything is reverted.
So I'm not starting to try to debug these weird, you know, kind of orchestration failures
on the, on the backend.
And so I don't know if I'm getting too far ahead of myself here, but I think there's,
there's a lot of richness in going from, hey, I'm NPX installing off the internet and
it's connecting to GitHub, you know, using my credentials.
And I'm just going, yeah, yeah, yeah, yeah, yeah, yeah, yeah, I've proven already paying
attention to building a system which is unlocking like AI enabled application development with
these highly optimized and highly reasoned about, you know, views that are tuned to the specific
workflow that you're trying to accomplish.
And there are a couple of directions that I want to go from here.
I think one thing that you mentioned that is worth digging into a bit more is that concept
of evals and the relationship to distributed systems practices where you have unit tests.
But for cases where you wanted to be even more thorough in the validation and verification
of those systems, you would likely reach for something like TLA plus for design for describing.
This is the way that the system is supposed to work and running that through some mathematical
proofs of the ways that it should behave in that distributed systems context.
And then the idea of JEPSEN tests for being able to see, okay, but how does this thing
actually break when I put it into production and really put it under extreme load?
And are those behaviors when it does break acceptable for the use cases that I'm planning
for?
And I'm wondering how you think about the ways that those two approaches map to this
newer world of these orchestrated systems using MCP and context engineering and these probabilistic
workflows and just being able to understand how do we actually do something of that same
degree of rigor?
Yeah, it's an interesting question, you know, and I thought about this a little bit.
Like, look, don't get me wrong.
I'm not like super deep on formal method analysis, but here's the way that I kind of reason
about this.
We have this, like, and I think this is a comment, I find myself falling into this trap
all the time, right?
Like, I grew up in distributed systems.
The first distributed system I worked on was Windows, NT351 clustering.
It kind of dates me, right?
Like, I've been a distributed systems guy since time immemorial.
And one of the things I've learned is, you know, we have this pattern where we start
to kind of try to apply what we know, right?
So it's like, hey, you're dealing with a stochastic system, the stochastic system is generating
imperative code.
I can at least reason about that imperative code, right?
Like, so this is the sort of what I think of as the sort of normal sort of systems engineering
thinking.
And I think there's some this value and veracity to that.
Like, you know, certainly these tools give us the ability to start expressing behavior in
terms of formal mathematical language.
And it's like, it's like, you suddenly have this, this kind of, you know, like, Amazon's
been really good at this, right?
Like they've hired up this kick ass team of PhDs.
they've done a bunch of formal method analysis, you know, the stuff that was previously reserved for
folks that were building flight controllers for Boeing or control systems for nuclear reactors,
they've built out the muscle to start applying that to things like the S3 serving path, right?
And they find some very, very subtle and very nuanced and very interesting things,
and they certainly are looking at ways to kind of generalize that, you know, I don't know what they're
doing, but I have to imagine that they're looking at this because suddenly you have the system,
which has PhD, you know, they behave like a Stanford PhD, right? Like they can actually do
the work to start describing these things in a formal method way. And so there are going to be
classes of integrations where like, I think this is kind of, there's always something that's new
in info, like this idea of, hey, rather than using formal MCP, just describe your sort of APIs,
you know, through a sort of JavaScript based, you know, sort of format or decorate them with an
open API thing, allow the agent to vibe code something, run it in an isolated sandbox, if you really,
really care about it, you know, apply formal method analysis to it and reason about where the
breakage is. But I think that's missing the point. And the thing that I come back to is these
osteochastic systems, right? Like what you're doing when you're inferencing is effectively,
it's a probability field or like, I don't know, I'm not, I'm not, I'm sure this AI experts are
listening in that are kind of gnashing their teeth and being angry with me because I'm probably
using the wrong words. But, you know, you've got a system that's basically, you know, like,
with a frontier model, well, what you think about it is it's, it's an encapsulation of the world's
public domain information, right? Like 4% of, 4% of, of the world's knowledge is in the public domain.
And I'm reasonably sure that all of that is being used to generate these frontier models, right? But
96% of it isn't in the public domain. And so, you know, a lot of what you have to do is start to
craft context that gives you the highest probability of, you know, during inferencing, you know,
finding the logically connected set of tokens that have the lowest cosine distance from, you know,
what you want. Like, it's a, it's a stochastic, it's a stochastic process. And the problem with that
is that, you know, anytime something changes, anytime you add, you know, kind of entropy and that
could come through a lot of different ways, right? Like, you know, I had the system set up and I was
using three tools and now I'm using five tools and now I'm using 10 tools and now I'm so to put
pressure on the context window and so the self-attention mechanism started decay, you know, that's
something that's changed that I wasn't necessarily aware was going to change user patterns change.
People start to talk about things, you know, new, new slang emerges like there's a lot of different
sources of entropy that start to show up in these systems. And so it's not enough and all the
parameterized data changes, like you spend a whole bunch of time running evils and getting things
just working, new model comes along, it's got new parametric data, what was optimized is no longer
optimized. So I think there's definitely a veracity in saying, hey, you know, there's ways to start
introducing more structure. There's ways to start reasoning about these things so that when I really,
really care about it, I can start to create more formal boundaries around it by isolating it or I can
start to create more determinism by describing that. But those are imperative coding concepts and those
only apply to the imperative parts of the system. The rest of it is still that it's the thing that's
actually driving most of the value creation is a stochastic system and that is given to the vagaries
of entropy. And, you know, there's a lot of sources of entropy. So I don't think you can ever give up
on evils, right? Like I think these, you have to change the psychology of building and running
these systems. The things that like, you know, you can get it working, but it's not like a distributed
system where you can reasonably assume it's just going to continue to work. You have to get it
working, but then you have to have a continuous evil framework in place. So like to get it working
after an evils, to keep it working after an evils, and you have to have the ability to constantly tune
it because the behavior is going to change, right? Like this is sort of what is the SRE for AI systems,
right? And maybe SRE for AI systems is other AI systems, right? Like, you know, like I'm not saying
that this has to be a human function, but you have to have the watches. Like, you know, and something's
going to, you know, it's not enough to just deploy it. It won't continue to run if you don't
continue to watch it. And then another interesting development that still seems fairly niche and
hasn't gained a lot of adoption or even really proven out its overall capability is the idea of
these large action models, as opposed to the current transformer models, where the idea is that
rather than them being for conversational purposes, their goal is to be able to take instruction and
then make some sort of action on it. And I'm just wondering how you're seeing broadly the evolution
of these generative models, the new architectural paradigms, and the interfaces that those models
presume changing the types of tooling that we need to be thinking about and whether MCP will be
eventually obviated if there will be other interfaces that we need to be able to support for
maintaining context for different model architectures, etc. You know, it is interesting and it's
something that I have certainly given some, well, I'm not an expert on, you know, kind of the
evolution of model architectures. Like, I do take the better lesson pretty seriously, right? Like,
I think that at the end of the day, there's no amount of, you know, kind of, you know, like,
like, this is the danger, like, and I think a lot of enterprise are already struggling with this,
right? Like, it's, it feels kind of hopeless, right? Because you start a, you start a proof of
concept on something, you think you can create value that's going to differentiate you versus your
competition, you get to a certain point, and then you're just overwhelmed by the next frontier
model comes out as capabilities are disproportionately different, you know, things are, you know, things are,
things are moving so quickly. And, you know, I think we, we certainly haven't seen asymptotic
convergence around capabilities to system yet. So what I kind of really focus on, and I think this,
this kind of, I always describe this in two ways, right? Like, there's the Emerald City and
there's the Ellabric Road, right? And we don't know what the Emerald City is going to look like yet.
We haven't seen it. Like, you know, we can see a glimmer in the horizon. We know it's going to be
fantastic, because, you know, every, every signal, you know, points that way. So the question is, how do
we make progress in the meantime? And what are the set of things we can do? And like, I think of this
as a kind of future proofing exercise, right? I think there's these hard, immutable truths about,
you know, bringing systems into into into existing environments, right? Like, you know, one, like,
you know, one hard, immutable truth is that, you know, what is regulated will stay regulated,
right? Like, you know, like, you know, the, the, like, you know, there may be ways to help you deal
with regulatory oversight, but regulations tail technology by a pretty significant margin.
And that regulated landscape isn't going to change any time soon. So there's a set of things that
you just have to be able to do, right? Like, you just have to be able to authenticate, authorize
a certain policy, you know, generate observability, auditability. You have to be able to track provenance
of things like this. A bunch of sort of Uki enterprise things that need to be done. And I think that,
you know, as, as we see this disruption, like, the slowest moving part of the disrupted landscape
is the thing that touches your existing systems, right? Like, you go talk to a bank and you ask
them to describe their workflow. There's always a mainframe, like every, every time, right? Like,
soon, you just, if you haven't seen a mainframe yet, you just haven't asked them right questions.
Like, like, there's always a mainframe in this somewhere. Technology is fantastically slow moving,
because at the end of the day, you know, a company is this weird and meshed union of its people,
its processes and its technology. And you, you can't change just one of them, right? Like,
they will have to kind of change together. Like, it's, and they are, that is what is running
the world right now. Like, what's running the world? Isn't these models? It's, it's the existing
systems. Like, if I want to get money out of a bank account, I, you know, I'm not touching a model,
I'm touching these existing systems. And so, I think the safest place to, you know, innovate,
well, not so much innovator, like, invest, you know, is like, obviously, you know, have access to
these models, like, figure out what they can do. But at the end of the day, you have to have this
interface and it has to be controlled. And it has, you know, you have to be able to reason about
it, right? Like, it's, if you're an organization, like, I spent a lot of time talking to, you know,
grown up companies and, you know, like, hey, you know, we can't have these things npx and store
random shit off the internet. Maybe they'll eventually, you know, do something, but we can't
have them running unsupervised code that's touching production systems. Maybe one day we will,
maybe in five years we will. But that's not going to happen today. Like, but what we can do today is,
is, you know, put a boundary around it, you know, start to reason about the identity so that,
you know, the action of this thing is, is showing up in its own context. We can reason about,
like, auditability traceability. We can start to, you know, it's a certain, you know, kind of,
you know, controls around those, those pieces. And so, look, 96% of the world's information is
behind the firewall. You have to be able to access it and you have to have a pragmatic way. mcp might
be it. Maybe it's not, you know, maybe we'll come up with some new cool thing, you know, next year.
But, you know, so that's, that's thing one. Like, you just have to unlock that in a way that's
pragmatic. And I think investing in an mcp based platform is the best way to do that right now.
But the second thing is that sort of 96% plus the 4% of public domain knowledge only represents
0.001% of the world's data, right? Like, that's just knowledge. That's stuff that's being synthesized
into, you know, human consumable kind of tokens. The rest of it is, you know, time series data,
video streams, like, all these other things. Like, you're not going to want to influence on that.
You probably can't influence on that. It doesn't even make any sense. So you're all going to need
systems that can start to translate that ocean of information that is not knowledge into
something that you can actually start to feed into these systems. And so I do think that, you know,
starting the journey with mcp isn't highly logical because there's so much obvious value right now.
Yes, the state of the art will change. It will be different in a year. It will be different in
five years. But you will still need to authenticate, authorize, observe. You will still need to
search policy. You'll still need to do all these things. Building your muscle on that now is sensible
because whatever new technology comes along, you will probably be able to reuse a lot of that.
And if you make a platform investment in space, the platform will fall to keep up with it.
And digging now more into tool hive and that platform question, what are some of the core
capabilities and core ideals that you're focusing on with that as a means of building that
foundational layer for this context engineering and context exposure?
to allow teams to be able to have a reliable means
of being able to actually interface
with that broad corpus of information
that isn't already synthesized into knowledge.
- Yeah, so I mean, the way I think about it,
and this is something that comes up
in every conversation I have.
There's four constituent pieces that you need
when you want to basically render
a lot of the context engineering narrative.
We call it just an MCP platform,
but it's a context engineering platform,
it's a platform that enables people to start building
these next-gen AI-enabled apps.
Part one, secure runtime.
And when I say secure runtime, I mean, like,
hey, fetch server is great.
A lot of people just install the fetch server
and you can access them.
Do you really want one of these agents
having unfettered access to every resource
on your corporate internet
that isn't behind the north system?
Or do you want them to be able to access that
using the users' auth, like probably not?
So being able to just describe a runtime environment
into which you can deploy a server
with all of its associated tools,
and then start to reason about that is pretty important.
And so it always starts with a secure runtime.
The second piece of it is, you know,
this gateways are dime a dozen,
but you do need a gateway technology, right?
Like, the way I think about this is you need a gateway,
not only as a way to, like,
a lot of gateway technologies are built around this idea
of like, hey, run all your, like it's not an API gateway,
like just run all of your servers through this API gateway.
And you can now start to, you know, assert security
and you can deal with authentication authorization
and you can have some basic policy assertion.
I think that's the wrong path, right?
Like what you really want is an aggregation endpoint
that you can start to map and recontextualize.
So like, when this tool is calling this system
with this thing, like these are the tool descriptions
that map those, those kind of atomic units
that are going to produce the best work.
Here's an endpoint where I can start
to apply semantic search, right?
Like, you know, like, hey, I've been using a frontier model,
let's find a tool selection,
but it's killing me on interesting costs.
I want to kind of thunk down to a lower model,
but I don't want to have to rematch my tools.
I need semantic search.
So that gateway then becomes a single endpoint
with semantic search, the ability to start, you know,
driving a lot of the tuning and composition narrative
for specific workflows or specific users
or specific groups, you know, based on, you know,
some other configuration setting.
And then the third piece of it is orchestration.
You need to be able to start snapping these things together.
You know, pushing orchestration into the model
is very expensive and it creates
the possibility of entropy cascade,
where you're asking something that's sarcastic
to do a set of relatively precise sequential tasks.
You're either going to have a lot of redo,
which generates token wastage
because it's just trying and trying and trying
or you're going to get an unpredictable result.
So the ability to start describing structured orchestration
and then transactional behavior.
Like, hey, I want this thing to do this action,
but I really can't afford it to, you know,
like I'm a missionary refund.
I really can't afford it to, you know,
mark this thing as an issue to refund
and then, you know, do a financial disbursement.
Like, those two things have to happen concurrently
and they're two different systems, right?
Like, so you need that, you need that sort of capability.
So, you know, I think of it as going beyond a gateway
to kind of virtual MCP construct.
It's a way to start optimizing and rendering tools.
And there's a lot of other things you can do in that space.
Like, you know, we work, you know,
things like context compression, you know,
other ways to start sort of do context orientation.
There's a lot of other systems that belong in that,
in that layer.
And the third layer is the registry, right?
So, you know, this is, this is, you know,
we've been working very closely with option communities
particularly around the registry specification
and the registry implementation.
But, you know, as you start to have MCP as a,
as a protocol, being able to start to describe, you know,
these are the sort of service that available to the organization.
Some of them can be run locally in a certain environment.
Some of them can be run as a sort of heroic hosted system.
There's life cycle associated with that.
And some of them are proxy because, hey,
there's some great SaaS-based MCP service.
But I want them all aggregated
and I need to be able to describe that.
I need to be able to make sure that I let my organization
know what can and can't be consumed
and what's available right now.
So the registry is an important part of that.
And then the last piece of it is, you know,
like you have to generate a console, right?
Like, you know, hey, I'm a developer, I want these servers.
Like, where do I go to actually start, you know,
setting up that, that, that, that, that surface area,
where I go to start kind of tuning it from a workflow.
You know, some of that will be driven by UI.
But over time, most of that will just be driven through,
through a meta MCP server, right?
Like, you know, like, that just happens, you know,
hey, I need a server to do something.
Can you find me one, sure?
Here it is, we can go kind of go from there.
So those are the constituent pieces that,
that we see a lot of demand for.
That's what we've built with two high.
Another interesting aspect of the tool equation
is in particular when you're using these AI agents
in a local context, particularly for software engineers,
where things like cloud code, co-pilot CLI, Gemini CLI,
are the forefront in terms of usage.
And they do have capacity to interact with MCP,
but they also have your entire suite of command line tools
available as well as ad hoc and arbitrary tools
to be able to bring in context, perform different actions.
And I'm wondering how you think about that juxtaposition
of MCP as a protocol for either curated
or remote actions versus just having a suite of,
hey, you can do whatever you want and figure it out
as you go as a means of kind of tool use.
- You know, it's interesting, 'cause like,
I always joke, you know, like with developers,
it's always safety third, right?
Like it's very hard to cancel a developer
on a safety first mindset, 'cause like really,
you know, developers love building, right?
Like they don't love locking down and hardening
and all this sort of stuff.
And you can certainly make good inroads, just, you know,
in an interactive model with kind of bash and whatnot.
But I think there's kind of these, you know,
there's a set of environments
where this becomes problematic.
Like one is, I mean, how often have you had, you know,
one of these things are in more of your file system
because you weren't kind of paying attention, right?
And, you know, there are, like I said before,
like the security stuff, people are getting shut down
because, you know, at the end of the day,
the cost of a security breach is dramatic.
The embarrassment associated with this dramatic,
you know, like, you know, people are starting to,
you know, at least the people I speak to
are getting authentically gunshot about,
you know, relatively unfettered, you know, kind of root
level access to whatever resources you want.
Because, you know, while you might have it in your head
that you can watch this thing
and make sure it does do anything stupid, you're human.
Like, that's a low dopamine task.
You're just going to tune out
and sooner or later it's going to do something
that's going to regret.
So I think that, you know, kind of helping recognize that,
but you have to provide an alternative
that is just as useful.
I think that's the key thing, right?
Like, you know, for developers who actually embrace this stuff,
the alternative needs to be as delightful and as capable, right?
So it's, you need to have a relatively robustive tools.
You need to, you know, start observing
a set of things that they want
and actually start to reason about loving the tools.
And this is something that is wonderful about open source.
Like this ecosystem will start to, you know,
like the best way to tap into a lot of people
is to start actually having an ecosystem emergent
and start to formalize it.
But there's also knowledge workers, right?
Like, you know, there's a lot of citizens
developers out there and God help the world
when they start to, you know, like, you know, like,
a lot of organizations, cyber programs
are built around the idea that like,
developers go through training.
And now you have citizen developers
that are basically, oh, I know how to do all this stuff.
And that gets kind of even more, more, more dangerous.
So I think, you know, one is this will be naturally correcting
as people start to just recognize that
pace of exploitation is outpacing pace of spend.
And, you know, the cost of exploitation
is becoming a line item that you need to worry about
as much as you're worrying about
the actual spend on the systems.
And then I think a lot of it's also just,
it's nascent, like we haven't fully shown
what this can do, right?
Like it's, it's, you know, like it's a curiosity.
But until organizations or people
individuals start sitting down and asking hard questions
around like, what are the actual nonsense verbs
that dominate my life?
How do I start to generate tools?
And I'm not saying I'm going to write the tools.
Like, it's entirely reasonable to have an agent
start to build those tools
based on the known set of nonsense verbs.
And that code can then be reason
about hardened, isolated, secured
and published and reused inside an organization.
So that's the sort of, you know, like I do agree.
I think most developers aren't attracted to, you know,
anything that's going to kind of, you know,
inhabit their ability to vibe code their way into productivity.
But I do think that there are options
that will initially satisfy,
but eventually delight developers,
give them tools that they can use
to kind of radically improve their life,
but also not get hurt.
In that context of security to your point,
developers always put it last
because we'll figure it out later.
In terms of what you're doing,
we'll tool with tool hive.
Obviously there is authentication at the access layer,
but I think another interesting piece
that you're bringing in is that question of
how do we actually expose credentials
to the MCP servers themselves?
Because that's also another piece
that in many of the MCP servers is left
as an exercise to the reader.
And I'm wondering if you can just talk to some
of the ways that you're thinking about
the overall ecosystem of MCP.
- Yeah, I mean, this is what we already started the journey.
It's like, look, I'm a grownup.
I want to use these things.
I'm not going to have API key written in plaintext
on my system where an agent can read it.
Like it's just not going to happen, right?
And so, you know, being able to start using,
you know, like, you know, we started with local,
you know, kind of serving people with a local system,
being able to use, you know,
one of the locally integrated, you know, kind of key chains,
like, you know, like one password
or the Apple integrated, you know, system.
So you write your tokens in there
and it just gets mapped into the files
by a system made of tongue sense.
And so, you know, we certainly started with that,
but the place where it gets really interesting
is on the server side, right?
So when you're starting to look at, you know,
hey, I've got a set of clients
and they want to start connecting to the server.
The MCP specification is pretty good
at the client's server auth flow.
But the exercise of saying,
here's a set of patents that enable you to, you know,
reason about you know, serve it to back end server, you know, all
is it's an exercise to read it, right? So, you know, I know, like, friends that I've been doing,
you know, quite a lot of work in space, the intra people doing a lot of work in space,
there are conversations happening in the community, but you have to have a reference
implementation, right? So what we basically, you know, did is say, what are the four or five common
patterns that people have? Like, you know, hey, you want to use an old-worth, you know, kind of
credential, like, how do you do a token exchange so that you can, you know, move from, you know,
a token that is, you know, scope for everything a user can do to a token that's scope to a specific
task. And like, what does that look like in a canonical way? And how do you implement that without
having to redo this for every MCP server that you're building? Hey, I want to use a durable,
credential to go access a database, you know, how do I inject that conditionally and how do I
say policy around that? So we've put a lot of time and thought into the authentication and authorization
space, but we're also, you know, just working with a lot of partners, right? Like, it's like,
hey, you're not going to want to deploy a new identity provider. That's kind of, you know,
there's a reason why, you know, criminal security and zero trust, there's been a lot of, you know,
adoption difficulties in enterprise because enterprises are living with what they have. It's
difficult to, you know, unless you entirely greenfield to adopt these types of patterns. So a lot of what
we've been trying to do is just figure out how to make it work with what you have, but recognizing
that there has to be a space for something that's, you know, like, you know, agents have to have
their own identity. It's it's not an ID, it's not a service account. It's it's a delegated
credential that has a certain set of attributes. You can kind of make it fit into some of these systems,
but eventually we'll figure out what that looks like and and we'll make sure that we can support
that over time. And then the other place we look at is just, you know, this should be obvious from
security perspective. You need to be able to do, you know, kind of malicious prompt detection. You
like, you want to make sure that you do PI data reduction. There's a bunch of things. We've made
sure that the system is, is kind of sufficiently principled. So you can just snap in whatever your,
your favorite is and over time, tool hive will come with a lot of batteries included options for
for those pieces as well. And as you have been coming to grips with the current state of the ecosystem,
understanding the usage patterns and the opportunities and building out tool hive to get in front
of people, what are some of the most interesting or innovative or unexpected ways that you're seeing
that toolkit used or just MCP more broadly? You know, it's interesting. It's this cork milk
guard. We did this and we've worked with other people that have done the same thing. It was kind
of a silly thing. You know, we wanted to build a knowledge server. And, you know, basically what I'm
saying by that is like, hey, a lot of people are buying green or they're buying these, you know,
these kind of first-class systems that integrate with all these systems. And I set my team an
exercise of like, what would it take to just build one of own? You know, like, what would this look
like? And it turns out it was actually surprising easy, you know, we just build an agent that runs it.
We go and take all of the content we chunk it. We had to do a bunch of tuning because, you know,
like, you know, the the quantum when you're chunking discord messages versus, you know, kind of
Google docs to make sure that the semantic, you know, relevant showing through was surprising. But
having MCP based knowledge server endpoint, we scored the knowledge server and you can integrate
into all your tools. And you can ask questions and it can basically see, you know, everything that's
within your, you know, the kind of public domain. It was a complete game changer. It's it's just
shockingly useful. Like, it's just wonderful to be able to, oh, by the way, what is this developer
working on? Oh, by the way, what are the top, has this been mentioned in discord? I haven't seen
anything on it. Oh, yes, it has, right? So, so that's just sort of immediate, like, unlock
of institutional knowledge through a relatively simple implement, you know, server. So we built
this thing in a few weeks. And it's just shockingly useful, right? Like, it just, it changes how
I work. It's it's changed how everyone in the company works. And as we started to talk to other
people that that becomes like, you know, sort of an anchor, the ability to, you know, sort of
redefine how, you know, support works, just like build a knowledge server and expose this is an
endpoint. And now, you know, like these things suddenly, you know, they just have rich, rich access
to kind of precise semantics. I just didn't expect that. Like, you know, I guess it's one of these
things I thought would be a huge, circulating lift. It turns out that a couple of engineers knocked
out something just damn useful in a couple of weeks. And as you have been building in this space
and growing a company around tool hive and this overall space of empowering teams to build these
AI native applications, what are some of the most interesting or unexpected or challenging lessons
that you've learned in the process? I think the biggest challenge that organizations face is,
I mean, and this should be obvious. So I think anyone who's worked in this space for a while, but
it's not enough to just start to expose tools, right? Like you have to really think like,
I've seen this a lot of folks were, you know, like they're like, okay, we're building, you know,
a system, right? And we like, that gives the problem like cursor is the same dumb in turn every day,
right? Like it's like they're incredibly enthusiastic, they're incredibly prolific, but they know
nothing, right? And like, you know, for an enterprise that wants to work, like they can't, they just
can't npx install these random tools, they just can't use those integrations. And so they start,
you know, kind of working on the assumption around like, okay, well, if we just created a server that
made this doc available, that doc is the, is the, is the specification. If we just did this and
we've just did that, you know, everything would kind of work. And you know, what's happening is this,
it's sort of, they're seeing kind of incrementalism, but they're not necessarily seeing a sea,
a sea change, right? Like it's, you know, you kind of have to start rethinking the SDLC and you
have to rethink like wave value engineering's happening. And I think the fact that organizations
are bringing a relatively kind of traditional mindset to this and on taking the time to understand
what problem are you solving? Where are the boundaries? Like let's run some very, very fast
experiments. Let's not assume that just, you know, kind of rendering all these endpoints as tools
is fundamentally going to change the workflow. Like let's actually, you know, start by trying to
understand what we're trying to accomplish and work back from like, well, what do we need the model
to know? Not just have access to what do we need to do to start to, you know, contextualize this?
Because it's interesting to see how you get these kind of like, you know, people work with these
tools over time. And there's this weird thing that engineers do where they start to like have these
sort of ceremonial sort of activities. And what they're effectively doing is, you know, do this first
because then it knows what packages. And now that effectively manually loading the context window
with, you know, by performing things in a certain order, it's getting to the point where, you know,
like they don't necessarily even realize they're doing it, but they know that it works. And so there's
this sort of process of kind of implicit discovery of how to render context, you know, in a way that
actually completes a task, but it's not being written down or formalized or reasoned about in a
first class way. And so I think, you know, you kind of have to start with an experimental mindset,
where you're just like, look, just get these things, put in a controlled environment, get them
working, try and see what happens, you know, play, play, play, play, play. And then I was the question,
okay, what, like, what's working? But why is it working now? And what can we start to do to kind
of work back from that to actually start describing the tools that will, we'll get us into that state?
And for people who are interested in expanding their investment in MCP as a core capability,
what are the situations where you would advise against using tool hives specifically?
Well, that's a great question. I would say if you, like, if you don't know what MCP is,
don't use tool hives, right? Like if you, if you, if you're not at a point where, like,
if you don't, if you don't know what problem you're trying to solve is, like, I can't help you,
right? Like it's, you know, this is a system that's really, you know, it's, it's, it's solving
two problems, right? Like it's solving the problem of, of control for, for a real world, you know,
real world company. And it's solving the problem of scale. Like when you want to go from one to 10
to a hundred to a thousand to 10,000, you know, service, it's, it's a, it's a good system for that.
If you haven't yet tried to just use the SaaS-based endpoints and experiment, if you're not convinced
that there's something there, you know, tool hives probably not the right starting point, right?
Like it's, you know, this is, this is a tool that is, you know, it's really for teams that are
invested in building cloud native applications that want to unlock the first party data in addition
to other, you know, sort of data. It requires a fair bit of operational lift. This is not a,
like a sort of an incredibly simple turnkey, just turn around and go kind of system. This is like,
no, I'm serious about this stuff and I need a platform to do it. And I, I want to operationalize it,
and I have to reason about authentication authorization. And yeah, I know I want to do this.
Maybe I haven't figured out perfectly, but at least know what MCP does. And I, I know what I want
to start exposing. I have a theory around what my first projects are. Tool hives a great option.
But, you know, I would definitely start by, hey, just turn on the, the integrations in, you know,
like all of these, all of these systems have native integrations. There's a lot of SaaS-based endpoints
out there. Start there. You know, see how far that gets you. If you're still, like, if you see the
value, but you don't know how to fully unlock it, if you get to the point where you like it,
but you can't use it because you're an enterprise. And if you're interested in like, okay, how do I go
beyond that? And I actually start, I want to start building, you know, in earnest. And I want to
start ready reasoning about how to not just, you know, provide a tool, but deliver a tool that works
for this agent all the time that has this, this kind of rigorous evil framework that has this,
this kind of tuned behavior that, you know, I've got a lot of tools. I need to start, you know,
reasoning about context pollution. Hey, I want to start driving, you know, kind of context compression.
There's a variety of these things I want to do. Come talk to us.
And as you continue to build and invest in and grow, tool hive and expand the overall
set of offerings from stack lock, what are some of the things you have planned for the near-to-medium
term or any particular problem areas you're excited to explore? Yeah, there's a lot of things
that we're interested in. I think, you know, we've got the core of the platform out there. By the
way, it's an Apache 2 Open Source project. So it's, it's stack lock, but it's also red hat and a
variety of other industry players are starting to participate in it. We're being very happy and
excited to work with Red Hat, I should mention in the year. They've put a fair number of
engineers and we actually have maintainers from Red Hat on it.
So red, I'd have the theory around where this sort of time goes.
But for us, it's about hardening the platform.
Getting to the point where it has--
I'm a Kubernetes guy, right?
So when I say platform, I'm very specific taste
around what a platform should look like, right?
So common API-based interface, well-factored, well-structured,
common naming conventions.
So I'd say thing one is, we're working very hard
on the platformification of the system.
So it looks and feels like an actual context platform
or a context engineering platform or an MCP platform
where we want to call it.
The second thing is, obviously, we
want to continue to enrich the capabilities at every level.
So right now, to cure runtime, we've
basically built out the proxy layer, the gateway layer.
We've put a lot of the VMCPs of the VMCP,
the virtual MCP constructs in there.
We'll continue to put a lot of effort into that.
So enriching the orchestration systems,
enriching the ability to support transactional semantics,
layering in and making sure that the pipeline architecture
supports capabilities like PI, reduction, turnkey,
prompt and resource security.
So there's a lot of work that needs to happen in that space.
The registry is, I think we're the first people
to have an upstream compatible registry
because we actually wrote most of the upstream,
or not most of it, but wrote a decent portion
of the upstream registry implementation
for the recent release.
We certainly contributed a lot as that piece was out there.
So the registry is there, but we'll continue to harden it.
And there'll be a lot of enterprise 101s associated
with that, we've worked with the community
to make sure that there's kind of extensibility pieces
in there so you can start to get an attestation.
And if you need to have signed images,
and you want to start reasoning about this thing
flowing through a more rigorous STLC, those pieces are in place.
So we'll continue to invest in that.
But the place where I'm most passionate about
is really what we call the cloud console, right?
Like the ability to provide that interface
where right now, hey, just wire in the VMCP thing,
and then people just platform engineers
or just deliver tools and you can use them.
But we need to create that self-service experience
so that for developers and citizens and coders
and for people that are, you know, on a state citizen code,
like everyone's gonna be a citizen code at some point, right?
How do you actually start to describe your work
in a way that's more natural?
How do you actually start to participate
in defining that interface in a much more natural way,
both through a kind of, you know, AI native experience
so that you can interact with the model
and it will actually support a lot of that work for you,
but also potentially having the right visualization.
So I wanna make that system
and AI native application itself, right?
So it's consumable through an MCPN point,
but then as you start to have to have like web app,
you know, like web MCP type, you know, modularity,
so you can actually start to have a really great experience
when interacting with it.
That's gonna be a big part of our focus
from the next level up.
- And one thing too that I saw just recently
from the folks at Century is they're working
on actually making their MCP server be its own little agent
where rather than exposing all of the tools,
it has an agent that takes the request
and then determines which of the tools it knows about
to then expose to your agent.
I'm wondering what your thoughts are
on some of that pattern as well.
- Yeah, I mean, we like that pattern.
It's, you know, it's the sort of, you know,
we started with just relatively simple semantic search,
the implementation that we have.
You know, this is ready just to deal
with the tool pollution problem,
but there is this other piece which is very interesting,
and you know, we certainly spend a lot of time
talking about it, which is like, what is the,
like, what is the, like, how do you acquire privileges
for a task, right?
Like, because a lot of these agetic tasks,
you don't know what resources have to act as a priority, right?
Like, so you don't know what they're actually gonna do.
You don't necessarily wanna give them
unfettered access to everything.
So there's a human and a loop process initially,
but eventually there will be kind of agents watching agents,
you know, and reasoning about a lot of these,
these classes of decisions.
I don't know where it's gonna go.
Like, I mean, it's, like I said earlier in this conversation,
like regulation and policy trails the state of the art
by a pretty large amount.
So I think, you know, for a lot of systems,
you know, there will necessarily be, you know,
some, before some class of work,
there will have to be human supervision for a while,
but I do like that pattern.
And I think it's very interesting.
And, you know, we'll, you know, as the,
as the kind of authentication authorization system
starts to kind of snap into focus,
we will certainly be looking at some of those patterns.
- Are there any other aspects of the work
that you're doing on tool-hive at StackLock,
anything about the MCP ecosystem more broadly
that we didn't discuss yet
that you would like to cover before we close out the show?
- We have a lot of ground.
- This is really great.
- Well, for anybody who wants to get in touch with you
and follow along with the work that you and your team are doing,
I'll have you add your preferred contact information
to the show notes.
And as the final question,
I would just like to get your perspective
on what you see as being the biggest gaps
in the tooling technology or human training
that's available for AI systems today.
- Oh, it's skills, right?
Like it's just the talent necessary to kind of take this,
you know, it's the mindset shift
where it's not about what the agent has access to.
It's about what the agent knows,
like it's how it's contextualized.
It takes time.
Like people will build that expertise,
but it's expensive to do so.
So I think the biggest gap there is skills.
And like, you know, like, you know,
we are, you know,
we provide for deployed engineering capabilities.
And that's the best way to kind of, you know, translate the skills.
And so I really think of it as a bootstrapping problem, right?
Because you need to be able to transfer skills
into an organization.
So we sell, we don't sell the product without, you know, FDE support.
Because we know that's what's required to get people to succeed.
But, you know, it's a bootstrapping.
You need to do that enough to get to the point
where the organization generates its own awareness.
And then you need to start providing the agents
that actually are able to perform those tasks.
But it's a, it's a, it's a coastal problem.
It's a bootstrapping problem, right?
Like, you can't deploy the agents until the organization
has the capabilities to run them.
You can't necessarily describe the agents
until you spend enough time working with the organization
to figure out what they're trying to do.
And so it all comes back to that activation energy problem
where you just have to have someone on call
that knows what this looks like that's done this before.
Because it will reduce your time to come by orders of magnitude.
Like, it's not hard to go from nothing
to a running version of Tool Hive with a well-formed set
of MCP servers in place.
But getting to the point where those are actually tuned
to workflow and are generating results
and are reducing token consumption
and are improving agentic task completion
requires like a different way of thinking.
And so that's, that's always the hardest problem.
It's just, that's zero to one.
The platform's easy, you know, we'll deliver that.
But it's how you use it that requires this kind of,
you know, so the skills and organizations
are struggling with right now.
- Absolutely.
- Well, thank you very much for taking the time today
to join me and share the work that you and your team
are doing on Tool Hive and in standardizing
and simplifying some of this overall ecosystem
of MCP and context engineering for people
who are trying to build these AI native applications
is definitely very important set of capabilities.
And I appreciate all of the time and energy
you're putting into that.
And I hope you enjoy the rest of your day.
- Likewise, thank you so much.
(upbeat music)
- Thank you for listening.
Don't forget to check out our other shows.
The Data Engineering Podcast covers the latest
on modern data management and podcast.net
covers the Python language, its community
and the innovative ways it is being used.
Visit the site to subscribe to the show,
sign up for the mailing list and read the show notes.
And if you've learned something or tried out a project
from the show, then tell us about it.
Email
[email protected] with your story.
(upbeat music)
PodcastTranscript AI