#42 PulseAI and the Hard Question: What Is Your AI Actually Doing

Let's design a product and also get a risk assessment of your actual AI traffic for free, no strings attached. How does that sound in exchange for your next 20 odd minutes? So, there are two universal truths about AI or artificial intelligence in most enterprises today. Number one, when we ask, are we using AI, the answer is absolutely everywhere. And when you ask, do we know what it is doing? There is never a direct answer or there is always a wishy-washy answer. And today's episode lives in the gap between these two answers. And by the end of it, you and I are going to close that gap on an imaginary whiteboard together. This is the identity navigator, I am Rohit, let's get started. So, quick housekeeping and three things that I want to have a full disclosure about. One, I know the founder, we are talking about today. His name is Amit, but nobody paid for this episode. There are no sponsorships, no affiliate link, no discount link. This is 100% my lens, his product is something is thin, I will tell you it's thin. Two, he's one of us, a career identity guy, consulting trenches, manage services, before he even built this thing. This is not a tourist who discovered governance in a pitch back last spring. And three, precision, because there are at least four companies running around with pulse in the name, including one selling dashboard to the sea source. Today, I am talking about pulse AI from IDM express, that's the letter, IDM, then express, and IDM express dot AI, and it does runtime AI governance. So, if you Google the wrong one, that's between you and your search history. Right? Okay, so the problem, because the problem is in the episode. So, let's assume you have an unread email with the subject, evidence request, AI usage. And trust me, if you are in cyber or finance, you will get that email one day. Now the auditor wants three things, everything sent to an AI model that touched customer data this quarter, everything that came back, and by each this season was made. Take a second, could you produce that? Now, let's not kid ourselves, both of us know what the answer is, we are amongst friends here, and the answer most likely is in no. You cannot produce that. Now, I researched some of the data to bag that claim, and I did get some, IBM's 2025 cost of a data breach report says that 63% of the organizations have no AI governance policy at all. Now, I'm sure between 2025 and now, many more organizations must have a better or at least some form of AI governance policies. And of all the organizations breached through AI, 97% lack basic AI access control, and I'm pretty sure that number hasn't gone down drastically. But 97% even if the number has gone down to let's say 70% that's still 70% lacking basic AI access controls. So let's look at the 2026 edition, does it make it better? Does it make it worse? So it says the share of security incidents involving shadow AI doubled year over year to 43%. Let's go to 2026 number say nearly half of the creative AI users are still on personal unmanaged accounts. Personal accounts with your data in them. So what do companies actually have in place? Let's inventory the state of the art. System prompts, developer guidelines, a static output filter, somebody configured in 2024 and never does the game. And if you're really mature, a point in time, quarterly review. So basically what most people have are a system prompt and a quarterly review, which is also known as vibes and hopes. Now what is it failed three reasons and you already know all three because you have lived there one system prompts are set at deploy time updating them across 40 AI surfaces means 40 release cycles and in the deadline pages teams override them every control that depends on a developer's patience has the shelf life of a sprint every no exceptions to filter pattern match affixed vocabulary with zero idea what's happening in context. So filters are kind of okay, but they check words and nobody's checking meaning and three and this one is in Amit's own word once teams control protects exactly one team. No cross application view, no single pain in identity we learned this lesson 20 years ago. Point solutions don't scale and don't govern and that's literally why our profession exists. And while all that was failing quietly, the clock changed, McKinsey's number, generative AI went from about a third of organization in 2023 to nearly 80% by 2025. One of the fastest enterprise adoption curve ever measured and the agents stopped typing and started doing calling tools MCP or maybe sometimes directly executing actions on their own at machine speed. So long time listeners say with me, if you are 100% certain what your AI agent is doing, it's probably not an agent. It's an RPA script with better marketing and we have discussed why many forums but also why are the identity and certainty principle that there are three things that essentially creates an agent, the memory which is the context, the brain which is the LLM and the hands which is the tools that it connects to. So basically what I'm trying to say here is if you are 100% certain what your AI agent is doing, basically it doesn't have a brain, it's just an RPA script with better marketing because agents by themselves, the core behavior is to change their non deterministic nature. So what are we going to do about it for a second, let's forget the puls AI which I'm going to talk about in this episode exists. For the next 10 minutes, let's assume it's just you, me and a whiteboard, we are going to design AI governance from scratch and I promise by the end you will have invented his product before I name a single feature. So I made big claims now, so let me see if I can stay true to them. So step one, one choke point, you can't govern what you can't see and you can't see 40 applications by visiting them one at a time. Here is the one place every AI request passes through, the answer is the network, put the control between every application and every AI provider, sort of a gateway and here is the constant that makes or breaks adoption, no code changes. The moment your governance plan starts with every team updates their SDK, you haven't long the control, you have launched a two year migration program with a steering committee and a mascot and a fancy dashboard. So apps keeps calling open AI and proper, whatever exactly like before and governance happens in transit. So we define our choke point, the network, what's the step two, rules that live in one place, not scattered across system prompts, they are centered and hopefully they are versioned. Organize the way people actually think like about what are the type of prompts you would write, your data handling, content, behavior, compliance, and they should be editable life. The policy owner changes a rule, it takes affects everywhere, immediately no deployment cycle. In Pulse AI, the name they rule set as the Constitution. Well, bold name according to my taste, but I have watched committees take a full quarter to name a shared drive. So no judgment on my part. That brings me to the step 3. Remember, we spoke about the network, when we spoke about the rules. So what the step 3 is? Step 3 is judgment. Because rules alone cannot read languages. Fire the intern and fire the kill. Look identical to a keyword filter. So for anything that deterministic rules cannot settle, you need something that can actually reason about context, which means an AI evaluating AI. And now you have got a new problem. No, actually, you have got an old problem, who watches the watcher. One model having a bad day cannot become your policy. So you layer it, a fast screen for the obvious stuff, deeper reasoning for context, and an independent check on the verdict before it's final. No single model gets the final word. Right. So basically, the thought process is I don't trust any model enough to let it be the only check. And then the step 4, verdicts with a spine. So allow it, didn't enough. Real life is messy. So sometimes the real answer is fix it and let it through. Strip the sensitive bits forward the rest. That's modified. Because sometimes the right answer is a human should look at this. That's pending a review queue built in. So we'll four verdicts approved, denied, modified and held for human review. And notice what pending does. It makes human in the loop scale because human reviews the exception, not the fire hose. Manual review of everything is how governance programs die. Manual review of the weird 5% is how they survive. Right. And imagine how rollback, roll-based access control and quarterly-user access certifications when they have scale have only scaled when we are certifying the exception rather than everything. That is why most of the teams take out birthright access from their quarterly reviews because they just add noise. It is the same thing over here. We are human in the loop is reviewing the exceptions. And then step 5 is scrub before it leaves the building. Every prompt let's scan for personal information before it reaches the provider. Redacted automatically. And the redaction itself goes on the record. And then my favorite mechanic or can I retoken strip fires trackable markers planted in your sensitive document. If one shows up in a prompt that request is blocked on the spot and in alarm goes off. Because a trip fire in a prompt means someone or something is feeding the crown jewels to a model. The token themselves stays encrypted never sent in the clear. It's a classic technique. The things to force folks popularize popularize that years ago. Right. Applied exactly where it now belongs. Now the step 6. And this is the step that actually got me who is asking every request should carry an identity assigned one per agent. Because the marketing chatbot and the finance reconciliation agent should not live under the same policy. And when your ledger says who did something that's attribution not archeology. So register the agents bind the verdict to the identity. We have had this for human for 20 years. Now do it for nonhumans. Which if you have literally heard any episode of the day is the hill I live on. So because any tool coming into your ecosystem will not solve all of your AI problems. We will have to do some housekeeping of our own as well. The discipline in development, the discipline in the rollouts. But here is the part that we all need to hear. And we do not like it. But identity is not the finish line. Being allowed to act doesn't mean the content is saved. And agent can be fully registered correctly permission acting completely within its authorized scope. And still in that single legitimate call send a customer's critical information to an LLM provider or a leak a secret from a system prompt or get manipulated by prompt injection payload hiding in the very documents it summarizing. Right or handbag output that quietly contains hallucinated financial advice. Now none of them is an identity problem. The agent was exactly who it claimed to be doing exactly what it was allowed to do. The failure is inside the payload. And a system that only checks is this agent authorized for this action doesn't look there. That's why the whiteboard the virtual whiteboard needs both halves the identity spine and the judgment layer reading every payload. Remember the identity and certainty principle. And who is asking and what is actually being said. So step seven is write everything down every request the verdict the reasoning the exact rule that fires the version of the constitution that was enforced at that moment. All of it in one queryable ledger which filtered by agent by verdict by time range exported and hand the auditors are report. And when the auditors ask for evidence it already exists. And as the guy who usually has to go thick through that evidence that is a life upgrade. And because we are engineers and you love bonus things. So let's have two bonus moves while we are still at it. Let's grade the answers. Every approved response gets code that relevance faithfulness hallucination. So governance stop being just was this allowed and becomes was this any good. You find out which application is confidently making things up. And I know some humans I would like this first. Cache the repeats near identical prompts get served from a governed cache instead of a fresh model call faster but most importantly cheaper. Yes. It could be a governance layer that partially pays for itself. All right settle down folks. Okay. So now let's step back and look at the whiteboard. We had choke point network constitution layer judgment verdicts which were four scrubbing and trip pious agent identity plus the payload check that identity alone cannot give you. And then we had the ledger the grading and the cache. So congratulations you have just built Pulse AI fortunately for Amit and unfortunately for you he built it first and maybe with fewer dashboard markers. So you know what Pulse AI is now you and I have designed it ourselves. Not that we are going to get any equity but who is Amit Mason. The by his own count nearly two decades in identity and access management. And the public paper trails back the substance. Real rules at recognized eye and shops. He had an speaker slot at identity verse. He founded IDM express out of my home state of New Jersey which has identity security plus round the clock managed services for enterprises. This July, Pulse AI launched under the IDM express dot AI umbrella alongside a non-human identity product called NHIDM, Boots Act or in Hamid's words the business funds the product. So no outside capital, no board act theater, so he smart, he's resourceful and most likely he's rich. And his origin story is not a lightning bolt, it's better, it's a pattern, it's something that you and I can relate to. Engagement after engagement, the same gap in every environment he does and in his words identity governance platforms are excellent at telling you who an identity is and what's it's allowed to do. None of them tell you what that identity is actually doing right now. Agents with standing access nobody had reviewed in months, quietly taking actions that technically fell inside their permission but that no human would have approved if anyone had been watching life. His summary line I'm honestly I checked whether he has been reading my rough notes is governance that knew the identity but was blind the moment of action. And that is the whole disease I recently wrote a paper on it as well granted versus doing. We audit the grant once a quarter and ignored the doing entirely. So I may have watched that whole for years, client after client until it was obvious nobody was building the thing that needed to exist. And he did the thing that I implore all of you to do. He built it. The proof was in this June's identity was IDM Express had booth 801 and I was there at the conference and they presented or the team presented Pulse AI in the non-human and AI identity pavilion. I mean it was on the speaker list that's when I knew he was at the conference and the demo in his telling the system intercepting and evaluating real AI agent traffic live in front of a room full of security professionals. And that is a hostile audience by the way we heckled quietly in compliance language. But the demo and the deployment matched the promise no code changes so team typically see governed traffic and their first audit records within days of connecting. And both of us know both you and I know in enterprise security days is usually a type. So before I go ahead with you know what more about Pulse AI and stuff like that who the competitors are a market analysis there is something that I think all of us needs to learn from them and I would all encourage all of you to see if it helps you as well. He has bootstrap this company scene a gap and build the product for this team. What's stopping you? I will just leave it there. Moving on, is Pulse AI alone in that space? And I will tell you that even if I was asked not to for the record he nobody asked me to say or not to say it. But I think Gartner files this under AI runtime inspection and enforcement. So witness AI plays here. Palo Alto's Prisma heirs gateway when GA this summer. Cisco is stretching AI defense to do so agent actions tie back to employees. So anybody who tells you nobody else does this is selling. Everyone this space is selling and that's what makes it a space. So what's the honest edge? Here is what I would actually put on an evaluation sheet the identity DNA. This was built by an IAM shop. Per agent signed identity isn't a bolt on here. It's a spine. The four verdict model modified and pending are where real governance lips and a built in even review queue is rare than it should be. Response qualities code in the same ledger as the policy verdict. I apparently not seen the Blake plate forms put. Was it allowed and was it hallucinating in one record? Go check me on that. It it deploys self-hosted even fully air-gapped which matters more than vendors admit. And there is a managed service provider model which is quite a big deal if you are a regulated mid-market shop that buys through partners. Because most tools in this category assume you have a 40% security team and maybe you don't and that's fine. Someone can run it for you. And the honest questions founders never answer. Who is this not for? If you are running one or two AI integration models you don't need it yet. A guard real library will get you by. It's built for the moment you have lost visibility across multiple teams and multiple providers. Compliance in terms of it. It is mapped to soft to ISO 7570001 GDPR, the NIST AIRMF, the EU AI Act and HIPAA. Evidence exports build for the audit cycle and not pulled it on after. My one push on this one and I told it to the team as well get ISO 420001 on that list. That's a certifiable AI management system standard and auditors have started asking for it by me. So I've told the team and I'm telling any other founders in this space get ISO 420001 on your compliance list if you haven't already. Cost model is pretty standard or the pricing model I found was pretty standard platform fee plus usage. So a base subscription tied to org size and request volume layered on top. So roadmap and this is my backyard. Today, pulse AI governs what an agent says and does. Next on the list, who the agent is, what it's permitted and just in time access. So just in time for nonhumans and I've been yelling about this frontier for a couple of years now and somebody is out there paving the road. They're also taking a small number of design partners. So if you want hands-on early access direct input into what gets built next, the door is open. And by his telling, the phone has started ringing. Big BC names, a big identity governance name, and the word strategic getting thrown around which for a bootstrap find the founder pointed one of two endings. A partnership or an early retirement on a beach in Jamaica. One more line that's important to all of us. And the pulse AI team states it as a matter of fact. It's not training on your data. It evaluates your traffic against your rules. That's the T. Here is here is the verdict. If you're evaluating AI governance right now, mention this episode and they will run a complimentary risk assessment of your actual AI traffic. Not a slight deck. A real look at what's leaving your network. No commitment. Free I think is a very good price for a mirror. You can find them at idmxpress.ai AmitMassand is on LinkedIn or you can email him at AmitMassand at idmxpress.com. This is the identity navigator. Just me. You and my and finally, a straight answer to what's your AI actually doing right now. Until next time, this is Rohit, your identity navigator.