Agent Skills Masterclass

Today on the AI Daily Brief, an agent skills masterclass. The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. Alright friends, quick announcements before we dive in. First of all, thank you to today's sponsors, Recall.ai, robots and pencils, Blitzy and super intelligent. To get an ad free version of the show, go to patreon.com/airdailybrief and if you are interested in sponsoring the show, send us a note at [email protected]. Now one other note, today's episode of course features, the one and only new Far Gas Bar, walking us through a masterclass in agent skills. For anyone who listen to my agent skills primer, it's a really good part too for that, that gets much more practical with a whole framework for how to use skills and how to use them well. Now you can get all of this companion data, including things like the anatomy of an effective skill over on play.ai/airdailybrief.ai. That's where we keep the companion experiences for this show. And if after that you want even more new Far, we have just opened up the second cohort for Enterprise Claw, which is an agent and agent team building program. I'll have links to all of that in the show notes. For now though, let's dive in and up your skills with skills. Alright, new Far, welcome back to the show. We're talking skills. How you doing? Good, happy to be here. Yeah, we are. Man, it is since the last time you were on, the things that matter in terms of teaching, being up to speed, getting up to speed with AI are, you know, some of them are obviously that there's fundamentals of teams and, you know, expectations and things like that that remain. But God, the last time you were here that stuff we were talking about feels like ancient compared to where we are now. The human element is the same, the technology is completely different. So what we're talking about today is I did a couple of weeks ago on the show, kind of an introduction in a primer to agent skills. It's a standard, a sort of primitive for the agent era that helps agents figure out how to do things that you need them to do in very simple terms. But obviously there's a lot more complexity in how you use them and how you use them well. And that's what we're going to be talking about today. So tell us a little bit about what we're going to go through and then let's dive in. Okay. So obviously you did a very good job in your skill episode. You talked about what they are, the entropic categories and the various things that are currently like a landscape overview. But today I want to go much deeper and make it more of an operator cut because I want to give people the actual playbook on how to build skills that work, what mistakes kill them and what organization and opportunity really looks like. So we made it fun like we always try to do and we structured it in a five level journey. So by the end, hopefully you will go from understanding what skills are to knowing how to build an organizational skill library and everything is accessible to you guys on the play at AI Daily Brief, which we will demo in a minute. Okay. So we have five levels from apprentice to architect. So to make sure that we are all on the same page and give a reminder of what skills are at the core skills are just folders, not just markdown files, folders that contain instructions, scripts and resources that give AI tools and agents the actionable playbooks to execute various tasks. But here there's something that many people are kind of missing and that is that skills are not just for agents to it. They work in two modes. An agent can discover the skills that you're enabled in the environment and it can do so automatically and invoke them on its own or assume and can trigger them manually either by using the slash commands and most tools or we can just provide verbal cues and the tools will know to pick up the skills that we intend them to use. So for example, you may say research this topic and it fires a very specific research skill that you build that is very specific to what you like in terms of doing the research. So that's something we'll also show in a minute and the very good thing about skills that are highly portable. You know, most of us have built many custom GPs or gems over the last few years. The problem with them is that they were locked inside the GGPT or the Gemini Enterprise Forever. They are skilled, basically solved that. They are folders that you can just take with you between tools. They are human readable. So there is no proprietary format and anyone in your team can open a skill file, read it, understand it, edit it and you don't need any engineering degree and you can just take it between tools. Why are we saying that skills is not only the present but also the future of AI and agent tools because we're already seeing that all major companies are supporting skills. Basically we counted about 44 tools and counting every day yet another tool introduces that they support tools recently. Notion said so and many other tools already announced that we of course include in the tools that support skills, the open-cloth, the claw, the cursor, windsurf, GitHub and many, many other tools they're all supporting that but not just coding tools. And then we have people that have been basically using and building skills effectively for a while and they will tell you that this is probably the complete game changer to how AI and agents work for you and also it's quite addictive. Like once you start realizing the power of skills odds are that you will create more and more and more. And I do want to flag out wanting very explicitly here is that third party skills wanted to be acquired from somewhere in the internet whether it's in open-cloth marketplace or other places. They are code and as such they can run with a lot of your agent permissions and if you download that it can execute scripts and sometimes it can be a malicious script. So be very, very careful whenever you're getting a skill that you have not verified the source. Read it very carefully and treat it like installing any software package on your machine and especially if it's a work machine. Be very careful and pay attention so you will not bring any malicious software back into your organization. All right, so that's the basic. Let's talk about when to build skills. So the question is when should you build skills and I wanted to start with three obvious signals. Either when you do something more than three times that's to me is a good indication that now is a good time to build a skill or you keep basically pasting the same instructions and getting very frustrated with your tool and that's another good one and also when you need a consistent output. But the two additional things that you want to consider. First of all, this is a great opportunity to standardize things across either the way you do the work or others do the work with you. It's a great opportunity for you to think of all the things that you ever wanted to be more consistent of or get more consistent behavior by others and just build a skill to get others to behave in the same way. And lastly, and that's something that often also when I'll W talks about skills are not just a way for you to be more productive. It's also a way for you to unlock opportunities of things that you always wanted to do and just didn't have the bandwidth or the ability to do so. So think outside the box of what are some of the research tasks that you never had the opportunity to do or what are some of the work and the business challenges that you could never ever solve because you didn't have the know how and the bandwidth. Okay, I want to talk about two other things. One is that skills cannot be 10 different things. So one skill per task, if you find yourself getting to a point where it's completely separate jobs, separate them to different skills. And lastly, when it comes to the question of reuse versus creating a skill, I know that there are many marketplaces out there and especially in the open-close system, we know that there are an abundance of them. Similarly, with the entropic skills repo, however, it's very hard to navigate some of these skill marketplaces and find the exact fit. And often you will find yourself wasting so much more time just trying to read what others created versus creating your own. So especially if you want to hone your ability to create skills, I will actually recommend that you sit down, build a skill for yourself, leverage some of the best practices that we're showing here. And at least you will learn how to do it well. Later on, you can of course go and search what others are building because some of these skills that people are building are amazing. But I would advise to lean on more heavily towards building at this damage. One note on that front, I agree entirely. I also think that by virtue of them being sort of marked, you know, just mark down files, you can also treat even skills that you download as templates, not things that you have to copy wholesale. So in the next show that I do this week, it's going to be a personal context portfolio. And I'm sharing a GitHub repo that has basically templates for 10 files about yourself. And it's sort of, it's not meant to, it's not something you would copy. It's about yourself. You have to use it like a template, but I think there's a lot of resources out like that. And so I think it sort of puts a fine point on the idea of wanting to have the skills to build because it actually unlocks using all of these things that are out there in different ways that aren't just sort of blindly copying it into your projects and hoping it works. Yeah. And by the way, versus custom GPs that were black boxes, if you were to use others, now you get the full visibility into how this skill is instructed. So if you don't like some of it, just change it. I just wanted to note that Claude created an amazing and tropical, rather created an amazing skill creator that they recently released. And I definitely encourage you to go and use it because it's generally impressive. It interviews you to extract your expertise. It runs evils. It does a bit testing and bench. marking. So if you are a cloud user, you can definitely leverage their skill creator tool to do it even better. But in case you're doing it on your own, or you just want to understand what is the anatomy of a very effective skill we created a list for you. So every skill should have some of these elements and I want to emphasize a few of them. The most important part is the beginning and that is the trigger. The trigger is how you instruct the tool on when to discover and when to basically fire this skill. And it's probably the most important line because if your trigger is not very precise or very mick, then your skill will just not be used and selected by the agent. So I would advise actually that you make it louder rather than quieter because the models will sometimes keep past more subdued descriptions. So trigger words, exact descriptions about when do you expect to be used and be more explicit than implicit here that will go a long way. And then we have the body and what most people go on with the body that they write pros and skills are like playbooks. So favored and numbered steps or bulleted lists. Claude and all of the AITools they really like structured instructions dramatically because that will also turn to be their action plan if it's very very concrete. So try to make it as literal as possible. That's how the tools like to follow the instructions. However, I want you to also set the right level of freedom. So if a task is very fragile like a database migration coding, acquiring something that has to be very precise, be very prescriptive with a step by step. But if it's more of a creative task like writing a strategy doc or something that is more open to interpretation, give the guidance, but do leave some room for the tool to be creative because if you overreloading the model, you will not get as good results. We also encourage you to make sure to include an output format. And here it's even better if you just include an output example. So show the model don't just describe. If you want a template include it. If the output is a table, show a table and headers. If it's a document, show the section structure. So that's very useful for you to get exactly what you want out of it. And another section that and tropical-commended very strongly is the GATCHI section. This is probably the highest signal content in any skill because it's the area where gets the model to go out of its own patterns. Because you're looking to put here things that where the model will typically go wrong or what assumption it might make that it shouldn't. And you need to say something like, I know you want to do X, but don't. Here's why. And every failure that I've seen is probably something that you should document here after you stress test your skill. A few things not to include are some of the classical prompting skills like don't include the persona and stuff like that that's not useful. The tools are looking to get playbooks. A few skill killers that you should avoid. First of all, it's the trigger. If the trigger is not well set, the skill will never be picked for usage. Second, over defining the process like we said don't railroad the model. Also don't state the obvious. Don't waste tokens on things that the model already knows. And we strongly recommend that you don't skip the GATCHI section because this is often when your skill will go off or will create a suboptimal results. And lastly, don't do like a monolithic blob. Everything crammed into one file instead of using more of a folder structure. So speaking of folder structure, the recommendation is to keep skill under 500 lines because it's a playbook, not the anticropedia of everything that you do for work. If you have reference materials or context that are very important for the skill, move them outside of the skill file into a separate set of files within the skill folder when it's relevant. If you also have very long input and output examples and you should include input and output examples, you can also put them in a separate examples.md files inside the skill repo or the skill folder that will help you a lot. And that's probably the most effective way to communicate the desired format. In terms of the discussion of whether or not I should append a bunch of files into my actual skill folder that will be bundled with the skill wherever the skill goes versus perhaps just pointing the skill to my other files and other systems. The deciding factor should be if this is something that is context specifically for this skill that should always come with the skill whenever I'm offloading the skill to someone else, then put it within the skill folder. Otherwise, when it's stuff that are more general about you or about your company, that can be pointed to an external source. Why is there always a meeting bot in your Zoom call blame recall dot AI. Recall dot AI powers the meeting bots and desktop recording apps behind products like Clouli HubSpot and ClickUp. They handle the hard infrastructure work capturing clean recordings transcripts and metadata across Zoom, Google Meet, Microsoft Teams, in-person meetings and more, so developers don't have to build it themselves. If you're building a meeting notetaker or anything involving conversational data, recall dot AI is the API for meeting recording. Get started today with $100 in free credits at recall dot AI slash AI DB. That's recall dot AI slash AI DB. Today's episode is brought to you by robots and pencils, a company that is growing fast. Their work is a high growth AWS and Databricks partner means that they're looking for elite talent ready to create real impact at velocity. Their teams are made up of AI native engineers, strategists and designers who love solving hard problems and pushing how AI shows up in real products. They move quickly using RoboWorks their agentic acceleration platform so teams can deliver meaningful outcomes in weeks, not months. They don't build big teams, they build high impact nimble ones. The people there are wicket smart with patents, published research and work that's helped shape entire categories. They work in velocity pods and studios that stay focused and move with intent. If you're ready for career defining work with peers who challenge you and have your back, robots and pencils is the place. Explore open roles at robots and pencils dot com slash careers. That's robots and pencils dot com slash careers. One to accelerate enterprise software development velocity by 5x. You need blitzie the only autonomous software development platform built for enterprise code bases. Your engineers to find the project a new feature refactor or greenfield build blitzie agents first ingest and map your entire code base then the platform generates a bespoke agent action plan for your team to review and approve once approved blitzie gets to work autonomously generating hundreds of thousands of lines of validated and to end tested code more than 80% of the work completed in a single run. Blitzie is not generating code it's developing software at the speed of compute. Your engineers review refine and ship. This is how fortune 500 companies are compressing multi-month projects into a single sprint accelerating engineering velocity by 5x. Experience blitzie first hand at blitzie dot com that's BLI tzy dot com. It is a truth universally acknowledged that if your enterprise AI strategy is trying to buy the right AI tools you don't have an enterprise AI strategy turns out that AI adoption is complex it involves not only use cases but systems integration data foundations outcome tracking people and skills and governance my company super intelligent provides voice agent driven assessments that map your organizational maturity against industry benchmarks against all of these dimensions if you want to find out more about how that works go to be super dot AI and when you fill out that gets started form mentioned maturity maps again that's b super dot AI. Let's show you a concrete skill that is slightly more advanced and that is a meeting prep skill obviously all of us know whether what we do we have to get ready for and meetings so I wanted to showcase an interesting skill here first of all description when to use when users say prep for the meeting meeting prep and so on so it's offering quite a few options so the skill will be picked up in almost all less scenarios in terms of context required so as part of the skill folder I bundle stakeholder context so either it's going to be transient or if you have regular stakeholders that you work with that can be fixed context file that comes with the skill it should get some email history for reasons so that can of course be pulled directly from the user systems calendar and the other open action items evolving the attendees in terms of the steps identify all the attendees from the calendar or the other inputs collect the context analyze the agenda LAN scenario analysis so part of what this skill does is kind of preparing you for what can go on with your meeting and generate a brief and the output will be structure that is defined in an attached file because it's a very long and specific output and in broad stocks the the structure will be executive summary and so on a few got you that can happen when you're getting ready for a meeting with such a skill sometimes they are assumes attend this in your already just from title and if someone is a VP they assume that they are the most important person in the room and attributes unnecessary wait for them don't fabricate company details don't prepare generic talking points don't skip the what could go wrong analysis and so on these are things that happened when triggering the skills without it so that's why they're here and the way this overall folder is structured we have this file We also have, like I said, a stakeholder context, an external pointer to be sure the course relevant skills. The brief format, scenarios, examples, and also there is a nested skill. And in this nested skill, there is a sub-skill of how to simulate the actual happening of the meeting. And that's a very, also a very cool skill that will basically come with the six to seven different scenarios of what could go wrong. We'll see, like, if someone is joining your meeting and has an agenda, how will you address them? Someone is asking you difficult questions, so it will literally help you get ready for difficult questions. If you use it for a sales call, it can come up with difficult questions around the sales and so on. So that's an example of how you would build a skill that also has context and perhaps refers to another skill. Moving on. So obviously, there are many, many skills that people have, but I wanted to include here four ideas of skills that might be useful for anyone who is a knowledge worker, which is most of the audience here. So first of all, as part of the materials that we provide you, we include an example of research with confidence. That's skill that not only does research that is very precise to what you care about. Same horizon, specific sources, but also it has a built-in fact-checking methodology where it will compare sources and do a deeper dive into specific things that seems off, as well as giving you confidence, scoring about how security is with the findings so you can decide how deep to go in. So that's one skill that I think every person who does any type of research, which is all of us should build or reuse. Another one that I really like is Devil's Advocate. This is basically a skill where we say it takes any proposal and systematically stress testing. What makes the version that we included a little bit more special is that it explicitly looks for blind spots and biases both on your side and on the AI side, because we know that the models have many of their biases, so it explicitly tries to avoid those. And it always ends up with something that is more constructive. So it's not just finding holes with anything that you want to do, but also helps you to be back to something that is actually actionable. And another skill that we created is Amonning Briefing. That's another classical one. It pulls together your priorities, calendar, pending item, relevant news. And the thing that makes it more powerful is that it binds your personal context files with the skills, including the goals, the current project, and stakeholders. And as part of the materials that we provide with this episode, we also included a poem that lets you create one for yourself that will interview you and make sure that you will be able to create your own morning briefing. And another one that I strongly recommend that you will build is a board of advisors skills. So either one or several of those basically will simulate perspectives coming from multiple expert archetypes. So it won't be just like think like a CFO, but rather you can think of all the different perspectives that will help you make decisions. So if you are a starter founder, then perhaps your board of advisors will have someone coming more from a VC background, someone from more of a entrepreneurial background, your imaginary advisor and various other perspectives. And you basically create a skill that gets them all to advise and assist you by providing various perspectives on any decision that you would like to make. A few more advanced patterns that for people who are already building skills for a while and want to take it one step further, first of all, having a dispatcher skill, which is a meta skill that reads all of your requests and routes them to the relevant skills. It's like a traffic controller basically, and it's very important where a new library of skills go past 10 or 15 active skills that you regularly use. Often I would advise that you will create this dispatcher instead of hoping that the agent will read to all the available skills and pick up the right one. And this is especially important when you have nuanced, similar skills that you want to be picked up in completely different scenarios. Another thing that you can do is you can change skills one after the other either automatically by having a skill that basically calls one skill after the other or manually you will take the output of one skill and it becomes the input to another skill. So in the examples that we've shown before, maybe you start with research with confidence and then the output you take it to the devil's advocate to poke holes in the research and then you take it to another skill that does an executive summary and deck preparation for you. The only thing here is that skills need a clean input and output, so that's an important thing in order to change them well. Obviously, recently we've been seeing more and more the emergence of loops, agentic loops and other loop patterns. So you can also create skills that create stuff like that that they will iterate, check, act, check again and then iterate. And it's becoming very interesting also for non-technical stuff because you can think for example on marketing and campaign optimization. For example, you will monitor your ed performance adjust the beads, recheck, flag when the specific metrics that you're following like rows or others drop, do competitive analysis and vice versa. So you have like an endless loop of someone that optimizes your campaigns. And you can also create skills that basically orchestrate multiple agents or multiple sub-agents execution. You can just explicitly prompt them to spin up multiple agents. Our research skill also does that, so you can take a look at that. And of course, the sky's the limit and will be curious to see if you have other advanced patterns that you've discovered that are working well with you. I want to make sure that you don't just create skills, but that you test them and make sure that they're working well for you over time. I think the easiest test for you is if you find yourself having to iterate after you get the output of the tool that used your skill, that means that your skill is not good enough. Because ideally, a skill should create a ready to use output. And if this is not the case, you have to go back and fix the skill. And this becomes even more important when you are about to share it with 50 different people. That's the case for you to treat it like any other AI product and basically run a proper evaluation. And the rigor, of course, should match the stakes. If it's something that also updates your CRM, then make sure that the skill is well tested. If it's customer facing, make sure that it's well tested. And there are some ideas here on how to test it. But in general, every time that you have a new model or that you have a different tool that will be using the skills, you have to go back and reevaluate. OK, let's talk about the organizational perspective. So up until now, skills were, at least it could be inferred that we're primarily talking about skills as a personal asset. However, organizations that are very AI forward already realize that skills are the future of how to streamline work and how to get everybody to get more value from AI. And as you can hear, this is where I get genuinely excited because it's basically the pipe dream of every knowledge manager that finally can become real. And you can think about it at the following. You can standardize the way work gets done. You can get a lot of the work done autonomously or to some extent. And you can bundle that also with organizational knowledge. So everything can be bundled into a single portable artifact that both the humans can read and new employees can onboard using that as well as the agentic tools that will be using them and doing the work for you. So what have seen happening in some organizations, they do skill hackathon where they create skills for their relevant teams. They are maintaining skills in shared libraries like they would maintain code. They make sure the skills are having clear ownership and use the cost various people. And those organizations are seeing massive uplift with the quality and results that they're getting. And the ones that are still not there, their people are kind of reinventing the will every time that they have a conversation with the eye or even with the create skills just for themselves. Eventually, and we already seeing that with the cloud core work and most organizations will have the set of plugins. So in cloud core work, we're seeing plugins for specific professions, but you can create a plugin which is comprised of typically skills and connections and perhaps some context for each and every department or if it's in every group in your organization. And all of a sudden, everybody enjoys the same worldview and the same goodness. So to be a little bit more prescriptive here, what I would recommend that you do at the org level, you will start with the discovery, running work audits or understanding where people do repeating work or where people are not getting optimal value from AI and where there are wish list that are not being covered. So you will have a list of opportunities to create skill. And then you will curate them and build skills using the best possible methods, maybe with Claude's skill creator or just some of the best practices that we discussed here. I then want to encourage you to do a lot of validation, especially if those are going to be shared across many people. Then perhaps the person who created the skill will replace with another person who created another skill and they will stress test each other. And of course, people who should use to poke holes in the skill itself, then you should package them into plugins and reusable elements. And lastly, skills have to have clear owners, whether they are AI champions or some of Gishmeter experts, be reviewed every time that they're being updated. And when they're no longer relevant or no longer serving us, they should be deprecated because otherwise, we will very quickly have a stale system that was amazing at the beginning, but is no longer the case here. So that's the five levels. Hopefully got you from a skill apprentice to architect. Everything we talked about, plus full skill template and a lot of bonus content and some advanced patterns are all packaged very nicely in the AIDB play. Everything is there. Feel free to go and take a much deeper journey using this artifact. Awesome. Thank you so much. One sort of just mental framework that I wanted to maybe close on is the last thing you said about deprecating skills when they no longer serve. I feel like even more, we're used to infrastructure, which is what skills are as being sort of semi-permanent or long duration. And I think skills feel like one of the first infrastructure primitives of the AI era that exemplify one, how iterative things are going to be to the sort of shorter half-lives that we have to assume for things that are valuable. And I think that what that means is, you're not going to have an initiative to design a bunch of skills for yourself and for your organization where you do a sprint and then it's done. It feels like it's going to be something that is just now a new recurring ongoing part of working with these systems and basically requires constant upkeep. I mean, is that what you found so far in your experience with them? Yes. And even in the materials, one thing that I included was saying that reevaluate skills in the following scenarios and when one month has passed. Because that's about the time horizon where things might become a little bit stale nowadays. So at least until, and I don't know if it's going to happen in the foreseeable future, but until things will be more stable and you will have a more self-healing system for skills management, you have to proactively go and revisit everything that is created, including by the way, the context is part of the skills. Often the skill itself will remain relevant, but maybe the examples or the context that the skill refer to, that's the problem of why we're not getting good results anymore. Interesting. We need a little agent that sits there giving a rating to how stale skills are based on when they were last updated. I'm sure people that the advanced organizations are already building these systems of automations, of skill reviews and suggestions for improvement. Awesome. All right, well, new far. Thank you so much for this. Sure, tons of useful stuff for everyone to dig in. Can't wait to have you back. [MUSIC PLAYING] [MUSIC PLAYING]