Stop Renting Generic Intelligence with Oumi CEO Manos Koukoumidis

Anytime in history, whether it's for technology or non-technology, when you care about doing the best job possible on something, you care about quality, when you care about cost, when you care about controlling your success, you don't go and somebody else is genetic, too. Welcome to the Talking AI podcast. We talk AI with both experts in the field and early adopters. I'm your host, Matt Page, and we're here to demystify AI for you so you can get some value from it. Let's talk some AI. Every company says its data is a strategic asset, and many then send that data into the same handful of general-purpose models their competitors can call it, too. And the answer is maybe useful, but the intelligence never becomes something the company owns. Menos Cook comedies is the founder of UMI, a platform built to help companies create, deploy and continuously improve specialized AI models. Before me, he was Google Clouds, Gene AI lead, bootstrapped the Cloud Palm effort that later moved into deep mind as Gemini. And his thesis is simple. Rit the intelligence that is generic and own the intelligence that differentiates your business. And today, I want to test where that idea holds, where it breaks, and what an enterprise actually has to build to make it real. Mattis, welcome to Talking AI. Thank you very much. Matt, very excited to be here and discuss about this with you. Yes, this is going to be a good one. I've been looking forward to this. But let's jump right in. When a company uses an off-the-shelf frontier model for a workflow that let's say differentiates their business, what's the bigger issue? The exploding cost of how expensive the top tier frontier models can be, or the fact that the intelligence technically never becomes their own? How do you measure those different risks in variables differently? Yeah, I would say all of the above and more. And that's the interesting thing that is not one reason that's multiple, extremely important reasons why it makes sense for enterprises, not to rent the most critical tool for their success, but to build the known their own tool that is specialised for the job. And as you mentioned at the very beginning, yet, most of them are still not doing this. But this is changing very fast. But let me start with one by one. You know, what are the main reasons? Why, as you also mentioned at the beginning, we don't tell enterprise, you should just go build your own AI for everything. It's not a matter of building and owning the most AI. But a matter of the AI that is most critical for your success, that differentiates you, that makes you what you are, that's the AI you should own. Or the AI makes sure you have a specialty that wouldn't affect your balance. So let me take it one by one. The first one is quality and differentiation of the service you provide. If enterprises want to innovate, which should be at the core of every enterprise, not just tech companies like us, you need to be constantly be compounding and improving that technology that makes you better. And increasing this becomes clear that technology is AI. And if you do believe that it's going to be AI, and you're just prompting the same model as everybody else, well, you're not going too far with innovation. You're pretty much using the same model. You don't have any mode. You have something that pretty much any other competitor can write the same or very similar prompt and build the same type of product. So when it comes to quality and differentiation, you don't have that if you're just prompting the same model as your competitors. And actually, as AI becomes the most critical component of what you offer, the leisure differential becomes because the core or the biggest part of the product is the same. The second reason as you mentioned is economics. The thing that is so, what's happening right now, it's widely inefficient in the industry, where enterprises are using like the Swiss knife for everything. They're using the model that was built to be good at everything for any task, because it's a Swiss knife. I can do anything, it does everything. So I'm going to use that. And while this may seem like the easiest choice, as I mentioned before, it's widely inefficient. It's a model that was trained to do 100 or 1000 things when you only care about one thing at a time. And the thing is that you're paying for that. And what we have proven time and again, and many entrepreneurs have proven independently not even working with us, but on their own, like Kersor, Intercom with Fin and many others, is that you can have a specialized model that can be 5, 10 times, 100 times cheaper, and as good if not better than this frontier models. Why? Just because it's specialized. And you don't need weights for 1000 things. You only do one thing. And the last reason, which I think we didn't touch so far, is about control of the technology. If I think it's become clear that this technology is key for your success, if your future depends on it, you cannot be renting somebody else's tool. That they can take away from geopolitical reasons what happened with Fable. They could take it away from you because they suddenly decided they're starting to compete with you. Like, let's say, anthropic, doing drug discovery, financial services, coding agents. And suddenly, they don't want to provide to that model to you anymore, because they want you to go directly to them, to use it. Most companies don't have an AI problem. They have a prioritization problem. There are hundreds of different places you could be using AI, but the hard part is figuring out which ones are actually worth investing in, which ones you're ready for, and what should actually come first. That's why we built GenROI at Hattrox AI. Instead of basing your AI strategy on a few workshops, wherever it happens to be in the room, GenROI uses AI guided discovery together insights from across your entire organization. It helps us surface the themes, patterns, and connections that are easy to miss. Then our strategies work with your leadership team to turn those insights into a prioritized AI roadmap, a way to business value, feasibility, readiness, and what actually makes sense for your organization. So if you're investing in AI, but still try to figure out where it can actually create the most value, highly recommend checking out GenROI. Go to Hattrox.com, backslash, GenROI, or just hit the link in the show notes. Or because they are just deprecating it, that's a problem we'll have with our company, we're working with a bank, and to say the deputy mini model is getting deprecated, we're being asked to switch to the new mini model. It's twice as expensive, not any better. And we have to do their work to change now. And last but not least, privacy security. You know, if you don't own the model, you don't control where it's deployed, you don't control data retention. So again, it's all these reasons, your quality, and differentiation, economics, control, and assetel people. The answer that you should build and own your own AI when you get about this, it's not new. Anytime in history, whether it's for technology or not, or technology, when you care about doing the best job possible on something, you care about quality, when you care about cost, when you care about controlling your success, you don't go and somebody else is genetic, too. You build your own specialist job. That was my follow-up question right there, because I'm thinking, what's the equivalent to this? Is there a time in history in the past? You know, the obvious one people think of renting something to think of cloud computing? I'm curious, is there another analogy to this point we're in right now? Because it's kind of interesting, you're going from using the API and using models and whatnot, to effectively renting it. And then you have people doing stuff on their local machines. Is there something in the past you would equate this moment, to or is it just unlike anything else we've seen before? So there is definitely lessons from the past, but also you could say this is very unique. And actually, if anything, those lessons apply even more to AI. When, even two and a half years ago, when that was highly controversial, highly, highly contrarian, now more and more starting to see this, was again, what happened with databases? Their Postgres SQL has more markets than Oracle. Why? Because more and more companies wanted that additional flexibility and control. Operating systems, right? All the internet, all the data centers, everything is running on Linux, not so much Windows. Because again, all the similarities, it's more flexibility to customize and optimize into your use case, lower costs, better control. And the thing that's interesting is that these arguments, even though they were true for databases, for operating systems and many more, they're even more true for AI, because there's one technology that is amenable to being trained, to be specialized, the specific use case, it's AI, even more that operating systems, even more than databases. It's kind of the fact of the thing that you do with AI. Yeah, so that's why again, what's happening for a while is very paradoxical, but I think it's a matter of time and it's already coming. The transition to the rights, right as it to the right equilibrium in the market is coming. Well, I think it's super interesting, because like the thought of a company training its own model doing this has just seemed so complex and only reserved for the frontier labs. I think that's what a lot of enterprises assume right now. I'm curious, before we jump into that part, because I think it's super interesting, and the fact that you can actually build this stuff in the platform, was there a moment in time where you realized everything the monologue you just gave me? Was there a moment in time where you were like, "Oh, this is going to be different." Were you still at Google when this happened? Was there like a moment you remember where you're like, "Okay, I need to go build this thing?" Yeah, yeah. So, for what is worth, it was at least a year before that GPD came out that I was arguing that he read the next, because I was owning all the natural language services at Google Cloud, that the next thing we should do is text-to-text followed by multi-modal. Text plus image-to-text. That was almost a year before that GPD came out. It was not called Gemini at the time. One of the very first customers in the very early days of building Palm, They, which unfortunately I cannot name them, they asked me about a major organization that I'm sure you would recognize if I were to say their name, they asked me can we get the weights of palm and say unfortunately this is not allowed by the terms of use we can say the weights of the model with you can we distill palm to create our own version of palm internally so we can freely do whatever we want is like unfortunately this is not allowed by terms of use either. And that was very early there was just again only a couple months after the GPT came out and that was the first time that I started thinking okay why are these people asking for all these things right it's a big model complex to build complex to manage maintain do the inference but you know yet they're still asking for more and then that's what I said you know what actually they're not going to be a one off because it kind of makes sense I mean. We build so many systems before yeah these are bigger models more capable but this doesn't mean that specialization you don't get benefits in terms of quality. Of course they may look as if they can do everything they can be good at everything but this doesn't mean they are exceptional or they have the right economics or the right controller enterprise need so that it was actually very early on and that was to give you an idea at least. I don't know generally of or February of 2023 just again just a couple months after the GPT came out that I was starting to realize this and yeah we're still very deep down into launching palm. Did you think this moment like where we are today did you take this was going to happen at this scale I'm talking just broadly because you know you're in it before even the chat GPT moment you know around what November 2022. Like in your wildest dreams do you think we would be here today in terms of the capabilities of the models the proliferation of it I'm just curious if that was even in in your bingo card now I'll be I'll be honest here I mean I didn't expect it would be as big. If you were to go back and look at some strategy talks I was writing a Google I was suggesting that this should be the next thing we invest a close all natural language services at Google Cloud. I thought it would be the best opportunity we had in front of us but by no means I was for seeing the scale that such as technology would have that being said it was not also the first time I was building this technology I had built something like that GPT in 2016 at Microsoft. After a review with to you was reporting to satia nadella I didn't exactly be presenting some ideas to him after winning cadet one week hackathon that Microsoft had say hey what you're presenting is very similar to this effort we want to start. You should join the team I'll give you head count and that's how I joined that effort that we're building a multi model multi model open ended chatbot you could answer questions about everything. We did embedded based retrieval back then that even proposed in prototype myself almost what people call right now yeah but the thing was and to the to give a lot of foresight to you know where is you to all the executives back at Microsoft in 2015 and 16 we called it as a experiment we used to refer to as an experiment because like we're not sure if the technology is there to make it successful. After almost two years it was proven that hey the technology is just not there yet but for me that's why six years later so you know what I could see how people were reacting to such technology six years ago when it was not as good. Now I could see from internal research at Google how much better had become I think we should do this again for me that was the thing we should do this again we did six years ago pretty much or now is the right time to do this again again that was before sensitivity came out. Yeah we didn't that is it took us wild so you're at the moment before even 2017 when the transformer architecture and whatnot yeah kind of playing with the exact exact we were using at the time for generation. I don't know if it's like 70 80 90 it was double digit million parameter LSTM they were not transformers at the time to you. So we're going to get into for those that are listening on Apple or whatnot you may want to switch to YouTube in a second we're actually going to get a look at some of this live before we do that. Give me an executive like an executive a decision tree like when should a company call just for an off the shelf model when should they be rallying to several different models when should they be fine tuning in existing one or when should they be like training. Specialize because I think this is where a lot of companies just get stuck in the minutia of like okay where do I apply what. Yeah now I'm glad you asked because quite often we say build on your AI but I don't want to be misunderstood that you should just go build everything yourself right. I think there's two rules I often say I think the first one perhaps the most important one which is consume rent what is genetic and build a known what is your differentiation. To be more specific let's say for example you're a bank since we're working with many banks if you're doing coding just use cloud code it's really exceptional model at coding and you're not the only one writing coding any other organization across any other vertical doesn't have to be a bank rights code. I'm probably has done an excellent job and us others as well to create models that are highly capable specifically for coding we put so much effort specifically for calling it's almost a specialized model almost for coding so just use that but if it's how do you do finance analysis. Calculate to risk or process mortgages things that you are the expert on you should be owning that AI because how will you can execute that is what we will sooner later differentiate you more and more from your competitors and give you the competitive advantage you do not want to be consuming. If you have a chance or cloud Gemini or GPT for any of these tasks many so it's right first one you cannot be that you're abdicating your right to innovate and you're waiting for them to give you an improved model when you are the domain expert on the field you have all the data operating that business for a long time. You just don't want to do that also your competitors may not do it and then you're going to be in trouble and again you have the expertise you have the data you should be building and owning that AI not letting on tropic or any of your competitors cuts up with you you should be building that your specialized intelligence your specialist your most important IP and mode. The other rule is besides differentiation is again if something affects your your your budget a lot and your margins you need to be using the right tool that is specialized for the job right I mean not using a model that cost you a hundred times more. And this means use the specialized tool if the specialized tool specialized model doesn't exist you should build it quite often this rule ends up collapsing with the first one because if you're doing something at high volume and affects your balance. It's likely that it's also where your differentiation often is so you should already be building that model because your differentiation is there or likely somebody else has built a specialized model like coding and then you just go use that. So anyway say rent what is genetic build and know what differentiates you or what cost you the most make sure you're using the right tool not the genetic one that costs you ten hundred times more than it should. Yeah that said perfectly and just reading it back to there's differentiation there's like the volume of the task at hand and then there's like this complexity element as well as kind of the this maybe trifactor of some of those factors and I'll say for listeners just based on when these episodes launch we had. A mod from wrist vantage on prior and he's literally you know PhD data scientist head of AI starting his own company in the legal field and he's his co founder is like a domain expert in legal and they're tackling this specific use case around procurement and legal and whatnot but they literally you're doing what you're talking about I need to connect them with you because they are fine tuning and training their own specific model like literally very tailored. Use case and it's speaking to exactly what you're talking about that I think this is like this will be the future very common in the future I think as we yes the only thing I would say there is let's say for example your character and Alice. Should you use Harvey's legoras or the start of that you mentioned or should you build your own AI yeah what should take the. I think likely should be building your own AI because. Especially for somebody like the size of Kecklen and Alice you should definitely be building your own AI if it's a small. Harvey and LaGoura right I mean I don't know they don't have their own models either in the summer like I don't know if that's in the road map but they're kind of your bypassing another. I don't know where the day will come that the topic may shoot them and say no actually yeah you know we are providing better legal a better legal model and enterprise are going to them because if they are a thin wrapper on top of on tropic yeah I don't do minimize what they're doing but. The harness or some application and prompt around the model is not going to be a very robust mode and we can get to that in a little bit later but it won't be a very robust mode. That's why to give credit to Kecklen and Alice they came out there's sermon a couple months back and said. There's many models out there right there's the Harvey's the legoras you know all of this and many more what the tropic provides but we are an exceptional legal firm at least that's what we think our customers hire us not. for the floor or you know the versus genetic quality of legal services you can get almost anywhere. So we're not going to use the same genetic AI as everybody else. We have the institution and all this. We have the capability. We're going to build and own our own AI because it's going to be a key tool for us to provide better services. We need to have better tools than our competitors. It only makes sense. There's a massive presence there with them doing this as a legal firm which I think is just super interesting. Yeah. Yeah. So let's actually get into the platform and I'll maybe set this up a little bit while you're pulling it out but effectively you've built this platform to where you can build a specialized model within it and monitoring and production. There's this continuous improvement element. This auto scaling deployments which is very interesting when it comes with scaling with GPUs and whatnot but like take us through an example and for those that are listening to this we will narrate the experience as we go. Yeah. Yeah. So just to give you a little bit of the motivation of what we build UMI. The mission for what we want to build and why we left from building Zemini to build pretty much completely the opposite was that again an enterprise should build on compound their own intelligence. It's going to be their most important IP and differentiation. Pretty much what Satya said in his viral post in X just two months ago. Yeah. And you should do that in a way that gives enterprises full flexibility in no locking and they have full AI sovereignty. That was our mission. Again pretty much exactly what Satya said. What also Alex Karab has been setting a lot in introducing CNBC and beyond. The thing though that was very clear is that and actually we're still hearing from some of the largest organizations out there is that it's hard. They are worried that building a specialized model for a task would take them months of effort and sometimes quite often many of them would reluctantly suggest they were worried we not have the right expertise to do that or maybe we have only a few people that can do that but not enough for all the use cases. So and we said you know a building this specialist say I should not be that hard. There's so many things that you can do to help enterprise to build better AI and why not use AI itself to help automate the development of AI. I tell people right now it may sound novel but I think we're going to be looking back after a year and say why did it take somebody like Umi to propose that why didn't we do that earlier and I tell people I don't know if it's because of lack of creativity of AI engineers we automated everything except for our own job or false sense of self-preservation I don't know what it is but it only makes sense to use AI to automate AI development. Do you see this as a form of recursive self-improvement because I think that the frontier labs are doing this right there. They're leveraging AI to kind of in that you get in this loop but do you do you see it as that? Absolutely there's a lot of elements of that which is how can you make AI understand better where is it falling short and how can it improve itself changing some hyper parameters in its training or changing quite often the data that it's being trained on but offering that as a capability as a frontier lab you could say that any enterprise can now own and operate. Sometimes we refer to this as the equivalent of cloud code for AI development right you can start with a problem that say build a model that does action automates all the steps for you while still giving you full control flexibility to change every single part and component you want. Quite often we describe it as hey this is your frontier AI engineer you know all the experience we had from building Gemini we put into this AI it knows what is the right process how it can iterate in prof quality and automates all the steps for you while still giving you full control and flexibility. So that's the high level idea. Next if you want to happy any time to also show you a quick demo about how it works and she can get an idea. If you want pull it up maybe we take a peek at it and obviously there's probably some elements that we can't cover in the podcast given the time duration but it could be cool to kind of see what this actually looks like in reality and while you pull that up I'm curious you're talking about opens and this is open source I'm curious like what went into your decision to open sources it's already over what 9000 stars on GitHub how does that decision play into this what was your thought process and saying okay we're going to open source this and what is the I guess the business model look like. Yeah yeah great question who curious yeah to be very very specific what you see right now you could say to be very specific it's open core it is built on top of the open source foundation and open source library that we have on GitHub over 9000 GitHub stars as you mentioned used by many organizations and on top of that we build this platform that you now see that exercise all the capabilities of the open source but put it into an enterprise platform that helps enterprises own all their assets govern them and the most important part automate the development which is the thing that makes the capability much better. Usually the way I describe the tube for people is if you are an AI researcher likely you want to use the open source even though we have some researchers that are using their enterprise platform but if you are an AI engineer that time to value matters then definitely the enterprise platform makes sense right yeah I mean yeah I would say the similar thing if you are doing research on how can you write better code maybe you should write it yourself not use cloud code but if you're an AI engineer you should use cloud code right to write your code it's almost the same story. The easy but the harness for for doing this in itself. Exactly sometimes we say it's the first of its kind AI development harness yeah it's because there's no industry precedent we do is yeah we've been describing perhaps in three different ways at the same time but yes. Yeah yeah nice so the given yeah let's keep working yeah when you go to me has you know things like here's your data your models the evaluations the deployments that you can do but the key differentiation is I mean besides having an AI technology platform that you can really find is the again the equivalent of cloud code for AI development you can say build a model that classifies bank customer support tickets based on category and urgency and you can start just with this prompt and hit go and then the agent that we have will engage with you in a discussion to understand make sure it understands fully the problem you want to solve and walk you through all the steps that you need to develop that model so we'll see the first point was okay face I want to clarify the requirements so I can build the model for you what about these different types of categories because you didn't tell me what categories you care about or what about these different levels of urgency and you can say yeah this look good you can give natural language to change things and it will continue going through this process from make sure and fully understands a problem and that starts walking you through all the steps of an AI engineer to be reasoning that through this and figuring out what things are needed to help kind of build a specialized model in a sense based on whatever prompt you give it in a sense right right just in the interest of time I'll show you another example which I had run before and actually I iterate quite a bit with the agent but the way all of these took me I don't know 15 20 minutes of my time but after he asked all the questions he said okay here's the problem we're solving the task automatically classify incoming bank customer support tickets the input comes as a raw customer ticket text there's no preprocessing and you're going to use this for an automated three-ass system so you told me and we are I suggested that the output should be in jation because it's going to be automated process the categories are going to be this and the different types of urgency levels are going to be this here's the steps first you define the value eight doors meaning what are the different metrics that define success your value success create your test data evaluate the baseline model which is what is the model you're going to start from then create training data to improve that baseline model find you not baseline model evaluate the model you just trained and then see how you can keep iterating and going if you want we can go in all these steps but it will say suggest you whether the right evaluation metrics then it will tell you how to create test data it will do the valuations for you and just going super fast and I'll show you at the end some other key points because we could easily make that demo last minute I don't know half an hour if we want to right all the type of this is exactly this is for the step for example where it already built for me the test data and everything yeah the value the baseline model where it said for your classification task a quaint 3.5 four billion model is good enough here how it does in terms of if you look at let's say for one score supposed to accuracy compared to sonnet it's a 63 comparing to another model effectively so you kind of exactly how does it stack up versus maybe what I would use frontier model of the shelf exactly exactly and you can tell it compare with zp team the big one or opus you know you can ask you to compare it with use at best out with any open or closed model but you can see the the quaint out of the box is not as good a sonnet right it's quite lower but it analyzes for you whether the problems and then when it goes in create training data it automatically creates training data to fix the quality problems it tells you how we'll create this data actually also even later how exactly it shows you the detailed recipe sorry maybe it's here how it's creating it's training data as well so you're not I guess I'm assuming you can supply your own training data that exists but it can also create as well exactly exactly so even before you start this process you can say you can upload any of your data structure or unstructured it could be just road documents and say you know here's example data that I have small amount of volume or large amount of volume and if you have large amounts of volume it will clean it up or improve for you have small amounts it will use it as the foundation to build more than expand on it. So it can work flexibly with any of those data regimes. But anyway, it will create the right data. I will train it for you. I will then tell you what is the right way-- sorry, after it creates the training data, let me go forward forward. I will tell you what is the right way to train. If you should use parametricifying tuning or some other approach. And you can see, after you train the model, this is from 631277. So it's better than Sonnet now on the category classification. For agency, it's still lagging behind. It's lagging behind Sonnet by four points. And so here's some issues that I still found. And I'll show you later how you can even go deeper into those quality issues. And I'll say, OK, what do you want to do? And so OK, iterate to improve further. I selected that option. So OK, I'm going to create new training data that addresses the failures of the model that's just trained, fine tune it again, evaluate, and then show you the results. And after it did all these steps, again, this is the last run, where you can see even the agency F1 score. Now it improved from 64, that was the first train. 56 was originally. It went to 64. Now after training again, it went 72. And now you have a model that is beating Sonnet. And it's costing you, I don't know, 10 to 100 times less than Sonnet to operate. And literally, all this process, it didn't take me more than 15 or 20 minutes. And this is the agent experience. Some key things I want to mention is that you have full flexibility and transparency for every step. When the agent tells you, here's how I'm going to evaluate. Here's exactly I'm going to synthesize data. He's exactly how I'm going to train. Everything is recorded as a recipe. For example, for the data synthesis, that was the recipe that the agent used to synthesize the data. And you can go inspect this spec. You can use the agent here on the side if you want to tell it how to change the recipe for you. Let me maybe close again. Or you can even go hands-on keyboard. This is our open source definition that we have for the library in GitHub that you can have full access to all the parameters, everything. So you can have full flexibility, full rock control about how you change things. And the key idea that goes to the open source question just a bit before was that we want to build a product that will have the automation. But it doesn't look queuing. It works with any open model. It gives you any parameter configuration, any option you want. And even one day, you can get the models, the data, and even the recipes for how every step was executed. And download and run it against the Apache 2 library. So you can have full control. If you don't want to talk to me again, you don't have to. There's many other fields I could show you. For example, you can see this is the Sonnet model compared to the fine-tune model, the version 2 that we built. I mean, you can show you all the data evaluations with the scores, all the confusion metrics, all the things typically you want to see. And a feature that strongly resonates with people is failure modes, where we tell you not just for the cases, the 50 samples that we tested, which one is failing. That means if it's failing in multiple tests. But what are common issues that we found? For example, there is cases where fraud is getting wrongly categorized. And then it gives you an insight about what are exact examples and what are common problems that I expect to see. So this is how you build the model. Maybe to wrap up the demo, the thing I want to quickly show you is the following. Actually, maybe, let me quickly show you this. Let's say we also help you quickly test side-by-side the model that you just built compared to another model. For example, the fine-tune coin model compared to OpenAI. And let's say you try with it side-by-side, just for the-- Exactly, let's a quick test. And let's say, for example, you say, I can access my bank account. Not just a quick sample. You can see this is the train model. And this is OpenAI. You can see how much faster the train model was. And if you look at the cost-- I don't know, you tell me. I think this is about two orders of magnitude cheaper than-- Lots of zeros after the decimal, yeah. Yeah. OK, now very last one to complete the demo. One thing that is very important, also Satya said in his post, is when you build this AI, you should have it compound as your IP. And that's actually something that we launched just last week, where when you have a model like this that you have built, you can create a deployment. And just take a couple clicks, check a deploy model, and you select it. One once you have deployed the model. Like, for example, this one, we show you graphs. Let's say, for example, you can see the last three hours of your traffic, how many requests we're going through. What was the-- all the typical performance stats. But the most interesting ones is the insights about what is happening in your model. You can say, OK, out of all these requests, I had in this window, I want to select this window and run failure mode analysis to see whether the common-- how-- what was happening in production? Where was my production model failing? Actually, there's one that I run literally a couple hours before meeting here today together. And that was this one, where, for about 200 samples that I issued, it shows you which ones were failing. Again, it shows you the failure most. Like, for example, for category evaluation, there were cases where, actually, the ground truth was mislabeled. So the model did the right thing, but it was mislabeled. Or cases where, for example, very important ones that one should know, where fraud cases were misclassified. Where-- And this could be fed back in to further improve the model. Perfect. Right on. So all it takes is you can click on them. Or you can even click on those as well and say, generate now a data set out of these failure modes. And then you can use the data set. Actually, it just takes a couple of clicks. You say, OK, generate that data set-- or actually, maybe I want to clean this, because it will take a couple of times to execute everything. But you can go generate the data set and then add it to your existing training set or however you want to handle it and train. So literally, it can take you, I don't know, three or four clicks, no more than two minutes of your time. Or even a minute if you have done before and you know where to click. And you can immediately train a better model. Actually, I did this also less than 24 hours ago, or maybe just a couple of hours ago, just to show you. And the model that I trained was this one. So after I trained the model and I deployed in production, I evaluated with my production data to see where the quality was. And that was the failure model you were also seeing before. It was based on this sample, a lot of those. And then I clicked on the buttons. It says, create for me, analyze the failure modes, right? And after you have them, click on the button, select all of them, create a data set, append to the existing training set. Now, train again. Literally, four clicks, it doesn't take you more than a minute if you know what you're doing. And then it trained for me a better model that you can see again, the scores from 84, became 93, from 76, became 86. Again, this is not a cook demo. It was a real use case that we just once sorted into a demo to show you today. So it was not something that we had to try 10 times, just in case we can get some good numbers to show you today. It was exactly as it works. - So this is really cool. It seems obvious that just using AI in this way, to build AI, it feels like the way I'm working now every day with everything else. So it just logically makes sense. - But I'm curious, when it comes to, you see you built the model, you're deploying it. - And you can actually, I'm assuming do that in the platform as well. What does that look like? Are you then accessing this model? You build via API that you can use in other ways or run on your own hardware, I guess, whatever you want to give in a set? - Yeah, good question. So it can work in a couple of different ways. For one, when you, after you have trained the model, again, it just takes, we say one click, it actually takes two or three. But you can click and say here's the model, deploy it, you can type a name for your deployment if you want to call it something. You can select the number of GPUs you want to deploy it. You can say five GPUs, so you can say zero to 10, in which cases if there's not traffic going to your GPUs, they cool down, you don't get sards. If there's a lot of traffic, you can scale automatically out of the way to 10. This is one way where we do one click deployments for you and you get an open AI compatible endpoint, so you can literally, literally no exaggeration, move from your GPT-clod or Gemini endpoint to an endpoint powered by the specialized model that took you 15 or 20 minutes of your time to train. You missed it a few hours, but only 15 or 20 minutes of your human effort into a new endpoint and you have a lot. Now, because again, we strongly believe in AI sovereignty, we make it possible so you can extract and download all of the assets. You can download the model weights and run them in your own local hardware with open source library that we have on GitHub, we've discussed before with your data sets, the recipes for how every step was executed so that you can have full control. So because if you ask me what an enterprise you have, do you need to have all four? Because if you don't have all four, you're still getting logged in to somebody's platform without you realizing or without them telling you. - So curious, let's take a step back and look at this so this open source future and potentials becoming more and more obvious. Like it's, you know, maybe it's already based this exists with hugging phase, but kind of a marketplace where people are using a tool like this, building their own specialized models and selling them or open sourcing them or whatever it may be. What do you think the future looks like between open source and the frontier labs? 'Cause some people think that, oh, the frontier labs are screwed and open source is gonna take over others. I've heard which I think is more logical is along the lines of both will live, right? And open source may get the vast breadth of, you know, token usage, but on the monetary side that may be where the frontier labs still live. 'Cause they're still on the frontier, holding the best of the best. What do you think that future looks like? I'm curious, this way it opens for me. - Yeah, so yeah, I don't think it's a, you know, we shouldn't say that's a false dichotomy as you mentioned. I think there is a future for both close frontier models and open models. If anything, I tell people for consumer AI, likely this frontier models make a lot of sense, or especially if you're building something like ZZPT, you want the model or Gemini app, or something that can respond to any question about anything. - Yeah, you need something that understands pretty much everything. But again, for enterprises that have very specialized use cases, internal institutional knowledge, again, it's their differentiation, it's key to their success. It only makes sense for them to build own compounded, their own intelligence. As we've been saying for a while, as also Satya and Alex Harff have been strongly advocating very publicly and very loudly lately, some very glad they are. So, and at the same time, you know, also for some enterprise use cases where companies like Anthropic build highly specialized models, like Cloud Code, I think it makes sense to. So, I think there's going to be, again, between consumer AI or different specialized models at these frontier labs, the Velo, again, like coding, that they're going to still see a lot of users and success, because the market is expanding, they're going to still most likely see their revenue increasing, but not at the pace that many people are anticipating, because it will make sense for more and more enterprise use cases to be built on top of open source AI. So, they can be both the articles and their postgres sequels. - And then you look at Nvidia too, with NivaTrad, they don't like to talk, if hype it up a lot, it's their biggest customers on the frontier labs, just another player in the space. So, I'm curious about this. Like, we just saw, you know, what you're building, and I'm assuming your team is small, like most native organizations that are starting up today, and like, I always go back to the Jack Dorsey hierarchy to intelligence and what he's building, and just rethinking what an organization looks like. But I'm curious, can you compare for me what it looks like in your organization today, versus Google when you were there? And how do you see the future of the organization evolving as AI just proliferates? - Any cues into how your teams are using AI, how you're structured, anything that's interesting that, you know, folks listen to take from. - Yeah, yeah. So, I think in that article, the dimensions like Dorsey makes some good points. For example, I do agree that if you have AI, most of the things are now recorded like meeting notes. Everything gets recorded, right, with AI transcription. Then the information sharing role that the manager will have gets reduced, that being said, I think what information sharing managers do is a little bit more nuanced. They don't just pass any information up and down. You know, they, so I think it's a little bit more nuanced what perhaps is in that article. But overall, what I see with I think is, you could say perhaps a little bit more. Obviously, that whether you're a software developer, designer, product manager, or manager, CEO or non-seo manager at an organization, you can be made more efficient. You have tools that can help you become more efficient. So, before you're writing a sum amount of code, now you can route a lot more. If you're designing some pictures, now you can design many more. And a lot faster, if you're managing people, you could actually manage more people. I do think there's some, a lot of human elements that cannot be scaled as easily. But I can definitely see that a lot of the work that I was doing as a manager could, I could get a lot of help with AI to be able to support a larger team. That being said, the other thing that we see, it's a little bit of some sort of collapse of functions. For example, the most critical features are still designed by our designer because she can do the best so possible, better than AI. But is she using AI in her process? - Yes, she is, she is. - Okay, yeah. - I mean, he's using it, I think, yeah, below. - That's the element where he has the overuse term taste, but I'm sure he has that in a sense of where he can apply that and get more from it in the context. - Exactly, exactly. And it's not like he gave a blank check to Claude and say, now go design this new product. He built the design system, I think as he calls it, the whole framework. And then Claude has a structure environment that fits our brand, fits what he likes and what he thinks is the right design approach to go build more things that are aligned with. So if you have a junior or designer and you tell them here's the guidelines, now go design this new feature based on these guidelines and these components that already bring this design for you. And that's how he can now, he's much more productive. And even now for small features where I said about the function collapse, for smaller features where it's a little bit more predictable how the design would be, engineers don't wait for the designer to tell them how exactly to do it. They go and build it. And if then our designer, Constantinos has any VM and reaction to it. He may tell them, okay, now we should go change it and fix it. But there's so much more that we do, again, because Constantinos can scale up his own work or engineers for the work that's easy, can just directly get it by consuming the same design framework that Constantinos has put in, but at least they know how they can operate on their own. We have ML engineers that go right, a lot more front end code to complete the end-to-end experience of the features they're building. Before they wouldn't go build front end code because they would be so slow and they wouldn't like the idea, perhaps, of writing a little front end code. But now they do. So. - Yeah, I think it was a Boris Cherny that was talking about this on a recent podcast or talk or something. But he was talking similar to what you're saying. And it's like almost he's seeing this convergence around function or purpose where there was, I forget the terms he used to be like, there was the maintainer, there was the builder, the prototyper. So they're kind of fitting in these purpose built roles, but they can do way more stuff up and down the stack. But they have like, they have, they still have a lane, but it's not specific to like, I'm a front end developer, I only write copy or I only do this, you know. - Exactly, exactly. This is why the monitors have become a little bit more flexible. And again, especially when it's not the most critical design or the most critical product decision or the most critical front end to develop that needs to be designed in a highly efficient, specific, safe way. And it's more about, okay, let's do a little bit more of the same. Yeah, and again, ML engineers go do that front end, based on the design framework from Constantinos to Claude, and they're using the UI components built by the other front end engineers, but Claude helps them reuse that to build exactly what they need. - Do you see any downsides to this or negatives? - Yeah. - In just in terms of organizations, you know, I can see some overdependence maybe in the, I don't know, I'm curious if you see any negatives, either emerging now or you think will emerge. - There is definitely, first of all, for one, there was, which I don't know if it was a negative, but it definitely got some cultural adjustment within the company to say, hey, you know, it's not just front end engineers writing front end code, it's not just Constantinos dictating every single pixel and design how it's gonna be, right, our designers. Because Constantinos are designer, which I wonder whether or not. - So, so it took some cultural adjustment there, but everybody, I'll say, that's the right way to do things, right, we can be all more efficient. The other one, let's see, in terms of downsides, there's definitely some dependencies being built. We have though push hard engineers say, don't just submit code for a poll request when, for a code review, when you don't know what the code does, right, I mean, you can say, "Cloud wrote that, I don't know what that code does," right, this is not okay. So, I think if you manage that, and also if you have this coding agents operate within a framework that you define, and you put a lot of effort into how you organize your code, so that Cloud Code is gonna go to use your components, it's not gonna go duplicate and triplicate and, I don't know, what's the word to. Do four times, five times the same code with a small variant, because it doesn't understand that it's the usable components that you can just replicate or adjust. You add up with a lot of, you know, inflated code that's hard to maintain, that people don't and deeply understand and then becomes a problem, but then we put a lot of effort to make sure we use AI quite aggressively, but at the same time in a way that we're more maintainable and we know what's happening, and any automations happening within our control and the frameworks that we define. - Yeah, and maybe the last question, is kind of sideways in, so I'm building my first iOS native app using codex, like this is what I do in my nights and weekends now instead of playing golf or doing other things like I used to, but it's, you know, very interesting to see it working, like, 'cause now it can access the simulator, it's running through and testing, and actually going through the flow of something, it has its own built-in browser and terminal where it's actually going and doing things in there and computer use has gotten so much better, especially when you're pairing it with a really good model, but all that to say, I feel like at times, the harness is almost just as important or potentially more than the model itself. I'm curious how you think about the harness versus the model and those two things working together. - Yeah, I mean, you need the body, not just the brain, and that's why I see the harness as, How can the model execute an interface with it? world, right? So you definitely need it. The thing though, because I've seen in the news recently about, we asked before about model routing or harnesses and all these things, all the values in the router or the values in the harness. Yeah, there's definitely a lot of value. There's definitely things you have a, you know, a better harness. You can improve the viral quality of the model. That being said, I think why many people were focusing on this and forgetting about the brain is because they had the misunderstanding that adjusting the brain and improving it, was impossible for them. So that's why I think you should definitely improve the harness, makes a lot of sense, making sure you can fit all the right data to the brain and make possible for the brain to actuate, to action on the world in the best possible way. But yeah, don't forget about training that brain because that's, that's even more important than anything else. Yeah. So to wrap things up, I want to try something new, calling it the three prompts. So it's kind of like a fire round. I'm going to ask three things. Let me know your thoughts here. So first one, and we're going to ask it to every guest. I'm just curious to see the difference in opinions and whatnot. But what's something, what is something cooler and usual that you're doing with AI right now outside of, you know, building it, fighting models with AI with you. So anything, it's going to be in your personal life, within work, anything interesting you're doing with AI. Man, you already disallowed me from using the example I want to use, but we'll say the most, I was more excited about them, enabling every organization to build its own AI using AI itself. That was together. So, yeah. So, no, but I don't know that I would say, how many? Like, many people, right? I mean, I'm using now AI for many, many day-to-day tasks I want to do from helping me through on an email. I'm going to write to analyze a legal contract before I send to my attorneys because I want a quick response on something. So, I wouldn't say anything terribly exciting, but definitely using it aggressively for almost anything. Just if you can make me a little bit more efficient, it's, I'm definitely down for it. So, my wife's in the floral industry and they still write things by hand for stuff they have to do and she was doing it other night and it was driving me crazy. Looking at that, I was like, I can build a system to do all of this for you and she won't let me do it. What's one thing most people are getting wrong about AI right now? I think we discussed that. I think, if I were to pick one, which is starting to change, but most people still get it wrong is that that renting AI is the way for you not to fall behind, that the way for you to succeed, that the AI strategy for your organization should be renting and consuming the AI that somehow this few handful of people only can deliver to you. If I were to choose that, I think that would be the most important one because this then dictates all the strategy for the organization and I think whoever sticks to that strategy is going to be disaster for the organization sooner or later. And this may be the synopsis of the conversation, but last one, what will feel completely normal in two years that just sounds strange today? I think it's a good say where you say, yeah, building and owning your own intelligence. I think I'm going to be looking back and saying, how come we thought that our most important IP, it was okay for us to rent it or outsource its development to some other organization because right now this is happening a lot. FD companies are getting a lot of success or people still renting these large models. I think we're going to make it back and saying, how do we go so long and we thought that our most important IP, it was okay for us to rent it or outsource it to somebody else. What else, the opposite of, no, we should build a known and continuously improve our own IP our own intelligence is going to look as the extremely obvious thing. In two years, we will reset, reconvene and see what the world looks like. Thank you for jumping on talking to me. Where can people learn more about UMI and finding contact with you? Yeah, I think if you go to www.umido.ai, we have all information about the product demos, a lot of resources, and also a link where you can go for free to get started, test UMI, even credits are on us to get started for now. And yeah, you can see you're going to be able to build your own model literally in minutes of your human effort. That's awesome. A lot of it. Manas, thanks for talking to me, I would be. Thank you very much, Matt. It was wonderful to be here. Thanks for listening to the Talking AI podcast. If you enjoyed the show, give us a follow or subscribe when you finish podcast practice. And don't forget to leave us a review. You love next. For more info on talking AI, visit TalkingAIpodcast.com. Quick break in the pod. I keep you're in the same pattern with companies I talk to. Clots helping employees move faster, but in many companies, the business itself hasn't changed. The value still trapped in isolated chats and experiments. And that execution gap is why four deployed engineers have become one of AI's most talked about deployment models. They have been with your team instead of advising from the outside. It's also why the FDE model is now central to every client engagement we lead at HATWORKS AI. As an official anthropic partner, we embed anthropic certified FDE's to identify high value business problems, build and deploy the solution and put governance and security around it, then transfer the capability back to your team. If your cloud rollout is still mostly individual usage, check out how HATWORKS AI FDE's work. At HATWORKS.com/claw-FDE. You can also find it in the show notes. Now back to the show.