Go back

From Hype to Impact: Deploying GenAI in Real Businesses with Chris Taylor of Fractional AI

47m 23s

From Hype to Impact: Deploying GenAI in Real Businesses with Chris Taylor of Fractional AI

Fractional AI, founded by Chris Taylor and his co-founders, operates as a generative AI transformation partner, assisting companies in deploying impactful AI projects into production. The company emphasizes a collaborative, in-person engineering culture in San Francisco to tackle complex projects through teamwork and shared problem-solving. It focuses on enabling existing non-AI businesses—particularly those in the lower-middle market—to leverage AI for automating workflows, improving operational efficiency, and enhancing customer experiences, arguing that these firms have a head start due to their existing domain knowledge and data. A significant part of the discussion addresses the challenge of AI hallucinations; Fractional AI mitigates this by establishing performance baselines, implementing evaluations (eVals), and developing custom AI agents tailored to specific tasks to ensure accuracy and reliability. The company typically begins projects with high-performance frontier models for maximum accuracy, then optimizes costs by evaluating trade-offs and potentially integrating smaller or fine-tuned models. The conversation also highlights the need for ongoing model management and cost optimization, similar to maintaining traditional software systems.

Transcription

9146 Words, 48643 Characters

English
♪ Another private equity fund cast ♪ ♪ Is about to begin ♪ ♪ So give it up your host ♪ ♪ Devon and Jim ♪ - 5, 4, 3, 2, 1, Thunderbirds are go. Welcome, Chris Taylor. The private equity fund cast, CEO, Founder and CEO of Fractional AI. Welcome to the podcast on this first Monday after spring ahead. So we're all a mighty chipper this morning. This is the day if you have pets or children, it's a bad day. Nobody's comfortable getting up this early. The dogs don't want to get up, the kids don't want to get up. This is life. So welcome. So we just talk a little bit about Fractional AI and your background, Chris, and how you jumped into this. Why you decided this was a thing to do now? - Yeah, so Fractional AI is a Gen AI transformation partner. We meet companies where they are on their AI journey and we help them get high impact Gen AI projects into production. Core DNA for us is our engineering team. We have a room full of A+ engineers all working on Gen AI workflow automations and new product development, building up best practices in this domain and figuring out how to wrangle this magic hallucinating ingredient into reliable production grade systems. And a little bit about myself and the journey. So I was on the early live ramp team back in 2012 where I met my two co-founders Eddie Siegel and Travis May. Three of us really enjoyed working together back then. Had a lot of, we've started various company since, had various exits and then at the beginning of the year, we are beginning of last year, sorry. We all had the opportunity to start a company together and we thought, you know, what are the biggest winners of this whole Gen AI movement is going to be existing non-AI businesses that can just use this technology to improve their operations and products. And we thought, hey, we actually have the right skill set to start that business and start a Gen AI services business and really excited about jumping in together and it's been going really well. - So I wanna poke that for a second, but just I think it didn't come clear in this conversation but in a previous conversation, one of the things we had talked about was you had all your guys dumped into one big room, which is counterintuitive to everybody doing the whole work from home thing, which, you know, look, I'm a lot of time software developer allows you one, but so it's always good if there are people around me. But you get some energy at a beat in the same room, talk a little bit about that 'cause I think, you know, we're in this world of, oh no, you should work from your house as opposed to collaborating together. Talk a little bit about the decision behind that. - Yeah, so the whole team is here in San Francisco working out of one big room, like you said. The decision really came down to one of these projects are gonna be really hard. It's a lot easier if you can tap your neighbor on the shoulder and ask for help when you get stumped or just brainstorm together, go grab a whiteboard and diagram out your solution. And so the thinking was, let's put everyone in one room, have them all working on different projects but create that collaborative work environment where you can problem solve together, learn from the best practices that everybody else is developing. And we take good advantage of being in one room. We do sessions about what's a week, we do lunch and learns and someone presents on a project that they're doing. And when you just go into that room at any given time, you'll see engineers looking over each other's shoulders and pointing to computers, it's a fun work environment that we have. - So interesting, so what we sit, we're buying a founder on software businesses, right? Lower middle market. So we think we're in the sweet spot of companies that can leverage AI. And this is interesting, you're in the belly of the beast, so to speak, and that everybody out in the West Coast is really developing AI technologies opposed to using AI. And we're of the strong belief that, hey, that's unbelievable that that works going on. But we're the biggest beneficiaries. I wouldn't want to be in the LLM business or tooling around AI, but I love being in the business of, I've got relatively small teams and we can use AI to make those teams better, more efficient, et cetera. So I think that's one of the things that you saw as sort of a hole in the market, is that right from a wire attack and the market where you're attacking it? - Yeah, I think we saw similar things. I mean, when we were thinking about what's the business that we're gonna start here, the need that we saw was specifically around, how do you best leverage this technology in an existing business like the one you're describing here? And what we see with those businesses, they have a tremendous amount of workflow knowledge of deep domain knowledge and how they do their operations. And that knowledge is critical. And in order to then automate those workflows that they're great at, automate the things that those businesses do well, they're missing the core tech DNA, the Gen AI knowledge to be able to do it. But our viewpoint is basically, if you're gonna start an AI first business, trying to tackle that, actually the first thing you would have to do is you have to recreate the entire business. So I think those businesses are very well positioned to capture the benefits of this technology. They just need to sprinkle a little bit of that tech DNA that Gen AI domain knowledge on top of their existing process knowledge on top of the existing data assets that they have in order to build the workflow automations themselves. But they have a massive head start over the startups that may be coming for them from Silicon Valley, I think. - So do you see, so given our size, I look at your customer list on the website and I think of very, very large companies have super and here I'm using my hands like people can see it on the podcast. Pretend you saw it, Jim put his hands apart. We see large companies have very complicated workflows and lots of document processing, right? A lot of, probably a lot of still manual processes. At our segment of the marketplace, I would say in general, we don't have as many document kind of processing problems. We have a ton of data problems, I would say, in our businesses, right? So we've seen a couple of use cases in the lower-mid market and be interested in seeing what you're seeing the same thing at the high end or how people are attacking it. We see at the low end of the marketplace, we're seeing, so this is my standard joke, you'll laugh at this Chris, but when we ask a portfolio company, what's their code coverage for test cases? They always start answering like you tell your dentist, when he says, "Thea floss," and you say, "Oh yeah, I floss every day." But dentist knows you don't floss every day, you know you don't floss every day and then negotiation starts and you usually end up around 30%. I mean, a third of the time, I may actually be flossing, right? So we have the same problem with code coverage. One of the simplest things coming out of the gates was, hey, let's use Gen AI to basically create test cases for old code, right, to sort of stabilize old code, which makes perfect sense, 'cause in a lot of cases, companies will found their own, they start fast, test coverage is something that always seems to be let's sort of left behind at the smaller companies. Despite what you go see what Google and Microsoft does, guys, a $20 million software company doesn't have those resources, right? So they do what they do. So that was a good test case, right? That's sort of step one. Then step two starts to be, what junior developers using it to develop, to generate code, right? Well, hopefully within the skeleton of what they're doing. And then we start to see the processing. It feels like the large corporations, it's a little bit of a layup that they have lots of document processing, even if they're a document company, they have a lot of complicated workflows between departments. That seems like that's always gonna be the best low-hanging fruit to start a project. Would you agree with that from a large company or do you see them doing more sophisticated stuff out of the gates? - I think most businesses have some low-hanging fruit in that area. One thing I try to encourage people to imagine is picture a world five to 10 years out where all boring paperwork tasks, anything that resembles boring paperwork, is done by Gen AI for us. And then imagine what are the things that just disappeared from my company? Like what are the tasks that just got off-loaded the Gen AI? So that's a great starting point. But it's not just limited to internal processes. Like one of the things that we also take a look at is what are different ways that, what are different workflows that your customers are going through that are onerous? If you have a client or a customer that's entering a lot of manual things on your site to get set up with a new account, there's a lot of opportunity to make your customers lives better with this technology. If you can infuse it into their experience, in the right way. - So let's talk about that a little bit because again, from an outsider's perspective, you hear a lot of people will say, we're worried about the hallucination. And if people are familiar with that term, it's when sort of AI is like super eager to be right. So even if it doesn't know the answer, it wants to give you an answer as kind of the hallucination. Like it'll make up an answer if it doesn't have one. And so how are your customers or how are you guys thinking about this from, hey, if I want to get rid of the person out of the process entirely, particularly if it's boring and not, think of it as the document processing or data processing like an old factory, like you're just punching that thing day in and day out. It's not a job anybody wants to do. It's a job that has to get done. How do you handle that AI? Have you seen that in practice? Like this hallucination effect, what's the percentages look like? And how do you think about dealing with that? - Yeah, so the hardest part of every one of these projects, is one, just getting your head around, how well performing is the existing system? No system's perfect. And so for your standard workflow that you've off-shoreed to a BPO, let's say, there's some accuracy threshold that it's hitting. It's not 100%. And how do you define accuracy? And defining it is challenging. So step one is kind of establish a baseline and understand how well is the existing system performed. And then also we have to get our heads around, OK, what are the key points of this process, the input and output pairs, and getting a clean data set to work with. And so you're trying to get to some source of ground truth early in these projects or as close as you can to it. The next thing you're trying to do is set up eVals. So eVals or evaluations, the idea there is you want to put in place the right metrics to tell you how accurate is this new system that you're building. And eVals can include things like, what's the hallucination rate? And what rate is this system just making up information and adding it versus an error rate? And you can measure each step in the process, break a process into all the different sub-components, measure each sub-component. And then ultimately, if you have a great set of eVals and a great set of ground truth, you can answer these questions with very easy metrics. You can just say it like, all right, here's the exact accuracy percentages of the existing system. Here's the accuracy of the new system that we built with Genai. And in many cases, the new system outperforms the legacy system. What we find is these projects go through this life cycle where early days when you build a system end to end the first time, it's going to be OK. But it's not going to be nearly as accurate as the existing system. And then after iterating on your prompts, iterating, you might find to do a model on this step, you continue to break the process up into further sub-steps to get more control over it. And eventually, this system is going to outperform the existing human-based system. And when you put it into production, you're either going to be on par or oftentimes actually have a higher performing system at the end. Yeah, and the interesting part is that corporations actually have that level of discipline as they look at projects. Because we see so many metrics that are just met-- sometimes they just created just so you have a metric to compare against, which doesn't really help you. I think one of the things we've seen is, if you're looking at this as the external agent approach, where I think a lot of people think of AI at the moment is, oh, I'm an agent that I can ask questions. We see a higher hallucination rate when you can ask it anything you want. Most of the systems we see coming online early are the AI is embedded in the back end of the system. It's not directly accessible by the customer, right? So you've got some sort of layer that says, look, I just can't ask you anything. I'm asking it domain-specific things, which in theory leads to less hallucinations, right? Because when you see them, you can kind of knock them out. It's been our experience. I don't know if that's been similar experience for you. 100%. So say the work that we do is in building these custom agents. Essentially, and think of a custom agent as an LLM system that is doing a specific task. And when you've narrowed what it's trying to do to a specific output in a specific task, it enables you to do the exact process that I outlined before of putting in place evils, understanding how well it's performing at this very discrete task, as opposed to a general purpose system, which is much harder to do all these things. Yeah, I mean, I think you would think people had learned that from voice recognition. We've been doing that for such a long time. And nuance, which was discrete speech, was easier because it's basically the United Agent thing. Am I talking to James? Broom, or somebody who represents James? Broom, broom, broom. You only have certain things you could ask an airline system. Otherwise, it was like, you could ask it any day that you wanted to say, I have no idea what you're talking about. People sort of forgot that lesson and going into AI. It's sort of a similar approach that you need to narrow the inputs in order to control the outputs, which I think is sometimes counter-intuitive, because they just see, oh, wow, I could just throw chat GPT up there. And away I go. So you and I talked about this a little bit. We've seen several different approaches. But you all start typically with, hey, use chat GPT or ontropic or something kind of mainstream commercial grade for sort of the first-- getting the first party off the ground-- as opposed to trying to go find an airline that you like. Is that a fair summary of how you think about going after the market? Yeah, in terms of how we build these custom agents. Yeah. Yeah, so I would say generally what we do is we'll reach for the frontier models first. So your anthropic, your open AI, best in class models. And the reason being, those are generally the highest performing models. And what you're trying to do early in the lifecycle building one of these custom agents is just get accuracy as high as you can. And so reaching for the best models to get accuracy higher makes sense as a first step. Once you get towards the end of the lifecycle of your project, let's say you get the accuracy high, maybe you've even gotten it higher than your existing system. And now you're ready to put the thing into production. You need to look at cost. And the variable cost of running that new process end and can be much higher if you're using the state of the art frontier models versus maybe the last generation. And so what you can do is you can look at the cost of that whole process by step. And it might have seven different calls to LLMs in that in the pipeline. And you can look and see, all right, which ones are causing the most-- which ones are driving most of the cost here? And if you have e-vows in place, what you can do is you can now swap out models at different steps. And you can make accurate cost accuracy trade-offs using-- informed by your e-vows. So you can say, all right, great. For this step, instead of using GPT-4, we're going to use GPT-3.5. And now we can measure the accuracy of the new system with e-vows. And we can look at the cost on a variable basis of running that. And then you can make the trade-off decision of, OK, does it make sense to swap that out and make that cost accuracy trade-off? There's other techniques you can do as well. This is one of the best use cases for fine tuning models is taking one of those smaller models, putting it into that step in place of the frontier model, fine tuning it for that specific task. And now you have a small, cheaper model that's fine tuned for that specific task. And potentially, you're getting the same accuracy overall at a much reduced cost on a variable basis. So that's interesting. So here's my corollary, right? Because I think people miss this. So years ago, when mobile apps first came out, the iPhone got popular. And we'd have a portfolio of it and say, hey, we're going to build a mobile app. My first response was always, for what? What's the business case for? And that sounds kind of smart for me. But there is-- customers want to meet you there. Is there a reason-- are there really valid reasons why you have a mobile application? And in the case, we're like, yeah, here's the business case. It's like, yeah, make sense. My following was always, now, remember, once you do this, you're kind of stuck on an ongoing basis, right? Because every June, Apple is going to stick it to you by coming out with a whole bunch of new things that you've got to include in this thing. And I think that's the corollary here is, hey, guys, if you had a database and you had a database administrator, you probably have to think about having a model administrator or somebody who spends their time thinking about this piece of it, right? Because over time, costs can go up. There may be an alternative model that solves this better, an open source model that you can fine tune. You may have to bake off one versus the other between the contract. You can cut or deal. You can cut with that particular vendor. And I think that's the thing that people don't think about. It's like, hey, I picked my model. I trained it. It's kind of going to be there forever. And that's really something I think people don't think about in terms of ongoing. So do you guys get involved in that kind of process where you build a system? They take it over. Then you're jumping back in on occasion as they are, OK, guys, it's time to retune a model. Or should we look at cutting costs out? Or usually when you hand the system off, they kind of take it over and they do it from their own end. Or do you see some kind of hybrid models? So we make the offer. Like, hey, we're here. Let us know if you need any help. And then nobody takes us up on it. It's basically happening. I think part of the reason-- part of the benefit of having great engineers is that you get systems that have minimal maintenance requirements. And then with these LL-UMPowered systems, swapping models in and out is one line of code. It's not a hard change to make. So if there's a new generation of models and you want to try those out in your system and see if it improves your performance, we've handed off the system. We've handed off the eVals. It's pretty easy for an engineer on your team to swap the new model in and see what the performance booth that you get is. And then on the cost front, one of the benefits of putting a system like this in place is that over time, you can get cost savings on day one from it and the costs of these models are going down tremendously. I think the last time I looked at specific numbers for that, the cost of running LL-UMPs was going down about 90% every 12 months. So that same system, if it's costing you $0.10 today to run it on a variable basis in a year, maybe it cost you a penny. If that trend continues. Yeah, it depends on whether you fill in your favorite large monopoly software company that buys out somebody and jacks the prices up. I'm not going to, I've been on both sides of that. I've been the jacker upper prices and I've been jacked with prices. So I'm already going to Hellcris. So I just want to wait and line with everybody else, which is great. It's just their competition. The good news is there's competition. Yeah, for sure. It's not open AI being the sole dictator of pricing forever. They have competition for my anthropic. It's part of the reason that everybody got so excited about DeepSeek is another legit competitor being added to the mix. So hopefully nobody has a monopolistic pricing power on this one. Yeah, you never know. I've been in the software market long enough that the pessimist and me says, if they could find a way to wall it off and jack your prices up, they sort of will. We just call the old CA model. We have A back when which probably predates you. That was CA's famous state which is bi-compes and jacked the maintenance up through the roof from those count. Especially we'll call it dead technology, maybe not the most popular product anymore and they would buy it and jack the maintenance up. Which on one hand was bad for the customer. They had a home for the software. So at least you didn't have to get off it. It wasn't going away, which was something. The pessimist of me always assumes that they're going to try to do that to us at some point. Yeah. So let's talk a little bit about one of the things that's interesting. And again, the mobile carler, I find useful that this is one of the few market changes where even my best engineering teams, and we get some, you know, everybody talks they have the best engineering team. It's like your kids the best and play whatever. But let's assume we all have really good engineering teams. Even the best engineering teams, you know, something new comes out like a new framework, service-eye framework, you know, or throwing it a cube. Most of them are pretty good about picking up something new and kind of rolling it in. I think the corollary for me here is it's very similar to the mobile market that you probably can't teach yourself, you know, mobile development and do a good job of it. Meaning, you might not design the product, the UI might be poorly designed based on what the mobile application is going to do. You don't take sort of advantage of the mobile app. We've seen the same thing here that some of my best teams like, hey, I'm not sure how I dive into, you know, AI because there's just so many things I just literally don't understand. Which I think is, you know, sort of the reason down tray, there's my bad Boston English pronunciation of French, for fractional AI, right? How do you, how does somebody get bootstrapped into, I don't know what I'm doing. Let me start on a project. You know, so typically what does that look like in terms of time, commitment for my team, what do the dollars look like? And at the end, ideally, they're taking it over. Is that sort of a fair kind of idea how the process works? So talk us through that a little bit. Yeah. So I think you make a good point. Like the learning curve is steep. Oftentimes what the curve here are the end to end process. So we'll meet a company where they are on their Gen AI journey. Oftentimes today, what that looks like is, hey, I have an inkling that Gen AI can be used to automate process X. We haven't done an automation project like that. Am I right about that? You know, can you confirm that? And how long is it going to take? And they've never scoped a project like this. They don't have a good sense for time on how accurate will the system be at the end of this? How can I be confident that the investment is going to pay off and that it's going to work? And you know, if you don't, if you haven't done a bunch of projects, it's very hard to answer those questions. I think part of the benefit that we bring to the table is we put a bunch of projects like that into production so we can give you pretty accurate estimates about, all right, this is going to be about a 10 week project or 16 week project. And here's the cost and here's some projects we can point to that are very similar that are in production that give us confidence and do a thorough job of scoping this that you're making an important decision ahead of time. And then for the build, what we'll do is we'll generally meet with the client twice a week. If the client has an engineering team that wants to get up to speed on building with Gen AI, they can join those calls twice a week and they can ride shotgun on this project and they can learn from our team all these best practices that go into Gen AI, the spoke Gen AI development. And by the end of the process, you know, we'll hand off the project. They'll have a good handle on everything, you know, over the individual components and they'll have picked up some of the skills from our team and some of the best practices from our team around developing with Gen AI. And so have you found, have you found customers doing, you know, we see some of these technologies, one of one of two things, either they pick something that's way too simple. It's not going to be a good enough proof point that it does anything or the first kind of pilot project and then you know pilot near quotes is an 18 month project, you know, like millions of people and thousands of lives, which we all know never works, right? Do you get, do you get those kinds of things and how do you get them, how do you talk them off the cliff? Like guys, really what we should look at is kind of a 10 or 12 week project that does this, right? Yeah. So I would say there's definitely definitely some pitfalls along those lines that we've seen. One of them is the like 18 month to, to two year core Gen AI infrastructure investment that we're going to make as a company before we do anything with it. You know, if you're investing in a year data readiness project, you're not doing it right. It's going to, it's not going to be worth it. What we encourage clients to do is really find that what's that quick win that you can get. Typically it's something in the ballpark of two to four months for a quick win that you can get into production. And what it does is one, it's, it's a great, you know, it's a low hanging fruit. It's going to be ROI positive. Two, it gives everyone in the company visibility into, oh, that's what one of these things looks like when you do it well. And so that's, that's one, one piece that we really encourage people to do. For that project, the oftentimes it is some form of a workflow automation. It's something where going back to what we were talking about earlier, it has some form of clean input output pair. You know, you're trying to go from X piece of information input to it wants, you know, you want why coming out of that, that system. And it is not this open ended or just going to build some central chat bot that makes us better at everything. You know, if you're, if you're looking at investing in core infrastructure or kind of a core model for your company that looks like that, I would, I would really press you with questions to make sure it's, it's the right investment for you. Not that it's completely wrong for everybody, but oftentimes, I think companies talk themselves into, hey, we just want to build one system that's trained on all our data that all of our employees can use for everything. And too often everything is a synonym for, I don't have a single use case in mind. And if you're going to expose this thing as a chat bot to people and you're not going to show them what to do with it, chances are it's not going to get very much use. It's not going to drive very much value. Yeah, so that's, that's an interesting point. So let's talk a little bit about, you know, when we've talked about this before, you know, UI around AI, alliteration aside there, you know, is the thing I think we don't know yet, right? Because I think we've seen sort of two models. One is the open-ended agent where you just, you know, can ask it, you know, type in English, you ask any question you want or, you know, whatever your language is obviously. And the other side is it's completely hidden that it's a, that's basically a cog in a wheel, you know, in a processing chain where it's just doing work, you know, in a way it goes. I think we're still, you know, I don't know if you've seen any trends between those two, like, alternative ways to think about a user interface when AI is involved. Yeah. So generally what we try to look for is what is the existing UI that is being used for this process? So if it, if what you're doing is automating something that's already happening, whether it's an existing customer workflow and existing internal workflow, whatever that is, start with the, start with the existing UI and start with the steps that people are taking and the fields that they're filling in and potentially piggyback on that UI and just build the, expose the functionality in a way where it's just completing those tasks the same way that humans are completing them today. An example I can point to from, from our project work, we built a product feature called AI Assist for AirBite. If you're not familiar with AirBite, they're, they're ETL software. It's basically a data middleware platform that you can take your data from Salesforce and you can move it into Snowflake and now use it for analytics is one example, but they have connections into all different types of software platforms and one of the big things they need to do is they need to support building new integrations with new platforms. So what AI Assist does is it makes that process of building that new integration much, much easier. So what used to take a human, you know, 10 hours of effort reading through third party API documentation, building an integration with that new system, now AI does that for you. You just give it the link to the documentation. It reads through the documentation and it builds the integration. So fairly complicated, if you didn't follow everything I just said, that's okay, bringing it back to the UI here. So we, once we built the underlying system, we were thinking through what's the right way to do the UI, what's the right way to expose this. And you can picture the input output pair here is input link to documentation. So it's just a link to us to a site that has information about this is how to set up an integration with Salesforce output and integration with Salesforce, which is basically a file which contains the information that this necessary. We started with let's do an end to end, you know, agent, you just it looks like Google, you know, you paste in the link to the documentation. It's one field. That's it. And then outcomes, it outcomes the file that you need to have your integration. And what happened was 90% of the time it failed 10% of the time you got a perfect integration back but 90% of the time it failed. And in those failure modes, it wasn't useful. And really what was happening underneath is that 90% of the time where it was failing to get all the way there. It was getting 80% of the way there or 90% of the way there. And and so this this like all or nothing UI was wrong. Then we thought about like, okay, well, do we expose this via a chatbot and it was like, well, that doesn't make sense. Like how would anyone use that. And we went back to, well, what's the what's the UI that people are using today to do this. Oh, this is the obvious natural place for this to fit. And the way that we we ultimately built this you you paste in the link and then it drafts all of the fields that of human currently fills out today for the human to review. And the human in now and it's 90% of the way there. The human can see that it's 90% of the way there. Fix the 10% that's wrong. Save it and it saves them a ton of time. And so, you know, you want to build these things into the existing flow of work like that whenever possible. Yeah, I think that's that's where we're that's the mystery. I think to a lot of firms is the way they first engage with with AI is through a chatbot of some sort. Right. So that becomes the natural default, you know, mode, which does really work. By the way, we've done enough data warehousing on the podcast that I think most of our audience knows ETL is extract, transform and load. But those of you who don't know that the what used to be. You know, great. So there were tons of companies that provided ETL at pretty low cost. Now you kind of, you know, you're either buying in formatica for, you know, a million dollars, you know, a lot of the smaller guys have gone away. I, by the way, ETL is one of the places where AI is going to be super effective in, you know, in generating that stuff for you, you know, whether using airbite or something else as well. It's like you're pointing it at, hey, this is the data. I got like figure out how to get it into a neutral format for me is a great use case for for AI in my opinion. Yeah. Yeah. It's a great use case. So the couple of things that we draw to close a couple of things in your website. I love you have you talk about. So you mentioned like a co investment strategy. So talk about your co investment approach. You know, how does that differ? And then a little bit of, hey, you know, is it bigger than a bread basket? So does a pilot project cost me $10 billion or is it five bucks? Like what's kind of the, you know, and I'm not asking you to expose pricing. You know, fits, fits will kill me if I, if I get you to, you know, to talk about too much details of pricing here, but talk a little bit of the co investment strategy. Kind of what costs and an effort to kind of get it out the door. Yeah. So, so I'll answer this in the opposite or so. So first on the pricing side. I would say bigger the price get the starting project typically is in the ballpark of about $250,000 on the one. We do projects that are that are a little lower than that. But, but that gets you your kind of quick win first projects out the door. And then what we like to do during that during that phase is, you know, that's a two to four month project. We get that quick win in. We also then work with you at the same time on what's your full AI roadmap. What are the other pieces of well hanging fruit. Can we help you scope those. Can we help you wrap your head around the full AI potential for this business. And and ideally turn this into to a long term partnership on the co investment side of things. You know, we, I think the big reason that we're doing this co investment strategy is that we're huge believers in the fact that this these projects are going to unlock a tremendous amount of volume for these companies. And we're willing to put our money where I'm out this and go at risk. And ultimately we're looking for ways to change our economics as a business to go away from time and materials models and to share in the value that we're unlocking for these businesses. And so for the co investment thesis. The the basic premises that over the next decade private equity firms are going to get very, very good at investing in businesses that have high gen AI transformation potential. You know, picking the businesses that are going to win in in this moment doesn't necessarily mean picking AI businesses. It just means picking the right business that can use gen AI to do the automation projects to improve its margins or to transform its products and services in a way that that make them much better for customers. And creates some kind of enduring advantage for themselves. And so as more and more PE firms are looking to employ that strategy and invest in businesses that are going to have very high upside from a gen AI point of view. We think we can be a strategic partner in that to bring those strategies to life. You know, we can help all the way from the diligence process to really understand. Help you understand the opportunity all the way through to building the actual custom agents that are going to unlock those margin gains that are going to unlock that transformative customer experience and ultimately flow through to the bottom line. And so as part of that were were were open to co investing alongside PE firms in those businesses where they have gen AI high gen AI potential. The model that that we've that we've looked at historically is we write a check for somewhere between 10 and 25% of the equity. And then ultimately for the services that we're providing to that business looking to do that under an at risk model where we were sharing in the value that we're creating. But but but we're not just charging on a strictly time of materials basis. So something on the lines of you cover the you cover the salaries involved. But then all of the margin for for us is put into out of the money call options such that if we're successful in the transformation. We get we get rewarded for for doing so if we aren't then we don't. Well, so first I would say you know, Oh Chris your your confidence in PE people being that smart is it's very misplaced. Speaking from my own drive. I think you know it's an interesting concept. The challenge of course is if you're a PE firm and you believe in using the technology and it's going to add significant value for your company. There's no way I would you know want to let you carve out a piece of that right. If I'm 100% confident that what the solution's going to you know add significant value. That's all my upside. Right. So I want to carve that out. You've got to be careful you don't get sucked into the bad portfolio company where you know, Hey, this is great. You know that maybe you can help transform operations, but it's not going to be the winner. Right. That's really the challenge in the PE side thing. I mean, certainly our deal structures are, you know, unless it's like a super high end biochop. We do a billion dollar deals that it's you can be very complicated in terms of structure. Most PE firms that are structures are not crazy like venture because there's not more than one round. It's got you know, it's a bio shop. You may have a club deal. We got a few guys in it, but it's not. It's not that complicated from a model standpoint. It's not as difficult to sort of follow the bread crumbs. Like how do I get paid in this thing? But on the other hand, if they truly believe in it, in most cases, they're going to want to capture that upside for themselves. I think that's true. I think that's true. But but I do believe that in order to unlock it, we are a very, very valuable partner. Oh, yeah, no doubt it's not, you know, it's not us going, Hey, there's a cool opportunity here. Can we co invest it's, Hey, there's an opportunity where the PE playbook, especially in many cases, there's a rollup opportunity that is sitting there alongside the Jenny I transformation opportunity. And so, you know, kind of the PE playbook plus our playbook at doing these transformations plus this company like is magic, but you need all three. And so that's that's why we think we have the right to be in a lot of these deals. Yeah, I just think for Marsa, you know, for if somebody's a true believer in that in the solution, then I then rather just pay you then, you know, then put you in the upside. Basically, that's, you know, when you look at the PE markets, the difference in PE and venture for the most part is, you know, venture guys can take nine zeros and they'll get one Google on the dozen matter. PE guys can't take any zeros, right? So you gotta get, you know, you gotta get at least get your money back. Maybe twice your money and you got to have a few four and four or five in there. I don't want to just the way the math works. Right. So in any case, where they truly believe it, I guess, you know, I say it from the if somebody's really saying, hey, you know, Chris, we really want your co invest here. You know, it's like be super careful. And I get sucked into what is there? What is their dog? Yeah. Okay. So what so closing thoughts anything we missed? That's a good question. I don't think so. Go back through some of the stuff that you would say. I think you know, when we look at a summer, I think, you know, one of the one of the things we I listed in here is, hey, what are the challenges, right? And I see the challenges in an AI project is very similar to again, kind of a mobile or any kind of major transformation change came, you know, new technology from client server to, you know, to 3 tier, whatever. It's, you know, either you don't understand that the company doesn't stand technology and they try and take too big a bite. You know, trying to do something too big too fast. Don't know where to get started. So, you know, how do you hire the people? How do you know? I mean, that's always the challenge and something. You know, any of our team. can hire another software engineer. Yeah, they know how to do that. How do you hire an AI engineer? Yeah, that becomes, do you even know what to ask them? Right? I mean, I'm supposed you could ask chat GP, but you should ask them. But that seems like this weird, never-ending circle thing that would probably blow your mind. Right? Yeah. So would you say that's typically like if you look at like, hey, the biggest challenge is they just can't scope it right or either too big or too small. I think the scoping, the scoping challenge is very real. And I mean, I think, I think most people these days have a general sense for, I, you know, my instinct is telling me there's, there's something here or here and they're usually right, but they usually don't know what to do next. It's like, okay, I think that's it, but how do I confirm that? How do I staff it? How long is it going to take? How do I budget for it? All these things are very hard questions to answer if you haven't done 10 similar projects before and and most people haven't done any. And so, so going from that, my instinct is telling me to start here to actually budgeting for it and making a real attempt at it is very, very challenging. And it's not, it's not just a matter of hiring one AI engineer. It's, you know, it's, it's, it takes, it takes some knowledge, it takes some experience from having done it a few times to really wrap your head around the full scope of what it's going to require. Yeah, I think we've also, you know, that's interesting because we, I would say we would see a couple things about it. One, I, with our companies, I always recommend, hey, if you're starting to think about AI, start to just list out the, the applications for it in your like, don't, don't hone in on one right away. Start out with, hey, where do we think this could be helpful, right? From an efficiency standpoint, or, you know, from a process standpoint, like, let's lay them out because, you know, the, the one you pick made up either one you think you would pick, right? Like sometimes the one that may have massive value in the long term is just be too big a bite of the apple to start with, right? So maybe start with something else. Sometimes there's just a perfect case in the portfolio. But, you know, if you come with one case, I can't judge it against anything else, right? If you start to think about a bigger picture, I think that's where it starts to be a little easier. And I would say the other, again, this is just my bias. The other thing we've seen is that if the first step in the process is let's go higher in AI engineer, right? A lot of them are coming from academia where the models were created, right? And this is one of the challenges. If you're getting an academician who's been working on models, you're probably not going to get a lot out the door. You know, researchers are researchers, right? It's hard to turn a researcher into a, you know, into a ditch digger. And I'm not, this is not an insult to ditch diggers. It's, you know, like day to day program is getting stuff out the door. It's very different than somebody's a researcher. And I think that's one of the things we've seen too is a portfolio company would say, "Hey, I'm going to just hire a, what they used to say they might have said, I'm going to hire a data engineer, data science guy, you know, to do analytics. You know, we're going to build our first data warehouse, something like that. Same thing here. It's like, "Hey, I'm going to hire my first AI analyst. Sometimes that's a, all right, to do what, right? Like, I'm not sure you're going to get anything of them or the research and research isn't going to help you, right? If somebody's like, so to look at your data for six months, I'm going to make the argument in this particular case that don't look at your data for six months. Like figure out a pilot project that you can do that's going to add some value. I think you're going to learn a lot more in this particular case than you would in any other way. And I don't know if you agree or disagree with that. I agree that I, one thing I would add to it is that Gen AI is new. You know, the chat GPT moment, it was pretty recent. And so, you know, as you're looking for support here and looking to add to your team, you don't want the person that, or maybe you do, but the 10 years of experience that someone has on the resume that says AI, seven or eight of those years are not relevant to Gen AI. And they might be relevant for other use cases, but if you have Gen AI projects in mind specifically, there just isn't that much history of this technology existing. And so, you know, for us, for our hiring here, we actually aren't requiring Gen AI experience when we're hiring our engineers. We're looking for generalist engineers who are, you know, top of their top of their field, who can learn how to build with Gen AI on the job. And it goes back to what we talked about with, you know, putting everyone in person here in San Francisco. It's part of the reason is you can come in here at Generalist. You can learn from all of the AI engineers we have in that room, learn all of the tricks, learn the playbook for building with Gen AI. And within, you know, within a matter of a couple months, you can be one of the foremost experts in in Gen AI because you'll be one of the few engineers who has actually built production grade Gen AI software. And so, when you're looking at your own hiring plans as a business, one way to look at it is, huh, I have some great engineers. Maybe they can just learn the skill set of building with Gen AI rather than trying to find the person that has a ton of experience because they just don't really exist. And so, I would look at that route. And if you're, you know, if you're then looking at like, well, how do they learn some of this stuff? Like one way is to work with a company like ours where they can they can ride shotgun on one of these projects. On other ways, you just invest in their own ability to go figure these things out. It's just going to take a lot longer for them to learn some of these tips and learn the playbook for it. But, but in any event, I would encourage people to look to Generalist software engineers who can, who are just top of their top of their class and can navigate the difficulties of building with Gen AI. All right. So, if people want to get in touch with the, how do they, how do they reach out to Fractional AI, Chris? What's the best way to engage with you guys? Yeah. So, you know, our website fractional.ai. You can request a free consultation there. Also, you know, feel free to just shoot me an email, Chris at Fractional.ai. Perfect. Chris, thanks for joining me early on a Monday. I know it's early for both of us. I'm on a modern time here in Pacific time, but, you know, we, we persevered through. Yeah. Thanks for having me. Perfect. Thanks again. Devon and Jim. Technology issues. The middle market. Pea back companies. They pick up with each other. And they don't take themselves too seriously. It's not venture capital. It's private equity. It's the private equity fund cast. Pour yourself a drink and have a seat. It's private equity fund cast. That equity is private. If PE excites you, let us show and do not hide it. Devon and Jim.

Podcast Summary

Key Points:

  1. Fractional AI is a Gen AI transformation partner that helps companies implement high-impact generative AI projects, focusing on engineering and collaborative in-person work in San Francisco.
  2. The company targets existing non-AI businesses, enabling them to leverage AI to automate workflows, improve operations, and enhance customer experiences without needing deep in-house AI expertise.
  3. A key challenge is managing AI "hallucinations"; Fractional AI addresses this by establishing baseline accuracy, using evaluations (eVals), and building custom agents for specific tasks to ensure reliability.
  4. The approach involves starting with high-performance frontier models (like OpenAI or Anthropic) for accuracy, then optimizing costs by potentially swapping in smaller or fine-tuned models, with systems designed for easy maintenance and updates.

Summary:

Fractional AI, founded by Chris Taylor and his co-founders, operates as a generative AI transformation partner, assisting companies in deploying impactful AI projects into production. The company emphasizes a collaborative, in-person engineering culture in San Francisco to tackle complex projects through teamwork and shared problem-solving. It focuses on enabling existing non-AI businesses—particularly those in the lower-middle market—to leverage AI for automating workflows, improving operational efficiency, and enhancing customer experiences, arguing that these firms have a head start due to their existing domain knowledge and data.

A significant part of the discussion addresses the challenge of AI hallucinations; Fractional AI mitigates this by establishing performance baselines, implementing evaluations (eVals), and developing custom AI agents tailored to specific tasks to ensure accuracy and reliability. The company typically begins projects with high-performance frontier models for maximum accuracy, then optimizes costs by evaluating trade-offs and potentially integrating smaller or fine-tuned models. The conversation also highlights the need for ongoing model management and cost optimization, similar to maintaining traditional software systems.

FAQs

Fractional AI is a Gen AI transformation partner that helps companies implement high-impact generative AI projects into production. They assist businesses at any stage of their AI journey by leveraging engineering expertise to build reliable, production-grade AI systems.

The team works in one room to foster collaboration, enabling engineers to easily brainstorm, solve problems together, and share best practices. This environment supports activities like lunch-and-learns and peer reviews, enhancing learning and project outcomes.

Existing non-AI businesses that can use AI to improve operations and products are seen as major beneficiaries. These companies have deep domain knowledge and data assets but need tech expertise to automate workflows and gain efficiency.

They establish baselines for existing system accuracy, use ground truth data, and implement eVals (evaluations) to measure metrics like hallucination rates. By breaking processes into sub-steps and iterating on prompts, they aim to outperform legacy systems in accuracy.

A custom agent is an LLM system designed for a specific task, which helps narrow inputs and control outputs. This approach reduces hallucinations and allows for precise evaluations, making it easier to measure and improve performance compared to general-purpose AI systems.

They start with high-performing frontier models (e.g., from OpenAI or Anthropic) to maximize accuracy. Later, they use eVals to make cost-accuracy trade-offs, such as swapping in cheaper models or fine-tuning smaller models for specific tasks to reduce variable costs.

Chat with AI

Loading...

Pro features

Go deeper with this episode

Unlock creator-grade tools that turn any transcript into show notes and subtitle files.