How Investors are using AI

This episode is brought to you by Portrait. It's the AI research system that I used to prepare for today's episode and for all business breakdowns episodes. Portrait was built by former buy side investors and they understand great investing isn't just about having more information from low quality sources. It's about having the right information organized the right way. And if you listen to the show, you appreciate diligence consists of many things diving into the history of a business, framing the nuanced competitive dynamics, tracking key signposts around your thesis. And historically, that would take up material time that you do not have. But Portrait is basically like adding an army of analysts to your team. It's powered by an AI system specifically designed for investment research workflows. So you get nuanced idea generation. Portrait assesses the same types of qualitative attributes that we discuss on this show. And that can help identify businesses which fit your frameworks. Portrait also customizes research report generation. And I used Portrait to generate a primer and lay out bull bear cases ahead of today's episode to help frame the conversation. And third, there's intelligent thesis monitoring. And that's where Portrait assesses thousands of data points across value chains. Each day, extracting the insights, driving the business. Again, all this work would typically take hours and hours and hours. It's at your fingertips now. Visit PortraitResearch.com to start your free trial today. This is Business Breakdowns. Business Breakdowns is a series of conversations with investors and operators diving deep into a single business. For each business, we explore its history, its business model, its competitive advantages, and what makes it tick. We believe every business has lessons and secrets that investors and operators can learn from. And we are here to bring them to you. To find more episodes of Breakdowns, check out joincollosses.com. All opinions expressed by hosts and podcast guests are sole their own opinions. Hosts, podcast guests, their employers, or affiliates may maintain positions in the securities discussed in this podcast. This podcast is for informational purposes only, and should not be relied upon as a basis for investment decisions. This is Matt Russell, and today we have a special episode breaking down how investors are using AI. This is a question I get from many of you, and while there is no shortage of content on the implications of AI, I know there's an appetite to learn more about tangible use cases, how to make sure you're getting the most out of these tools, how to think about advancements in the technology, and ensuring that you're keeping pace with the innovation curve. So, my guest today is David Plan, founder of Portrait Analytics. Now, David and Portrait have been a partner of business Breakdowns since last year, but I specifically asked David to do this episode because one, he is really front and center to how investors are using and applying AI, but two, and maybe more importantly, him and his team come with a background in investing. So, while the conversation doesn't really focus on Portrait, you'll hear references to what him and his team are building and how they've shaped it for investors, you'll very much understand when you hear David talk that he is someone who understands the pain points of an investor. I think everyone will find something in this episode that will benefit them in their day to day, and I would love to hear the feedback now onto the episode. David, I am excited to have you here to finally do this recording. I think over the holiday break, many investors were spending time trying to figure out how they can get more out of AI tools. It's constantly evolving, but at least first snapshot in time. I think you'll be able to give us some really good perspective on use cases, some of the skill sets that you can use to leverage the most out of the technology, but I actually just wanted to start out with your background because I think this is really unique and can speak to why I find you such a valuable resource when it comes to thinking about this through the investor mindset. Can you bring us back in time prior to Portrait, what you were doing and how that led up to starting this business? Yeah, absolutely, really pleasure to be on here. I spent my career as an investor in a few different seeds before starting Portrait. I started at Barclays on a trading floor in the special situations group. I was then at a long short hedge fund called Slate Path Capital, where I was a generalist. Most recently I was at Boutpost, which is a large hedge fund based in Boston, where I was a generalist on the public market team doing both public equities and just credit. I think I probably first got bit with the idea of using AI within the investment research process when I was at business school at Stanford. This was like 2015 and 2017, so deep learning was very much in the zeitgeist. We were still pre-transform or read up until I graduated, but it just seemed inevitable to me at some point that this technology would be really impactful for a lot of the research workflows that I went through and after spending a lot of time studying and exploring and prototyping, I got the conviction to throw myself full time into this project. That's my background leading into starting Portrait. When you think back to some of those stints, whether it's Boutpost, Slate Path, even potentially on the trading floor, can you think of any scenarios where you were spending a significant amount of time that were pain points that are maybe easily solved by AI today? We'll get into some of the very specific use cases, but I'm just curious, when you look back at that time, some of the workflows that you had where you could see the biggest lift today, versus where it was 10 years ago. It's very different today. It's interesting because the output of an investment research process is ultimately a decision and hopefully a high-quality decision. It's very unlike other fields where there is actually a widget or a service being provided. When I thought about the investment research process, there were aspects of it that had friction, but that friction was helpful because it helped me build conviction personally. I was one of these guys that could never really outsource model building. I always had to build a model myself in order to feel conviction in recommending a position based on that company, but there were certainly aspects that I'd say limited me in terms of how productive I could be in finding and researching the best ideas. Essentially I would bucket it all into the category of there's just a ton of information out there that I could potentially consume, consuming it in a way that was efficient and additive to the research process was challenging just given hours in a day. I would put it in the three main categories. There's certainly idea generation, so finding the ideas that best fit my mental models. I knew there were probably dozens out there that were totally in my sweet spot, but at any given time, it was very hard to know if I was working on one of those. There's certainly the initial context building that goes into ideas, especially as a generalist. Not even developing a differentiated insight, but just getting the table stakes on how a company and an industry works and a narrative and business breakdowns has always been a very useful resource. Whenever I'm working on something that where there's an episode, that was always a huge help. I'd say lastly, just monitoring an investment idea. As a generalist, it was often challenging to stay on top of not just the companies, but the surrounding ecosystems, the customer suppliers, competitors. If someone lived in a given sector, they usually were able to put together a real-time mosaic based on all those data points that they're absorbing about what was happening in the industry and so forth. And I always found that really challenging as a generalist. There was more than one earning season when I got smacked because I was the last person to realize that end-market demand was weakening or suppliers were pushing prices through into those types of things. So ultimately, I would say those are the types of workflows, high-level information processing to help triage new data points. Broadly speaking are the types of things where AI can be really useful and we can get into some specific applications of that. But that's how I would think about where AI can fit in a research process without replacing the parts of the process that are important for building conviction. Yeah, I think that really resonates and seems to align with the people that I've seen getting the most out of it. A lot of it is productivity and efficiency, which on the surface level, sometimes people can poo poo. But when it comes to making decisions and having clarity of thought and being able to take in the most valuable information, I think there's a lot more to it. So I'd like to the idea of getting into some of the use cases. And I actually wanted to start backwards in terms of the buying process and reference what you just brought up there in terms of position monitoring. What are some of the tangible use cases of AI today that you think investors are using or you would recommend they use to just monitor what's in your portfolio and stay on top of it as effectively and efficiently as possible? I think there are many good solutions today for staying on top of what is happening with an individual name. There's plenty of ways you can add it to a watch list on any number of services and staying on top of all the new filings and transcripts and news related to that specific company. What I think historically has been hard but now as much simpler is being able to cast a wider net of pulling in relevant data points in a much sparser set of data. So for instance, let's say you own Expedia, one of the key investment factors for Expedia is always going to be what's happening in the hotel ecosystem and how volume is trending, how price is trending or what OTAs are doing with their distribution strategy and how it relates to OTAs. The challenge historically is if you're an internet analyst or you're a journalist following Expedia, you probably aren't going to want to subscribe to every piece of news of what Mary ought is saying. You don't care what their profitability necessarily looks like or if they're adding new units to take share from Hilton, what you really care about is any specific data point which would add to the mosaic around what is happening in the hotel landscape with respect to consumer demand for travel or if they're seeing trends and market share shift with respect to OTAs. But historically, it was impossible to pick up all those data points unless you really consumed every piece of news. There was no smart filter on top of all that data. I think with AI today, and obviously the company I run portrait, we have solutions that target this, but even outside of portrait, there are plenty of ways today to now much more efficiently pick up those data points across that surface area such that where today a certain level of understanding was unique to someone who just covered travel. You're now able to pick that up much more efficiently as it relates to your investment thesis. Yeah, I can definitely speak to that as a transport analyst trying to monitor truckers and knowing that there was valuable information coming out of CPG companies talking about freight costs. I spent many late Thursday nights while the rest of the world was enjoying men hadn't happy hour control, effing through these various transcripts and earnings releases. So I think that is a major one that has evolved and just the ability to gather all of that information in a much more efficient way and not control, effing the heck out of things. When you next move into the building up a research on a name or the pre-buy research process, getting up to speed, this is another one where I think it's the widest net of possibilities. But when you think about maybe through the lens of how you used to approach it versus how you're seeing other investors approach it, what are some of the use cases that you're seeing people implement to really get the most out of it and maybe build better conviction or gather more information, whatever it might be. It's interesting because I was someone who and still do love doing the pre-buy work. My wife likes to joke that my favorite Saturday morning activity is reading a 10K and the thing that's interesting, what I was mentioned before, where to me that's an important part of the conviction building process is building up that context and feeling like you've read all the material and you can start piecing together the various data points. Where I think AI can be really helpful in that process is two things. One is getting you enough information to know whether an idea should be killed. I used to do this accounting at the end of every year when I was an investor. How many ideas did I really look at? A remarkable low number that made it through to the deep research stage and a lot of times I could have killed an idea much quicker once I'd surfaced enough information to understand there's maybe an existential risk here I'm never going to be able to get past or the management compensation is such in a way where I'm just never going to feel comfortable that they're incentives are aligned. I think where AI can be really helpful in that process is giving you enough baseline context to be able to triage the idea of, oh yeah, this is passing the initial sniff test. I want to spend more time on it and maybe spend 12 hours reading through all the historical content. I take the other thing that has been really helpful is just expanding what gets moved up earlier in the pipeline in terms of types of analyses. For instance, I mentioned earlier CEO compensation, one of the things that I would do if I was really getting into a name is go through the last five proxies and try to map out what are the metrics that the CEO is being comped on and how have those change and the weighting of those change a lot of times that's a useful signal and to what the board is thinking about. That's a bit of a pain now portrait is a click of a button. So things that historically would be more in like the deeper dive category. Now I can move up and that to me is one of the most powerful use cases of AI. So as an analyst, you're able to turn over far more rocks in a given period of time and you would otherwise and spend more of your deep research creative time on ideas that you know have already passed your initial sniff test and are worth that time. When thinking about really tapping into that, would that be having a list of non-negotiables in terms of whether it's compensation elements or something else that you couldn't screen through quickly or is it you don't necessarily know what you're going to hit that might break it, but you're just going to get there faster through the various iterations of getting up to speed. I know it's a little bit technical, but I'm curious mostly if there is a way that you think that you could templatize that versus not. Oh, yeah. There's two getters. There's definitely things that you can templatize. So for instance, I mentioned CEO Compensation, there's certain accounting things I would look for with respect to like aggressive revenue recognition or changes in certain assumptions that would inflate figures. And then I'd say there were patterns that, or if I saw, that would turn me off. So one analysis that now is very trivial, but took a lot of time historically was I would go back through the last three, four years and lay out every piece of guidance that the management team had given both obviously the hard specific guidance, but also anything qualitative or soft like, oh, we expect revenue to accelerate sometime in the second half of the year. And I think building up a picture of management's guidance style and credibility is really important. And that's the type of context that someone who's followed a name for a long time intuitively has that being fresh to a name I wouldn't have. That's a situation where if my thesis is based on some sort of turnaround execution plan that the management team is undertaking, but I can do the work and say, oh, actually over the last three years, every year they've said they expect margins to expand and never happens. That would be enough to maybe kill the idea, depending on other circumstances. So I think that's another cool example of it helps. I think where AI can be really useful is surfacing patterns that are sorely were pretty painstaking to put together and now can happen for more easily. Yeah, 100%. It's a very valid point to on the credibility. I think there's a trust level and once investors show they don't really trust a management team or a stock. It could be a free fall and you see that quite frequently. Yeah, and it can be really subtle too. There are times where if you just look on the Bloomberg screen, it says management beats guidance every quarter and then you actually dig into it. You realize when they give four year guidance in the Q1 call, they end up revising it down quarter after quarter after quarter. So by the time they get the Q4, sure, they're going to beat their numbers. But if you're building your model looking out a year, you probably should shade whatever management saying at that point subtly, yeah, there's effectiveness and just kitchen sinking and taking all the pain in one quarter and that can be better than just consistently revising down. So there's all types of things that to your point feel they're definitely quantitative in nature, but the ability to extract that information was not always easy. And I think that's an entire category where AI is showing up. The last section is in that sourcing of new ideas and maybe something you alluded to before where you have certain frameworks or characteristics of businesses that you really like. And it's not always easy to find all the companies out there that fit that or to know if you're looking at a company that might be fitting that. So where in the idea sourcing, idea generation, are you seeing AI be most effective? I will just imagine to me personally, it feels like there's always a step before you get to sourcing you usually introduced to a name for other reasons, but I'm curious where you're seeing this coming into play. There's two main ways I'm seeing investors use AI for this type of use case. One is understanding companies that are exposed to a given trend or a given development. So for instance, when tariffs were first announced, I guess April of last year, there was a lot of investors, especially on portrait, trying to figure out which companies are going to be most exposed. And maybe the second order thinking is even trickier, which companies, for instance, have a predominantly US based supply chain and their competitors have an international supply chain, like that type of analysis. And AI is really useful for that, even out of the box. There's just a lot of world knowledge built into a modern state-of-the-art model about its first and second order effects and who might be affected and so forth. The more nuanced and I'd say difficult case and where we spend a lot of time has been working with firms to find ideas that fit like a nuanced definition of a mental model. So for instance, in my past life, I had this mental model of companies that were previously high-performing and there were really attractive elements of the business model that you could believe in and underwrite. And there was some sort of hiccup, it could be something macro related or really bad product cycle, some sort of execution mess up. And the market is now revaluating the franchise value of that business. And a lot of times I could isolate the research that one specific headwind and figure out if it's temporary or not. And if it was temporary, then you could buy it. That's a pretty nuanced thing to find. It's really qualitative in nature because the headline numbers are going to look bad whether it's temporary or not in the near term. So finding ideas that fit the mold of that is challenging for two reasons. One is that just requires a lot of qualitative reasoning along with the figures to do that. But two, I'd say even enunciating that clearly is hard. When I ask a lot of investors, what are the mental models that you think about when finding a really attractive idea? Some of them have been very explicit about writing down, I'm looking for attribute A, B, and C. But a lot of times it's more of a. You know what you see? And actually figuring out how do you translate that into a query for lack of a better term is challenging. And that's one area where we spend a lot of time at Portrait as well as helping folks look at their history and figuring out like, okay, what does a 10 out of 10 idea look like for you given your historical trading context and firm and all of that. So I think the combination of those two things is difficult. But when it works, it's incredible. There are a few feelings as exciting in this business as getting served up a pitch where you're reading it. You're like, oh my god, yeah, this is really exciting. Clear my calendar. I'm going to spend the next week just figuring this out. That's a really magical moment and something AI has been really helpful with. Yeah. I can tell you from doing the show, I get to experience all these different businesses and I'm trying to draw the patterns and match them up and some of them come more simply than others. A mission critical part of the supply chain that's a small percentage of the overall cost, but they're a dominant player. They have all the market share pricing powers should be there. That's one that's emerged. And I think gained a lot of appreciation over time. And then there are the more subtle ones like what you described before, which really articulating them can be more challenging. But that is an interesting approach and actually something that I've seen a few investors who have a differentiated framework, try to lean into finding ways to take that proprietary knowledge or pattern recognition that they developed over time and get this tool that can get the most out of it in terms of uncovering new opportunities. I think one of the things that you've made clear to me and other appearances that you made and as we've talked is there are skill sets to get the most out of AI. And I don't think they're always that obvious or at least they weren't obvious to me. But I was open, we could just talk through some of the things that seem to make a massive difference in terms of whether investors getting the Google search better quality output versus really building on what they're doing, becoming massively more efficient. And the first is just the prompt writing. So how would you articulate how important prompt writing is? Some of the basic things that you think benefit investors from thinking about when going about prompts and just feel free to wax poetic on it. It's an interesting question that the answer changes every three months in terms of what makes an effective prompt because the model capabilities change as well as the tools available to the models are constantly changing. In the investment research context, and I think this is true for other contexts as well, but particularly the research-based workflows with large language models. The mental model that I think still works really well is imagine you are writing an email to somebody maybe overseas who's going to work overnight and is going to be doing a task for you. And soon they're smart, but maybe lack a lot of context on you. What information would you want them to have to be able to do a good job? Whatever you end up writing is probably a pretty good starting point for an effective prompt. And to dig a little bit deeper, some of the subcomponents, when I write prompts today, I usually outline a specific task and why the task is happening. In the same way where if you give an analyst, build this cost curve, they say, okay, if you say, hey, build this cost because I think it might be shifting and that could imply something about future changes in the pricing, that's helpful. So I usually provide some background context, I provide a task, depending on the nature of what I'm looking for, I might specify an output. I might say, hey, I want your output to look like this. Have an intro paragraph, then outline these data points, show me a table that has x, y, and z. That sometimes is useful. Sometimes that can be hurtful in that you're constraining the model's output the same way of human analyst. You might want to give them some slack on the rope to be able to be creative and how they want to synthesize the information. And then I usually have a section of my prompts where I have specific task guidelines and then also just general domain knowledge. So for instance, in terms of guidelines, these are things that I would just think about, hey, maybe I'm doing that guidance prompt, make sure you capture soft guidance that doesn't have a specific figure on it, because otherwise if you say just capture all guidance, it's just going to capture a specific quantitative guidance. And in terms of transferring domain knowledge, I have like a set of bullet points that I try to encapsulate. Here's how I think about being an analyst and some of the things you might learn through your experience. One of those is I always remind the model like, look, you're going to largely be reading commentary from management teams. They are always biased positively. It's important you take a skeptical eye to anything they're saying. And even something that simple really helps because the models have been trained to want to be helpful and that usually influences them to be more positive than they otherwise should be. So just little things like that where you could imagine taking an eager college student who's excited and believes in the good and trying to impart a little wisdom, actually, management teams tend to inflate things a bit. Just drop that up. Defining the task, the context behind the task, some specific outputs if I want it, some guidelines and some kind of domain context. I think is a really good starting point. And if you can send that to a human and they read and understand the task, you're probably going to do pretty well with the model as well. Yeah. The level of instruction, if you almost humanize it, that was one of the initial things that I think you mentioned months and months ago that made a material change for me. And then on the idea of context and where we'll use an LLM, assume off the shelf, LLM for this question. But when you're thinking about when to just give them massive freedom versus when to load in documents, how does that change the scale and the output quality in your experience? So I think it's very test specific. There's a few different dimensions of which it depends upon. One is, is the nature of the task something defined and structured and maybe more quantitative? In which case, yeah, it can be a pain if you're using an off the shelf solution, but upload the documents. Because otherwise, generally speaking, an off the shelf model is going to have to go crawl the web. And that's an imperfect, inefficient way to gather a bunch of structured information. You'll most likely get hallucinations, which is why, of course, companies like Portrait exist because all that stuff's preloaded. But to the extent that the goal is something more exploratory or creative, it can still be really useful to upload the relevant documents. But I would explicitly instruct the model, hey, I don't really know which way this answer is going to go. So pull on threads, follow it, chase down leads. And in that case, you can steer the model, especially modern ones that are quite responsive to do a lot of web searching, where this matters as well is how important the task is with respect to accuracy to usefulness. There are certain tasks where if you're 99% accurate, you use 0% useful, and that's like self-driving cars, where the cost of an error is exceptionally low. So I think of the model building, like if I presented a model that had an obvious flaw on it, to my boss in any of my past roles, that would be pretty bad. I certainly did that on occasion. Makes two of us. Yeah. Maybe you're just doing an industry survey of what's the history of this industry and how is it evolved to these three players? If there's a factual error here or there, and it's earlier in the research process, and it's more of a triage exercise, that's okay, I'm fine with that. Having some calibration, depending on where the task is in the process, the importance of the accuracy, and how structured you want the output to be, I would vary in experiment. Yeah, it's quite interesting to see where things matter based on precision versus being directionally right is where all the value comes from. On your point on experimentation, I'm curious how much this matters, and this is something that I mentally struggle with sometimes as well. You mentioned the technology is constantly changing, so by experimenting or getting better with certain skill sets that eventually the models are going to be able to do on their own, how much difference do you see in investors or yourself when you experiment with different things in creative ways? When you know that there's a standard, decent quality way that you can solve it already, I know it's a broad question just about experimentation as a skill set, but I am curious if you've noticed any patterns with the clients that you work with or you yourself and some of your teammates. In an evolving technology like this, I think spending some 15% of your time on experimentation is really important because unlike past technologies, when the underlying models are changing, but too, the capability frontier is evolving as well and is quite jagged, and there are a lot of undiscovered for lack of better term capabilities that you can pick up on well before others do, and so having a process by which folks experiment and really spending some time pushing the models can be one helpful in its own right, but two gives you a better intuition of where the models are today and maybe where they're going. So, to add a little specificity to that, I and we at Portrait have plenty of tests like this and even me personally, I have some tasks where I know the models can't do this today, or maybe they can do a small percentage of the time, but anytime there's a new model that comes out, I run my suite of 10 different exercises across it, and I one can get a feel for how much of a leap to this model is, but two, I might now need to update where I think the edge of that capability frontier is, and a lot of times, once you start doing that process, it kind of builds on itself where you might have an experiment of maybe it is building a cost curve and how that's shifted, that's a type of task that can take a long time and be challenging, and now the model can start doing it. Well great, now you start having ideas of well, I wonder if I layer in the long-term cost curve or some marginal cost curve, and there are different things you can start coming up with for ways you could actually apply it within the research process today. So I would say for anybody, certainly companies do this, but individual investors have a suite of 10 different things where if the models could do it would be great and just constantly run these against the model, and once they aren't able to do it, save that template and reuse it as part of your research process when working on a name. So I think those are some of the things that where experimentation can pay a lot of dividends. I would buck at this, an experimentation, but one of the things that you laid out when you talked to Brett from fundamental edge was just writing prompts in different forms. So start out with the most simple prompts that you know is probably going to give you a very broad mediocre answer, then take it layer down in more detail and specify more detail, and then see how the responses change based on how you're instructing it via the prompts. I did it a bit, and it made me appreciate how it's sensitive to maybe the ordering of questions, or if I specify the ordering of questions, and what I'm aloting into the context window, and it doesn't just impact that single exercise, it makes you appreciate. This is a technology that really doesn't have many buttons that tell you, oh, this is four-wheel drive that I use in this condition. It is just a chat, and you can have to play around with it yourself. And just on that, I think one thing that's really cool that I mentioned earlier with the way I write prompts is I think about sending an email to someone overseas. The difference with an LLM is that the response is for all intents and purposes, instinct. The real insight, I think, comes from doing exactly just describe, which is iterate on this. The cost of sending a single query is trivial. Start simple, start adding complexity as it's helpful. It's really like a two-way dance that end up in a spot where the AI really is acting like a useful analyst, and you don't need to wait overnight to get the content to then see how I get feedback, you can give that feedback real time and adjust things, and that becomes super, super powerful. Your prompts are loaded into portrait for primers or whatnot, and you see the level of detail that go into those, and it's actually quite helpful, it made me feel a little down on myself in terms of what I was prompting prior to seeing those, but I think, yeah, it definitely changed true, taking it up a level. I have seen quite a few individuals who are getting a lot of value out of AI, but it almost feels like it's siloed. You could have two individuals at the same fun that use entirely different tools, and they're getting value, obviously, the enterprise or the fund is benefiting from all of it, but it doesn't necessarily feel strategically aligned, or there's too much collaboration in regards to it. Have you noticed anything in terms of best practices for funds just to get some type of adoption that's aligned or collaborative? Because investing is ultimately a lot of the research is building conviction and people build that in different ways. It's challenging to mandate changes in a research process because if I was at my last job and the head of the firm said, okay, now you must use only cell side models for your models. I can certainly sympathize with, I know a lot of firms have been trying top down to say, hey, we don't want to fall behind on AI, you need to start using such and such tool. And I think that can sometimes work in certain ways, but in other ways, it can backfire and people push back. What I've seen the most adoption and firms that are taking the most advantage of this, I'd say is finding the right balance between having firm wide initiatives that don't force people to change their behavior while letting individuals experiment and figure out where AI is going to be additive and reducing friction without negatively impacting conviction. So for instance, just using Portsha as an example, there are firms where at the firm level, we spend a lot of time building idea generation screens that are super bespoke to them and their process. And we just run those like an outsourced analyst. And that's really value out because all we're doing is we're not asking anyone to change the process. They just get a bunch of really interesting pitches that now they can spend time on if they want. And similarly, the thesis monitoring has become a really big thing. And a nice thing about thesis monitoring is it doesn't require a change in process. All we're doing is we take a ticker and maybe your investment thesis and then we're just pulling in data points that at least our system believes could be really helpful and incremental to your thinking. And again, that doesn't require anything new, it just requires looking at a new insight when it arrives and ignoring it if it's not relevant. Whereas I think for the actual day to day, asking queries, building outputs, it really just depends on the user. And obviously, working with a firm that specializes in this and has built software specifically for this vertical can be really helpful for driving adoption because unlike a general tool like JGPT, someone building software in their space can customize the UX around the specific workflows. But ultimately, I think that adoption has to happen bottoms up and people need to feel comfortable with it to be able to continue making high quality investment decisions and to the extent people have changed the process and many people have, they've done so because they feel trust in it. And I think building that trust has to happen on the individual level. Yeah, it's actually an interesting perspective that I somehow hadn't thought about. But when it comes to any type of investment research and investment process, people do it all differently within the same fund. They might have the same core things that they look for, but the path to get there is quite different and even model formats. I can remember many of arguments over having a single specified template and people moving off of that. So, yeah, I think there's a lot to that and maybe that'll put my eyes a little bit on the practices that firms or individuals can do in terms of investing in things that maybe aren't obvious in value today. But as the technology evolves, maybe you'll be able to leverage more and I mentioned before we talked about. There's the famous example of the sports analytics teams that had been tracking data well before sports analytics were a thing and they were only able to really benefit from all those data practices decades into the future. But the teams that never kept up with it, it's really hard to go back in time. So, are there exercises or practices that you think funds or individuals can start to implement that will pay off in time? Obviously, everything we spoke about earlier with experimentation certainly applies here, but I think one thing that is going to be really important and already was historically, but I think carries a lot more weight going forward is just the importance of documentation of thinking and decisions and research. Essentially, these models, the usefulness of them rises exponentially with the amount of context they're given. This is the model, context length expands and as their agentic reasoning gets more heavily utilized to do longer running tasks and they ultimately shift from being a helpful research sort of something that lives within a firm and can execute a research process, their ability to do it is going to be a function of how much data they have on how you and your firm operates. So I think firms where they have documented memos and even write up short paragraphs and why they made a trading decision. It's hard to know ex ante which data is going to be used in how, but I think it's a pretty reasonable bet that having that data at a minimum is helpful for the humans, but will certainly be helpful for the machines because I think, when we go we're speaking about adoption, plunging is loading in that context. It's the same challenge you go is when you hire a junior analyst needing to teach them everything about how you think and your past trading history with a given name and all that information just takes a ton of time to convey and so to the extent that that data can be captured in real time. That forms an enormously valuable source of IP that will make AI unique to you and again, it's a little hard to know upfront how that's going to be used and it probably takes some creativity to figure it out, but hard to imagine two or three years from now that every piece of data isn't being used within a model that is operating within the context of a fund. It's a perfect example. I could speak to on ramping at places where all they had were the memos, which were the buy and sell decisions for positions, which are valuable in some ways, but they're usually overly polished versus those that after earnings, they would have their quick blurb. Maybe Amazon entered the space and was something to monitor. And then a month later, they had a meeting with the management team and it made them feel a little worse off. And then another quarter comes out and it's clear, Amazon's impacting them. And the memo might just say, Amazon's movement, but to see the thought process and how the team makes decisions, there was something very valuable in me getting up to speed. Again, I think it goes back to that lens of almost thinking of it like a team member in the same way that you would give them the most ammo to align with how you work and what you do. Going back to some of the quick hitting questions thinking about down the road and where we're going, I already brought up context windows. We'll use LLMs again, just thinking about some of the most used tools. When you think about where investors or where you personally find the context window advantage has the most impact or how you adjust for the challenge that I think many investors face, which is I want to get as much information in there as possible, but sometimes unlimited by it. How do you approach that? And how much do you think you should worry about that as an investor? It's super dependent on what tool you're using and what context you're using it and the task to go a little deeper on that. The context windows obviously have been expanding, although really in the past year, they've been flat with Gemini at around a million, GPT, I think around 400,000 and opus or clawed around 200, where the models have gotten more capable is using that context. Well, I think historically model struggled if you loaded in, I don't know, the last five, 10 Ks. And let's say a 10 K is like 100,000 tokens, so 500,000. The models historically would be really good at say, if you say, what was SGNA in 2023, they call that a needle and a haystack problem. They would struggle if you said, build a three statement model using these. The model needs to apply, able to tension across lots of different pieces of the context window simultaneously. I think that still is a roughly helpful rule of thumb, which is the models are generally, if you're using a ton of context, we need to do better if you're loading that in because you're looking for very specific data points. That being said, I think the change versus if we had this conversation three months ago is models have gotten very good using the context intelligently. If you're thinking about building a project in portrait or notebook LM or something like that, I wouldn't really hold back. I would give it pretty much the entire corpus, and you're seeing this a lot in software engineering where clawed code and other agents at coding tools are very capable, exploring super long context in a way that is efficient and focused on the task at hand. In the same way, I wouldn't give a junior analyst a task and say, these are the only four documents you can use. You want them to ultimately have the agency to choose what content they're going to use, and I think that holds true. All that being said, if you're going to use the raw model itself, you do not want to overload the context window and use 70% of the context window. You will see a degradation in any complex task. In that case, you should break it down, but hopefully the much easier way to do this is use a tool that manages that complexity already, so you don't need to worry about, okay, what percent of the context window am I utilizing and so forth is how I think about that today? On memory, I think it's obvious where, as memory has now been implemented in a lot of these tools, it customizes you in that iteration, again, I'm going to overuse the lens of the employee overseas that you're creating a better, more aligned working environment. Are there any non-obvious impacts that you think memory will have? I think the interesting thing about memory is today, as it's implemented, it's far worse than the human's memory. It is super limited. It's expensive in the sense of having deep memories takes up a lot of context. It lacks a lot of the nuance of our memories are really useful, both for specific facts, but more so for the abstract concepts that they create and that we learn. When you think of those memories a lot of times, that connection is where they become really powerful and that's become a pattern recognition for investments. In the near term, I think memories is a useful tool for all the obvious reasons that you can think about in terms of just adding more context that persists in the context window. Longer term, everyone knows all the fallibility of human memory and this is a little bit of my pay grade in terms of the actual ML research and how memories being incorporated in different ways, but you could imagine any one of the architectures that have been experimented today with memory and having short term memory and longer term memory and the ability to look up memories if implemented correctly, both in terms of how the models are trained as well as the harnesses around them. Imagine having a model that has lived every single one of a firm's investment and maybe things that even passed on. There's a world where it becomes so much more powerful than any given human. I look at some of the investors I worked for and they just have this insanely eerily intuitive judgment about things. A lot of that is not just memory-specific facts, but memory that comes from an experience and scar tissue. There's a world where that certainly could exist in models and I think in some sense people talk about AGI being the ability to be generally intelligent. I think a really interesting test for a model of its AGI is can it predict the future? You need a really good world model and understanding of lots of variables to predict the future and that's what an investor does. In memories, clearly a really important ingredient to that. I think near-term on a little bearish on just how big of an impact it's going to be relative to just copy-pacing that context into a window. But longer term, I think that's the unlock where this goes from being the useful tool to the real core driver of a lot of the research or firm is doing. Well, I'll separate. Near-term, I'm almost taking away that it might be more valuable to limit the memory therefore reducing the context window and investing more into the prompt that I'm using. I know they're not like for like, but ultimately memory is customizing to you and if you could just put that in the prompt, that might be more beneficial than sucking a memory. It's a near-term convenience. So in other words, if there's knowledge you want to impart in every single time you're not going to have a model, yeah, that should live in memory or if there are aspects of working on a name and you've maybe done 20 queries on a particular name, you maybe want some sense of the memory in living in the model. So if you reference something back or it already knows it's did XYZ analysis, it can do that. I think that's where it's useful. It's a bit of a shortcut for just loading in context that you don't have to repeat. I definitely would recommend spending a lot of time being thoughtful around the extent you can manage the system prompt in off-the-shelf tool or anything like that. I think we're just very early innings from a technical perspective tapping what it means to really utilize memory in a thoughtful way and just like illustrate that and these models have such incredible world knowledge by essentially reading the entire internet in their pre-training. So they clearly have the capability to store in their weights a lot of knowledge and then learn important concepts from that knowledge that leads to things like reasoning. So it's just a matter of time before that same paradigm applies to continual learning on the job and that to me is going to be really, really fun. Yeah, I do think about bridge rotters really been recording all those meetings for all of this time. What all of that input could do eventually and what happened with that technology. I don't even have an idea of what it could be, but it just feels pretty unique and seeing how people are testing this even with their own personal recordings and creating their own things. The last thing I wanted to talk about because it gets brought up, it's a buzzword. It has very practical applications, but agentic AI, the definition gets a little bit fuzzy sometimes, but just thinking about agentic AI as a topic and as it applies to investors, if you could just talk about where that's going, how you would even categorize that within the various tools and anything else you would mention on that. So there's lots of definitions of agentic AI. I think of it essentially as the model has the capability to reason, reflect, take action, reflect on those actions ultimately in the pursuit of a goal and there's obviously different levels of scaffolding and constriction around what the model can and can't do. What's really exciting especially now is agents have started to work and interestingly, when we first started portrait, we tried and we did actually have an agent that was running using GBT4 and it was so hard to get the work and the way you got it to work was you had to be so prescriptive about what the model couldn't, couldn't do. I think our system prompt was 30,000 tokens or something crazy. Really had to limit it because at any given time, the model, if it started going down the wrong direction and it didn't have the ability to self correct and be thoughtful about what it's learned and how it should adjust its plan and that required just a ton of prompting. It's cool today and I think what's happening in software engineering is the leading edge of what's going to eventually happen in investment research and really any other knowledge field is the models have now become smart enough to do longer running tasks through using tools and updating its thinking process as it goes. You're seeing this with things like cloud code and codex where people are leaving these models blown for multiple hours and it is dynamically operating like a software engineer to understand a code base, make changes to it, test things out, see an error, go look at the trace and figure out why there's a problem, make a fix. That type of thinking is certainly really challenging and that now works. The nice thing about code is it's the perfect environment for these models to figure this type of capability out because one, it's all just text files at the end of the day. It's all contained usually within a repo. So all the context is locally available and it's super verifiable. The code either runs or it doesn't. Investment research is obviously much different. None of the things are true. The context is in many different forms and very broad and some is more accessible than others. It's also much more qualitative in terms of output but the form factor of Keny's models reason iteratively and arrive at complex answers that require on the go changes of plans. The answer is yes and it's just a matter of the engineering work to get those models into the right context so that they can operate like a junior analyst and ultimately like a senior analyst doing long-running research. It's definitely still a buzz word but there's a lot more meat behind that buzz word than there was maybe a year ago. The point on code and I often bring up chess, these constrained environments that are still very complex but they're constrained by certain things. You tend to see more of a clean iteration cycle and more productivity that comes out of it and the more complex and adaptive the system. The harder it is to control the evolution and so it's one that I think will be interesting to see how it all evolves. But this has been fascinating. Thank you for going on some more philosophical tangents in addition to the very tangible use cases. It's been a pleasure. So thank you for sharing the knowledge with us. Thank you. It's been really fun. To find more episodes of Breakdowns, ranging from Costco to Visa to Moderna, or to sign up for our weekly summary, check out joinkolasis.com. That's J-O-I-N-C-O-L-O-S-S-S-U-S.com. [BLANK_AUDIO]