Stacie Mintz – Turning Qualitative Fundamentals into Quantitative Factors (S7E33)

Hey everyone, Corey Hofstein here. Before we get started, if you're into capital efficiency and smarter portfolio construction, mark your calendar. The second annual return stacking symposium is coming to Chicago on October 28, 2026, headlined by Cliff Asnuss, founder and CIO of AQR. It's a full day of conversations on how to stack returns dreams, put your capital to work more efficiently, and rethink the way you build portfolios. Through your advisor, allocator or just curious about return stacking, this is the room to be in. Head to www.returnstacks.com/symposium to learn more and grab your spot. That's returnstack.com/symposium. Now on with the show. All right, Stacy, are you ready? I'm ready. All right, three, two, one. Let's jam. Hello and welcome everyone. I'm Corey Hofstein and this is Flirting with Models. The podcast that pulls back the curtain to discover the human factor behind the quantitative strategy. Corey Hofstein is the co-founder and chief investment officer of newfound research. Due to industry regulations, you will not discuss any of newfound research as funds on this podcast. All opinions expressed by podcast participants are solely their own opinion and do not reflect the opinion of newfound research. This podcast is for informational purposes only and should not be relied upon as a basis for investment decisions. Most of newfound research may maintain positions and securities discussed in this podcast. For more information, visit thinknewfound.com. My guess this episode is Stacy Mints, managing director and head of quantitative equity at P. Jim Quantitative Solutions. Stacy has spent 33 years at P. Jim and she's been there for every defining moment of the firm's quantity effort. A first strategy born from a clients challenge to move beyond indexing, a 1999 decision to abandon Bara and bring the risk models in house, surviving the Quant Quake of August 2007 and the post GFC realization that in a crowded factor world, it's not enough to be a Quant. You have to be a different Quant. We dig into what P. Jim's fundamental Quant label actually means in practice. From a financing factor that asks how a company funds its growth to a factor taxonomy that includes an unfamiliar linkages group and unusually for a Quant shop excludes momentum entirely. In the back half, we turn to the frontier, turning qualitative signals like board composition and innovation into systematic factors, building models that assess emergent shocks like COVID and AI in real time and why Stacy calls LLMs bazookas. Tools powerful enough to blow up what already works, which is exactly why you start with the insight and only then reach for the tool. Please enjoy my conversation with Stacy Mints. Stacy Mints, thank you so much for joining me on the podcast today. Excited to have you. If I have a great Quant equity guest in a while, it's a perennial topic, but sometimes I go through seasons that I'm excited to dive right back in with you. So thank you for joining me. Thank you for having me. I think it's exciting time to be a Quant. So I'm happy that we're getting together on this. I'm biased. I think it's always an exciting time to be a Quant, but particularly in the last couple of years, I think it's got a little bit more exciting. Excited to touch all those topics. So we'll start, I guess, rewinding the clock because you have actually been with PGM for correct me if I'm wrong here. Just a little over three decades. You started your career at PGM and that gives you the wonderful perspective of getting to see how the entire Quantitative Equity group has grown within PGM to become what it is today. And you told me a story in our pre-call that's sort of the original Quant Equity strategies started as a challenge to PGM from your largest client requesting you to move beyond indexing. And I was hoping you could take me back, maybe talk a little bit about that context, but also at that time, what was sort of the evidence that you and the rest of the team were looking at that convinced you that Quant Equity was feasible? Was something that you could really do? At the time we were a Quant Manager but focused on multi-asset strategies and this happened to be one of our multi-asset clients. Their active managers didn't have the consistency of performance they were looking for and we were managing an index fund for them. At the time, there was a lot of discussion around behavioral biases and how they were explaining some well-known anomalies in the markets, such as the low PE effect and high momentum effect. They were anomalies that people were aware of, but this is where investors are starting to connect this idea of behavioral biases that had been discussed for decades to those anomalies. Thinking about things like investors being overconfident in their ability to choose stocks, being loss of verse or anchoring to the price that they bought a stock at and not being able to sell even if information changed and went below. Not only was that describing these anomalies and these are anomalies, traditional fundamental managers had been taking advantage of for decades, but it brought to light this idea that if it is a persistent anomaly, can you take advantage of it? That's really what this client was interested in. We always love a challenge so we dove right in and we did a lot of our own research that focused on those biases and where they were most relevant. What types of stocks were these biases most relevant for? That was a framework that we brought to the marketplace and wrote a paper, Behavioral Bias, Valuation and Active Management back in 1999. This strategy was launched in 1996 and today we still have a lot of those same concepts. In our strategy, the market has changed a lot and our strategy has changed a lot, but having that core philosophy to our process is something that's endured through those three plus decades. One of the things that you did that I think is unique in the course of history compared to other equity quants or at least certainly did a lot earlier than other equity quants is you actually moved to your own internal risk model back in 1999. I know everyone loves to talk about the alpha signals, but for equity quants, risk models are really, really important. The industry standard for a long time was this off the shelf, bar a model. Our team pushed back on that and said, "No, we don't want to do that. We want to build it in-house. Arguably, it could be for a variety of reasons." So can you take me back? That was very early in the game, in my opinion, of people doing this in-house. What drove that decision? What was the motivating factor? We had built an alpha model. We had a lot of confidence in. We realized that your portfolio is really the combination of obviously your alpha model and your risk model. The risk model really was having a big influence on the portfolio. And specifically, we had insights that we were trying to get into the portfolio. And some of those insights, the risk model saw as a risk and would unwind them. So back then, it was really looking at growth characteristics of a company, which were a little different than momentum and then value characteristics. And we had them balanced in our model, wanted them to be balanced in the portfolio, but the risk model was only seeing the value piece as risk and unwinding it. So our portfolio wound up being imbalanced. We fought it and fought it and made it work. And then we decided, what are we doing here? We just need to do this ourselves. And this has really helped us through time because as information improved and models were improved, obviously, these off-the-shelf models improved. We had control over how our risk model was evolving. And that in the end gave us the opportunity to really tailor that risk model to attack those risks that we really thought were important and balance them the way we wanted to. And having our own risk models really helped us with customization for clients even today. So it's easy to integrate other ideas or values or information into our model because we do have our hands all over the details of the risk model. And brought up what I thought was a pretty illuminating example on our pre-call where you sort of said, well, what happens if you and a competitor both like two names, two stocks equally, and off the shelf model might nudge you both to concentrate into one of those names. And if you sort of propagate that view across the entire Quantico system or all the competitors, everyone ends up concentrating in the same name. And so that name becomes potentially a lot riskier than it looks. And obviously for all the crowding dynamics, I think that may have arguably been a factor related to like August 2007, Quant Quaker, whatever we're calling it nowadays. Can you talk me a little bit about how PGM went through that period with its own in-house risk model, how things played out for you and how market dynamics looked? It was definitely an advantage. At the Quant Quaker, you start with some unwinding of a Quant hedge fund and then it snowballed into something that was a broader, deleverging event. It really did highlight the connectedness of the Quant strategies and did bring to light this idea of crowding. And for our strategies, we did much better than we would have done if we were in or using one of those off-the-shelf models because of exactly what you described. something that looks risky because of historical data that doesn't mean you can extrapolate that into the future. That means it'll be less risky going forward. So what our risk model did at the time would be more of a diversification engine and have us diversify that idiosyncratic risk instead of trying to predict which stock was less risky. By using a different risk model, we were naturally just a little bit different than others. So clearly an event like that affected us as well, but it did give us even more confidence that moving in the direction of our own risk model was definitely worth it. You described the GFC period as driving a realization for you and the team that in a crowded factor world, quote, "It's not enough to be a quant. We need to be a different quant." So you have the risk model working on one axis. Maybe we can talk a little bit about the alpha model for a second that being another axis of differentiation. What did different really come to mean for the research agenda going forward coming out of that period? I think anytime there's a market event like the GFC always have to kind of go back to the drawing board and see how your model did and dissect what needs to be different. This was also a period of time where smart beta was becoming a bigger thing. So investors can go out and buy these simple exposures in an ETF form at a very low cost. So if they were going to allow us to manage their money, we really needed to do something bigger than what was simple. So we took that opportunity to really focus on how to be differentiated using alternative data sources, looking beyond these simple factors that look at the biases like momentum or PE. We invested a lot in determining what a quality of a company looked like. So not just R O E, but looking at the company from different angles to understand whether this company could achieve the growth that it was planning to achieve or whether it was a real good value company or whether it was a value trap. So we spent a lot of time looking at more quality factors. But I think what was more important is how our research agenda evolved. We were really looking at things not just probable alpha sources, but also things that were very different cheated and that were scalable and feasible. So what we have today is actually an event that we have every year in December. We call it our research shark tank where we all get in a room and our researchers get up and they pitch three or four ideas and kind of like a shark tank situation. And we all get a chance to ask questions to poke holes to challenge them. That's how our research agenda comes together. But what we're really trying to do there is look for things that align with our philosophy, our scalable, our feasible, our differentiated, our relevant. And I think an environment like that where you're getting input from everyone in the team to come up with those different ideas is really important. The most senior to the most junior researcher along with the portfolio management team all chimed in. And I think that's how you get some really good ideas that will ultimately make you different from other quant strategies. I want to drill down into that philosophy idea for a second because PGM describes itself as a fundamental quant shop. There's a lot of firms that might claim that label something like quantum mental. And so I really want to get an understanding of when you say fundamental quant versus just quant, where is the real distinction lie to you? Is it in the types of signals who use the process of validating those signals versus a fundamental story or at the time frame you're targeting? What does that really mean to you? What we're trying to do is find companies that are just good companies that anybody would want to invest in, right? They have good growth prospects, they're high quality, they're reasonably priced. It's very much trying to replicate what a traditional stock picker manager would do. In doing that, we're really looking for every factor in our model to have a really solid theoretical underpinning. And the reason for that is because if we find a signal that works that we want to add to our model, we want to understand why it works. That will solidify that we think that this is a good factor, we understand why it works, we understand when it works in different market cycles. And it gives us confidence that that factor will work in the future as well. So we find that space in our model is very valuable. If you've got a lot of good factors and you're looking to add something, you're taking away space from something else. And you want your model components to all work in harmony with each other and balance each other well in different market conditions. So all of that being said, being a fundamental manager keeps us away from some of these ideas that maybe back test well, you're not quite sure why they work. You put them in the model, they're taking space away from something else. And then maybe they may not work in the future. So we have a very carefully curated list of factors that have true rationale to them that if I explained to you exactly what it was, you'd say, oh, that makes sense. And I think that gives us comfort that we're building a resilient model that makes sense. And in periods of stress, you know the companies in your portfolio are good companies and that they should be resilient. Maybe we can ground this idea with an example and you know, pre-call you mentioned one of these factors is the financing factor. This was sort of unique one, one of the first times I've heard of anyone talk about this particular idea. I was hoping maybe you could walk through what that factor is and how it might illustrate the distinction between being a fundamental quant and a generic quant dare I say and how that actually looks in practice. This was one of the additions to the model after the GFC and trying to look at a quality of a company and instead of just looking at some basic quality metrics, we were interested in the concept of how a company finances its growth. So if you've got a group of growth companies in an industry and you've got companies that finance their growth through organic cash flow, what our research showed is that those companies tend to outperform ones that consistently tap the markets to get more capital. And the rationale behind that was that companies typically have opportunities with diminishing marginal returns. And a company that has abundant access to external financing might be tempted to fund increasingly lower quality projects, whereas a firm that is bound by its internal cash flow will really have to face the decisions on what do we really want to invest in what really makes sense for this company. It was an interesting concept and when we tested it statistically, it worked very well. This is a good example, two of a factor that's evolved over time. We started with a very simple approach to this and over the years we still have that concept in the portfolio, but the factor has evolved quite a lot as there's more information and better techniques to use to get at that concept. One of my favorite questions to ask equity quants is about the taxonomy of their characteristics or the factors they use in their portfolios. I think when you talk to your typical equity quant, you're going to end up with sort of a hierarchy of value and quality and momentum and maybe defensive whether that's statistical or more fundamental quality. But you do have a cohort of what are more statistical or price-based properties often, not necessarily the case when you talk to a fundamental quant. So I'm curious, how do you think about organizing your factors internally? What's the taxonomy that you use and what's each group really meant to capture? We're looking for those companies at a high level that have good growth prospects or high quality and reasonably priced. So we do have categories. What's interesting is when we talk about these categories, they sound kind of simple, but the model itself is really complex and there's so much interaction between these individual pieces that the taxonomy sometimes is really just used for a sort of organization to talk about our strategy and to share information with clients, performance-wise, attribution-wise, the groups that we use are growth. So we're looking to find companies that have good growth potential and take advantage of investors that are underreacting to new information or take advantage of slow information diffusion. We're looking at linkages, which for us is another way of looking at the growth of a company and trying to understand how information is flowing from one company to another and how a shock to one company might affect other companies, whether it's a positive or negative shock. We look at the quality, so how well is a company run? Does it have solid management? Is it a competitive threat to other companies? Is there something in the options market that signaling something is concerning here that maybe our models aren't picking up elsewhere, so getting information from other informed investors? And then of course, valuation, looking to make sure that companies are reasonably valued or maybe they look undervalued under-appreciated. An important part of these groups, the growth, the value, the linkages and the quality is that we use them dynamically based on the kind of company we're evaluating for companies that are much faster growing, younger companies, we're going to tilt more towards those growthier factors, whereas if you're a mature stable company, we're really going to focus more on those valuation factors because there you're not paying for some future growth. Your return is going to come from the realization of a mature income stream and getting that income stream at a discount. I would suspect that our listeners likely have pretty strongly established mental models for generally what growth and value and quality. I mean, linkages is one which doesn't get spoken about very often and likely is a novel factor category for a lot of folks to think about. Let's hope when you get spent a little time there discussing what does this idea of linkages capture and why does it earn a place among these more sort of classical factor groupings? If you look at our growth factors and our insights there, we're really looking at the company directly and we're saying what are signals that are telling us something about this company about its growth prospects. What linkages is doing is just trying to come at that from a different angle. It's saying how is this company connected to other companies in the marketplace and how can that give us an informational advantage, meaning there's a shock somewhere in the market and that shock is going to trickle through. A simple example of this is like an industry. You can think if something's happening in an industry, you're going to see this whole industry move together. There are shocks that happen within an industry that truly do affect every company in that industry. This is a more sophisticated way of doing that because these days, everything is so interconnected across industries, across countries. You're finding other linkages to other companies that are not as obvious. One example is customer supplier relationship. If you've got company A and a shock happens to company A, say something really positive, you would expect the suppliers to company A to benefit from that. That can go several layers down. The suppliers to the suppliers of that company A should do really well. You can map out different relationships, even beyond customer supplier relationships. You can measure the strength of those relationships so that you can measure how impactful that new information may be across different companies. We found that this is another layer of mapping information and how it's affecting different companies around the market. In your taxonomy, you notably did not include momentum, which maybe isn't a surprise being a fundamental quantia of momentum being arguably a non-fundamental measure, but it is unusual for a quantia. You do often see that even fundamental quant firms do find a way to introduce momentum in somewhere, and other, even if it's just in trading and execution. Curious as to your thinking about, is there anything lost in leaving momentum out, or is it captured through some of these other fundamental factors you look at? It is captured through a lot of the other fundamental factors, but one of the reasons that we did not include or automatically include price momentum in our models is because as a fundamental quant, price momentum has a lot of elements to it that are related to the company that you're evaluating, but there are a lot of elements to it that are not related to the company. It could be, we were just talking about an industry shock that may or may not affect that company. It could be speculation, let's just call it noise. You've got the price movements of a company. Some of it is relevant to that company, and some of it is just noise. What we do is we pull out that relevant piece by looking at things that are actually happening to those companies. Key events for that company and how is the price moving around that event? How are investors reacting to that event, that's specific to that company, and then ignore all of the other price movements that they're less relevant? By calling out that informational piece from price momentum and leaving the noise out of it, you get a much richer signal. We find that our measure, we call it information momentum, is as effective over time as price momentum, but it has a lot less of the crash risk. About half the downside. When we are building factors, not only are we focusing on the fundamentals, but we're also looking at making that factor as attractive as we can from a risk and return trade-off. We have an element of risk control built right into the alpha model, trying to find the adjustments that we can make looking at all the detail to get the best factor we can from both a risk and return perspective. You mentioned that this strategy first went live in 1996, so we're talking 30 years of live performance and 30 years of doing research. And surely you've looked at some of the same individual names that have been in that basket. It must be 8,000, 10,000 different ways. Realistically, how many truly orthogonal signals do you think exist for things that ultimately move company fundamentals? It's probably less than you would think. Our models, depending on the region, have maybe 25 to 40 signals, or I would call them concepts because a lot of them have a lot of components to them. And each one, as I mentioned, is carefully designed and are in the model for a real reason, because they have a benefit alongside each other and through different markets. We want to make sure all of the signals in our model are there and have an impact. If you have a model with 400 signals in it, how much is each one really contributing? And how can you really dissect what is driving your performance? So that's something that's important to us as well. Having a meaningful weight being tied to an economic underpinning, for us, the number is more in the 40s as opposed to the 400s. I want to talk about some areas and what you've been leaning into for recent research. And one of those areas has been how information flows through the market, trying to get a read on a company before it actually shows up in those traditional income-stab and balance sheet type numbers. Given that goal and objective, I'm curious to ask you a question. I've been asking a number of guests lately, which is where do you think the largest source of alpha signal is living today? Is it in accessing unique data that maybe no one else has access to? Is it engineering unique features from that data? You know, right, getting a different look at the data or looking at it in a unique way? Or is it in actually forecasting more accurately or developing more complex or more complete forecasting models from the same sort of data and features everyone else has? I think it used to be the data. I think it used to be people getting that unique access. But these days, I think the data is pretty much available. Data is expensive, though, so being part of PGM, which is a very large organization, being able to get that data at scale is important. So I think that's an advantage that we have. But today, I think the data alone isn't enough. You have to be able to take that data. Number one, sort through what's really relevant and what is it. I mean, decades ago, it was hard to find data sources. Now, people are emailing me every day with a new data source they have, but you really have to fair it out. What is very valuable and what isn't. But I think what matters more is what you do with that data. So how do you turn this into a factor that is going to be a complement to your model? So you got to figure out what kind of tools you're going to use in order to analyze this data. You have to figure out how you're going to transform the data. Are you going to normalize it? What time horizon are you going to consider? Are you going to time wait the data? How do you deal with missing data? To us, all of those little decisions make a difference. And so you've got to take the time to go through and make those decisions thoughtfully. Not just try a bunch of different versions and see what looks the best. So paying attention to those details is important. I think it's also how you use those factors. So I'd mentioned before that one of our original concepts for our strategies was using factors where they're most effective. And for us, we make that decision at the company level. So again, faster growing companies who are earlier in their life cycle, we're going to wait those growth factors more and for more mature companies, we're going to wait those valuation factors more. And that's really at the company level. That really increases the power of our factors. New ounces like that and thoughtful approaches like that that are pretty intuitive when you think about just investing 101 really do make a difference. So I think it's not just the data you have, but how you use the data and then where you use those factors and your decision making for different stocks in the market. Can you dive a little bit into that idea of waiting those factor scores on different companies differently a little bit? I'm curious how much of that process is discretionary versus purely systematic and where you derive the evidence from. That's really important. that's the appropriate approach to take. It is all systematic, not discretionary at all. The concept comes from the idea of just evaluating a stock. So, the price of a stock is going to be driven by mainly two components. You buy a company because of the future growth of the company. And if the company has strong growth, that price is going to go up. You'll get future earnings. The second part of the equation is the existing earnings. And is the company priced appropriately for the existing earnings? Or is it a value stock and a discount stock? If you're looking at a fast growing company, you want to pay attention to future growth prospects much more than whether that stock is expensive. You can think of a lot of stocks in the industry right now where they might look horribly expensive relative to their current earnings, but their growth potential is huge and that is why you're buying that stock. So our models want to act more like a growth manager when we're looking at that stock and not really worried too much about how expensive it is. On the other hand, when you're looking at a more mature company, even if you're hearing hype for management that maybe there's some super-wow growth coming around the corner that can really trick investors to bumping up prices artificially or discounting historical bad news for a company and it's really beaten down and it's more of a value stock. For those more mature companies, we want to be more price sensitive. We want to look for bargains. Quality is still important, but this future growth piece is not as important. So in that end of the spectrum, we want to act more like a value manager. So when we're going to pull our portfolio together of stocks, we're having stocks we like, because even though they're expensive, they're good growth prospects. So those are the growthier names in our core portfolio. On the value side, those maybe they're not great on the growth side, but they're good bargains. So that's why they're there. It's taking a basic economic concept and applying it down at the stock level. So in practice, what happens is we evaluate a company's growth rate relative to the universe that we're looking at. And then we adjust the weights on the growth in value factors based on where they fall along that spectrum. So that's happening every day at the stock level. Now when we put the stocks in the portfolio and aggregate it up, you'll have a nice balance between your growth and your value factors, but they're coming from different types of stocks in your portfolio. In the last couple of years, we've had these examples of emergent fundamental shocks. We had COVID, what's your whether we'll call meme stock, mania, fundamental shock, but you know, meme stock, mania more recently, I think AI is a good example of an emergent fundamental shock. These are kind of a regime change that's arguably difficult to capture as sort of a standard static set of risk factors. And in our pre-call, you talked about trying to figure out how to make your strategy more resilient to when shocks hit the market, figuring out which parts of the market are most vulnerable and building custom factors that get at things like, well, how innovative or cutting edge a company is or how resilient they can be to these different sort of market shocks. Can you speak a little bit about how you go into building models that can help assess emergent shocks in real time? The goal of our model is to produce as consistent of a stream of alphas we can in different market conditions. So when we're building the model, we're really focusing on resiliency. So using different sources of alpha and different types of factors that work different in different market conditions. And you can do this by using a variety of data sources, different types of tools that you can use to evaluate the adaptive waiting scheme that we talked about. But the key is looking at companies from different angles and trying to understand those information shocks before they really show up in the numbers for that company. As far as a larger shock to the market, I guess by definition, it's hard to predict a shock because it's a shock. It's not something anybody can predict. But I think what's important for a manager is what you do during that shock. Having a solid investment philosophy and sticking with it is important. I think about when COVID hit and everything was turned on its head and only certain types of stocks were doing well. And the key is to stick to your investment philosophy, not to throw your model out the window. You had mentioned before about manager discretion. I think manager discretion needs to be pretty limited because we've all built these models together. We believe in these models. We built them to be resilient over time. We know that there are times in the market when these models don't work and we know that they will recover. So don't go in there and kind of throw the baby off with a bath water or do something with your human biases to mess the stuff like we're prepared for these situations. We own good companies. They'll hang in there. Now that being said, there are times where portfolio managers need to intervene and that's where experience of a portfolio management team is really important. Example there was when Russian invaded Ukraine. Our models were like, wow, Russia is really cheap. Well, that's the kind of situation where the experience of the portfolio management team all working together need to step in and say, you know what, maybe we need to make some adjustments here. But then I think the third thing about these shocks is learning from them. Sometimes you go back and you're like, well, there's nothing we could have done about that because this is how we manage money and this is our philosophy. But other times usually there are some really good insights that you can then work into your model, find ways to adjust it so that you're more resilient the next time the next shock comes because the next shock is always different. That's an important part of the research process. I want to talk about some of those learnings and I specifically want to use COVID as an example because I've heard you say that COVID compressed everything that long horizon factors collapsed into short windows and that a back test that looks really great through 2020 might actually be more of a red flag than a feature. I think that's an interesting perspective. How did that period change the bar for factor acceptance for you? Looking for factors and using back tests is such a valuable tool in the quantitative space, the fact that we can take an idea and test how it worked back in time is incredible. Back testing is an art as much as it is a science and I'm sure you've heard the phrase like I never saw a back test that wasn't good. But I think that that is a powerful tool that we need to continue to evaluate as if we were actually managing these strategies back in time. So COVID was such a market shock that when we're evaluating something that is based on our philosophy that is fundamentally driven, I would expect that it wouldn't necessarily work in 2020 because if it does, then it's not in concert with our philosophy because it was such an interesting period. I like to see a back test in time that does have periods of weaknesses so that I can understand what those weaknesses are and I'm more prepared for the future when there are market events that I understand what the weaknesses of this model is. I mean, the scariest thing is a back test that does well every year back in time because that can't possibly be. So what's really going on with this back test. I think one thing that's happened that we had to kind of accept was that we have shorter histories for some of these new factors or new data sets. So it used to be we wanted to see 20 years of data so that we can see how a factor did through time peaks and valleys. But now we might have information that maybe only goes back five or six or seven years and we just really need to understand that factor better and understand again the theoretical underpinning to get comfortable with it and maybe we use that factor at a lower weight for a while to get more comfortable with it. But that's definitely something that's changed in our evaluation too is that we need to be comfortable with shorter data sets than we did back in the day. Top of mind for most quants that I talked to today is the role of large language models in the quant process. You've called large language models bazookas, tools of enormous capability with a real chance of blowing up what already works. What do you think the right discipline is for working with these tools? How are you thinking about integrating these tools into your research process? We like to start with a concept, a data set or a concept that we want to add to the portfolios or to the models and then evaluate the best tool to use. And so typically the simplest approach is really the best approach. And so we want to start with something more basic and then get increasingly more complex to understand where the sweet spot is. So don't just jump in and use LLMs through your whole model and there are risks there that you want to manage and the best place to manage those risks is during the research process and really taking the time to understand what the tool is doing for you and how that should be integrated into the rest of your models. How you evaluate new factors with those tools is really the important piece of the puzzle. I know one of the areas that you've begun applying large language models and even small language models is to these non-financial quality metrics and this is an area you're leaning into. Things like board composition and ideas like innovation, things that are squinted. you're in harder to measure and don't necessarily come with a hard numerical value for you to easily plug into your standard quantum process. Can you talk a little bit about how you turn something that is more qualitative into a systematic signal, especially taking into account that you don't necessarily always get the breadth and level playing field across industry sectors and geographies that your typical traditional quantum process would want to need? Well, one of the advantages of using these language models or even natural language processing, which is something that we've been using for decades, is that there is a lot of availability of text data on companies around the world. So you actually have more data available consistently for companies. Most companies have some sort of earnings transcripts or there's some news about them in the world. So you can get text information. The challenge is how do you extract information that's valuable from all of this text and then how can you compare it across different companies? There's a lot of attention to detail around extracting the right information and making sure that you are comparing apples to apples with the information that you've gathered. And then once you've done that, turning it into a score that's comparable to other companies in your investment universe is much easier. But it's that middle step that's pretty difficult. The more sophisticated our models get, the more insights we can pull, and the more important that intermediate step is to make sure that the information that you're working with is really what you're thinking you're getting. And then that's able to really be that apples to apples comparison so you can make that investment decision. Maybe we can explore that with a specific example, something like board composition. Or maybe we can start with, what is the fundamental thesis behind why board composition would matter? What are you looking at with board composition? And how this is sort of working through the pipeline? We want to think beyond just board credentials. We want to understand how boards are connected to one another. So this kind of gets back to that linkages idea. If you've got a really valuable board member, somebody who's really good, really talented, they're going to be in demand amongst different companies. And so they likely sit on a couple of different boards. In looking at that, we can understand how these companies are connected to each other and how information flow may happen among those companies. So what we find is especially for companies who are struggling, if they do have boards that are well connected, they tend to have more resiliency than their peers. That's an example of trying to look at that both in a qualitative and quantitative way and understand that board quality and also that board connection to understand how information can flow among these companies. One of the common concerns among the ones that I talked to when talking about using large language models is that they bring their own unique risks, whether it's explicit hallucinations, there's this black box of not knowing exactly what's happening in the version of the large language model you're using, the risk of a look ahead bias, having the model trained on data that you're trying to not incorporate at any given point in time. I was hoping you could expand on how you are thinking about incorporating large language models into the research process, given sort of the lengthy list of potential risks that go along with using them. Hallucination and the look ahead bias memorization is definitely a challenge and there are some funny examples where you're trying to ask an LLM a question and keep blind what that actual stock is and the LLM will figure out what stock you're talking about even though you didn't tell it and then come back with supporting evidence because it knows whether that stock did well or not. So being able to control the data that you're working with is important. So during testing we tightly control what data we're looking at so that we can dissect the output better. Model selection is really important. We need to choose models that have been published recently enough that they incorporate the modern LLM techniques but that they were long ago enough that there's some period of out of sample testing that we can do. We also use multiple models so we can ask those models similar questions and see if they're coming up with similar answers. And that's really part of validating the response of the model. We'll also look at asking the same model the same question and see if we get the same answer back. Also if you change the question a little bit do you get a hardly different answer or do you get something that kind of reflects the degree of the change in the question. There are a lot of nuances like that or additional levels of review that you need to do to get comfortable with the models but I think what's most important here ties back to again us being a fundamental quant that we want to make sure that the insights were extracting with these LLMs tie back to the company's sales or the company's earnings. So being able to make that connection is important and again makes us more comfortable that the information we're extracting is valuable and the way we're extracting it will be beneficial going forward. All right Stacy we have come to the end of the episode and I want to finish with the same question I've been asking all of my guests to finish the episode this season which is what is something outside of work that you are currently obsessed with. This could be an idea it could be an activity could be a book or show music just something that has you completely enraptured at the moment. This might be a corny answer but my kids are in their 20s now and their adults and we are starting to have fun with them as adults so we go out to dinner and we were just on a vacation together and we play games together much different from when they were little. I'm really thinking about how does that relationship evolve and how do I keep it robust while you know still being their parent that's honestly taking up a lot of time and it's a lot of fun. Like kids live in the city we're in New York City all the time that's something I'm obsessed with at the moment. I think any parent will agree that that's probably an obsession throughout the whole cycle. I somewhat at the very beginning of the phase with very young kids at home. I'm thinking about the same things just a very different part of the age cycle so I love that answer. Stacy thank you so much for joining me this has been great. Thank you for having me.