David Li, CTO – WashU IMC (EP.69)

I'm Scott McDonald and this is Investment Management Operations. This show explores the inner workings of some of the most innovative institutions in the industry. Through in-depth conversations with leaders across operations, compliance, legal, finance and technology, you'll hear firsthand how key operating partners run their business in an ever-changing and complex investment landscape. You can join our mailing list and access capital-allocated content at capitalallocaters.com. My guest on today's show is David Lee. David is the CTO at Washington University Investment Management Company where he's led a lean data-driven build of the Endowment's tech stack. David's path runs from Harvard Management Company to independent consulting to wash you, bringing a builder's mindset to the role. For those not familiar with Wash Use concentrated co-invest heavy approach, Ted sat down with CIS Scott Wilson a while back and that conversation is replayed in the feed. David shares how modular architecture with Snowflake at the center gives his team cleaner data faster changes and less technical debt and how clear governance and practical security can make for efficient workflows. We also get into AI adoption in the Endowment world, where to use walled gardens, and how to spot tools that look great in demos, but fall down on data quality. For merging in season institutions, David shares concrete advice on bi-versus build, vendor selection, and running pilots that surface issues early, plus how to set expectations and give business owner's real responsibility so technology actually matches how investors see their portfolios. Please enjoy my conversation with David Lee. David, thanks for joining me today. I'm excited to dive into all things tech and where we are today, but first, I'd left to go back and hear about your background and how you made your way to wash you. Absolutely, Scott. Thank you so much for having me. It's a pleasure to be on here. If you could time travel back and tell college me what I'm doing right now, I would be utterly bewildered. It was not on my radar at all. I was the kid who didn't know what they wanted to be when they grew up. I still don't know. Except in elementary school, I knew for sure that I wanted to be a scuba diver, but it turns out you don't have to do that as a job. I went to wash you. It was between that and University of Washington, I guess I really liked the name Washington. I was originally in STEM. My original plan was to be a doctor, turns out that would have been a terrible choice for me, but I didn't know it at the time. And when I did actually figure that out, I knew that I still wanted to do something technical, something in science. I studied physics and PNP, which is a program at WashU that combines philosophy, neuroscience and psychology together. I really liked doing the brain sciences. Started doing neuroscience research out of college, and I just realized it wasn't for me, heartbreaking, but I didn't like that lifestyle. Just the full commitment. I loved the philosophy side way more than the neuroscience side, and I saw the senior folks in the lab and their lifestyle of writing the proposals, the grants, and then doing the research and writing the papers. I'm like, "I just don't want to do that for 30 years." This was 2009, not the best time to be trying to figure out what's going on, and I was like, "What am I going to do with my life?" My wife, Kristi, and I, we have this joke that I've been tumbling uphill for the last 15 years because I'm not a planner, it's just worked out really well. Right out of that neuroscience job, I went with my aunt and uncle, and they had this company that was doing legal work relating to finance. It was a good time to be doing legal work relating to finance. I didn't have any formal finance training or experience. I did have some experience with legal documents, doing copy editing, and also working on computer models, so that's how I got into that. As I was working there, I met the COO at Harvard Management Company. He encouraged me to apply for their analyst position, even though I was not their standard person coming out of State Street applicant. I applied, I got the job, I drove across the country, by myself, found an apartment when I got there, and I went in on Monday. What were you doing at Harvard? This was a time where it was really exciting because this was a huge machine. It was 200 plus people, it was an amazing place to cut your teeth. I loved it there, and I started on this team that managed one of the internal investment systems. To be honest, I'm not sure they really knew what to do with me at first, and to be fair, I did not know what I was doing at first. This was a baptism by fire because I had no finance experience beyond this legal work that I had been doing. For the first six months, I was going into meetings with my notepad and I was literally just scribbling down words, unfunded commitment, tranche, and then I would go back to my desk and I would look it up on Investopedia, and that was literally what I was doing for six months. I had no idea what I was doing. I did have some ability to add value, and I had some experience looking at these documents from the legal world, the work did revolve a little bit around that. Six months into it, I moved to the risk team, so I was focused on the liquids and alternatives that worked out really well for me because I did have this experience with private equity, so we did real estate natural resources, interesting things, and that's when I started to get that spark, I would say. This is a lot different than what I thought finance would be. I thought it was a lot of like, hey, let's pick some stocks. There are some really interesting PE vehicles in here. A little while into it after that, I had been enjoying my time on the risk team, the performance team asked me to join. I was hesitant because I loved the risk team, but I decided yes, and they were focused on reporting, and it ended up being really good for me because I'm the kind of person who sees a page full of text and sees one word being a different font. It's just the way my brain works. When I was there, I learned the importance of optics, so not just getting the numbers right, but telling a story with those numbers. It's funny because when I was in STEM, I had a lot of friends who were in business school. I thought that they were all in board rooms, talking up really complicated ideas. I learned that it was way more qualitative, way more optical than I ever expected. One time, I distinctly remember, I was at my desk, and the CEO came up during a board material preparation process, and told me that this shade of purple in my bar chart was too aggressive. I'll never forget that. She was completely right. It was too aggressive. I ended up being one of the first people to learn Tableau, that's what set me on this path of learning that it's interesting to build these graphics to get this story right, get the data right. I had some folks that were pushing me a little bit more towards front office of being kind of an investment professional. I realized around that time that I'm really not interested in painting for gold. And much rather sell the shovel, keep everything running smoothly than be that person who's trying to dig up the gold. This was 2014 or so. My now wife and I, we got together, and she had this opportunity to move to Hawaii because she's a physician, and there's this program there is very competitive where you can go there for a year as the beginning of your residency. She got that. I basically told Harvard like, "Hey, I'm out of here. I love you guys, but I'm out of here." I was very fortunate that they agreed to take me on as an independent consultant. I started my own one-man shop, moved to Hawaii, and it was one of those things where I was getting up at 5 a.m. and working, and then 2 p.m. I'd be done, and then, "Hey, let's go hiking." It was amazing. Was it the same job that you were just a contractor? Not exactly. I wasn't responsible for any of the day-to-day performance anymore like I was before. I pivoted more into being the Tableau consultant and building things, converting things that were in Excel, putting them in Tableau, or building on new capabilities, it was Tableau focused. It definitely helped to have that understanding of how the office worked, but I wasn't involved in any of the day-to-day beyond maintaining Tableau things that I'd built. Consulting life, it suited me really well. I love solving problems. That's the piece of the physics education that's still in my bones. I can't stand when you have to solve the same problem in the same motions over and over. That's why I said I would've been a terrible doctor, and when you're a doctor, you banish someone up, you tell them how to fix their problems, and they come in the next day because they won't do it. My wife's a doctor, she's okay with that. It's amazing, but sometimes we'll get pop-ups on her phone that come up all the time. She's okay dismissing them over and over, and that ability to be able to shrug off that persistent annoyance is an amazing personality trait. She makes her a wonderful wife and mother, but if I see that, I'm immediately in the settings. I'm hunting down the settings. I can't stand that. I'm all about that, using engineering solutions to solve everyday problems, reducing the everyday friction. What's the secret to a happy marriage? It's having at least two bathrooms. That's the kind of approach that I'm always taking with everything. What was the opportunity that made you say, "I'm going to actually explore this and move away from the consulting piece of it?" When I had moved back to St. Louis, randomly one day, WashUIMC, they reached out about building a tablet platform, so the COO there had gotten into a cab with one of my four boss at Harvard. They were trying to build something out at that time. It was a traditional siloed model, so there was a PET team, and there was a public's team, and there was a hedge fund team. They wanted to build something out for the asset allocation and risk group. Got involved there. It was an easy yes to start working with them in 2018 when they hired a new CIO, Scott Wilson, in addition to a generalist model, they wanted to redo the tech stack and they offered me that CTO role, and I couldn't say no. You basically got the role from a random cab ride. That's exactly right. That's the tumbling uphill. They were in the same cab in Seattle, and my old boss was like, "Hey, you know David Lee isn't St. Louis. It's unbelievable." Such a random walk. You couldn't make that up. Tell me about WashUIMC, what are you doing today, size of the company, all that fun stuff. Here apart of the university, it's a department, the AUM is as of the last annual report, $13 billion. The team is 22-25 people, depending on junior level folks and summer analysts. It's roughly evenly split between investments. operations. So we have a relatively large operations team and we do full shadow booking here. So pretty much everything comes into our office and we have hands on it. The investment team is a generalist model. We have coverage universes between specific managers, but there's a lot of cross-pollination across different asset classes. The approach here is one of the things I love about the organization, which is that it's highly concentrated. We have this deep research approach. So we really want to know what our managers own. We want to invest alongside them wherever possible. We're really looking to build that tracking error and find those opportunities that are interesting and asymmetric somehow. To go back to that idea of why did I take this job, I hesitated because I did love being a consultant. We really use something special going on here. It feels like a family on a mission. We love that environment. Coming in there is really fantastic and I love the forward looking viewpoint on tech and this investment strategy and the board of directors is really special. I've brushed into a number of them over the years. This one is really special. And going from a specialist to a generalist framework from your seat, at least that feels like a big move. So different people, process, technology, integration, tell me more about that period of time shifting from one type to the generalist. That is a big move. It was important for changing all of the outcomes that have to do with the endowment. First of all, the team ends up having to change. So we had a lot of folks who ended up leaving. We did keep most of the operation folks and a lot of the junior folks. It's an entirely different mindset than what a lot of people have. In terms of the technology, we were in 2018 on a platform called Backstop and it was an all-in-one solution. Every team had their own piece inside of there. As we're transitioning to this generalist model, what became really evident was that we needed a higher velocity of research availability and communication. What was happening at Backstop was that we were managing all sorts of things in there, including the notes and people were taking their notes in every note and one note and pasting them in maybe a week or two after and that definitely will not work for a generalist model where you have this deep research. The first thing is we said we needed to get this new research management system. We ended up picking Bipsing for that. That decision was essentially made for us that we were going to go with this best-to-breed model. So it's somewhat fortunate that we had this key turning point, this tipping point, to go off of and that caused everything else to be able to fall into place. What does your tech stack look like? Since we're part of the university, I don't really do anything infrastructure or help desk apart from helping people with random things. The university helps take care of that, which is fantastic. So you can think of me as sitting on top of that and doing everything above infrastructure. We also have shared responsibility on cyber security with the Office of Information Security at the University and we have an external consultant as we all say this year to help with cyber security. The best way to understand how the stack works is that the best-to-breed model and we're all about compartmentalization. We're trying to keep specific systems in their place and also make them somewhat replaceable so that we don't enter the situation like we had in 2018. We're trying to rip everything out at once. You can think of it like a space station where there's these modular nodes on the outside but a centralized core. In the center we have snowflakes that is really the hub of everything. On the outside we have the individual systems that different groups within the organization use for their specific tasks. On the investment side we really try to make BIP sync the primary focus point for their work. It's mostly research management. We also have CRM in there. We've piped up a bunch of other things from the performance system into there where we have tablo served up in this eye frame that's inside of there. We have salovus for the portfolio management system. Once again we're doing full shadow booking so there's a lot of operations folks doing work in there. A lot of power users. We have MFiles for structured document management so this would be official documents that are coming from managers, legal documents and things like that. We also have box for excel files and working files. We have our external data providers. They're more like market data. We have the pitch books. We have Bloomberg downstream of that more consumer or communication side. We have tablo which is what I was brought in to create and then we have Slack for real-time communication. We have O365. We have a number of other things like notion and development tools. The way that all of it comes together is that we have AWS as our integration platform. Essentially there's a lot of custom code and mostly Python running on AWS Lambda that shuttles data between the hub and snowflake and the nodes back and forth so we do an ELT type data pipeline to post up in and then we're doing most cases something like a reverse ELT to push back out to systems in a format that makes sense. It feels like we're in this moment. There's a lot of integration talk. Different nodes are actually trying to work with each other. Where do you think we're going with that? One of the big changes I would say that has happened and it's continuing to happen over the last few years is that people are starting to look at things data first. It very much used to be product first where you would have a product something like a backstop. The data lives in there but it's not really meant to come out of there. There are APIs that are somewhat supported. They give you the option of doing things like how to send you a cryptid hard drive with your data but the data was even on after thought but it was siloed. Now I think one of the most important technological side back end transitions that's happening caused by AI is that everyone is becoming aware that hey this data is critical. It used to be before there was a little bit of learned helplessness where if the data is in the system it's fine users will get used to it but now with these large language models with people wanting to bring their own data into the context that's not an option anymore. So I'm definitely seeing a lot more vendors they come out and even if I'm not in the call they'll come out saying hey we have an API where API first here's all the ways your data come out even before they talk about the platform they might start talking about how the connectivity works. That means two major things one is that's becoming more commoditized where you're just expecting that. That's becoming a must-have which will tend to drive the cost down. However that also means that as you're getting your hands on more data as the availability increases certainly the demands on storage will increase and the demands on transfer. Certainly you're going to have to pay for it at the end of the day but it remains to be seen how that's going to be bundled. So I don't know I would say for sure that your storage or transfer specific costs will go up. There are ways that you can optimize for that. It's more that people are starting to apply business and engineering solutions to how do you treat data as part of the service. It's going to be really interesting about how that gets bundled or how that gets charged going forward. Everyone's talking about AI. Tell me about your vision. Talk about adoption. I'd love to hear what's on your mind on that aspect. It's an exciting time right now but there's also a lot of uncertainty. This must be something like how people felt during the industrial revolution except they had 80 years. We're probably looking at 10 to 20. Who knows before I'll gonna have jobs at the end of this? I mean but people thought that back in the day. I don't blame the Luddites for what they did but now I'm literally looking at my kids. Are you guys gonna have jobs? I don't honestly know. It's a kind of time where we're all just trying to keep from crashing. I think a lot of us haven't even looked at the big picture problems yet. Are we going to have people like me in 10 years if there are junior level developers trying to make sense of what's right in front of us. It's white, knuckle driving right now. Keeping our lanes and not crashing to something. AI is really promising but at the same time I at least feel somewhat unsettled. Probably everyone does. There's this trust problem at all stages of the problem. There's this trust of obsolescence. Am I going to get replaced? That's foremost for a lot of people. Look at the products themselves and everywhere you look there's a trust problem. The inputs. The data quality becomes a problem. You can't rely on poor data to get you through anymore because the AI is sensitive to any data quality issue because you'll pick up something and who knows if it's the right thing. In the service level you get these huge question marks about privacy. Is the LLM vendor going to train on my data and am I gonna get in trouble for this? What do I tell my partners who are interesting me with their data and then of course with the output it's very much still a garbage and garbage out situation except now the garbage out looks and smells great. You can't even tell based on looking at that. You could also have gold going in and garbage out as well. It's honestly a really tough situation where these AI are so good at outputting something that sounds amazing but as soon as you get past the service level who knows if that's actually the right thing or not. One of the hardest parts for me is the use cases are so blurry. People come in and say I want to do all this stuff. I saw this in a YouTube video and trying to convert from that putting the sky into the actual actionable technology is difficult even more difficult with AI. In terms of our strategy we've been trying to navigate in such a way that we are responsible stewards of the information that our partners give us. We really value that trust. We've intentionally been slower than average and adoption up until very recently we had this model where we only allowed our data to go into AI systems that were a walled garden which would be where the vendor is running AWS bedrock or something like that where the data wasn't going back to the LOM vendor. That was at least solving some of that trust problem going forward our strategy is much more let's lean into it a little bit more. Let's be sure that we understand the governance be sure we understand the walls that we are going to set up and who has access to our data which we did a little bit more as open AI is somebody we can trust and give our data to. How hard is it to get to that point to know if it's a walled garden? Understanding whether a specific product is a walled garden or not is relatively easy because most vendors will be willing to tell you and if not then I'm not willing to work with them so we do a technical due diligence on everyone and it usually is pretty easy to figure out if for instance sending API calls or if they're running bedrock that's more logically isolated environment. Now the question of whether it's a good idea to go outside of the walled garden or not that is tough to answer. I feel like a crazy person sometimes I'm the guy who's got my foot in the brakes and trying to hold everyone back and any success full prevention may feel like a waste. It definitely has felt that way. I'm glad we're getting to the point where I think most of the industry is catching up to where everyone else wants to be. Everyone's trying to be at the top and talk with the data. I still want to talk about the data and data hygiene. This is a big problem. I'd love to get your perspective of the magnitude of getting to a great place with data in hygiene. It's that Warren Buffett analogy. The water's gone out and now we're seeing who's waiting without pants on. It's a little bit like that because it is really brought into stark contrast. Whether you have a real data strategy or if you were just getting along, people are learning that you can't build this tall building on a crumbling foundation. You can build it but it's going to come collapsing down and the user's going to find out on day one that the data is not there. That erodes a lot of trust. That's something that as a technologist you just can't afford to give up the trust is everything. I have always luckily had this approach where I really want to go lean and keep the data clean. We've been very lean for a lot of this journey. It's been just me in the office as a technologist. A lot of this stuff was me building it by myself and I get in there. I code every day. I might be in a little bit of a more advantageous position. I have to implement everything. When you are that solo cowboy, you see the results of your labors really early on. I learned really early on that data cleanliness is not a project. It's a lifestyle. You have to keep it clean otherwise it's going to trip you up very quickly. The best way potentially of thinking about it is it's not like you're going on a diet where you're just going to clean everything up and then look good for the beach and go out there. It's very much a lifestyle. You've got to keep it up all the time otherwise it is just not sustainable. You don't want to get in that situation where you want to move systems. So you need to do a bunch of work. Pack up your data and then get it ready to go because you should always be in a state of motion. You should always be ready to go. That's the only way to keep up right now. The things that work for us really well are front loading as much of that data governance as possible. Really understanding what are the best practices making sure everyone understands that making sure that people have the right tools to keep the data clean. I'm a strong believer that in human engineering solutions, things where trying to get people to do the right thing is easy. I like to say that my goal is always to make the path of least resistance the same as the path the best practice. I like to build these organic loops for people are seeing the benefits of them doing the right thing and then I'll encourage them to do it again. For instance, we have this document manager system called M-Files. One of the hardest things a technologist has to do is get people to input metadata. It's literally the hardest challenge. A system like M-Files is great because people have to enter the metadata. That's one thing. But then they immediately see the value. They see oh my goodness. I see all these different types of dynamic views that come up because I feel doubt these fields and that encourages them to fill it out some more. That's I think one of the only ways to do it without having a really strict approach and the strict approach in my opinion it doesn't really work. What about security, privacy issues related to all this? It fits with data, cleanliness, data hygiene as being an unsexy but critical piece of the foundation. You need to lay that first and make that part of your everyday. It's not an afterthought and it's not a once in a while thing. It's really important to make sure that the expertise exists. It's really easy to get lulled into this sense of I know what I'm doing. I'm pretty smart. I don't use the same password. I'm good. But having outside eyes is critical to making sure that this works. We lean very heavily on our chief information security officer, Chris Shull at the University about that and we also lean on our cybersecurity consultant a lot to help us understand what are our risks, understand the frameworks that we can use to analyze the risk. What's the entire surface of all the possible attack factors? How are we making sure that our AWS instance is secure? We are a strong proponent in having external help and doing penetration testing, making sure that we are at that right level where we have efficient security that is also easy for people to use. One of the most important things that people don't realize is that all security is about practicality. You can't have security if it's not practical. You can talk about having the most sophisticated security system in the world, but if it's not practical, if it's not convenient, then people are going to find ways around it. One of your internal people is going to put a hole into that wall trying to build that up, making it a conversation between experts and internal folks making sure that everyone's in the loop there is critical. So I want to talk about the vendor landscape. There's so many options out there. There's new tools out in the market that we haven't had before and also the anchor stores at the mall are changing a little bit. How do you balance what's interesting? What do you need to have versus what you're exploring for maybe a future node? Every organization is so different. This is something that I saw as a consultant. It's really challenging for me because I want to give this general advice that someone could pick up on, but it is so specialized in terms of what is actually useful for you. It's especially daunting if you're coming from a place where either you don't really have a text stack. You have a bunch of Excel files and a bunch of people who know how those Excel files work most of the time. Or if you're in a situation where you have a monolith where you have one system that does everything, it's really hard to switch. This is one of the big advantages of compartmentalization having been spoke model that we have where we have these systems that are somewhat interchangeable because then you at least know where you are. You know what Venn diagram circles do I have right now? If I want to pull this circle out, what am I covering? What am I over-covering? What am I not covering? It's really helpful to at least get to that point. Even if it means you have one giant Venn diagram over everything like we had with Backstop pre-2018, it's really critical to focus on those use cases. Find out what are your must-habs. Do some workflow mapping and figure out what do people do on an everyday basis? Make sure you have that information before you go into it because if you're starting with the product, you're definitely going to pick the wrong product. You can pick the one that looks the best and all the demos look fantastic. But the insights never look as good as the box. Anytime there's something really flashy, I always go back to basics. Go back to the foundation. What do we need? What do we have right now? What can't we live without? When you're picking vendors, I like to think about it as this idea of closed-ended creativity. You know, there's open-ended and closed-ended creativity. Understand the difference is important because when you're picking a system, you don't have an infinite number of options. You're not painting a picture, you're building a bridge, you have a start and an end and there's a gap in the middle and you need to figure out what fits in that gap. Once you start seeing it that way, I think it's much easier because you can start reducing that group degrees of freedom. You can start saying they don't have the level of data availability that I need. They don't have an API, so check that one off. As I'm seeing a bunch of these, I know that I want to have this. This piece needs to be in my bridge. Trying to reduce the degrees of freedom, trying to figure out where you start from is really critical to figuring out if the vendor's right one. As you said, it's incredibly difficult right now to keep up. There are vendors I hear about all the time and even as a person who does technology full-time, people will tell me a name I'm like, I've literally never heard of this name before. Getting that foundation laid out and understood is critical before you try to put anything fancy on top of that. How much is brand awareness matter? For most peers, it's very important, especially the ones that don't have a dedicated tech team. I think it makes a lot of sense because deciding when to act on something is a challenge, especially if you don't have somebody who's spending all day thinking about that thing. Most of the people in the peer group are going to be quite averse to being the first ones on there, and I think that's smart. Being a leader doesn't mean you have to be on the leading edge all the time. A leader can say, let's wait, let's gather more info, let's see what happens. You don't have to be in the vanguard. You can be in the reserve. With technology, there is very much often a first mover disadvantage, and that's true for the vendors. It's also true for all of us on the LP side that are trying to implement these. You could have a product where you implemented a data scraping product, and this was 2021, and then shortly after Chatchy PT comes out. Now you're in the situation where, well, I really wish I would have waited, but you can't always wait. That brand awareness is honestly huge, for instance, with BipSync. To my knowledge, we were the first E&F that was on there. Originally, it was focused on hedge funds or on long shirt equity that space. I had never heard of it when I started in the seat. One of our current COO's friends had recommended it to her. We essentially did the hard work of testing it out. We were impressed at the get-go. We definitely had a lot of faith in the system, but now you can see that on our peer group, it's everywhere. Everybody's talking about a lot of people have onboarded it. A lot of them saw what we did and said, hey, this is a good idea. Let's go with this, especially for folks that don't have a technology team. Brand awareness, it may be 90% of the equation. Going back to the whole bespoke approach to building your tech stack, I want to talk a little about finding that right fit. How do you think about Bipers' build, points, solutions, all in one, all that? That continuum is absolutely scale dependent. When you're a large organization, your scale is just different. You can afford to have a tech organization that's maybe a thousand people. At that level, you can have a lot of overlap, a lot of redundancy. You can take risks. Let's say you need a database administrator at DBA. If you're in a big organization, you might have 20 DBAs. If one of them ends up being a little bit of a dud, it's fine. You can find another one. You can try them out, but at a small firm, you don't have that. Even on a team like mine, we don't have a DBA, but let's say you have one DBA. If that person's not the right fit, you might be stuck with them for a while, and that role is not getting filled. It's very difficult. Small teams, they are constrained. The sense of they have to do a lot of buying. The building is often not an option. Even if they happen to have the monetary resources, let's say they were to try to go out and outsource to a consultant, which is something that I think people do a lot. In that situation, it can be helpful. I'm not going to say the consultants are useless, but consultants are good for certain things, not as good for other things. When you're outsourcing, a custom build of some kind of critical system. You're getting customization in exchange for that money outlay and also creating another point of failure. You're outsourcing the work, but you're also offloading that piece of your risk management to someone who has minimal skin in the game. We have not really considered that to be an option. We are always thinking about what's critical to keep in-house. Not just products, not just workflows, but what very specific things do we really need to have in-house that increase our enterprise value, that increase the robustness of the organization and the future proofing. Usually that has come down to things like really understanding our data, creating a viewpoint of the world through analytics that reinforces our internal view of the world. That's where the build is for us. Otherwise, we try to keep things modular. We buy the best piece to cover that need. Most of what I'm doing is integrations. For people who don't have any appetite for integration, which is a lot of people, all in one solution make a lot of sense for them. I'm definitely not going out there and telling people not to do backstop or dynamite. That's absolutely not the case. It makes sense for a lot of people, people who don't have that appetite, but you do end up with trade-offs. You do end up with a situation where it's not perfect for everyone, but if you can get around that, if you can satisfy then that can work well for some organizations. For organizations that are really technology forward and willing to invest, they're seeing the potential value of applying technology to create this structure of systems and analytics and technology that can provide a lot of value and create another stepping stone for leaping out into that next layer of gaining value and building that organizational power. What about the overlay of the technical capabilities of your investment team or the operation to the users of all this tech? How much does that matter? If I have a team that's really technical, that matters. Does having a more technical savvy team help with the path that you follow? It changes the path for sure. Having spent a lot of time talking to peer organizations working within them, what I found is that from a CTO's seat, it can be helpful or it can also be challenging because Edwashi, we have folks that are incredibly smart. Some of them are technical, but as a general rule, they tend to not get their hands to dirty with the technical aspects. They're more than happy to let me handle a lot of that and to work on the level of business understanding, which I think is a great place to work. The advantage of working in that kind of space is, as long as you also have the business capabilities, having come from this world of endowments, as opposed to purely a technologist, I have that background, so it's quite comfortable for me to say, "Okay, I understand the business, I understand your needs, I can convert this into a technology solution, as we go, we're going to iterate, we're going to make sure that this is what you have in mind." Converting that broad idea, that drawing on a page, if you're lucky, into an actual technology solution is not difficult for someone in my seat, but it probably may be more difficult for somebody who doesn't have that business side. Now, on the flip side, I definitely know of organizations where they have very technical people, they might have data scientists even, who are working in some kind of risk capacity, for instance, who are doing really sophisticated modeling. You have this other problem, you're trading essentially a no assistance from the business users on IT problem. Into this problem of, now there's this shadow IT, where you have these folks that know too much to be safe. They're helpful, but they know a little too much, they're a little dangerous. Trying to manage the content that they create is a different challenge altogether. When two people are technical, it's great because you can build some really cool things, managing those really cool things can be a challenge. And we're at this moment too, we've got a lot of legacy issues, we've got AI coming in, there's a lot of note takers that are coming into the market, we've got document collection, data scraping, that's getting better. How do you balance all these things and what's important? It still goes back to understanding your use cases and drawing out and understanding what are the pieces that you need, how do they fit together, because there are so many options out there. It's a little bit like going to Cheesecake Factory and seeing 30 pages of menus, what are you going to pick? It's really hard. The first step is to compartmentalize to understand how are these pieces going to fit together rather than being focused on, hey, this specific product looks really awesome, let's do that. It's honestly becoming even more confusing, unfortunately, because the capabilities have grown so much, different products are becoming very similar to each other. Those lines are becoming much more blur than they were before. When I first started here, we were looking at traditional OCR products for doing document scraping. There are some compelling solutions that were out there, but nothing like what we have now and the decision that we would have made five years ago would be very different than the decision we make right now. There is now this opportunity for teams to come in and with very minimal effort with minimal training on their own part to really have that full suite of surfaces from collecting the document all the way through getting the data out. One of the big changes here is that we're seeing a lot more of these all-in-one solutions that are becoming pretty compelling where they don't come at it from the perspective I'm going to give every different user, every different option, but we know what you do. We're just going to connect the dots and as long as you're okay with how we connect the dots, then it's going to work great. A lot of smaller organizations are going to see a massive boom from this. We're okay fitting into this. Give us what we need, we'll adapt a little bit. For groups that are a little bit closer to us, where we have that technology appetite and we really want to make it a very specific way. A contrarian team needs contrarian technology. Some of those solutions are not as good for us. If we're not in that sweet spot where our use cases are really nicely overlap by that mindset that that vendor has, then we're going to be fighting against their optimization. That's been a tough one for us. I'm figuring out the balance between having this nice solution where we have this white glove service versus the white gloves aren't handling our data the way that we want them to. It's very difficult right now. Being laser focused on those use cases is the only way to make sense of it all. Must be confusing for some vendors when they have a conversation with somebody like yourself, because you guys are in a unique position where you have a concentrated book. If you have SMAs, that changes what you get. The needs and requirements are very different than if you had a thousand managers and you're just trying to keep up with the flood of information that's heading your way. It is a challenge across all stages of the investment management life cycle because when we are looking for managers, most of them don't fit into this mode. Most of them aren't willing to give us that level of service and not willing to get into the structures doing co-investment. On the operation side, we are dealing with these often specialized structures that we have to have to fit these vehicles in so that's a challenge to the operations team. The finance landscape is so interesting. A lot of people that are from the outside, they see it as one monolithic thing, but what's your inside of finance? It's so different. I get all these people reaching out and they think that we're some kind of reinsurance firm or bank RIA. This is nothing like what we do. I have that conversation a lot of. This is really interesting, but we don't do this at all. This often happens as well with people who might think that just because we're highly concentrated that we have this really intensive internal training platform and we don't. We lean heavily on our external partners to do the transactions and manage the investments for us. A lot of times we'll get these contacts from vendors where they're like, oh yeah, let's talk about our trade management system, how we deal with clearing houses. We don't deal with any of that. There's a lot of thank you, but no. That's also applies to recruiting as well. It can be quite difficult to try to find people that fit this mold of we want this lean team where we have technology journalists as well and they have spent a lot of time as I've been recruiting trying to talk people out of the job. Are you sure you want to do this? This is what the lifestyle is like. It's not like you're going to a hedge fund and you're going to be working on high frequency trading. This is a very specific way to look at the world. I want to turn and talk about where we are a little bit of a hype cycle. Do you have any definitions in your mind what is good AI today versus bad AI? For any AI part that you're going to use, especially one that's going to generate an output, there are minimum requirements. You really want to have transparency. Mostly when it comes to AI, how much can you trust it? How can you test the inputs? How can you test the outputs? There's certainly good and bad AI in terms of the purpose, the potential for bad actors to go in and use that for deep faking and trying to convince people that they're the CFO asking for some transaction. Scepticism is warranted. The product never looks as good as it does in the box. A lot of times people will go in and they'll see some amazing demo. They'll onboard the product and they'll realize that the demo environment was carefully manicured. If 10% of the data is in the wrong place, then it's going to pick that 10% on data. You're left with this amplified trust problem, especially if you don't have that level of transparency. I'm always looking for that transparency. I'm always looking for a partner that understands this is in a situation where I'm going to throw everything in and then I'm going to be hands off and just you come and give me the output. I'm going to be there every step of the way. If I can't get in there every step of the way, I'm going to be very skeptical about the product. Another thing that we like to look at is what our long-term user interface is going to look like. One of the struggles in the space right now is this idea of siloed products versus generalized products. Every product is going to have AI embedded into it. I remember the day when Adobe Acrobat AI came out and it was plastered all over Acrobat. My first instinct was, oh my goodness, what are you doing? I can't have my legal documents in here next year AI. All that is happening and the pace of change is incredibly fast. There's this question mark about, are we going to use the AI products that our existing vendors already are providing to us? Or are we going to look for a top-down solution that will make all of this look more unified? The first inning of this AI push, it was a lot of we're just gonna give you access to your data in the products. We experiment with a lot of these. We have Slack AI. We've had some success with the M files AI product, which is called I Know, which goes in and runs a GPT on top of your documents. We've experimented with BIP syncs AI and those kinds of things. In every situation that's been a place where I thought, this is interesting. I feel like the user experience is not gonna be what I want people to have because I want people to have a unified experience when they're talking to AI. I don't want them to have to think about, oh, my data is siloed over here and having those vendor provided AI products tends to create those data silos 'cause it's to put up those walls. Ever since the GPT came out, I've been really excited about this possibility of having a unified chat interface. I believe strongly that we want to have one interface where people can go to look at their data across different platforms, not to have these piecemeal solutions. And originally, the main one that was out there was Microsoft co-pilot. I never had that much faith in it because of the way that they were approaching it. And we're not really a Microsoft shop. For somebody who's a Microsoft shop all the way through, I'll use all Microsoft products, it is a super power. But for us, it's a little bit outside on the edges and most of our peers have not had the best experience with it probably because of that data cleanliness problem. I'm still waiting. We just stood up a chat GPT for enterprise environment, looking to see if that could work. Also interested in players that are clean, they're out there and that's where they've started from with this idea of we're gonna give you one service. We also have snowflakes in the middle and the data's all already there. So snowflakes coming out with some really interesting and compelling user facing products. There are a lot of folks in that space right now, but I always lean more towards the long-term solution has to be one that's more unified. - You as a developer in your background, is that create challenges around technical debt? - As a developer and somebody who has to run really lean, I am always thinking about technical debt. That's what I'm mainly optimized for. The only way to have sustainable growth in your tech apparatus is to minimize the technical load and the technical debt. Anybody can build really fast, but after you get past six months, 12 months, you're gonna start feeling that drag if you weren't careful about how you're building. And a lot of it has to do with figuring out how to compartmentalize that complexity in the most efficient way. We have snowflake in our hub and spoke as a center and it's not just because snowflake is a data platform. SQL and snowflake is part of our strategy to manage a complexity. Complexity exists, you can't get rid of it, it's just where you're gonna put it. Originally we had tried this thing where we put that complexity into the data pipeline pieces that move the data that's more of an ETL framework, but that was a disaster because the rate of change in those different data transfer needs is too high. We ended up going to an ELT framework where we're loading the data raw in and then we're manipulating it downstream. That's completely flip the script on how fast we can iterate, iterating in SQL is so fast. One thing that I've learned in terms of managing this tech debt is nowadays you have to minimize the innovation iteration cost. That's failing fast. That's how you make it so that you can be tactical and not be stuck in a system for years and years and years. The keys to that are understanding your data, keeping your data clean, having this framework where you can pick up and go at any time. Snowflake is part of that. Managing that tech debt is very much also about staying lean and being focused on the right solutions. Technology has this amazing potential to be a value ad. A lot of organizations started out thinking about technology as being just a cost center, something you've grown about. Your level two thinking is maybe we can make this more efficient and level three tech can be something that can empower us. Understanding where that empowerment happens or where it should happen is important because you can throw technology at anything. It will create a lot of technical costs potentially. You want to make sure that that technical cost is giving you a return to your investment. Working closely with the investment team or the stakeholders and understanding where are their pain points and which ones are the ones that best fit technology solutions is very important. It's even more important than going out there and picking the system that seems like it will work the best. Starting with the fundamentals, starting with the people, starting with the workflows and understanding which ones will fit into those well is huge. It's not just tech debt. We can increase the velocity of the technological innovation as much as we want. We can make this situation where we have compartmentalized systems able to be switched out as quickly as possible, but humans can't handle that speed. System fatigue is absolutely a real thing. We can't be changing systems as often as we're changing tech vendors and that's especially true now where the pace of technology innovation is so fast. You're looking at a product in six months and it looks completely different. One thing has helped us a lot in terms of managing not just tech debt, but also that human ability to handle change is trying to keep the front in as consistent as possible. We try to make BIP sync that core interface, the key workspace of choice for the investment team where pushing things from other systems into there, abstracting that sausage-making apparatus from the end user, we're decoupling the system of action from the system of analysis, pushing things really into that system of analysis. That's really what managing a best debris stack is all about is trying to make sure that people see you what they need, keeping their UX consistent. Things like BIP sync, Tableau, we're building our own reporting platform. If you were to swap out Solovis or something like that, that could remain relatively unscathed. That is really important for making sure that humans aren't the ones that are suffering when your technology velocity gets really high. - Why BIP sync over Solovis or other PMS system where you're actually looking at performance and returns, why start there? - Backstop, Dynamo, they're great. The main problem you're running to is not that they are worth that innovation. Rather, it's that the user and client base is too diverse. You have a million different organizations that are vastly different from each other, that have vastly different needs. They're all pulling in a million different directions. And that creates a product that has to sit somewhere in the middle that comes with a lot of trade-offs. The same is true when you're thinking about how users are using systems. It's a good idea to break down silos, especially when it comes to making your data more available. Liberating your data. One thing that I've noticed in the last few years that has caught me by surprise is that I don't think the same is true for governance and for the end user experience. We definitely wanna get the data out. If you think about breaking down the walls in terms of governance as well, you end up with a situation where there's just too many cooks in the kitchen where people are asking for too many different things and you're never gonna satisfy all of them because anytime you design a system or you onboard a system, you're making decisions. You have trade-offs, whether you're aware of them or not and whether you have made a decision or someone else has made a decision, there's these trade-offs. You always end up with this situation where the quality is suffering because you're having to make these trade-offs. I started thinking a lot more about re-putting up some of the walls of the silo for governance perspective, identifying key individuals that own that process, that own that system, routing everything through them. When you empower people in that way where you're bringing the data they need into the systems they use, even if that data doesn't make sense to be managed there, it creates this opportunity for them to get to know their space cleanly and communicate that to you as a technologist clearly as well. We're trying to make BIP sync a place where they can go to get 90% of what they need versus operations going into silovis most of what they need because we really wanna have a situation where silovis can be optimized for the operations team for how they look at the world, the way that they handle legal entities and transactions, which might be different from how the investment team looks at it. So if we can keep those two lenses, those two workspaces somewhat separate, but then user integrations to length between them, it allows us to have all that data the place where it needs to go, but not have as many of those trade-offs between different types of users. - I would guess that's where Tableau also plays an important part. The reporting piece is an underappreciated aspect of all the things that we do. - As somebody who lived in Breeze Tableau for 10 years, I absolutely believe that. The Tableau side, I think, has to do a lot with customizing the viewpoint of the world to how we look at it internally. You can do that with external systems with silovis. None of this is a nox silovis. It's a very powerful system. We really wanna have that flexibility. So for instance, we have our own customized attribution reporting. We have various other types of reporting that live really only in Tableau. The beauty of that is a number of things. One, you can avoid system lock-in. If all of your reports are built in silovis and you wanna move off in silovis, you have a hirky lay-in task in front of you. If you can get it done, your users have to get used to using another system. Abstracting that, having a display layer, an output layer that doesn't rely on those systems that works off of a semi-canonical model that is based on your internal view of the world, that smooths out that experience. The second thing is you can be very specific about how you look at things. You don't have to think about this report is really designed to do this thing for everyone. If I can customize it and try to get what I want, the problem you run into with that is a lot of times you run into these rough edges where the report is really not meant to do what you wanted to do. And you might not realize that unless you're a power user. That's one of the key problems that we've run into when we've had non-operations folks that are in the performance management system is the output may not be how they look at the world. If we can get into Tableau or we can get in Bipsink and we can say, hey, I'm gonna output this data really the way you look at it, you don't have to have that question mark. This is how we talk about things. It's not how the vendor talks about things. - Could this be an example where generically here's our asset allocation, but exposure wise, I'm gonna take my hedge funds which are all tradable marketables for the most part and lump them in with beta. - Yes, and this is one of the problems that we run into with this custom attribution model. The way that we manage our asset allocation is not like a traditional model. We have risk assets and we have diversifying assets. The divisions between those aren't clean. We're not even saying that all of equities is risk asset. We have some blend of those. That blend, we can output that using a property in celovis, doing analytics on that can be challenging. The analytics that are in the system may be based on having some strict one-to-one relationship between your asset class selection and your risk profile. We have the ability to get outside of that. We want to look at the world this way. This hedge fund is 60% risk assets. The rest of it is diversifying assets, and that's something that we can do when we have that customized platform. A lot of exciting stuff there. How do you think about budgeting? How do you advocate for all these interesting things? There's half the haves. There's some must-haves and there's nice-to-haves. How do you advocate for the right budget? One of the reasons why I took the job and left consulting was that it's a very technology forward organization, one that understands that to have these outcomes you do need to put some time and money and people toward the problem. It's probably easier for us than it is for the average organization. We definitely want to have a compelling use case, a compelling reason to onboard anything. We're always thinking about how can we cut the cost? Can we use this as a replacement? Can we swap something else out if we bring this in? When it comes to managing costs and keeping that from ballooning out of control, what's your Venn diagram look like? What are the things you're trying to cover? Knowing that in mind will give you an idea of if I were to increase the spend, what's that covering? How does that affect how I think about the rest of it? If you don't have that framework in place, any spend, all it is is more. You don't know what that's going to be. Understanding what you have is the only way to make sense of all of that. Looking forward, what do you think the tech stack vertical will look like down the road? What excites you these days? There's so much going on where in this transitional phase, it's so hard to even see right in front of your face sometimes. The best example of this is Glean recently. I had a peer come over to me. Did you see this six months ago? This does look at the same product. It's hard when you're somebody who needs to have those forecasts of what things are going to look like because things are changing so rapidly. AI has made a huge change. Everything that we're going to be doing going forward will have AI somewhere in the equation, even if it's not the forefront where I was going to be thinking about, is this something that we could apply AI to? Something more broadly talking about the E&F space in general, a lot of all-in-one solutions are coming out that will be attractive to smaller shops. We've talked to folks like Flayed and Finn Pilot, their solutions, could be very compelling for shops that just want to come in and say, "Do everything for me." That white glove has changed dramatically into being something that can be compelling and fit quite well for a lot of organizations. That's really interesting. I'm not sure that that'll end up being where we go. We have a contrarian view. We have all these custom things that we like to do and we like to get them exactly right. When you're at a level of resourcing like we are, those solutions are probably not foremost in our thinking, but we're definitely thinking about how AI and able products and AI first products are coming into the stack. We started out trying some of those AI products that our vendors were offering to us and some of them have been successful and some of them have been less successful. We're trying to find places to put AI into the workflow so people can really focus on the decision making but also be focused on the quality as well. We're always looking at that. One of the most important things that's going to come out of the next few years, especially with those all-in-one solutions, is going to be more clarification about what use cases AI can really put a dent into. One of the hardest things right now is we just don't know. We don't have enough experience to know which use cases are going to work. Which ones are never going to work because the data is not there or we need to wait for the next leap in AI before we do a certain thing. It's going to become a lot clearer and that's going to help direct our decision making. We're looking into all these options for document scraping, document collection. We're also looking into finding that fronted AI interface, try to pick that winner. Is it going to be open AI, is it going to be anthropic, is it going to be something like Glean. I was also thinking about how do we use AI to scale that front end and the back end of the same time. All of this to say that the tech stack is going to look very different but at the same time, many things are going to stay the same including making sure that we have that solid foundation for data. So that's going to cause the core to probably not change that much. Sounds like we have some homework to do in the industry on the data side. That's for sure. There's so much going on and you have so much on your plate. So it's a natural question to ask you is how do you stay productive and focused and excel in all these things at the same time. Juggling is not easy. I'm not the best at saying, hey, this thing is going to be done on this specific date. I don't like to say no and so a lot of times I'll just say yes, yes, yes and I try to get things done when they need to get done. That also means that trying to manage all those projects and tell people exactly when they're going to get done is difficult. I tend to put that effort more into I'm just going to do this right the first time and make sure that it's right for you. Even if it takes maybe a little bit longer than someone's expected. I have a couple of interesting things to say about productivity. My entire life, I've been a terrible procrastinator. Honestly, I've pulled so many all nighters in school. Then advice I give to anybody who is a procrastinator. That has worked out really well for me is I've turned into productivity by procrastinating on one project by working on a different project. That's honestly been a huge vessel for me to get work done because a lot of times I'll be sitting there like, I don't want to work on this right now. I happen to have a lot of irons in the fire and so I'll just switch to a different one and I'll be like, oh, this is interesting and that'll reinvigorate my brain. People have described me as a lazy person and I will describe myself as that. Bill Gates has this quote about when he wants something done the most efficient way that he gives it to a lazy person because a lazy person will find the way that requires the least effort to do it and I've leaned pretty heavily into that. The way that I do that is I'm always optimizing for how to minimize the expectation value of the total amount of time I spend on something. Doing things right, the first time means you want to fail really fast in the beginning when you're in skunk work. I do a lot of that but then eventually over the long period of time when I'm trying to do is not have to come back and do it over again. The best way to save time and money is to do things right the first time and so I've used my status as somebody who hates to do things twice as a power to come in and say, I'm not going to be satisfied until this is the way that I think is going to be sustainable. Both of those things, which originally I would say were character flaws, I think I've managed to find a way to turn them into something that's productive for the organization. Is it hard to manage people's expectations if you're in a CIO seat? Is that hard? Yeah, if you're thinking about the three big challenges that I have to deal with, one is getting people to do the right thing. Second one is managing technical debt, which I was talking about. Third one is managing expectations. This is something that Consulting taught me a lot about managing expectations. It's huge and it's not just about under promise over deliver but it's really being honest about what you can do and what the product is going to be. The key for me has been trying to work really closely with the team to understand what they need, what they really want to have. Even then, it can be a challenge just because a lot of people have a lot of different expectations. They have a lot of different needs that goes back to my idea of centralizing the management and the ownership of specific things that's been really helpful just because originally I was trying to do this thing where I would talk to everybody and try to collect all this information about, "Hey, you want to see it this way? You want to see it this way?" I would love to promise to do it that way, but then I would run into the situation where I would change something. All of a sudden, someone else would come back and be like, "Hey, wait, why did you change this?" I like that the way it was. I was a human thermostat getting pushed in every direction. What I learned is that the only way to manage the expectations across a team of any size over a couple is you really have to have those key stakeholders, champions who are the ones you go to when there's only one way you can do something to go talk to them about this is the way it's going to be done and a little bit of that responsibility sharing in terms of managing expectations between you and them is helpful. There's a lot to learn here. I have a lot of work to do about making sure that everybody knows where things are going and they're satisfied, that's a constant process for me, especially with the way that we do things of trying to juggle a lot of things at once, but hopefully my team, they see the progress being made there and understand that it's all a good effort of trying to make sure that we can get people all the things that they want if possible. On top of it, you have understanding what's involved. Some of these things, it sounds easy, but it's phenomenally hard data labeling and the number of things that you're going to ingest and how do you tag it and label it is just a mountain of information. That has been one of the biggest challenges for me as somebody who is technical who comes from STEM background and used to talking to people in terms of art. A lot of feedback that I got my first three to four years here was that people don't want a much of this tech chart, take them out of the wheat and I love to explain my thinking and I love to talk about the details that can be overwhelming for people who aren't living in there from a day to day and so one of the hardest parts for me I think has been trying to make sure that people understand the complexity that's under there, not just for the sake of understanding my plight, but also getting them to be open to the idea of talking about the details because when you're starting with, hey, I want xyz done, you have to convert that into a hundred different steps when it comes to the actual technology implementation. One of the things that I tell everybody that I've ever hired is that you're going to run into the situation where the business owners don't know the technical implementation and they may not want to know. You have to make sure every step of the way that you're bringing them in and helping them to see where you are and where you're going. The analogy I like to make is your cab driver and you get into the city, someone gets in and they say, take me somewhere warm and they go to sleep. You drive them down to Florida and they get out like, oh my god, it's so humid here, what are you doing? Try somewhere else and I'm like, oh my goodness, there's so many bugs. If they told you ahead of time, I want to go somewhere warm where it's not humid, where there's no bugs, you would have taken them to Arizona or whatever to start with. They don't tell you that. I tell everyone who starts with me that you really got to help them taste as you go or help them feel the pain of choosing this option. How's that going to affect their use case? How's that going to affect their life? - Explaining that to the users is difficult. It's one of those soft scale challenges that I'm continuing and developing because it is very difficult. - I like that analogy. David, this has been really insightful. And I'd love to turn and close with two questions. One is what advice would you give to an emerging manager from a technical perspective? - Don't go chasing waterfalls. Don't run after the flashy things. You really want to start with your team, your workflows, your use cases, figure out your must-haves, map out the things that you do and how that fits in the systems, and then figure out where things go. Don't start with the product. Always start with the use case. Always start with your needs. The best way to save time and money is to do things right the first time. Sometimes doing things right means that you're gonna fail early on. Give yourself time to sort things out. Sometimes the best way to do things right the first time is to fail a bunch in the beginning. You will never get things right the first time. When you're implementing something, there's always a bunch of unknown unknowns. You're gonna go into there and realize on day one, oh my goodness, I didn't think about this. Where are we gonna need this? Don't put that pressure on yourself to get it right the first time and make sure that you build that into your release plan so that you're not pushing to go live too early. You don't want the moment when the average users are being the system to be when you figure those things out. It's important to spend a lot of time thinking about your use cases and your best practices that you want to enforce. This can be difficult when you're looking at new products because you just don't know what you don't know. This is an absolute challenge. I've told people all the time, I really wish I could dump my brain into yours so you know what you're getting into when you're onboarding Solovus or Bipsink or whatever, but it's not something we can do yet. When you're comparing products, when you're in this green field situation, I think it's really important to narrow it down based on what degrees of freedom can you exclude? What are your must-haves? What are your cannot handles? Always run POCs, always get your hands on the system. Your goal in those POCs is to flesh it out, poke at every button, find the unknown unknowns before you sign wherever possible. I am a huge proponent of front loading that work of discovery of making mistakes as much as possible. - The other question I have is what one book article or other resource you commonly refer to people? - One of my favorite authors in the space that I live in is Edward Tufty. He's considered the Galileo of graphics and graphic design. I think he's currently Yale as a professor. He has written a bunch of books about information graphics and how you present information. It's like adult children's book in the sense of, there's so many pictures gorgeous to look at, and they tell you all sorts of insightful things about why you shouldn't use 3D pie charts and things like that. I couldn't recommend it enough. Even though it's something that's maybe not in your core flow when it comes to investment management, you'll be really startled at what amazing insights you can get into the little changes in how you decide to present information that make a huge difference. I also think it's really important and really valuable to talk to peers. Get outside of your comfort zone and understand how other people are thinking about the world. And I don't mean just talking to peers where there's a similar level of sophistication. I talk to other CTOs all the time. I also love talking to peers where they're just getting started in their journey, where they're not that sophisticated in terms of the technology they're using, because those are the cases where you have these really interesting use cases and that always gets my brain going about, what can I take from that to apply to what I'm doing? How can I jog myself out of the loop that I'm in using those new use cases I've never thought of and apply them to come up with some elegant solution that I hadn't thought of before? - Beginner's mind. - Exactly right. - This has been a great conversation. I learned a few things. Thanks for joining me. - Absolutely Scott, thank you so much for having me. - Thanks for listening to the show. If you like what you heard, hop on our website at capitalallocators.com where you can access past shows, join our mailing list and sign up for premium content. Have a good one and see you next time. (upbeat music)