S6:E10 | Building Compliance in an AI World | Compliance in Context

[Music] Hello and welcome to the Securities Compliance Podcast presented by the National Society Compliance Professionals. We're at a's our mission to help you put compliance and context. I'm your host Patrick Hayes, part of the CalPhe Law firm and on today's show we discuss how to build a culture of compliance in an AI world. At some other best practices firms are using now to build AI into their respective firm operations and compliance programs. Moving into our headline section, the SEC raises the qualified client threshold for performance-based fees and SIFMA re-erges the SEC to overhaul its communications retention rules. And finally, we close up today with another installment about takes where we review a recent enforcement action involving fraud and registration charges against three venture capital fund managers and their owner. The SEC recently issued a notice of intent that would adjust for inflation the dollar amount thresholds under the Advisors Act that permit investment advisors to charge performance-based fees to quote qualified clients. Under the rule, an investment advisor may charge performance-based fees if a qualified client has a certain minimum net worth or a minimum dollar amount of assets under management with the advisor. Since amendments in 2011, these minimums have been set to update every five years. Past updates in June of 2016 and in June of 2021 push the thresholds higher to reflect inflation. But 2026 adjustment is now expected which will impact private funds relying on 3C1 and advisors to high net worth clients that are charged performance-based fees. Qualified clients are authorized to be charged performance-based fees such as performance fees and incentives or carried interest. A qualified client is defined by two financial tests which the SEC is seeking to change. The net worth test which is the advisor must reasonably believe that the client has a net worth above the minimum threshold excluding the value of the primary residence prior to entering into the advisory contract. The current minimum net worth is 1.1 million and the new minimum net worth will be 1.4 million. The other test is the assets under management test. Here the client has at least the specific minimum dollar amount under management with the advisor prior to entering into the advisory contract. Again the current minimum net worth here is 2.2 million and the new minimum net worth is 2.7 million. What does this mean? Well essentially any existing advisory contract executed before the change will only be able to rely on the threshold at the time of signing however after the SEC issues a final order new clients and investors must meet the new thresholds on or after that orders effective date. Obviously you want to review your compliance program for any private funds that might be relying on the 3C1 exemption and the advisory agreements again similar to kind of the 3C1 exemption under private funds. You want to confirm any references to qualify client status in your advisory agreements and contracts that may need to be revised. Look for other eligibility forms and make sure that your compliance manual and any related policies and procedures and operations have also been adjusted accordingly. Moving to our next headline, SIFMA supplemented a previously submitted letter recently urging the SEC to modernize the agency's communications record keeping framework. In the new letter SIFMA provided survey data which showed that the record keeping rules impose enormous costs on broker dealers and investment advisors while generating far more data than the regulators need. In the original letter which was dated from last October SIFMA asked the commission to amend the communications rules to clarify the types and scope of communications that must be retained under the federal securities laws and to provide safe harpers for compliance with each of the communications rules. These rules are applicable to broker dealers, investment advisors and security base swap dealers including the Exchange Act Rule 17a4, Exchange Act Rule 18a6, the investment advisors Act Rule 2042a7. In the supplement of research SIFMA and the SIFMA asset management group found that large firms retained as many as 28 million communications on a single weekday. Over one trillion communications per year while smaller firms retained up to 250 million annually. 44% of surveyed firms reported they had not permanently deleted communications their own policies deemed non-essential citing potential enforcement risk. In addition annual storage costs range from 80,000 to approximately 37 million per broker dealer and from roughly 500,000 to more than 10 million for investment advisors. Firms also reported the voting between 2000 and more than 42,000 staff hours annually to retention compliance separate from supervision and surveillance. SIFMA attributed the over retention as it called it to rule ambiguity and enforcement risk arguing the framework encourages defensive data hoarding. SIFMA said that firms relied on between three and fifty third party vendors to capture communications across email, text, chat and video and represented a very fragmented architecture. SIFMA said results from applying legacy rules to a rapidly expanding ecosystem. SIFMA further warned that the burden is growing as firms address compliance questions raised by a generative AI. SIFMA then proposed narrowing required retention to communications involving registered persons with survey respondents estimated would reduce volumes by 15 to 45% and generate annual savings of up to 5.5 million for the largest firms. It also urged the SEC to reduce the retention period from six years to three which advisor said could cut storage costs by up to 50%. One time limitation costs were estimated at between 6,250 and 500,000 depending on firm size. In the original letter SIFMA had proposed to eliminate the requirement that a third party, such as a cloud service provider, must file a third party commitment to provide access to the broker dealers' documents of under Exchange Act rule 17A4-1. Shout out to Fairbrew Investment Services and Northern Rose Fulbright for providing some of the content of this update. As we move into the interview section of today's show, we are going to be talking about a topic that probably many people really can't escape from not only every day, but probably every hour without hearing people talking about artificial intelligence or AI. Obviously the past few years have been a crazy entrance really of AI into most people's lives. Certainly there were lots of us that had been delving into related topic areas over a longer period of time, but I certainly think the adoption rate of AI and how it's really impactful not just in areas like investments, but certainly even in areas like compliance as well and how in operations and how firms are using it has just increased, you know, what feels like not just 10 fold but 100 fold. So to help guide us through such a substantive topic, I am very very pleased to be joined by Mr. Eric Olson. Eric is a stalwart in the compliance community, helping lead the charge and being involved in lots of different organizations across the space. Eric, I'm so glad that we finally get to do an interview, especially on such a fantastic topic as this. Thank you so much for joining the show today, really looking forward to the conversation. Thank you for having me. It's great to be here and thank you for those kind words about my background. I appreciate it. Yeah, if you win mine maybe talk to us a little bit about, we'd love to hear just a quick, you know, kind of bio where are you now and tell us a little bit about about your background. Sure. So I'm the chief compliance officer at Advaiton Asset Management. We're about a $25 billion ETF sub advisor based in Alfredo, Georgia, just under 40 people. Been inviting for almost five years, so I guess be five years in summer. Before coming to Biden, I spent around nine years at ACA Group as a managing director and heading up their registered investment company compliance practice. Before joining ACA, I was at the Lincoln Basin for six years, heading up their internal compliance examinations team. And before that, three years at the SEC and what was the office of compliance inspections and examinations? Now just the division of examination and started my career, George, George Asset Management, doing fun accounting and fun legal. So a little bit of a curve there, but yeah. Well, no, I look, I think that all of that experience is going to be something that is going to benefit our listeners today as we really dive into, again, what can be at times a very tough to tackle subject matter area. But I love the fact that you've got both experience on the regulator, side, experience on the industry side, and in working inside of firms in a variety of different roles. And I think, you know, understanding that they're going to be different legitimate business interests, depending on the particular constituency that you're dealing with and what their focus might be. That's all gonna be super helpful. So, what I'm gonna show is we dive into the topic, and I think one of the things that I know you and I had been talking about is you hear the phrase a lot culture of compliance, right? And we know some of the other catch phrases that often go with that, tone at the top, right, and other stuff like that. But certainly in an age of AI, in an age of artificial intelligence, I think understanding how firms can continue to foster that culture of compliance can be something that's challenging. And so I guess just to kind of kick off the conversation, more broadly, right? But, you know, what are some things that you're seeing? How are you seeing firms that are successfully using AI both inside and outside of compliance? >> Yeah, sure. I mean, like AI, we know it's this big, mysterious thing. There's lots of definitions of, we can start AI and go in between and what it is and what it's not. So it's interesting. But it's still in technology. It's still the next evolution of something, the way back, if we're going back through our careers, right? Starting with email and instant messaging and teams and Zoom, like we're on now, right? They've all had to be integrated somehow and where you kind of start is, what is it, what kind of do, where is it fit into the firm overall, of course. But then as, you know, we practice compliance professionals, legal professionals have to understand where's it fit into the regulatory scheme that the SEC gives us or if you're a broker, you're listening. What's the scenario to you? >> So in some ways, I'd like to believe we, as compliance legal professionals can get our business colleagues to crawl first in looking at what, in this case, AI, what is it going to do for us, what it can do for us, what we want to do for us and crawl and get into that comfortable, what is it, what are we doing here? And we'll talk about other areas, but the biggest part is governance, the AI governance structure. I know we'll hit on that and a little bit in our conversation. But that's the crawling part. You got to have that set up and what that looks like first. And then you can walk a little bit to rolling out some AI and what that looks like. And then, you know, the analogy, then you can start running. Then you can start building where there are big, generous, very gentick, but starting at the basics and then working away up. If you skip to the running part, you're probably going to fall flat in your face. You carry a child analogy through here. And every step of that way, it's training. And just like everything else, it's training not for compliance or ourselves and Leo ourselves, because we always have to have the education, but it's the training then the business. And in some cases, in this case, might be even training us, whether it be the business technology training us, but then we have to reciprocate and train them about the regulatory concerns and whatnot. - And that is super helpful context. And one of the things that I really love that you alluded to there, it really speaks to again, like walking, sorry, maybe even crawling before you're walking, before you're running, 'cause Lord knows in the compliance phase, we definitely do not want to be falling on our face, or firms just from more broadly. - Don't worry, we'll be able to keep the running right away. (laughing) - That's fair, that's fair. One of the things that you mentioned, to me, the crawling and maybe even into the walking part, is also like, I think it's important for firms to do some of that front and work, have some of those really substantive conversations around. What are you going to use the AI for? What are some of the purposes in being super intentional about what it is that you're going to be using it for from a business case perspective? Because then, to your point about, hey, let's figure out if we can get that stuff together first, like thematically, where it's gonna fit into our firm, that's gonna give us the ability to really start to do some of the walking, and then ultimately even do some of the running into kind of like on the execution side. - Yeah, well, it's new shiny toy, right? It's the swooped up a new shiny toy that, as we always hear, our neighbors down the street have and we don't have the same way they do marketing or something like that. So yeah, I agree with you. You've got to figure out what is the use case for us, because in us, you, them, it's not gonna be equal. Even though we all do asset management, as we know within the product line up and the strategy we offer, it's not all equal. So we do have to do that in analysis. What do we use? What type of firm are we? Like I said at the top, we're about 39, well, we are 39 people, that AI use may look totally different than a shop that's 1000 people, right? Not only just how you use it, what you're using for, but even how you even get to implement it, you know, if you have 1000 people, you probably have a decent tech department with maybe some people that can actually build out these internal offerings, or partner with the public offerings and come together and something versus a smaller shop that maybe don't have to dedicate expertise or tech staff to do that, and maybe just have to acquire the enterprise version of kind of the public offerings and make it fit. - Yeah. - So yeah, that is that idea. And I think the other thing that we've all seen in the last year, year and a half, whatever we've been truly talking about AI, maybe two years now, how different it is. So even if you come to that decision now of what we wanna use it for or what we think we wanna use for, even the platformer, I guess, in this case, our platform is plural, that may be different, three, six, nine months from now. So that has to be nimble in there as well, which is why you just can't straight run, because then you may be ahead today, but you may be behind tomorrow. - Yeah, so that all sets up really well with then, okay, if we've started to have, you know, some of those more high level, more strategic conversations around what do we wanna use AI for? How do we think it can benefit our firm and start getting into some of the more, I'll say robust conversations. Have you developed an AI policy? And if yes, you know, can you talk to us a little bit, kind of, you know, in a general sense of the steps that you took to kind of help complete that process? - Yeah, the answer is yes with some caveats. So we did take the baby step out of that last year, or so where, you know, because we are Microsoft's sweet users, right, co-pilot is basically in there. So we gave everyone the ability to use co-pilot for work related stuff. And in our acceptable use policy, which is IT on policy, we had a section dedicated to large language learning models and AI and what you basically the limitations are. Basically co-pilot or boss, here is those of the finer points, you know, put restrictions around, trying to get backdoor access to, clawed or a Gemini or a Chatty PT, whatever, except. So that's been kind of the last, again, listen, call a year. And of course, people want more, which is fine. And the constant pullback is yes, we want more, explain that to us and us, really meaning my firm, kind of IT and compliance to help us understand. We're not against it. Like that's, that was kind of the reaction. We just want to understand what you want to explore. And to the part of, we need governance first. So now we're at a part, in a place where we're about to roll out that governance structure. So we've been lucky on two fronts. One, I have a CEO that has laid out how he sees AI broadly. Not very direct, probably that he sees AI being additive to our day to day, kind of being that extra bit in kind of, for individual or department. Not replacing, but additive, he's kind of laid that out and kind of has the expectation that if you're not using AI in some way, you just, not with the time. And that could be simply doing searching or enhancing something you wrote a little bit or something like that. And he's even laid out to everyone that he even takes a time, more or less every morning, to teach himself that AI, going on YouTube or something like that. But learning how to use AI and integrate into his day, let's say maybe building a spreadsheet in Excel. The other lucky thing we've had is, we have a lot of, we have a lot of smart people who want to serve. But we do have an individual and one personal call out that is very tech savvy, very knowledgeable, very inquisitive about tech in his personal and also professionally. He's also got a quant and coding background as well. So he's kind of this all one person who wrote the one that has jumped on this. And he's really driven hard on where we're going with this next kind of AI stepping stone for us, which is the governance package. And so in him stepping up a little bit and working with myself and also broader our IT risk and information security committee, which he's a part of, compliance is on there, IT broadly is on there. We've got to a policy that will come out very shortly. So scrap what I said about that kind of acceptable use, policy limitation and enter in this brand new standalone AI policy, which lays out broadly speaking, okay. What are the appropriate. tools. So you're getting enhancement. You're not just limited to copilot. You're getting Gemini, you're getting Cloud, you're getting Chattity BT, you're getting some code X, you're getting some others. But we have a standing list of here's what's available to you at present. It's going to talk about what's acceptable and prohibited use cases. So, okay, using Cloud Code to help code things behind the scenes. I'm using, and we've actually kind of implemented this already. We do a lot of DDQs and 15C responses for where it's your funds. And so we're actually using Cloud a little bit on a special use case to help populate the initial round of responses before humans get involved. So those are some examples of acceptable use. What's prohibited? Economist, investment decision making, and trading. We don't want the human out of there. We're not setting it as off to do that. We'll talk about data protection of course, a very big implementation part of this, the data protection, access controls, configuration standards, some of these are very powerful tools. Before they roll out, we want IT involved in limiting what can be done. We don't want anything walking here. And with a wrong code, you can basically be destructive to the system, wiping out the entire file system. So that's kind of in there. Monty reporting, training, which is obviously probably come back to training a lot in this conversation about what is there for employees. And of course, reference other documents, other IT policies like the AUP I mentioned and the information is carry policy. Along with the broad governance policy, I mentioned the list of tools. It's kind of another part of that and also how to repost one. If for some reason again, where two years out, there's another whiz kid out there that comes out of another AI tool, how can we get it? And then even another part, a third part is the employee questionnaire certification. This is, yeah, through typical acknowledging you've read, understood, received in the policy, but also you understand the approved tools and the data protection around them, the access controls, the security practices. Limiting in that, this is a work related thing, not personal, business device sort of situation. If you're going to use it to code, so if you're getting special access to codex or code code, special access stations, but also special training before you get those. So everyone's going to get the other four pieces, I guess is training, right? AI use generally speaking, governance policy generally speaking, but if you're going to get or wants the ability to use like the codex is or the code codes are well, there's another level on top of that, given how powerful they are. And I think most people have been set around saying, I want to go code. That's not something most people are thinking about on a day every day, but now with these tools you can. So you can't go from like, I never thought I code ever before to let me start up and spin up a whole application. So I don't have to do my morning routine anymore. Pretty powerful. Priming at a couple of steps in between to someone who like I said has, you know, been coding for years in different languages like Python or whatever. We're probably going to step in there a little bit. And there's other nuances there. There'll be a model kind of coding group that's going to, you know, spot check, but those are the building blocks that were ready to roll out. And I think we're going to do it as here's the policy. Yeah, here's the tools, here's the training, here's the questionnaire, and oh, yeah, here's the official piece of the tools. Yeah. Number one, thank you, because I think, you know, just thinking about that entire process, that is a really fantastic outline for a lot of our folks just to be thinking about as, I mean, I'm guessing a lot of our folks may have already started this process, but certainly as they look to enhance it or for those folks that haven't, I think that's, you know, excellent, excellent content. I think a couple of things that I wanted to grab on to and that I think are really important here, both from a tangible and an intangible kind of perspective, from an intangible perspective. One of the things I love about the fact that you, and your firm have taken this on and kind of taken up the mantle to put together the policy, is it's immediately showing, I mean, just by the very act itself, you're immediately showing people, hey, we're not running away from this. And also employees, we're trying to help meet you where you are, right? We're not saying, hey, look, this is dangerous, like in handcuff you over and over and over again, quite the opposite. In fact, what we're really saying is, hey, like we recognize there is significant utility here. And we want to help empower you and invest in you to help continue to enhance your own skills and services and the stuff that you were doing. But we also recognize that with this new, fantastic utility, there is also some risk here. And so we as a firm have invested the time, energy, money, and resources into exploring these different tools. And these are the ones that we have vetted and found that are going to be able to provide meaningful benefit without necessarily overexposing us on the risk, right, or conflicts or other related perspectives. And I just think from a messaging standpoint, that's so powerful to the employees to start with so that they recognize, hey, look, this is, this is my firm. They're investing in this technology, they're investing in me. And also, this is where I know if I want to continue to be successful in my role here, I have to get more knowledgeable about this subject matter area. I agree. I mean, and we're giving a lot of the tools, including, you know, obviously some of the vendor applications that we use that have their own AI. We're greenlighting those as well. But we are listening. And I think we've done, or all the managers, I think in some way have done this to their employees. I know I've done it. Like just a general conversation about AI. How do you look at it? How do you feel about it? You know, how do you use it? How do you use in co-pilot now, or business wise? What are you using? Personally, just to get that sense of exposure and make sure no one's, you know, scared of it and sees it as a villain to their job. This is not right. It is, you know, it is additive, at least the way I say it in our world. And then we, I think when we roll this out, we're going to see groupings of people. We're going to see the people that said, oh my gosh, thank you. I've been waiting for this forever. You know, I'm glad to have it. You're going to have a middle of the road that are going to enhance you. Maybe try out some of the different things other than what they have so far. And then you have the pocket. They're like, okay, great. Glad I can get to it. But not necessarily something I want to use yet. And they'll catch up eventually. But it's there. And we have to, you know, if they, if we want employees to invest in the company, and we have to invest back in the employee. Yeah. And that's obviously there's always financially, right? That's one of other podcasts for you. But it's also this in training overall. And AI is just another form of training, right? You know, for my team, I pushed them hard on getting certification. They put them on the hard and do webinars, getting at the conferences. AI is at every conference. AI webinars out there now. Compliance, obviously overall. But yeah, we're in this more. It's another thing. And with every new regulation, every new invention as far as product in our industry, or obviously now another tool, that's another piece that someone can grab onto and become an expert if they want to. Because we can't all be experts. So if you want this, and it's the same thing, the market will come out a couple of years ago. And here you go. There's a new expertise for you, especially your younger and your career. Grab onto that. You know, privacy, SP, whatever, been AI, grab onto it, be that leader for our team, you know, help us think about. And the other thing we want from our team to to your point, we want their ideas. It's not so much here, senior management saying, all right, here's AI and here's how we're doing it. Yeah. What do you think? What is, what's going to be beneficial for you? How can this be additive to your day? Because you compliance, you operations, you invest in managing, you're going to come up three different answers. So what works for you guys? And we'll implement that. So it is a round, you know, is a, you know, top down bottom up kind of approach as well. Yeah. A couple of things you said there that I think are really awesome to reinforce. Number one, I certainly, and I know I'm sure it'll come up in other parts of our conversation today, but just all the training and the other stuff that you're going to continue to do around it, I totally agree how super, super important that is. And part of that is also because I mean, again, people, sometimes people view any kind of a, a more conservative thing is like they view AI like the bookie man, right? They're like, it's, it's scary to them and, and they're worried about what their employees are going to do if they get involved with it. At the same time, like when you go and do code of ethics training right now, at the end of the day, you are relying on people given the training that you've conducted that they're going to disclose all of their personal brokerage accounts that they're going to tell you about all of their outside business activities. They're going to inform you when they make political contributions. And of course, there are some things we can do to help supervise that or test against some of that. But at the end of the day, you're also very much relying on employees to incorporate training, the training that you've conducted into their day-to-day business operations and then making them certify to the notion that, hey, I've abided by the firm's compliance policies and procedures. So I also love that idea that you've added this element to your quarterly certification, right? As as something that again, for those folks that are going to be engaging in AIU, which we are encouraging and we've provided these tools for you. But again, you're all going to continue to again sort of I like you would any of the other stuff that we have at the firm that you are doing you're engaging in those activities in a way that is going to be compliant with our current policies and procedures. Yeah, and you know looking at that certification the other day of the question or certification it's a mouthful. We really look at it's the drafter of it through the kitchen sink in there and it's hard to roll it back, especially as like that initial aggregate you know here's here's the you know you turn this in you get the keys kind of approach I mean maybe someday we'll turn it back and maybe we'll obviously all certifications are always adjusted over time but yeah this is a pretty big one just because of the pure power of what AI can do and what you can do with it which is why I think everyone has been so careful for the most part and maybe slow some forms again you know have a different different speed and others but why you have to look at it and of course I think the other part too and we can't I guess anymore use this as a crush to slow it down was the former SEC administrations you know pda proposal and what was or what wasn't you know AI or generator or whatever that was our crush to slow everyone that we don't know the SEC is going with it we don't see see so with the new administration I think we have a better picture what we know how that proposal anymore but we have the bigger clear picture of how open they are to new technologies new ways of doing things so we've got to roll with it right and we can only look at potentially having a federal regulation or an AI but on privacy versus state by state yeah but you know we just have to roll with it but yeah the training and this is a new it's an interest for training across folks right but then the whole techie side of it is training the actual AI itself sure which is probably above both you and I's head that's for sure that it's you know that training so in a way when you look at the AI it's another employee again the thing on how you're using it especially the putting a gentle AI in use it's another employee so yeah you do have the training just like you would you know in train the new college grad that you're having start tomorrow and the human element in it it's there but yeah it's time to change in. When you think about and you mentioned earlier that your CEO right has been a really staunch you know supporter and advocate of this which by the way is awesome just on its own because again I think that you know speaking of setting a tone from the top I think that can be really you know effective and helpful with regard to the messaging that you all have with with the employees at the firm but even in addition to the CEO I guess you know and you've talked about some of the other you know areas certainly IT would have to be one of the areas it's involved but what what other invoices what other voices that you know inside the firm when you were coming up with the policy right did you make sure to incorporate in that process or did you engage with the help with some of the drafting just so that again for anybody listening to this podcast they can get a feel for hey you know here are probably some of the areas inside my firm I want to make sure that you don't necessarily want to have to use the analogy too many cooks in the kitchen right you're going to end up with a pretty bad stew at that point right but at the same time I think it's important that you engage the right voices so that you can meaningfully you know have good representation across the firm. Yeah so that that IT risk committee I mentioned all the short in the name of it does have a cross section of the firm I can mention IT its compliance it's our COO it's the gentleman mentioned that is really leading the charge here in AI and his you know his current role it's ahead of a portfolio management trading so it's very across the firm already to name a few people in there so they'll see it from that regard how much of input they had in writing the policy how maybe a little bit more limited compared to maybe what compliance had in working with the drafter but the whole final say it's a committee to you know let the governance out the cross sectional piece that you're looking to I think comes into play with the tools that were letting out the door right the the again enhancing co-pilot but the club the club code the chat to be to the code X Salesforce Einstein Gemini because everyone's got their little choice you know the voice of choice because you know these things have voices and personalities but what they want to use is for clearly the right now you know as we record this in end of April so if we listen to us three months now this can be a different answer but you know code is the leader for the coding aspect right and most of our people there coding things around the firm one access to so there was there was that input but as we were going to this and looking at this a lot of behind the scenes just having conversation with the voice again how are you looking at I were you looking for AI for or not for you know what your feelings for so it was that because what we got out of that to is and I've read articles about this idea and we've talked about this internally is having AI champion or champions right and that could be the firm which is you right now are are C.L. and this other gentleman but also the parts right you know so not to have too many cooks in the kitchen so we don't need maybe the entire compliance department being in the front of the line to champion what compliance needs we need one person maybe to represent compliance bring things to us us to him or her for that matter in champion you know and get in line for things and maybe it's a skill set maybe it's a level of seniority wherever the kitchen be but you need those champions to come back to maybe before the re-hands the policy to further enhance the tool sets and that regard and that's where we'll see people step up to that maybe share that vision about AI being more powerful then again AI what I'm supposed to do with this. Yeah right it's amazing how a little a little change in perspective can make such a dramatic difference in and what the overall I would say positive impact or benefit can can be and we saw that we saw one person that you know was maybe a little skeptical about AI and they got shown one of the tools and what it could do and 180 them like oh yeah okay yeah I want that now. I think some people right or has it ever skeptical like a lot of things and not just AI but you can show them some positive you know nature of it and they'll they'll swing around and they'll be all great too. Yeah we've touched on a couple of these items already but I am interested from an operational perspective and when you think about the key considerations that firms should consider again but kind of before implementing AI into their systems and processes and you've you've already hit on a few of these which are awesome right you've included some of the key constituencies you've you know engaged with your IT risk kind of InfoSec Committee you've you've done that employee training. Are there any other questions that firms you know should be asking themselves you know again just as they think about what before they start implementing AI into their systems and processes. Yeah we touch on a lot yeah I get right and so it's hard it's hard to avoid going back to them and bring them up again. Yeah. It's in what not but yeah that I think that's the groundwork and what you said before you know really where do we start what what is where we have to start somewhere what do we want to use this for and then you kind of get the tools in there and then of course the testing and in training but right also I think right the AI especially if you're building the territory of the edge it didn't it relies on data so I think part of this is where's the data what is the data is going to use where it's coming from hopefully it's internal right hopefully it's not you know PII or MMPI the throw acronym is that you but you've got to control against that and you also have to make sure your data is organized in a place where it can't be used so there's a lot of other internal control mapping data mapping that you have to consider too like if your data is a hodgepodge of stuff it's probably AI is probably not going to work for you once you get your kind of house or water but again obviously you know confidential information protecting against that of course model testing right if you're using different models you know how are you testing it to and because we'll back to the old school client investing right and you know black boxes and we want to avoid black black boxes we want to have it inside to what these models are doing and how we can talk about how we can test them but you do have to look out for everyone's favorite term hallucination right before I think even bias and it's creeping there depending on what data what you're doing you know how are you doing that how are you avoiding how you designing against that books and records you know we had talked about this in another conversation about what is a AI generated books and records you know is it the output is it the input on the output is it this transcription the summary the transcription question marks for everyone but those considerations right of what we're doing and then lastly and I know we're kind of stealing a thunder or maybe another question you had a mind but the school is a but the school is your part to me is the last bib and bob because all the other things have to be answered first right what are we using it for how we're using it what information are we using what is the output and then I think the school is your cons because then it's targeted right is it targeted at clients to be using it over there the target over here because we're using in here we targeted in a marketing collateral because we somehow created something there so I think that's the last bit but you know We can't forget about the disclosure in all this kind of broader operational aspect. No, you're absolutely right. In fact, let's stay. That's a perfect segue. I mean, I think that's let's stay on that for a little bit because that certainly is going to be one of the biggest areas in my mind as far as impact, right? To after you've incorporated AI into your firm from a compliance perspective, I would welcome your thoughts on some of the impact to the disclosures that you're making and the various disclosure documents and other materials. What about format DV? What about adjustments to the compliance manual? Talk to me about some of the impact that you've seen across your program and specifically some of the kind of material documentation incorporating that AI into the firm. Yeah, so I think for the disclosure part, really nothing has changed just yet. Again, nothing has broadly been enrolled out versus again, the co-pilot with some exceptions for some of those I mentioned earlier. But there behind the scenes kind of work, right? It's either quick research on some new regulation or a regulation outside the U.S. that you're not familiar with. It's behind the scenes coding that people have always done, coded some function using Python, but now you can use an AI tool to either enhance that coding or add to that coding or correct the coding. So there's nothing really quite basic that we have to, I feel, the scroll. So we're not using it for investment in management or research. We're not using it for choosing marking materials or statements or having forbidden numbers in marking materials. But I'm sure various aspects of that could come. But we're small enough and I think there would be good enough culture where before we got there, if people would take the breath and like, okay, we're about to go here. Why do we need to do? Where do we need to go? Because we've actually, again, going back to the DDQs at 50Cs, we do a lot of them, like said, and one of the questions on there are you using AI for investment management purposes? Right. A lot of boards, a lot of counsel are using that question. And it's pretty easy after now because no, we're not. But, you know, that's another form of disclosure. When do we have to change that answer? If we ever have to change that answer. But that's where we're going to have to go. To make a compliance manual aspect, again, the AI governance policy, I think first forms IT policy. Obviously, there's a lot of compliance input in there. But I don't know if a message fits in my compliance policy up into the point where you're going about books and records. And is there a tweet in language there or can we actually believe, rule true or true to the book's record? We'll see. Five, this is broad enough to address anything we may actually do under AI. Look, we've jerry rigged emails in there 20 years ago. We can write down AI in there now. But that's a perspective, right? But, again, it comes to what are we using for? What is it? I think the untold thing is what are the prompts in the AI equivalent to a book and a record? I agree that probably some of the outputs are, especially if it's line facing investment management, research, right? The same way you want to capture your notes on anything you're doing. It's in a hand note taker in some regard and obviously the enhancement to the investment process in that regard. But are the prompts in there? Because in some ways, I mean, I make it a Christmas for this, but in some ways, AI is basic. It's just an enhanced Google search. Exactly. Depending on what you're doing. And we never required people's Google searches to be maintained. But when does it turn from being that kind of suit up Google search to being the next books and record creation? Even though I just said everything I said before about policy coming, tools coming, not sure I have the answer. Yes. I know we certainly don't have it from the regulators broadly, but we'll figure it out. We'll find something. There is a 10 page long books and records table in our 5th manual. That's somewhere, probably some of the stuff, both a bit in, but it's going to be additive, right? You know, marketing can certainly jazz things up using AI. The end result is the actual jazzed up, you know, technical in there, marketing material. Right. If they start creating content in there and spit out actual statements that they clearly wouldn't need support, right? But you look at marketing, if you're going to make a statement in there, whether you're rowed by hand or, you know, used it from another source or used AI, you're going to have to source it and you're going to support it. So that part of the change. So there's, I think there's ties in between there, but, you know, there's things like no taking or the digital no takers on zoom and team calls or whatever kind of call. That's the tricky one because it's the whole thing. Is it just a trick? Right, right. Is it summary, whatever? Is there a recording above? Yeah, all of that. And that, that's certainly one. When you hear people talk about AI and especially on the no taking tools, there's a lot of questions that I think, you know, are still swirling around kind of those different items and different takes, you know, depending on your natural kind of inclination there. I loved what you said a moment ago too, just about, you know, I think a lot of people, again, because it gets so lost because of how crazy advanced AI has gotten just in the last three to four years that Google searches is traditional AI. Like that's, that's the, that's the OG. That's, that's the true, track, track AI stuff from the Google searches. This is absolutely part of that. But again, we want to necessarily have to think why I have to save all of my Google searches as part of the official books and records of the stuff that I'm working on from an advisory business perspective. So I'm glad that, that, you know, you, you, you mentioned that. Another question for you, are you seeing firms struggle with or, you know, did, did your firm ask itself the question of, are you buying or building, right? Because that's another one where we, we see firms that are, as they probably start to begin, that, that certainly seems like another threshold kind of question that they might have to ask. I don't know if they struggle with it. I'm sure we've talked about it. And I'm sure I'll be the part, when we lose it. I think every department started about it. You know, I've sat through probably in the last two weeks, a lot of vendors talking about their partners, whatever, you know, whatever it is, compliance, marketing, whatever kind of powerful vendor due diligence and a lot of them, not all of them, but a lot of them are bringing in, or there's an AI tool back in, in the back in there or in the additive AI tool. And marketing review, e-commerce review, or two of the bigger ones that have an AI version, right, no of it. Yeah. So it does come down to that. That is interesting. And I'm obviously focused on the clients. I know how we're seeing vendors come at it. I'm sure other aspects of our firm would see that from their vendors as well. So it does, it does lead to a good question of what is easier in some regards, what is quicker, what should I buy, something that already is set, you know, that's what that's my need. And I can just buy it. It's whatever the cost is, but it has this. Or do you want to kind of do that in-house, spend the in-house capital to do it and maybe modify the way we want it? But it's a good question. And I don't think there's any easy answers. It can be, it's my opinion, it's facts by facts, case by case, basis of what is the right situation because there may be something, you know, and there's been articles right in the last few weeks around how AI is going to overtake all the SaaS entities out there. And I mentioned the SaaS was signed before. That could be one of them in that conversation. And how they've turned the narrative, it seems. No, you're not replacing us. It's enhancing what we do. You can't, we're not easily replaced. So they've taken that approach. But again, depending on where you are, you may not be able to replace a vendor. You may have to add on to their AI version or whatever. But if you're a young nimble and have their resources, you can probably build something if you so choose. And I know, but again, I know we have two ideas and compliance at least where we can not build in the sense of building a whole program, but build in the sense of making things a little more seamless to grab something from one place and move it to another place and do some easy work, if you will. And then the human gets involved. I don't know if we have the resources to build a whole, you know, flats, kind of compliance management system where I know there's, you know, half a dozen vendors out there that do it. Yeah. No, I think, look, I think the way you frame that is really, really spot on. And I think, again, part of what I gather from that and that I think is so important. It relates to one of the things that you mentioned and that we talked about at the top, which is that you as a firm have invested in this area. You have taken the time to conduct due diligence and research various AI platforms. You are now better able to identify the types of information that the AI is going to be handling. The types of areas that using AI can help benefit you. You mentioned a couple key ones there from a compliance perspective, obviously, that's what my heart lies. But that, you know, marketing and advertising, compliance testing, other stuff. And those are really great areas where you can start to leverage AI to help enhance, like and make your role as the artist. of the stuff that comes in and gets you, it helps get you to the end a little bit quicker in a little bit more efficient manner, but still making sure to appropriately address all those items. And so I just think that, like what you articulated there, that seems like the, that's the juice that helps get you that culture of compliance. - Yeah, no, it's, like, if you have the culture of compliance already just take AI out, it should slot right in. - Right. - If you don't, this might enhance the wall wall less nature of the firm, which hopefully that's not the case. - Or actually, or there could be a thorough option, may actually change it around, because now they have the ability to use that to the benefit of using it, "Hey, I have a question about compliance. "I don't want to say ask compliance." Well, I have the, I have the compliance menu right there, so let me just have AI scroll through 300 page manual to get me the answer I need. - Exactly. - So maybe I'll make a more compliance. - Exactly. Well, and to the other point that, you know, this also relates to something that we talked about when the fact that the firm has invested in this area and has, is trying to meet the employees where they are, if you don't do that, because it's so pervasive, right? AI permeates every level of, I mean, starting to permeate every level of life if it hasn't already completely saturated it. And so the idea that we can try to give it the Heisman and totally run away from it is just really not a viable option. So rather than, because if you do that, then you're almost putting yourself in a little bit of a corner where it's likely your employees who can easily get this technology right on their phones and do so in a way where they're not even using the Wi-Fi inside your employer location and they're just using their mobile, you know, connection, they're going to start continuing to use it. But this way, if you give them options, if you give them opportunity using tools that you've appropriately vetted, that feels like a much better landing spot. To again, continue to help foster that, you know, culture of compliance inside the firm. Yeah, I agree. Because you're just, hey, here it is, use it here. It is that sense of, we don't know what we don't know. You know, critical trading is the best example. We assume everyone tells us everything, but you, this case is over the years where people, they want to do bad things. They're not going to tell you about them. Yeah. So yeah, but it is giving back in that sense, faculty employee of, we are investing in you. We are here listening to you. Yeah. Right? Because what's the worst thing that in my opinion, what firms can do is they can do an employee survey and they get the results and they don't do anything about the results. It's worse not doing the actual survey in the first place. So this is the same thing. Like we hear you, this is something you want. Yeah. And we're going to, you know, we're going to get at you, you know, make the time, but we're going to get to. So final question. This has been an exceptional conversation. And now I'm going to ask you to put on your, you know, best suitsayer cap here. You know, as you look into the future, where do you see AI going? Obviously, like, I mean, I'm almost laughing. Even asking this question, because if you had asked me this, you know, six months ago or a year ago, I'd never be able to tell you that we were here where we are in the present. But where do you see it going in the future? What steps are you taking now to maybe try to help accommodate that, that changing environment? Well, yeah, I mean, you're right. We can't predict the future because as I said earlier, whatever we said today will be obsolete by the time this rolls out. The podcast drops in a couple of weeks. Right. Exactly. Well, it's there's going to be like, what are you talking about? No, but I, but that does have everything with your set. Right. And it goes there. What we know is compliance professionals, we have to be flexible. So we can't, we know whatever governance structure we put out today and whatever policy we wrote today is going to be changed going forward. So we have to be flexible. That means we have to be flexible in the fact that we know there's going to be new labs, new agents, new view, viewpoints, pros and cons of using AI. We know there's going to be a change that who's in favor, who's the best at this for that. Because of the next voting of it's going to trump whatever without there. So it's being flexible. We just know that it's going to be, I think we know there's going to be change. That's about all we know. It's going to be constant change. And we just have to keep up with it and be flexible enough to, if our employees demand something new because that's the best new agent. We got a role with it. I think we're going to see AI across everything that we do. And that's professionally and personally, because again, going back to the coding, right? Giving the person that has probably never thought they would ever code anything ever. The ability to make programs, images, videos, gaming apps to whatever level you can be sophisticated. That's pretty new for everyone. Right. And so, and that's going to enhance people professionally and people personally. But I think thousands of things we can probably guarantee. But I think the one thing that will lock stuff, listen to whatever the future brings is that AI champion we talked about before. Or champions for that, for the matter who can stand top of this, who can help lead the department or firm to in that direction. You know, because I was looking at it. I'll stop here, but I was looking at the 2024 AI benchmarking survey that ACA group and the NICP did. Yeah. And I probably did one 2025, but I have the 2024 version in front of me. Yeah. So back then, you know, the use was about 39, 31% of the respondents said policy procedure development. You know, to then about 30% communication training and testing was about 27% and so on. I'm sure the 2025 version was a little different. The 2026 version will be different. And that's not even how we use AI, but that's internally how we're using AI will change and investigations and whatever it was down to chain. But I think that's going to be it every time we do one of these conversations or surveys. Those new rules will tweak. Yeah. And we'll get better. And the other thing I can promise will make our jobs harder. [Laughter] Easier and harder at the same time. Easier and harder at the same time, which is to make everybody feel a little good about job security. Yeah, you're going to need other more people to help interpret that. Eric, this has been absolutely fantastic. And I can't thank you enough. I really, this is such a, it's such a challenging topic in one that I certainly see the compliance officer of the future having a really, really strong working knowledge. But for a lot of folks that may have grown up in this area, it may not have been something that was necessarily intuitive for them. And so I think you've been really, really helpful in providing a lot of that context. But I will say I did lie a moment ago. I actually, I do have one more question for you. Something a little more fun. That was the last, you know, technical hard question. This one's a little easier. But, you know, as we dig into, you know, we're in spring now. But certainly I can feel the warm weather coming. What's one of your favorite things? What's one of your favorite summertime activities to get into here? Well, assuming I can actually get out there and do it. Yeah, without, you know, delving into AI class. Yeah, this time of year, it's definitely getting by a pool or the beach. You know, I'm the East Coast or so the beach. Okay. Certainly trying to get the golf course. Yeah. But there's a lot of moving parts of summer. So I think that out of all of that, I think the two things that are really hitting hard in summer is I have a has to graduate my younger son. Congratulations. Thank you. And we're going to the World Cup. Oh, nice. Everything else I said, we got to fit the World Cup into every four years past. That is absolutely fantastic. I can't wait to hear all about how that experience was. I have never been to a World Cup, although I have certainly heard wonderful things and have friends that have gone obviously. And I'm sure that everyone's going to hit a little different being right here at home. But that's awesome. Well, I hope you have a wonderful time on that trip. And again, thank you so much for coming in today and doing this, you know, podcast with us. It was absolutely fantastic. And I look forward to having you back on the show here at some point down the road. I appreciate it. It has been great. Great conversation. Thank you for the opportunity. And I hope everyone can take something away from this. Thanks. Okay. The final part of today's show features another segment of Outtakes. As a quick reminder for some of our new listeners, if compliance were a TV show, think of this as the bloopers reel, where we look at entertaining times humorous and often unsettling activities carried out at financial services firms, that hopefully provide us all with a roadmap of what not to do when facing a similar situation, we're trying to avoid a similar compliance breakdown inside our respective firms. Essentially, leave these activities on the cutting room floor and outside your compliance program. In today's outtakes, recently the SEC issued an order, instituting and settling administrative and cease and desist proceedings against three unred district investment advisors. Respondents managed several venture capital funds whose investors were mostly natural persons. And according to the order, the respondents caused the funds to violate Section 7a of the investment company by failing to register the funds, which did not have an available exemption as investment companies. The order also stated that the respondents failed to file registration statements with respect to securities offerings made by the funds which violated. Section 5A and 5C of the Securities Act in addition to the SEC. We see found that the respondents made materially inaccurate statements in omissions in marketing materials provided to prospective investors in the funds. And according to the order, the respondents provided misleading assurances that the investments in the funds had, quote, minimal risk and limited downside end quote, and were quote low risk high return end quote, because certain well known institution, certain well known institutional investors were co investors alongside the funds. The order also noted that the institutional co investors at the respondents touted had in fact, had not in fact invested in the funds portfolio companies, except for one investment in an earlier funding round that had occurred years before any fund had made its investment. The respondents also allegedly misrepresented the funds performance in a marketing deck, which stated that the portfolio company was exited at a multiple of invested capital of 025x, whereas none of the funds had earned more than 2x on any such portfolio company. According to the order, the respondents also included a quote, wins and loss chart, end quote, in a marketing deck that was distributed as late as June 2023, which admitted portfolio company investment that had been sold earlier in 2023 at a significant loss. In addition, the order stated the respondents submitted material negative information about the financial condition of three portfolio companies in marketing materials for prospective fund investors and then written updates to existing fund investors, which included positive projections regarding such portfolio companies businesses. According to the order, each of the three portfolio companies eventually failed financially and the funds investors suffered a 100% or near 100% realized or unrealized loss, amounting to tens of millions of dollars. The SEC also indicated that the respondents failed to disclose certain material conflicts of interest, arising from the respondents own significant investment in the funds portfolio companies. For example, the order noted that although the respondents disclosed that their owner's family were large investors in certain of the funds portfolio companies, they did not disclose that the owner and his family could potentially lose millions if the funds did not make additional investments in subsequent portfolio companies. They also failed to disclose that one of the funds portfolio companies paid equity compensation to the owner for fundraising and for other services where the amount of such compensation increased based on the funds raised, including from the private funds themselves. And that such incentive compensation ration could give rise to a conflict of interest. As a result of all of this conduct, the SEC found that the respondents had violated the anti-fraud provisions of the Security Act and Exchange Act and sections to a 6-4 and to a 6-2 of the advisors act and rule to a 6-4-8. The respondents consented to the season to assist the order and to pay a total of 1.7 million in Discordsment and Prejudgment Interest and a civil penalty of $600,000. I think, I mean, among the list, the litany of items that were at fault here. I think it's critically important just to remember that across all of your compliance program, whether it's identifying key conflicts of interest and making sure we disclose those, whether it's in the marketing and advertising that we're pushing out with regard to, you know, for private fund investors, the slide decks and other stuff that we're doing or more generally, if you're a wealth manager firm, just the general marketing advertising that you're doing, that we're not, you know, being superfluous in our language and putting in their statements that we cannot substantiate. Or that are really difficult to further qualify. We continue to see across both exams and and enforcement actions like this. The SEC is laser focused on marketing and advertising and in particular, substantiating any types of statements of factor performance based claims. So firms would do well to make sure that they're marketing and advertising and any relevant conflicts of interest and other disclosures are appropriately provided. And that will do it for today's show. I'd like to thank our sponsors, CalFee and the National Society of Compliance Professionals and extend a big thank you to our guest, Eric Olson, for sharing his fantastic thoughts and expertise on building a culture of compliance in an AI world. Please join us again next time on the Securities Compliance Podcast where we help you for compliance and context. Please check us out on LinkedIn. You can search for compliance and context podcasts or on X using the handle at CompliancePod. You can like us and subscribe to us on Apple Podcasts or wherever you find your favorite podcasts. We're going to compliance and contextpodcast.com to listen and learn more.