AI Risks and Realities for Nonprofits: Traps for the Unwary and Tips for Avoiding Them

I'm Cynthia Roland and you are listening to EO Radio Show. This is the third in a short series on using AI for impact and helping nonprofits work smarter in the digital age. In today's episode, we're diving into the risks and realities of AI adoption in the nonprofit world. From securing donor data to preventing bias and managing tool sprawl, we'll explore how nonprofits can protect their mission while embracing innovation. I'm again delighted to welcome back Sly and Kirstie from BDO and we're here to help you navigate the responsible use of AI. This episode is loosely based on a recent article by BDO, the top AI risks in the nonprofit sector, which is available in the links to the episode. Welcome to the EO Radio Show, your nonprofit legal resource. Brought to you by the Exempt Organizations Group at Ferrella, Bronn and Martel. My name is Cynthia Roland and I'm a partner at Ferrella. I'm a business and tax lawyer with more than 30 years of experience advising clients on nonprofit and charity law. Through this podcast, our lawyers and guests will discuss a range of legal and business issues, impacting the nonprofit world because we understand you work hard every day to make your community a better place to live and do business. Many of our programs focus on the basics and at times we'll do a deep dive into narrow and complicated legal issues. Again, welcome to the EO Radio Show. We're glad you're here. Today's guests are Sly, Atai and Kirstie Ternan from BDO, which is one of the largest global accounting firms and has one of the largest exempt organization focus practices. Kirstie Ternan is a principal at BDO Digital where she serves as the Digital Market Leader. In this role, Kirstie focuses on expanding BDO Digital's market presence by leveraging her expertise in data-driven innovation and strategic digital solutions. Kirstie also spearheads BDO's focus on AI for clients. She leads initiatives that integrate AI technologies into business strategies, helping clients leverage AI to optimize operations, enhance customer experiences, and drive growth. I'm also delighted today to have Sly Atai here again. Sly is a director at BDO and his work focuses on assisting exempt organizations with various aspects of forensic analysis, investigations, and building sound internal controls. Well, it's good to see you again, Kirstie and Sly. Welcome back. Thank you, Cynthia. Yes, well, thanks. So just to recap, in our first two episodes in this little mini series, we built the momentum or reflected our sense of the building momentum for AI as a very useful tool for nonprofits. In the first episode, Kirstie gave us her real world experience with her startup of a community nonprofit, a nonprofit as the steamery, and how AI has enabled small nonprofits like hers to begin to thrive. And then in our second episode, we talked about how the nonprofit leaders and board members can approach AI adoption strategically and responsibly. And we talked about a five-step plan for launching AI as a tool in the nonprofit leadership toolbox. I love this. The five-step plan being educate, strategize, build a foundation, manage the change, and then iterate, go and grow. So with that, I think at least the three of us are very enthusiastic about AI. But now I think we're going to, like, put a pause on the momentum and talk a little bit about the risks and traps for the unwary. And this stemmed from an article that Sly is going to talk us about a little bit to kind of set the stage for why we're switching gears from enthusiasm and how to wait a minute, slow down, maybe not. So with that Sly, talk us through a little bit of where we're going today and then we'll get Christie's perspective. Yeah, absolutely. You know, like Cynthia mentioned, we sort of went through first what are the positives of working with AI from a nonprofit perspective. And then we started to kind of walking back a little bit and saying, okay, how should the board think about this? How should management think about this as well? Now, you know, we're taking a further step back and saying, overall, what are the risks for nonprofits that are increasingly turning to AI as a solution to help them fill resource gaps, you know, help with making processes more efficient, especially when resources are very hard to come by these days. Reason why we kind of have that structure is we wanted to inspire people. We wanted people to feel like, you know, this is a great momentum shift, I think overall in the industry where there are a lot of great tools, but there need to be guardrails, right? And I think there's a responsible use of AI that's sort of underlying this entire series. And hopefully that's coming through clear. This episode today is sort of based on an article that cursed in the BDO team had published back in August, which will make sure it's in the show notes that was called top AI risks in the nonprofit sector. And it breaks out a couple of different categories of AI risks that organizations need to consider. And if you haven't seen the article, I would check it out because not only does it give kind of the high level summary of like what the risks are, it includes a practical checklist under every single area. And those areas that we're going to go through today are data security and oversight, bias, AI systems, ethical governance, and stakeholder trust, and then vendors for all in tool oversight. And we'll go into each one of those just to explain more of what it means and kind of what the risks are within those areas. But again, check out the article if you haven't had a chance to yet. So first, we'll start with data security and oversight. And so there's a couple of different considerations here. One would be a nonprofit's use of AI and donor engagement and fundraising. We talked about a couple of use cases where there are large nonprofit organizations such as UNICEF that are using AI and using it to make their facilitation of data more efficient. There are risks there. There are risks with how your data is being used, where it's going, who's getting access to that data, from a security standpoint, how secure the platforms that you're feeding that data into. Do you have permission from your donors or however other external parties, your soliciting information from to include their data in those tools? I think those are really important questions in general. We also wanted to touch upon the risks of these open source tools and generative models, given that they're so new and a lot of them do lack sort of proper security protocols because what's the tech kind of phrase that's really famous? It's like break it, or build it, break it, fix it. I think that's sort of like a motto in the tech industry in terms of just wanting to move fast and disrupt processes, which is great. But usually security is one of those things that's an afterthought. You come back and you do the security later on when someone asks to do it. It's not usually a proactive thing. Inform consent is also another big topic area where you want to make sure if people are feeding you data that you're eventually going to feed into an AI model. You want to make sure that you're able to actually use that data as part of what you're training your model to do, as well as understanding what your vendor, your AI vendors that you're working with, what they're going to do with that data. How do they process it? How do they keep it secure? It's an open-source model. Should you even be considering putting your data into those models and understanding what are the risks there? Have they had any issues in the past? Have they had any security breaches really understanding the different vendors that you work with? And then lastly, really a checklist for the board to make sure that they understand what's the high level in terms of what your policies are going to be. I think this is something we talked about in the last episode that can be sort of a high level, especially at the start. You don't have to have a 50-page AI policy from day one. No one's telling you to do that. Having a policy, I think, is a really important thing from the beginning, so you can set up some initial guardrails, just sort of like the most important things. And then over time, that policy will, just like AI will, will develop, right? You'll adapt to your risk tolerance and also your organization's culture. And most importantly, your mission and objectives, right? You want to make sure that whatever policy you adapt is allowing you to use AI in a way that's still driving your mission. I think Kirstie, if you wanted to chime in on any points there. Yeah, I mean, this is Cynthia speaking. Kirstie, I was just, it occurred to me. I wanted to point out also that we're hoping from, for your perspective, not only as a board member of the steamer and a small nonprofit, but you also sit on the board of BDO. And so you see it from the outside risk managers, what can possibly go wrong perspective. So definitely hoping that even if you can't share particular war stories, but you can shed some real world insight into these really important points. Yeah, those are two very different spectrums, a board experience. Yes. I know what we tell our clients in the non-profits side around data security and oversight. You know, a lot of times I'm very excited about how excited non-profits are about taking AI and just the industry interest, which has been maybe even more than a lot of the other industries, which isn't typical for non-profit, right? Like we finally have something that can help us. It's really exciting to see people grabbing onto that. Although a lot of times when we grab onto that, you know, the first instinct is to chase that efficiency. And we've got this great tool that can take our notes, turn it on, plug it in, go, go, go, right? And then don't sit and read the terms and conditions or don't have your legal team review them or don't have security black you because you've got all this process in place. It's easy to go. You're out there trying to save children's lives, maybe as your mission, right? Whether or not their data gets leaked during the process isn't probably top of our list of things to worry about or hold ourselves back with. So when we're chasing that efficiency, I think the things that we want to be careful about is making sure that we're really analyzing, understanding that chain of custody around donor data. Now with this teamery, I'm not as concerned about that. I don't have a lot of donor data. I don't, you know, I'm more on the startup phase. And so my use of AI doesn't have a lot of security concerns because everything that I'm putting out there in AI to build my mission, to build my materials and so forth, I want out there, right? So not as big of a concern, although every third party tool, you know, if you do nothing else, as you're clicking through those agreements, make sure you're throwing those into a tool, maybe even use AI to review those agreements to help you highlight where those issues are going to be because they're going to be hidden down in the fine print. It's going to be really hard for you to be able to identify, right? And then the other thing I also suggested, just make sure you run a media search. Has anybody else had a data breach with that particular organization or that particular software? And what were the issues around that? I think that's really important. But I think about even something like a, you know, we had a nonprofit and they had a generative AI donor chatbot. And so this would, is on their website and they would interact with their folks on their kind of individual donor portal. And so they could talk with their donors, they could give them information and they had a lot of information about their history to be able to do so. But website chatbots that are generative AI fact are very different than your old FAQ chatbot, right? So there's a whole different process of how you test that chatbot. What you're giving it, one instructions, how you make sure that it doesn't say something that you didn't want it to say or release some information. So that's a whole new way to learn how to code those and how to test those that you got to be really careful with. You know, AI oversight in general, the security side can't be left solely with IT. Thinking about the board's role here is critical. You need a governance checklist and that checklist. We do have one in that article, a slide reference. But you need something that's going to cover vendor due diligence. It's going to clear consent and you're going to have documented accountability with that. And so really having that responsibility around the security for larger organizations needs to be an alignment with the leadership mission and your compliance programs. And without that oversight, your efficiency gain really comes at the expense of trust with your donors. Great points there on data security. Next, we had on our list bias in AI systems. So what this means is for me, it's a couple of things. One, it could be biased in your data. Your data might be skewed or have, you know, anomalies or outliers in the data that make it so that it's not representative of whatever population you're feeding it, right? But there's also bias within the tools potentially that you're using. And you need to be aware of these biases, such as most of us don't actually know how most of these LLMs are trained, right? That's sort of part of their proprietary software and the millions of dollars that they've spent getting their tools to a certain point. We don't actually understand it. I feel like a lot of these vendors also don't understand it when they get asked about it in public. They're just like, yeah, we know we've just trained it for a long time, right? But we don't understand what are the underlying biases that might be built into those tools as well. So at the end of it, either way, if it's your data or if it's the tool, you might be getting biased outputs. That's a really important consideration given that you might be using AI to make grant-making decisions, right? To review grant applications. And so if your data is biased or your tool is biased and you're selecting grantees or at least down-selecting to a more narrow subset, using these tools, that could be a problem. It could be a problem where you're giving an unfair advantage or disadvantage to certain types of organizations when there's no real justification for it. I think that's a really important consideration. So for us, I think, again, considering if you are using these tools, it's critical that you keep your own data clean, but also try to get as much information you can from these LLMs on how they're training their data sets. And I think that's sometimes it's even comparing the outputs and making sure that you test the outputs themselves to see that there's no clear disadvantage that are coming out for certain types of organizations when, again, it's not actually based on any qualifiable factors. It's just based on the data or how it was trained, but of course, I'd love to get your thoughts on bias and AI systems as well. Yeah, bias is becoming kind of a day-to-day concern for a lot of organizations that are using you. Think about the grant-making example that you gave. Think about just considering resumes or candidates, volunteer candidates, whatever it might be. There's a lot of tools out there where you can put all of that past history into them, and then they can come and help you do that prediction for what's going to be the most successful, right? So like you said, answering the questions around the third parties that you're using, what training data are they using to train their systems? Are they using your training data? What demographic information have they used? Even being able to put some of these models through a system that allows to check for things like DRIP or an accuracy. So over time, you don't just create a model, train it, and then it's forever good. Let's say even if you do have good data in it, things are going to change over time, whether something as specific as geographic bias. Let's say first started making grants, all of your grants were made to a certain location while you'd expand and you've gotten bigger. If you kept that same model over time, those local, that first geographical region is going to get more bias towards it. And you just have to continue to make sure that you're testing these things on a regular basis. And that requires a constant review of these models and the tools and the data that you're using. Now, one thing that you're going to have an issue with if you are creating a model where maybe you don't have a ton of history or you don't have a ton of diverse data. Let's say for instance, resumes. And all you have is, you know, you have a position that you've only ever hired male candidates for. You may only have the training data set of their resumes. So what you can do is you can go use AI to generate synthetic data to supplement that data set to say, hey, generate an equal number of resumes with a different gender or with a different geographical presentation, whatever it might be. You might be able to use synthetic. Now, synthetic data can only take you so far, but it is good at helping us fill in the gaps. When we do have data, we just may not have perfect data for it. Any other tips you can think of on the practical steps to reduce bias before we move on to the next topic. Because this is just a real concern for me and I can't really even fathom how to check for the bias. I mean, I hear what you were saying about those particular things, but in general. Yeah, I'll say the article itself, like I said, it does include, I think it's like a nine or ten bullets right under this category of giving organizations practicals tips on how to reduce bias in their practices using AI. And there are tools too that you can run some of these data sets through to just assess how bias they are. So make sure that you're checking those out and you're reviewing how they're going to provide that type of bias measurement for you. Also, making sure that you're including as much data as possible to reduce the focus on one specific feature or another. That helps a lot too. And right now, there's, you know, there's tons of data brokerages to where you can buy additional data to update your current data set to give you a lot more to predict off of, you know, weather or even demographic information about people and so forth that can help supplement that data set and create a much more diverse representation group. Oh, that's interesting. I didn't even know such a thing existed. Well, there you go. So what's our next topic? Slide. Where are we going now? Yeah, next we had ethical risk and stakeholder trust. So again, you know, you want to make sure that there are ethical concerns around using AI, using it responsibly. This goes to your employees, to your board, to your donors, to your beneficiaries, everyone, all your stakeholders. We want to make sure we don't jeopardize those relationships because of your use of AI and cause more harm to the organization than it is benefiting your organization, right? We've talked about this already a little bit, but like one of the most important things, making sure your use of AI aligns with the organization's mission and values, just like every other big decision your organization makes, right? You don't want to do things just because they're easier or more efficient. You want to do it because it helps you fulfill your objectives in a better or more efficient way. We also want to make sure you're safeguarding information, protected information. If you have hip information, as well as transparency and communication, right? So, you know, we want to make sure that we are covering within our policies our use of AI, what we can do and what we can't do, what the organization is okay with doing, as well as, you know, in terms of transparency, I think a really practical tip for most orgs is keeping inventory, right? Keeping inventory of what tools you've approved so far, what are people using, keeping them updated. That'll allow you to do sort of like that tracking, the risk assessment type work that we've talked about earlier in this episode. If you don't do that, people are just going to be out there sort of the wild west where there's so many different tools that are being put out every single day and again, they're probably not too worried about compliance or risk. They're worried about functionality. They're worried about sales. They're trying to get more users in the door. And there's some of them are very effective, right? They're great tools, but at the same time, you want to make sure that you have your IT, your board, you have a good grasp on what's being used. It's not to say that you're going to restrict it or anything right off the bat. It just makes sure that you have an inventory in case something does happen, right? In case somebody does hear about a data breach or something else that's popped up like Christy mentioned earlier, but I don't have Christy any other thoughts on ethical governance or making sure that you can establish and maintain that trust with your stakeholders. Yeah, I think that trust, you've really got to take this approach of just radical transparency from the start, right? So publish these. If you are using AI, publish some clear explanations of how it's being used. What's being used? Give beneficiaries an opt out choice when their data is used for secondary purposes. You talked about this a little bit of established that mission first filter in every decision, right? If people first is our core value, then how is that applying when we use AI to get rid of 10 people, right? Is that really the people first mission? If that's how it shows up in our operational actions, you know, long-term non-profits have got to embrace that ethical governance, and that's going to strengthen their donor loyalty and community legitimacy across the board. Yeah, well said. You alluded to the next topic, vendor sprawl, I love that term, but there are so many tools and we're not going to talk today about tools we recommend or that are top rated. I think anything like that is going to be short-lived anyway, but because of the nature of the growing industry, what may be perfect today might not be in a little while, and so you want something else, but is that additive or, you know, how do you control all that? So walk us through the thought processes there. Yeah, I think I probably got a little bit ahead of myself in the last section and started talking about this a little bit, but I think they're all related, right? All these sections are definitely related in the way, but yeah, I mean, there's a ton of vendors, like I mentioned before, they all do really great things and the functionality is, it seems like it's getting better and better every day, but that means that again, your risk profile is ever expanding, and you want to make sure you have a good grasp of what are the vendors that our employees are using. Are we comfortable with those vendors and how often are we checking back in to get an inventory? Different orgs with different risk tolerances will take different approaches, right? Some orgs will say, these are the only software's we're okay with our organization using. Other vendors will say, you know what? Just keep us updated, right? We want to make sure that, you know, we understand what tools you're using as long as you're transparent with us, kind of like Christy mentioned as well. It's also your employees' responsibilities to report those things up to management to the boards that you're you're keeping them in mind. I think in general, you know, you do want to understand the risk profile of who you're working with so that over time as things of all, then things change, hopefully organizations are focusing more on compliance. You're able to work with those organizations and make sure that you grow with them, right? And hopefully the eventually the market for AI services and these software does stabilize a bit too so that there's not as much, you know, variability. But I think in general, it is a good idea to kind of keep tabs on what's going on. Make sure that you don't have too diverse of a risk of vendors that you're working with all the time. Maybe cut that down to a smaller list or come up with a certain list of criteria or categories. I don't know, Christy. Any of the thoughts on just the wide array of vendors that are available in today's marketplace when it comes to AI? Yeah, I mean, two things. One, use AI to test your vendors to question your vendors, to ask them the questions that you don't know when you don't know what you don't know. Yeah, that is a really good question. I often do that when I'm dealing with people that are experts. What did I not ask you? And then I think as you keep an inventory of what you have just to create as much efficiency on the contract process as possible, lower your spend as much as possible, keep an eye on the vendors that you're engaging with. Make sure you understand their product roadmaps. So let's say you signed up for Microsoft Copilot. Microsoft Copilot is coming out with something specific for their nonprofit teams that focuses on something to do with finding grants for your organization. If you are aware of the roadmap and you have a good ongoing relationship with your Microsoft representative, you're going to know that those things are coming and you're not going to go engage another vendor to build the same thing or build it on your own when that might be showing up in your system in three months anyway. These things move so fast. You really have to recognize that a lot of these large vendors are putting these things into their tools already. Now, there may be an opportunity or need to implement something even quicker, but it also kind of helps you when you do have to build what you're building for. Are you building for six months? Are you building forever? Are you building for a two-month period where they're going to have something soon and you just need a band-aid? That can really help you be as efficient as possible with a third-party contract that you engage in. Over time, there's going to be a lot of overlap, I think, between these different tools and consolidation. I think you see that just like in technology over the years anyways. This is the evolution of the new age of AI startups. I think over time, eventually these will kind of come back together and hopefully you have a couple of big players. You want to make sure that you're being efficient with your cost as well. Most of these tools are not free at this point. For Microsoft, if you use Copilot, that's usually baked into your enterprise licenses, but other tools usually not. You have to pay. You want to make sure that you're not duplicating different costs that you're incurring for a different AI software as your models that you're using. Then I think too, monitoring adoption is really important. We have a lot of clients now that not only have Microsoft Copilot, but also have an open AI chat GPT and a team environment. You'll see some adoption in one versus the other, which may direct you to how you renew or how you supplement those tools with additional vendors. It's not always going to be one tool or another with AI. Sometimes you do have to have multiple tools, just because people are going to use one more than the other because they like them. I'm wondering about that myself as I look at what's happening in the legal industry, and I imagine similar things are happening in the nonprofit world, that there are specific AI tools within our legal research tools are building out AI models, and the same thing is happening in other industries and other professional environments, where there may be a generalist kind of AI tool that you use for general questions and then specific ones that are built specifically within your industry. I'm just thinking out loud. I'm wondering if the universe of tools may start to split that way that you need specialized ones for certain industries and general ones for certain questions. I've certainly used both in the recent past. It'll be interesting to see how this evolves. I think there's no question that it's going to be an ongoing task to keep track of all the tools that you are using and aware of the ones that are coming down the pike and careful in disengaging or ceasing the use of either obsolete tools or the ones that just aren't right for your organization. But I think I digress a little bit. Where are we going next in our sort of move towards the conclusion of our traps for the unwary, which if we'd started with this, we would have turned everybody off in episode one. Hopefully we haven't turned them all away now from the enthusiasm in the first few episodes. Yeah, this was really just takeaways. I think we went through all the main points, but I think we were going to try to apply sort of our own unique perspectives in the form of case studies as well as just our experience. So for myself, I'm a risk management professional. I work in internal controls, fraud, prevention, detection, forensics, investigations. So I kind of come at things from a risk perspective for myself. I work a lot with the co-so internal control framework. I think anyone who's worked in finance or risk or any really operations focused role understands that framework. If not, we'll put it in the show notes. But the main principles of that start with your control environment, which go up to your risk assessment, your control activities, information communication, and then your monitoring. So even if you don't know what all those different things mean, you can apply that framework to your work with AI. So control environment, one huge part of your environment using that word is your policies and procedures. We talked about that already, right? Your board should pretty quickly put together a policy to say what's allowed, what's not allowed, and then adopt that policy, adapt that policy over time. In terms of risk assessment, we already talked about that on several different levels, right? Doing risk assessment for your own org, looking at risk assessments for the different vendors that you use, you need to understand what their risks are. Those will also change over time. Over the next three months, six months a year, there's tons of different softwares, new capabilities, you need to be keeping up with that. Three would be your control activities. Those are the actual internal controls you would put in place, right? So depending on what risks you assess, you need controls to mitigate those risks to make sure that nothing bad comes out of this, right? In terms of information communication, we talked about transparency. I was being super important in this process, and that's at all levels from your board, to your management, to your staff, making sure that your transparent and your use of AI, but also transparent with your external parties as well, right? We talked about your beneficiaries data. You might have that in your data sets. Your donor data, you may have that in your data sets. You need to make sure that your transparent and communicating well with them. And then lastly, as monitoring, we talked about that a lot in this episode where you're keeping up with what's your use of AI looking like? How has it changed over time? Who are the different vendors that we're using? Do we need to change our policy a little bit? Is it too strict? Is it too loose? Do we have too many risks going around? Do we need to add more controls? So again, if you're not familiar, you should definitely check out the coastal internal control framework. But I think from my perspective, as a risk management professional, that's how I think about these things. And I think that's how you should apply them in your ever evolving landscape of AI tools that are out there. But I also passive back to currency to talk about our experience with the steamery, but also her perspective in general. Yeah, and I think just in talking about my difference in the BDO board governance and the oversight where building out governance policies for 170 firms across the world, right? That's a very different oversight governance policies and procedures model than what I have for the steamery. And the steamery, I think, what I'm excited about for nonprofits is that we can do so much more. We've all had a mission to change the world, right? But there's never been a more exciting time to be part of changing the world when you can do with fewer people and less time, right? That is pretty exciting for nonprofits overall. I think the point is not that to fear AI or here's all the things you have to be concerned about. It's to treat AI adoption with the same rigor as you are going to treat your financial oversight. You want clear accountability. You want transparency. You want these ethical guard rails. And you have to protect that trust while unlocking this real innovation, this opportunity that you have. You don't need a fear AI, but you do need to manage it with discipline. And I think when we start figuring out how to do that with even a small startup nonprofit like the steamery, there's going to be so much benefit to what we can all do together using AI as an enabler there. I feel like some people are, I think it's off-putting to say we're doing more with fewer people. It's not just that. It's that things that could not be done because we don't have the people to do them are enabled by AI, which I think is a different way of looking at this. We're not in the nonprofit sector. Many missions are not going to be compatible with, let's just try to do this without humans. It's not so much that is that the power of it can do so much that with a shoestring budget and mission that needs a lot of information and a lot of things synthesize, these tools can really help that. That's the reason I'm excited about it. It's not just about financial efficiency. It's about enabling the mission. This has just been terrific. I hope that our risk management version of the episode here isn't too off-putting, but actually it makes people feel like they can be empowered to actually use these tools in a responsible and mission-oriented way. So thank you both so much. I really appreciate this and I think it's going to be super useful. Thank you, Cynthia. Thanks, Cynthia. That'll wrap up this episode. If listeners are looking for the article or the COSO International Controls Framework, check out the show notes. The show notes also have links to our Ferrell and Brown & Martell YouTube channel, where we have organized playlists on various topics for nonprofits, including a special one that will have, that'll have just these EO Radio Show episodes on AI in one convenient playlist. I'm Cynthia Rowland and you've been listening to EO Radio Show, your nonprofit legal resource brought to you by the Exempt Organizations Group at Ferrell and Brown & Martell. If you have suggestions for topics you'd like for us to discuss, please email us at [email protected]. That's [email protected]. Thank you for joining us. Until next time, make a difference.