Making the Most of Open Source in AI

As technologies, we do have to realize that the technology we build is still used. People can use it for a lot of good things and people can also misuse it. And it's the same with every single piece of technology that has been developed with the internet, with code, with encryption and all of these things. Now the question is what action do you take? Hi, I'm Derek and you're listening to the A16Z AI podcast. We're weeding into all things artificial intelligence, with our in-house team of experts, as well as the founders, engineers, and researchers working at the state of the art. In this episode, we're talking open source. So get ready to hear from some seasoned vets about what the AI community can learn from previous battles over the benefits and boogie men of developing in the open. As a reminder, please note that the content here is for informational purposes only, should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. For more details, please see ha16z.com/disclosures. There are a few terms in the world of AI, if any, that invoke more of a reaction than a simple floor letter word. Open. Whether it's industry debates over business models on the actual definition of open, or the US government actively discussing how to regulate open models, seemingly everyone has an opinion of what it means for AI models to be open, that good, the bad, and the ugly. But to be fair, there's a good reason for this. In a world where many developers have come to expect open source tools at every level of the stack, the idea of powerful models locked behind enterprise licenses and corporate ethics can be disconcerting, especially for a technology as game changing as AI promises to be. It's a matter of who has the ability to innovate in the space and whose release schedules and guardrails they're beholden to. This is why, back in February, A16Z convened a panel of experts to discuss the state and future of open source AI models, led by A16Z General Partner Anjane Midha, the discussion featured three panelists who've thought a lot about this topic in a battle scars from decades working open source. They discuss how today's AI moment compares with previous debates over open source, share their definitions of open, and offer advice on how the AI community can best ensure an open future. Anjane kicks off the discussion and hear some quick introductions to the panelists in the order they speak. First up is Percy Liang. Percy is an associate professor at Stanford, an instrumental member of the AI community. He's involved with numerous AI-related groups at the university, including the Human-centered Artificial Intelligence Lab and his director of the Center for Research on Foundation Models. Percy and his team at CRFM maintain the Popular Helm Foundation model benchmark, and he's a co-founder of Together AI, which is building open and transparent AI systems. Up next is Mitchell Baker. Mitchell's the co-founder of the Mozilla project, executive chair of the Mozilla Corporation, and chair of the Mozilla Foundation. She has received numerous awards and accolades over the years for her work on Mozilla and the Open Web overall, including induction into the Internet Hall of Fame. Among other things, Mitchell is now focusing efforts on Mozilla's product and AI strategy. In October, Mozilla authored a letter in support of open source models that was signed by more than 1,800 people, including many well-known AI builders and researchers. And finally, we have Jim Zemlin, Jim's executive director of the Linux Foundation, which probably needs no introduction with regard to open source software. It might be worth noting, however, that the Linux Foundation also manages a variety of projects and groups beyond the Linux operating system, including the Cloud Native Computing Foundation, Risk 5, and various initiatives around AI. These now include PyTorch, as well as the LFAI and Data Foundation, that houses dozens of individual projects. Thank you so much for being here. Before we jump into questions, I just wanted to go around and ask you why you got into open source. I think it's easy to go Google you guys and read up on what you've done, but I think it'd be fun to start off with just why. You know, what is it that drew you to this space? So I've been an AI researcher for 20 years, and for most of that time, we were just things that didn't work, and most of the energy was spent on design models and algorithms to make things work. And I think in the last three years that I started thinking a lot more about the social impact of AI technologies. And then that, you think about particular biases or particular harms that these models could potentially have and so on. But then I realized that something crucial was missing. If you zoom out from the individual model and properties of it, it's really thinking about how these models are built and who have controls the models and so on. Then you realize, oh, wait, actually you have a few big corporations that are developing them, and the amount of openness has decreased over the last three years considerably. And so in the last year, I've been thinking a lot more about open models and their implications for how society should develop AI. I'll leave that for you. Well, today, a lot of open sources, software, development method where you go grab a library, grab what you need and use it. But in the early days, I think it was a very radical and unacceptable idea. And it came out of a sense of actual opportunity. I'm in the words at the time we're freedom. It was a very philosophical movement of developers about what is the nature of my work, how much does my employer control it, how much, you know, if I'm doing the work of my lifetime and I change jobs, how much of it can move, and how can one collaborate. So it was a piece of opportunity. And the control and the large picture, I think, came later. But it really started with, you know, you have to be in these large companies and you have to have the right job and you have to be high enough in the chain or be lucky enough to have the exact right assignment in your work to do anything interesting. How is it that you can actually find the things that are interesting and engage and not have to have a particular title or job? So it was really a different, like it was that social question. And of course, there's a lot of governance and who controls it. How do you build a community? What's the world I want to live in? You know, and that the software development process was pretty intimately tied at the beginning into the kinds of questions that you raised. Yeah, I got involved in in open source. I was working in Silicon Valley at some startups and I eventually got to introduce some open source developers. I worked at one of the original open core companies, a company called Covalent Technologies, which was a little too early. But I got introduced to the open source community and it sort of created a confluence of two important things in my life that really made me want to be a part of the community. I grew up the son of a computer programmer. My grandfather was a computer programmer. He was one of the founders of Kray Research. That was on my mother's side. My father's side, my grandmother was the founder in 1953 of a nonprofit called Opportunity Workshop, which trained developmentally disabled adults. My uncle, who I'm named after, was disabled. Vocational skills so that they could have a burden life and contribute to society. So that, she was a huge influence on me. So the nonprofit side, kind of intersecting with the tech side, made it this insanely attractive thing to do. And so when I had this opportunity to go work on this thing, which is now called the Linux Foundation, I just jumped at it. And my wife, who is in the audience right now, I met her at about the same time on a blind date. She was graduated from Harvard Business School and was working as an executive here. And I told her, you know, I work at this open source nonprofit and the look of disappointment palpable. But we got married and said, here I am an open source. Thank you for sharing that all three of you. I think it's a perfect tea up for the first question we were going to discover, which is to do a bit of a retro throwback and talk about historical parallels between the open source AI movement that we're all in the middle of right now. And previous open source movements. And they've been multiple of them, plural. So, the first point, the idea of this fud today versus historically around open source, right? The fear of the uncertainty, the idea that this all is danger in open source somehow. So what's the same and how's it different? Maybe you can offer a clarification. So what exactly is open source AI? Right? So we know what open source software is. We have definitions. Thanks to the open source initiative. But what is open source AI? You know, you know, if a pipe is a pipe torch, well, that's code. So that's separate. And intuitively, we think about models. And I want to maybe spend some time dissecting that a little bit closely. So first of all, models encoder, obviously, different objects. One is is is legible. One is just a bunch of numbers. One has data backing in or the other one doesn't. So I do think that we should think more carefully about what what open source means. And I guess maybe one closest analog would be that open source. So I don't even actually, I don't use the word open source models. I use the word open models just to be a little bit less targeted is, for example, a llama. It's like releasing a binary. You get a bunch of weights. You can execute it, but you can't actually directly inspect it. You can modify in the same way that you do with open source true open source. I would say would be you have the data, you have the code that is used to produce a model because that would give you full transparency and reproducibility. So I just want to make that distinction clear. And of the true, you know, open source models, they are actually very few. So there's no bloom. There's now Omo and a few other small ones, but the main ones that I think of your building all these amazing apps on, you know, Mr. and Lama are not open fully open. They're more open than some of the API providers, but they're we could maybe demand a little bit more openness from that. So I just want to put that out there. You know, we could, everyone here is welcome to use their own definitions. If you've got a different one as you as you answer please, you know, feel free to clarify what definition you're using. Well, I might pick up on that and say I think the definition piece is actually important over time as a group to get sorted out of one of the things that was very, I think, critical and the success of open source. And it was a fight. And it was a fight on a lot of different reasons. It was clearly the Microsoft in particular, but the big companies were worried about it. But really it was a fight because people didn't understand it. I'm sorry to interrupt you, but if you could indulge us, could you spend maybe just a few, like a couple of beats talking about how close we came to losing that battle just so that everyone understands what the state of the war was when you mentioned Microsoft? Actually, it's a good question because people will often say, oh, you know, it's so much harder today. But the last big fight, if you imagine, I won't use AI as an example now because I'm going to, I want some established companies, but if you imagine that the operating system, so Apple and Android and Mac and Windows were all on company, and you imagine that all the apps that you use in Google were the same company, and you imagine that all the server capacity of Amazon were the same company, and you imagine that all the connected pieces of Facebook were all in one company. That was Microsoft. And it was like a 95% market share globally across all those categories, including the browser. And so that was the piece that the internet came to disrupt, but the internet alone wasn't enough to disrupt Microsoft at that era. It took open source. It took a nonprofit. I mean, like, Mozilla's a nonprofit because the market could not sustain any competition to the juggernaut that was Microsoft, and a nonprofit's a hard way to do business in the market, but for some things, it's the only way. And so that's why we are, maybe you too. So really, that close to the internet, as we understand it, being completely controlled by Microsoft, client, server, apps, cloud, compute, you name it. And as I say, it took years. It took us how many years, does Mozilla fail? Four or five, six years of failure before we finally got to the key to unlock it, which everyone knew was impossible. And so yeah, it's pretty close. And just to address the, one of the points about the FUD, if you just take us back in time, what were the prevailing kind of doomsday arguments about why that was actually a good thing? I mean, that's maybe one of the most important messages to think about tonight for this audience. And I'm kind of on Percy's spectrum, I'm the open maximalist here in that. I want all those things open, the data, every aspect of creating a frontier model. And what we heard and what you're going to hear is it's not going to work. There's going to be, and there's going to be a million reasons why. The first one that I heard was, you know, who can write, who can afford to write professional software for free? Who can spend three person years fixing bugs and maintaining software for free? Does anybody know who said that? That's a famous quote. Bill Gates in 1976 in an open letter to hobbyists, where he, you know, people were basically just copying docks for free. And then it's, okay, we have a security risk, a national security risk. We can't possibly have open source cryptography technology. Phil Zimmerman needs to go to jail because we've got this, you know, security risk. And then it'll be another risk, you know, while this is going to blow up the world eventually or in the wrong hands, this little dude terrible things. And none of that stuff happened. And so keep your eye on that prize. Like don't don't believe it. There's lots of people who have motivations to tell you why you're not, why you're going to fail by working in the open. The business reason, a security reason, an existential reason, which is the latest one that we've got in the open AI arena and don't believe it. So I'm going to try to piggyback off of the three things, you all say, which is the role of definitions, right? And at a time when a lot of policy makers, a lot of regulators, especially across the country who aren't as close to the ground as the open source community is, are making some of the most important decisions about this wave of computer infrastructure. Do you have any advice or learnings from last time around or when the internet battle for the browser was going down about how to, to both have the nuance that different definitions require, open weights versus open data versus open source while not confusing the message for regulators, policy makers, et cetera, because those do our trade-offs, right? The more nuance we introduce, the more confusion they seem to encounter. Well, one of the things that the open source community managed before was to be able to live with each other across a spectrum of definitions. I mean, we fought, right? That fight came out in the licenses, in the open source licenses. And there's the, you know, the permissive academic Berkeley MIT licenses and then what was the GPL, the most touch-the-moche code and then there's a range of things in the middle. And so we fought for a long time. Like, these are really not-down fights. Like, this is my constitution and this is my community. But, but event, there was a canonical definition and people could be at different places on the spectrum. And so I think right now it would be very useful to have an open, something, not open source, but open. And as a community to say, okay, like, this is full open, what you're, you know, talking about, this is, you know, some middle piece and, you know, have some places along the spectrum and be able to be united about some piece of it. Now, there's going to be differences and we're going to like, probably fight them on ourselves about whether the maximalism is a one-to-way, but I think in the environment today, which is much more high pressure than early open source, being able to accommodate the nuances within the community and understand throughout that spectrum, there's something about open that we're looking for. So there's some broad umbrella that most or all of us could attach to knowing that there's some distinctions inside. So I'm going to ask you for this verse. You since you suggested there be some, you know, some canonical understanding of the spectrum. What is the best way, like, as we start to get our act together and a little bit more organized, especially when it comes to educating policymakers, decision makers, and so on outside of the cortex community, is there, in your mind, a way to get consensus to rally people around a canonical set of definitions here? I mean, I feel like there are a few concepts and it's not that complicated. So there's open weights, means the weights are very available. I mean, you know, it's also the definition that is used by, for example, the executive order. And I think this is the thing that people fixate on. I think it's worth considering the fully open. And we think a lot about at 0FM transparency, which is not just open weights, but it's also data and all the training, what are the data labor practices, and other things beyond just the weights, which I think would benefit transparency and make people both policymakers and businesses make better decisions. I don't want to come back to the FUD example of the point you brought up. FUD is natural. When you have a technology that is going so rapidly, of course, there's going to be, you know, sort of confusion. And I think we, as technologies, we do have to realize that the technology we build is do a use. And it's the same with every single piece of technology that has been developed with the internet, with code, with encryption, and all of these things. Now, the question is, what do you, what action do you take? What action do, do, users, developers, governments take? And that's where I feel like some of the decisions are not fully, a lot of it's based on speculation and uncertainty because we just don't have evidence. For example, I think there's a lot of concern about these models being used to generate, you know, disinformation or helping people build by weapons and so on. But I think it's important to take into account. And all of that is true. You can probably, you can prompt Lama to to get it to tell you some things. But the question is, okay, what do you do? Do you shut down Lama to? I don't think that particular case makes sense because if you take a look at the full ecosystem, well, there's other ways you could potentially get the information. Maybe it gets the information faster to you. And now, you have to think about, okay, does that, how much extra risk does that involve? And then, you know, there's also dissemination of misinformation or the manufacturing of the, of the, um, by weapon. And maybe regulation should be targeted more downstream as opposed to upstream on the actual raw model. So these are discussions that need to be had about appropriate reaction. But, but yes, of course, these technologies can be misused. And I think that's, that's sort of a given, but how you respond to that is something that needs actually more nuance and scrutiny. Yeah, I think open source communities are very good at a lot of things. One thing they are not good at is explaining collectively to policymakers in simple terms, like having a more unified and clear voice is something that I think is pretty difficult. It's kind of like the Linux desktop, right? There's just too many versions of it. And I think there's, there's an opportunity there, uh, both to educate policymakers on the nuances of the technology stack where things are more or less open. Like is there too much concentration at the hardware level, at the building blocks level, at the data level, what, where is that? I think there's also an opportunity, which is classic, like when tech makes a problem, the answer to it is always more tech. In this case, I think there's a little work we could do to head off some of the most immediate concerns that are happening in real time that that regulators would look at. Non-concentral, sexual imagery would be a first one. We've already seen this, you know, in Hong Kong and recently in Taylor Swift, where we can build technology to help at least mitigate that risk, right? And open source is super good at that. I know there's probably people in this room who are building tools for, you know, authenticity for detecting deep fakes, for intersectional bias, for example. Like there's a lot of things we can immediately do there. So I think they unified voice and then building more open source tools that can help mitigate some of the immediate risks that policymakers concerned about our opportunities. I do want to add to that. I mean, I think there's a great point that there are actionable items that we can take as a community to. And I think the response to all, you know, these tools are dangerous isn't like, okay, don't worry about this information. You know, don't worry about NCII. I think the appropriate response is that, yes, we can build tools to help detect the stuff, to help secure open models, make them safer, evaluate with their biases or other problems with them. And just like in software and in internet technologies, you know, we have spam filters, we have other, you know, security measures that have been in place to allow this whole ecosystem to flourish. And open source does not mean the wild west where everybody just does whatever they want, willy-nilly, but it's just purposeful and it's structured and but we have to take actions to make it so that open community can thrive. So I've got a question, you know, Percy's point is essentially look, we got to start doing stuff more effectively, right? And Mozilla has had this incredible track record of helping the community figure out how to organize effectively. And so I was wondering if you could just as an example walk us through what happened, kind of like the back story, if you could take us to the war room a few months ago when the executive order came out and you organized the community around the letter, which coalesced a bunch of people, there was lots of noise, a lot of folks were not happy what came out, but somehow you managed to capture and communicate, sort of communicate, but there was one place we should all rally around. What happened there? Well, first I'll say, you know, a lot of hard work behind the scenes. And Mozilla's been engaged in trust for the AI for three or four or five years now. Obviously the interest in it has grown dramatically in the last couple of years. So we were pretty hard to have a connection both to developers and into the community and also with legislators as well. And so that letter came from a fair amount of long term hard policy work came from a growing number of connections with research in Mozilla fellows and the Mico program and some of the work that we're doing at MoCo to engage with community peace. Then it came from understanding like what do we think is the key thing we can all agree on. It's that work to try and boil things down to the essence of what's really important. And then it's a fair amount of outreach and I think a lot of trust. And so all of that in the moment behind the scenes we actually have years of connection, years of research, years of engaging, and the Mozilla fellows program has got these years of people who are deep in the technology who aren't building consumer products but are trying to understand and building and supporting those folks as well and their platform with regulators and policymakers. At this point, Anjane took a couple of questions from the audience which we'll recap here for brevity. The first question was about how to think about the seeming complexity of some open source licenses like parsing commercial versus non-commercial usage as well as concerns over copyright that might arise model providers open source their training data. Okay commercial licenses versus non-commercial licenses like this is an area which doesn't really exist in open source software. Right open source software there's no usage restrictions and you can't permit prohibit people from making commercial use of things. I think it's like one of the areas where open source software struggles is how to support the maintainers. I think a growing problem you know with open AI pieces like that that that cartoon is very real and so we do it Mozilla you know trying to figure out what can we do to support the maintainer level at that infrastructure level. So I think we're going to see more and more commercial licenses because it is a kind of an issue in open source but then you're going to get back to what the definitions are and is it really open. Yeah it's kind of confusing out there right I feel like it's still the early days where people haven't really figured out what kind of licenses make sense and all the licenses have a bunch of extra clauses of you know meta's licenses like if you have over 700 million users then you can't use this without paying us and then there's open rail where you can I think many of the licenses have restrictions on use and so I think it's just people are exploring things and people change licenses you release the model and then people complain on Twitter and then those are oops okay now you can use it not commercially so I feel like this will you know high temperature regime will maybe a neo and will figure it out. If I could add one last thing and that is one of the big open source licenses the new public license was universally understood to be almost unintelligible in parts right like and and and it got why why why usage anyway because they common understanding what it meant developed so you can have a difficult license if you've got something else that pulls it along but there is a lot of work to get a good one. Yeah it is intimidated to speak about licensing and copyright on the stage with an attorney so I'm going to do my best. I think one of the keys to open source licenses was the many to many concepts you like you didn't have to go take a license out with anybody and vice for you know it just was this very nice way to share which is why we've kind of congealed around even smaller sort of licenses you know the Apache two license for for stuff that generally gets shared these days because it's just easy right you can just mix code much easier. I think one area so that's a good concept to think about we have our kind of thoughts at the Linux foundation on areas that aren't actually covered by copyright licenses like you would find it open source the first thing we we started thinking about this was a long time ago when when machine learning and AI started becoming prevalent which is how do you deal with data now in frontier models and large language models we're using copyrighted material that's part of the data but a lot of data actually isn't copyrightable and it doesn't have clauses from a creative comments license or or open source license around downstream liability and some things that you might want. So we worked with about 60 attorneys from industry and we asked a lot of government folks about how to create an open data license which we now have a community open data license which has a copy left version and a non restrictive more permissive version which I think people should start looking at and using for large data sets so that that's one thing and then we started trying to create data governance alliances we have one in geospatial mapping where we I think we raised about 30 million dollars to bring together a coalition of open geospatial data it's called the Over-Term Maps Foundation where we regulated under a data governance regime normalize it have it under open license those are like those are kind of the things we're trying to deal with right now around those issues. And maybe one last thing that he reminded me I wrote a license of the gazelle public license and that was very intentional to be how can you be truly open source 100% across the board open source and to be intentional about wanting your work to be used in commercial settings right and so we weren't particularly evangelical about it over time I think companies have got more comfortable but there was a period of time when that license was written where there was a need for exactly that how to do both and so they're I mean it's possible it's hard you know it takes some time but the final question before wrapping up the discussion was what ongoing attempts to regulate AI at the national or international level including the fast-moving AI act in the European Union and where would be good even lucrative areas for developers to focus given the scope of these regulations I'll start and then you can fill in you know we've spent a lot of time looking at the AI act that the EU is contemplating at the last minute moving generative AI into it it's kind of crazy now to have an AI act that doesn't involve generative AI and the place that we found that we currently in the current environment that we think is the most productive is to focus on like connecting whatever regulation occurs to the actual action that's happening you know like the magic ward is proportionate so that an open source project is not like the creator and deployer and giant business of these large companies and that while you might choose to regulate them in a pretty significant manner it's important to look out what's the open source community what's the use of it what's the piece that's happening and to tailor your regulation to that because the regulation is coming and and so as I say it's this magic order of proportionate which is the opening that's we think's legitimate that's why they listen to us but about how to bring open source and the myriad of different players that are connected and know one of us is the you know the giant company that should be responsible for the deployment and impact of it. While you're passing the mic to Percy I'll say the last look at the AI policy from Brussels that I saw was a week and a half ago you'll be familiar to you Percy were together it had been leaked I'm linked in like a janky PDF that was kind of hard to read so I am not like I can't speak to the most recent iteration of that particular policy what I would say from an open perspective for policy makers is I think there are legitimate reasons for regulation but put the burden of regulation on those who most equipped to deal with that burden and in most cases that's not an upstream open source developer working on high torch right we saw this exact same thing in the Cyber-Risillian Act in Europe I think the administration in the United States has done a really good job of understanding that nuance for cybersecurity here and I think that that's the number one thing I would say to policymakers there is you know look at where responsibility can be taken and apply it where it's most equipped maybe one thing I'll add is the AI act and also many of other you know policy proposals demands and type of requirements for evaluation and which is completely underspecified it's as you must retain your model or you must evaluate improperly but I feel like one issue is that that technology has gone so as grown so quickly that it's helped pace our ability to even evaluate what models so much of machine learning's history we have these benchmarks whether it be M-ness or ImageNet or glue that was sort of a North Star and progress would happen and now you know the open or the what's a GPD 4 models are well passed at an explorer of capabilities that are not tracked and and so I think one of the things that really needs to happen is that the community needs to develop better you know evaluations for models I'm there is an MLU and LMSIS and those are you know we have gotten us pretty far but of course you know every conversation is like all evaluations broken but you know but nonetheless what else are you going to do so so I think that is something that could be very productive and also to do in the open in the sense that if you have good evaluations that are shared that builds maybe an industry standard so I've been involved in this effort with ML comments on developing a suite of safety evaluations built on the home framework that we developed at CRFM as one attempt but I think this really needs to engage the entire community so that we have a better notion of what we are going and this you know is useful obviously for policymakers but also for developers you know when you when new model comes out how do you know what it can do and what it can't when you better evaluations to to know yeah I agree I mean I know there's folks here from Common Crawl I think that's a good example of coming up with ways to get a more transparent way of like dealing with questions as regulators ask them I think the cybersecurity stuff is a decent analogy here what policy like when log4j happened you know people policy makers they got their attention and what they're saying to all of us is hey you know we're concerned about this and in some degree get your act together or we will for you now there's some things there where government actually matters and like the only regulators could do this but getting your house in order first isn't a bad thing like let's work on more transparency on better tools to manage risk that are open source like let's get ahead of some of these issues as per C sujess I'm like 100% on your page on this like do more to get ahead of these issues is is I think important yeah I'm having a bit of a reaction to the world in which like a handful of commercial entities race forward and then in the open source world it's let's clean up so so I think like the evaluation understanding is a must but I also think regulation or not some way of making sure that that's true across the work especially because the the first set of consumer products are going to come from those large organizations we see it all ready right and so there has to be some way as well to address that piece because one of the things that was really hard and open source the first time was to actually get products that consumers would use the conventional wisdom that everyone understood was that open source was only forward developers and I see and hear that today I think it's part of the the discussion if you really want to use something you know you go to so called open AI or you go to Microsoft but like the real products you wouldn't get that out of open source and you know that was true for at least a decade until Firefox I mean it was the when the open source movement came into the mainstream that was because it was this product that people used and so getting these tools and evaluation in the products that people are actually using you know from the main players I think will be equally important to maybe open source world will do better faster but then there's the question of better technology even safer technology like you have to win in the market as well so there's something about what is it maybe open source leads the way but the benefit of that for the community I think is a is it maybe you've got an idea on that well I mean I think of the tooling as sort of or you know orthogonal to whether it's open or closed if you have evaluations it works well on both open and closed models there's also sort of interesting dynamics happening with you you know it's people look at appersario attacks on models and there's this paper that show that you can attack an open model because you have access to its weights you can actually transfer that attack to you know GPT-4 which is actually you know very nice a surprising result that showed the utility of open models not just for the open ecosystem but also to you know find vulnerabilities in in closed models so I think the interaction between the close and open model I feel like it's pretty interesting to explore you know on as a closing note you know your question I think was use it used an interesting word you say what is the most lucrative right and I just wanted to remind folks that sometimes the most lucrative things aren't necessarily the things we should do and at least we I mean we talk about a valve a lot and we certainly don't I do think what we we're hearing a lot at least of the firm we have it we have this debate a lot is clearly a valve or a problem a lot of the largest labs are spending a ton of money to put out new models and everyone's eager to know how they perform including regulators but the last thing I think we want is a replay of the you know 2008 financial crisis where you have moodies or or ratings agencies being paid by the investment banks and I think it's going to take a lot of vigilance from folks like us and you guys to you know to make sure we steer in the right right way even if that means doing things that aren't lucrative and turning down as opportunities as an industry so I know I'm representing the venture cap we guys in the room but sometimes we do think that long term saying no to some of the short term opportunities that industry is pushing for is the right thing and you guys are all helping us work on stuff that's really important so so thank you so much there you have it I hope you learned something from that insightful panel discussion about open source and make sure you subscribe so you don't miss our upcoming slate of episodes