Ethics Works: A day in the life of an ethics worker in tech

[Music] This is Deb Donig with Technically Human, a podcast about ethics and technology, where I ask what it means to be human in the age of tech. Each week I interview industry leaders, thinkers, writers and technologists, and I ask them about how they understand the relationship between humans and the technologies we create. We discuss how we can build a better vision for technology, one that represents the best of our human values. Today I am speaking with Sarah Fairweather about what it's like to be an ethics worker. Sarah Fairweather is the senior program manager of ethics at Wellside Labs, shaping responsible AI for synthetic voice technology and designing policies for Wellside Labs, ethical AI deployment. She leads the effort at Wellside to ensure that every team in the organization is equipped with the tools and skills they need to make ethics informed designs and decisions in support of responsible innovation. Before Wellside Labs, she was the director of Professional Learning at Code.org, where she designed equity-focused K through 12 professional development experiences, and she co-led the company's first equity working group. Hi Sarah. Hi Dan, thanks for having me. Oh, it's such a pleasure. This is really exciting for me. I have had many interviews with people who I consider to be ethics workers. It's always a delight to get to hear what you're up to, what you're doing, what's going on in the sphere of ethics work in the industry right now. So Sarah, I wanted to interview you for these reasons, and because you are what I call an ethics practitioner, you oversee ethics at Wellside. What does overseeing ethics at a company look like as a day job? How do you spend your work days? Yeah, such a great question. So I can start by saying that at a higher level, I get to work across our organization to make sure that we're all a holding argument to responsible technology development and worth. And just for context, herfax are the thing that Wellside we build high quality, natural sounding, trustworthy, and research. So they get to use those voices, writhing, like marketing content and learning development programs. Do you really bring that content to life? And they really trust us to deliver a high quality plus work. So at Wellside we define ethics as our mission plus our values plus the principles of responsible AI. And specifically for us, those principles are accountability, fairness, privacy, and security and transparency. Well, how did you get to your work currently overseeing ethics? What path did you take to get there? Were you kind of like a mini child ethicist or philosopher? Did you start off thinking about ethics as an undergraduate, hoping to apply ethics one day to tech? Were you in tech first? What led you to overseeing ethics as a day job? Yeah, and while I've had a winding career, but the common thread's dropped. I've been a focus on education, and corporate responsibility, and leadership. And I've had the opportunity to bring that experience into my work with several different tech-focused organizations, including Amazon, Co.Or, and Wellside. So at Wellside I actually originally came on board to manage and expand our voice active program. And I think one thing that really sets us apart from other voice programs is that we actually work with real human voice actors. They are the ones who give us confound to use their voice data. So getting to work with them really gave me insight into their needs and concerns on their interests. And I really bring that perspective into my work now on that whole. Such a good reminder to me that we have a responsibility to keep what will help us create the product, right? The voice actors and to our customers who've been re-optimal for the product every day. They trust us to build and support a safe and high quality product. And I, of course, staff that's really placed a key role in keeping that trust. And Dad, but originally you asked me, you know, what my day-to-day looks like. And I think one really cool thing that I found about this job is that it's in one of the different things, right? I get to consult with our machine learning team and our voice development teams on real practical steps they can take to support principal walls like counts, parents, and accountability in their work. I partner with our trust and safety team to do things like crafting our election, integrity policy, and always trying to keep an eye on the regulations that might impact our product or impact our customers. These are products. But I also get to do things like read, leadership development experience for our people managers and help them think about how to be inclusive managers and how to build a culture. What are some of the ethical questions that come up in your day job, working at well-set in particular, and imagine that there are some conversations about deep fakes, some conversations about how and to what extent you define authenticity in a world of AI-agmented voice. And the larger concerns about the ability for people to validate information on the internet and in their information ecology is given the influx of generative AI in particular. And the ecology of increasing amounts of information that might not be true given the prevalence of this technology and its capabilities. Yeah, that's exactly right. We definitely received those questions and hear those concerns from both our customers and our voice actors. Well, I would say some of the major questions on our lines while at the time, center around data and voice actor relationships and the potential methe subargoes is that I can get into those. You know, with data, you read about this all the time now to build these AI models, a ton of data is needed. So we are always thinking carefully about how we source our data, how we use it, how we store it, and how we protect it. And we're really trying again to acquire those principles of accountability, transparency, and security, and thought about how we have our engagement. I mentioned voice actor relationships. It is so crucial to us that we are treating all voice actors as professional. And to us, that means things like making sure we have their explicit consent before we ever start recording without before we ever use their voice data. And that involves us sitting down with them to say, this is what we do, this is what we build. Here are the terms of service that we hold our customers to, which includes prohibiting a whole list of content. But this is how we try to protect your voice from the seeds. And we also talk to them about the risks of having a high voice, right? I think we do a good job of eliminating the potential methe subargoes, but there's always a possibility that a voice could be misused and we're very transparent with voice actors about that. So they can make a really informed decision about whether or not they want to build a voice with us. And then finally, we also really focus on compensation for voice actors, right? We pay them for every data that they provide to us, but we also provide them with an ongoing revenue share connected to the use of their synthetic, whoa, we're thinking a lot about how to limit the potential methe subargoes in for things like election interference, the spread of disinformation, inciting violence, etc. I do agree that there is so much out there now that it's hard for folks to know what is true and what to believe, right? And so it's really important to ask that we limit the potential methe subargoes for harm. And she did that. We have a really robust content moderation program that we're actively using to block the methe subargoes before audio is ever created. So I don't talk about this that much on the show these days, but the real kind of genesis of my interest in ethics and technology was in 2017 in which I joined a movement called tech stands up and that movement was a kind of microcosm of a larger, I think, movement happening across the country and across the culture, a kind of tech lash or a backlash against tech, a phenomenon that I think really has its roots in the post 2016 Donald Trump election when the formerly kind of lauded tech giants. Remember, if you think back to those long ago Obama years, tech overlords and tech workers were seen as kind of quasi-saviors and embraced in full by that administration. But in 2017, they were suddenly made, I think, to seem suspicious by a public who saw that AI could manipulate democratic elections, that social media platforms could manipulate the public into thinking or even acting in certain ways that maximize profits for those companies that were tearing into or maybe even ruining our social fabric tearing it apart. And that a privacy was being encroached upon by data harvesters that made us users of information and social media services into products. And I think it was in that moment, at that stage when we started to see ethics actors hired by the industry to combat, in the sense, this PR nightmare and to keep profits up. Again, this is me being cynical. or to generally combat the harmful consequences of tech. Can you give us a sense of how you think the field of ethics and tech as a job category has changed or evolved or grown over the past several years? Am I right? In pinpointing this in the 2017 moment? And if so, what are the changes in shifts or even movements that have happened in the industry in the past six or seven years? Yeah. Thank you for sharing how you got into that. Actually, it's really interesting. I think there's definitely going to shift over time. And that what I think is really important to highlight is that key folks in this industry were pushing for ethical practices in tech generally and around AI specifically for several years. But so much of our work in ethics today is informed by their leadership and efforts and willingness to ask the hard questions that we should all be asking. Right? So I do think that is public awareness and interaction with tech has increased. The demand from our responsible and accountability on the part of tech companies has rightfully increased as well. And I think in belief that this field will continue to involve us more business leaders realize that ethics can and should be part of an organization culture and something that both customers and employees expect. So not just something that's used to fix any PR understand, but really something that becomes just the fact part of the fabric of how the company operates and what employees and customers can come to expect about that company. I guess we would push back on that a little bit because one of the more recent phenomena or stages of this development is the moment in 2022 when Elon Musk took over Twitter fired the bulk of his trust and safety teams and many of his ethics workers or people who would I think be called ethics workers broadly speaking and saw that didn't damage his profits too much. And in fact other companies saw this model and saw the steadfastness or even the increase in revenue and profit and followed suit. And what I saw in that particular moment was a widespread seismic shift in the industry away from ethics work combined I think with some correct criticisms and some overblown criticisms about the relationship between ethics and equity and the rise of DEI movements movements that were and continue to be somewhat activist and oftentimes in friction with the business model of some of the companies and the culture of some of the companies. So I guess I'm really pointed to 2022 and the time since is a moment where ethics work seems to carry a kind of job precarcity and tenuousness in its grasp in the industry that we really did not see in that moment between 2017 and 2022. Do you see that shift as I see it or do you see it differently or if you do see it the way that I see it what has happened since 2022 and now has there been any recovery or are we continuing to see ethics work in the industry maybe losing the grasp that it had between 2017 and 2022. I appreciate the pushback and do want to acknowledge that I think I think is showing up in different ways and it's valued in different ways depending on the company the leader at Valera. But I will say from my perspective from our experience at Valera said what we are hearing from our customers is that ethics on the part of the AI provider is becoming increasingly. When customers choose to work with us they often ask us to go through a very detailed security review digging into how we collect our data how we store it how we use their data they ask us questions about the people behind our voices they ask how those folks are reputed they ask how we're ensuring that those places are authentic we are getting more and more questions like that every day from clients to choose that which to me is a sign that there is a growing interest in and value placed on ethics in the broader business. I do believe that equipment to ethics starts back to top to be sustainable it needs to be ingrained in a company culture so that it can survive things like charity or budget cuts or reorganization at Valera. If it I would say that the message I'm hearing both her customers as well as employees who seek us out is that they are making they are choosing to work with us in part because of our state of commitment to that. So to follow up on that a little bit when I think about the quote ethical tech features and functions in industry when I teach my course on ethics and human values in tech I typically include things like law and compliance and governance some of which I think are the rubrics of the things that you have just mentioned. What if anything makes ethical work and ethical oversight different from law or compliance or governance in your view and what kind of frameworks does ethical point of view offer that maybe these arenas do not such a great question and I know actually there's a growing body of course is about that. So I think they're so important these just actually they're so important. I would say that in my opinion law and compliance and that right I read it to you and for methods but they're not interchangeable companies of course meet adhere to laws and regulations and they need to become on and our commitment to security and to compliance standards is one of the reasons like I said that customers choose to work with us but the conversation that I often have internally with folks is that just because something is technically legal it doesn't mean that we're called right we often need to take a step prior there to make sure that a choice that we're doing is both legal and ethical so for example you mean we hear about this in the news right as some folks choose to use publicly available voice recordings from a celebrity to create a synthetic voice that's technically legal right now but it also re-believe that without explicit consent from the person behind that voice it's not ethical to actually build that synthetic voice so of course law and regulations are important and we adhere to them but for us to use state true law or commitment to respond to the book maybe should we also have to throw in that extra light or ethics and respond AI trends in her work well what what should the role though of ethics be in business should ethics slow things down so that products are cautiously tested before they go public is the role of an ethical worker to kind of throw sand in the gears or should ethics speed things up make the gears go faster so that businesses have ethics thought out ahead of time and can ship products quickly without fearing egg on the face or recalls or something like that yeah great question firmly believe that ethic can be and needs to be part of the culture by an organization like I've said how a company operates how let me decision how to communicate I have talked a lot with our team about the clack clack if we want ethics to be sustainable yes of course you need someone to be leading those efforts but it needs to be infused into how we operate everyone needs to be able to see that they have the responsibility and the opportunity and the agency to support ethics right if the person or charge of ethics needs you need to be sure that it's part of the culture's work going and the tier question and you know sometimes sometimes ethics can be perceived as it of like slowing things down right but I think that if it can be baked into systems and processes like just like budgeting you use or testing QA are baked into those things if you can invest time up front in that work you avoid all the time you'll need to put into cleaning up methods later right so it might very can slowing down or worse but I think that once you get those systems in place you will find that you're actually stating off your bark and saving time while on that I wanted to pull a thread out of one of the first things you mentioned in that response which is that ethics work needs to be everywhere typically one of the things that I see in some of the larger tech companies is that ethics work is kind of siloed you have the DEI people you have the ethics people and they live in their own little huts and then everybody else is cut on the main campus and I think that sets up a situation frankly where you have the ethics team making recommendations that are oftentimes not just siloed but kind of done in a sort of ivory tower situation and then they're given over to the business and then the business led by the CEO has to make a decision between a ethics recommendation and a business decision that would be advantageous as positioned by for example the marketing team or the the business team and choosing between the business teams recommendation and the ethics team's recommendation the ethics team says you have to institute this policy and the business team says yeah but if you do that it's going to cost us 5% growth and the ethics team says yeah but this is our ethical recommendation the CEO then has to say well do I choose the ethics team or the business team right and if they see massive profit loss down the line potentially with the ethics team they may choose at, but ultimately the CEO is liable to maintaining or increasing profit. And so team ethics versus team business in that kind of situation, team ethics is always going to lose. I take your point that ethics should be in all of the processes or maybe spread out across the processes, but how in actuality does that work? That's the first part of the question. The second part of the question is where are the critical junctures of the critical points in a product development life cycle that ethics work is crucially important. What are the critical stages at which ethical interventions should happen or need to happen? So I can answer the first part of the question, click green, I'm happy to take into that more. I would say, you know, in my experience in every place I've worked, they have always been priorities that shouldn't be in competition with each other, right? Quality versus quantity or speed versus accuracy or being the best versus being the first. And to your point, in the start of environment, we're always also facing the pressure of demonstrating that we can be successful and viable and investment where they ensure. Right? So I believe I agree with your point that it is tough for an efforts to be successful if they're often in their own wing of their building and they're not integrated. And so I think that an important part of my role is to really reduce the perception that a commitment to ethics causes friction with other commitments or necessarily causes friction with other commitments and that actually helps us being more productive and meet the more successful. I think it's your point about making this work really attainable and practical. That's really how I've tried to focus on that with our teams. So for example, recently I got to sit down with a product team and I really talked into their knowledge and expertise, right? They told me about their stages of product development and then I shared with them the principles of responsible AI and together we brainstormed really concrete, relevant questions they could ask at each stage of their product development to make sure they were staying aligned to our principles. Right? And maybe that's getting to your second question, which is that I believe, and I based on my conversations with our products, I believe they also know and think that ethics should be applied at every stage of product development. Right? From the time you sit down and start and start ideating our thing, you're thinking about what our equipment's are, what could go wrong, who we're trying to help, who could be harmed by their idea. And those those questions are repeated throughout that process, right? All the way through user testing to launch and beyond. And maybe that sounds idyllistic, but I know that we apply design principles throughout a product development experience. You know, we're thinking through the budget, product-like experience, we can also be thinking about those ethical principles throughout a product development experience. Overseeing ethics, how do you decide which ethical principles should govern decisions about how to operate, how to develop a product, or how to balance competing values in a hierarchy of values competing for primacy, in what is ultimately a business operation? How do you come up with a code of ethics? What are the ethical principles that you prioritize? Yeah, great question. So I've touched briefly on the fact that we focus on the principles of accountability, fairness, transparency, and privacy and security. And just like we ask, you know, to just propose all of our business priorities, compliance with other legal requirements, we also ask to just propose all of our mission and values. And if the answer is no, we don't just stop there, we don't just talk about the trust, we dig into what we need to be true, what needs to be true of that design or that decision. And then we ask if the resources required of those changes are an investment we want to make. Okay, so I saw that on your website, well said, has a code of ethics. How did you come up with that code? Were there debates about what to include in that code? What was the process of deliberation by which you arrived at that finalized code of ethics? It's definitely been a journey for us. The wealth that has started in about 2018. And our founders met at the Allen Institute for AI, which has a focus on it for good. So we've always been rooted in that. But last year, we really began to focus on the principles of responsible AI. And that pivot came from a recognition that we needed to be more concrete and specific about what we meant by ethics, right? What that looks like in each team members day and day and our expectations for everyone joins well, but you know, the idea of AI for good is inspirational and catching, but it can be hard to apply that to data, data decisions if we're not being really specific. So when I took on this role of leading our ethics work, I really started by grounding us in our mission and your values. And then proposed some responsible AI principles that are common done by large ethics teams, right? Upplaces like Microsoft and Salesforce, they all reference things related to privacy and security, accountability, etc. Right? So that approached really a lot us to build on our existing foundation. And then you know, as a process of running out by all of our stakeholders, getting buy-on, getting import tweaking things, our leadership team of course waited on everything. And then once we had everything different, I invested a lot of time in socializing that definition approach with folks across the organization. And I'd stress that I think like with everything, our push to ethics will continue to evolve as we grow and mature as a company. But this code of ethics, as you say, really helps give us a foundation on which we can build and on which we can be making decision. When I introduced my course on ethics and technology, I got a lot of applause, but also some criticism about both the title and the focus of the course on ethics. And I think that that criticism is quite reasonable because what it pointed out was that anytime we're talking about ethics, and by the way, most business leaders and CEOs love to talk to me about ethics. Because when we're talking about ethics, we're talking about what companies can voluntarily do, oftentimes in ways that boost their profits or in ways that are aligned with their business practices, not what they must do by law or what they will get punished for doing or not doing if they don't follow regulation. In other words, law is the opposite of ethics in so far is law is not an opt-in system. If you stop at a red light, the example that I sometimes use, you are not making an ethical action, you are following the law. If you don't stop at a red light, and somebody sees you do it, then you will get punished for it. So in that sense, ethics is the opposite of law. And anytime we're talking about ethics, we're talking about what companies can voluntarily do or not do, not what they must do. I'm wondering if you could explicate a little bit of your point of view about the relationship between ethics and law. And perhaps we can also get into some of the newer legislation that is being passed, including the outcomes of the AI Bill of Rights that came out from the administration in late, I believe December 2022, as well as GDPR and a number of different movements to try and codify some of the governance practices and some of the practices, best practices as well as ways of mitigating harm that will ultimately, I think, be instituted as regulatory practices. I think one of the most solid examples I can give is in describing how they work with our voice actors. I think if we were just to follow the law, it would be much harder for us to recruit unretain and keep long-standing relationships with the voice actors that we were if we were just following the law, we could ask voice actors to record with us and then use their voice for whatever purpose we like. We can allow customers to use their voices for all sorts of harmful use case there. We could use their voices to impersonate other people etc. We don't do that. We apply our principles of accountability and fairness and privacy and security to our work with voice actors. And we limit the use of our voices to things that we think are not harmful. We think there's a lot of did you shop or are voices for things like learning development and we don't think there's a viable use case that we've had support for our voices for, you know, inciting violent and personning of its recognition. But so then is it your point of view that these decisions to not use your product or your technological capabilities in ways that are seemingly objectively harmful for violence, for example, for impersonation or for false information? Should those be ethical decisions or should those be legal prohibitions? I would love to see similar regulation, I believe, that we're starting to see hopeless irregularization around, for example, using AI, two-spread and different permission, right? And at a high level, we really welcome thoughtful legislation and regulation around responsible innovation, responsible AI development, how data is collected, transparency around that data, etc. But we're not going to wait for regulation to do the right thing, right? And as cheesy as that might sound, that is our ethos, right? Like, from the beginning, we've been committed to responsibility innovation. We know that's why our customers choose us. We know that's why actors choose to work with us. And it's really important for us to pull those standards. And as legislation comes, and as one regulation comes, I think we're in a good place to be able to adapt to those things because we have been meaning on these principles since our beginning. Is well said challenged by, or does well said have difficulty navigating the existing regulatory frameworks such as GDPR or other kinds of, I guess, national or international frameworks that limit how you can use data or store data? Are these frameworks helpful or harmful for the work that you do and for a company like Well said to navigate? I think that these frameworks are necessary. And in the long run, helpful, I think, again, our customers rely on us to follow frameworks like this in order for them to use that. But we want to honor those needs. I think for a small company, there can be quite time-consuming and often costly to incorporate those frameworks. But to us, that's ugly work-while investment. Again, the demonstration of argument to responsible data use and storage, privacy, that are all. I want to talk a little bit specifically about the code of ethics and the three major pillars that form the kind of core of that code of ethics. And those are just to explicate them, accountability and fairness, explainability and transparency, and privacy and security. So why these three in particular? What is it about these three principles? And the way that these three principles are really pivoted around the idea of what should be open information and what should be protected information. What is it about these core areas or these core principles for your area of attacker for ethics and tech? Bradley, that's so important to the concept of an ethical code of conduct. So those principles are commonly used principles in the world of responsible AI. I think there's a lot of good work out there already. We don't need to re-end the wheel, right? And many of our companies need on some version of those principles. To me, there are two primary reasons why those principles are so important to us. One reason they're so relevant to us and likely to tech in general is that so much of our product centers around data. Certainly, the voice data from our actor. Also the vexon tent that our customers share. Or, right, but also how we source our data and the quality of the data we source, what are our, right? All of that is vital. The other equally important reason is that we've recognized from the beginning that our product depends on humans. Right? I keep coming back to this. The voice actors who provide us with their voice data and the customers, you really depend on. So for not carefully upholding your commitments to accountability and transparency and finalists, and pedocene security, we're really at risk of not managing our data responsibly, and potentially causing harm to the people who trust us, right? So I think all of those principles were the sunflower around data. And they're a good fit for what we do and how we operate. What are some of the challenges that you face in your day-to-day work, or in ethics work broadly speaking? There will always be some perception of ethics as flowing things down, or maybe the idea that the role of ethics is just telling you what you can't endure. And I think there's much wider ethics than that. And I think that one of the most challenging, but also interesting opportunities on working ethics is that in my mind, if it's done well, it's really built into the culture, right? And so just some examples. Like a product team can develop a beautiful list of questions to ask at each stage of the development process to make sure that they're aligned to our effort. But if the culture of the organization is one that is just focused on speed at all costs, then a product team member really faces a risk if they try to slow things down, they discover bias in the future and multiply that, right? Another example is that if we have a culture where folks are punished for making a mistake, a team member just will have an incentive to admit when they made a practice in security, right? When they spend the day average. So that's why I keep coming back to the importance of building ethics into the culture, and that takes time, right? You have to get by in for a lot, which is another challenge, but I think it's the right approach and like the right kind of challenge to be tackling. So for the past couple of years, I have been undertaking a National Science Foundation grant award-funded project to study the ethics and technology industry and to connect the industry and its needs in terms of a workforce to curricular development that would allow for and enable the next generation of tech workers to undertake ethics and technology study in order to become ethical technology workers. What in your view are the major skills or experiences or credentials that a ethics worker should have in order to get into and succeed in this industry? What's the background? What's the degree? What's the credential? What's the experience? What's the skill that you think or multiples of these that you think that people should have when applying for or attempting to get into these jobs? That's a good question. And probably that's a presently. I would say there's no one, but I would encourage folks to gather a broad range of experiences to go into these roles, right? I think ethics in tech is about knowing about more than just the tech, right? It's about understanding how humans work, their needs, their concerns, their values, and how all of those things interact. So if you can belt in understanding of those things, if you can practice interacting with other people, building consensus, finding opportunities to collaborate, those are all really important skills to bring in. Of course, it's important to have sound communication, it's helpful to understand at a high level at least, and you can always learn, right? But understand how the technology works, understand potential biases that might come into their technology and the impacts of those things was all important. But I would say often we do fall to like, "Oh, have a computer science degree, you know, super valuable, but don't forget the human side of that, right?" I think coming in with an more well-rounded background is really helpful. The press on it a little bit, if you were hiring an ethics worker, would you want them to have a computer science background? What do you want to see on that resume that would get them past the hiring manager, the automated resume scanner, into an ethics in tech job? Yeah, that's a great question. Certainly, we depend on what we're trying to tackle, what our goals are. I'm not someone who is a strong communicator, someone who values inclusion and thinks about how humans are impacted by the work we're doing. It would be lovely to have someone who has a computer science background or can speak that language, and understands how a product team work. I think those are really important, understanding the priorities of a product team, the pace of work, et cetera. And why? Why are those things important? What do they communicate about something or are they unable? Well, I think that, for the way that I'm approaching this work, it's really important to be able to work with a different group of team, a different group of people, be able to quickly suss out the priorities on each team, priorities that each leader has, and meet those thoughts where they are, help, and then get their buy-in for this ethics. So again, to me, it's less about what you have studied, what your degree is in, and more about your communication skills, your art-to-ball skill, your organizational skill, etc. I think we have time for one last question before we end for today. A lot of students from around the globe, some of them who I think are probably hoping to go into the ethics and tech workforce, and students really from, when I say around the globe, around the globe, I think we're in over 80 countries now, listen to this podcast, and I think a lot of them listen to this show because they are budding technologists and budding humanists, who want to understand how those two things fit together, and how they might better fit together for the future of tech and human values. What would you want them really this global cohort or this global cohort of students, particularly those who might go into work with AI as well as those who will go into tech work broadly to know or understand or consider as they navigate their careers? Yeah, I really love this question. I have that deep thing. So first I'd encourage folks to really be open to the idea that it is possible to innovate responsibly and that ethics help this company can succeed. And I think our growth outside is evident for us. I'd also encourage folks to explore what agency they have given their experiences and transposition access to power. That agency and opportunity will likely shift over time as they grow in their careers. But I want people to remember that we all have the power to contribute in some meaningful age change. It's important to not lose sight of that. And then finally I'd say that tech has known among other things for innovation and creativity and optimism. But sometimes when the industry is asked for approach development in a way that includes the stakeholders or if we're impacted by that development, why in a way that prioritizes the reduction of potential risk or harm. We hear things like, oh, that's going to take too long, but that's too hard to scale or investors want to cheap prioritize something else. So I'd encourage students and everyone we need to consider what would be possible if we applied that seems to be out of innovation and creativity that tech has known for to the challenge and opportunity of building tech that really takes humanity into account. I know I said that that was the last question, but I have one last question. Given the vision that you just laid out, are you optimistic that we'll get there? And if so, why? I am cautiously optimistic. And I think I am because I get to talk to so many candidates who are interested in working out well-fledged and often these are folks who are relatively early on in their careers. And they so clearly speak about our values and our approach to ethics as something that draws them in and something that is the deciding factor for them as their choosing company. But I believe that we are going to see a demand for this kind of innovation become more large present and allowed across the industry in a real take time. But I do see that in the folks who are coming to work with us. Well, thanks so much, Sarah. Thank you, Dad. Thank you for the opportunity. [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music]