The Ethics and Technology of Teams in the Age of AI

[Music] This is Deb Donig with Technically Human, a podcast about ethics and technology, where I ask what it means to be human in the age of tech. Each week I interview industry leaders, thinkers, writers and technologists, and I ask them about how they understand the relationship between humans and the technologies we create. We discuss how we can build a better vision for technology, one that represents the best of our human values. Today I'm sitting down with Projol J.J. Gata, the CEO and co-founder at OnLoop, about the ethics of teamwork, collaboration, and providing constructive feedback in the age of AI. Projol founded OnLoop in 2020 to create a category called Collaborative Team Development or CTD to fundamentally reinvent how hybrid teams are assessed and developed after a decade of frustration with clunky traditional enterprise performance management and learning processes and tools that were either hated or ignored by his teams at companies like Uber and Accenture, where he spent many years. Projol holds an MBA from Stanford University. Prior to founding OnLoop, Projol spent three and a half years at Uber in a variety of roles including leading strategy and operations for business development globally, leading strategy and planning for the APAC Rides business, and serving as general manager of the Philippines Rides business. He's passionate about providing constructive feedback, about developing teams in line with ethical outcomes and processes, and creating organizational structures where teams work effectively, cooperatively, and equitably. Hi, JJ. Hi, Deb. So, JJ, I wanted to have you on the show because your work really focuses on team collaboration in work spaces, particularly in the tech industry. And the more I research how ethical concepts and practices are operationalized in tech culture, the more I understand the importance of integrating ethics across the entire team. You can't be siloed, there can't be a separate ethics team working on ethics, and then we're referring to the entire team they're finding. It really has to be integrated across the entire system. I'm curious about your experience. What has been your experience? I think that ethics can often be deeply personal, and people have their own basis of what's ethical or not ethical based on their life experiences, things that they've been taught, things that they've been exposed to. And I think as an organization, I 100% agree that it is very, very important to have a foundation of ethics that is true to that organization. And that in startups or younger organizations is often set by the founding team or people that bring that organization together. So even before it goes into product development practices, I think it is even more important to integrate ethical understanding of people in interview practices, in hiring practices to really understand that you are bringing in with people into the organization that have the same ethical baseline. Because if you don't, and I spent 300,500 years on an organization called Uber, and Uber has been covered in the press in a variety of ways, you will find yourself in a situation where there might be a deviation of what an individual might do and what an organization may or may not condone. So I think this is a fantastic topic. I really so appreciate the work that you're doing around it. And I think that more founders and leaders need to explicitly think about what is the type of organization they're building, what is the ethics of that organization, and how to better lay that down in every single practice across the org. Well, not to ask you to throw anybody in particular, an organization or company that you've worked for under the bus, but just to make sure that we're saying roughly the same thing, what I'm committed to in ethics is certainly, as you say, the idea that individuals have their own preconceptions about what ethics are, what ethics should be, or what ethical practices to put into an organization or to operationalize. But even if we are all entitled to our own moral suppositions, there are better and worth, it's just a moral questions. And there are better and worth company cultures based on better and worse decisions about how to answer to those moral questions. So for example, I would say a company organization that makes the ethical decision that men matter more than women as people to perhaps keep safe or to allow to use product if the product is meant for both men and women and people who don't identify in either of those categories. And if it's meant to be equitable and somebody decides in the organization or the organization decides that their ethical principles are that men are going to benefit from their product more than women and they're okay with that, I would say that that's a worse answer to a moral question. So again, to say people are entitled to their own moral propositions, but certainly what I think you're saying is that companies need to be selective and choose the about who they pick so that they are clear number one on what their company organizations are or myth or and two that the people that they hire matter that the people they hire should have a kind of ethics that align with that of the company. And the second thing you said and I hear what I did a little bit more is that you've worked for a company which we shall not name specifically that you felt did not have that kind of alignment either in terms of hiring certain personalities to cohere with a certain set of articulated values or didn't have the articulated values to begin with. I wonder if you could give us a sense from your negative experience of what it looks like when a company doesn't have the kind of commitments to ethics and personalities to substantiate that that you're describing. Yes, so before I answer that, when I was at Stanford Business School getting my MBA, we had a we had a we're an entire course on ethics. And one of the distinctions I did draw out in that class towards the beginning that there is a deep difference between something that is ethical on ethical versus something that is legal or illegal. And people often try and equate the two. But actually if you ask me, there are things that can be legal but unethical. Loving practices can be very interesting one to talk about in that regard. And then they can be things that are illegal but are ethical like disrupting criminal or sort of harsh taxi practices around the world that were disrupted. But over as a company was illegal in many markets because we brought about a transportation structure and approach that many countries were not ready for. And if you had not taken an approach where we sort of took matters into our own hands, we wouldn't have safe and reliable transportation around the world. Now it gets especially tricky when a large organization is going that's law in many countries. And when you go against the law, you sort of bring people who are fighters for a cause. And especially in those situations at scale, it becomes incredibly blurry as a line to be very, very clear in terms of what is okay or what's not okay. I think the point you brought about to the treating men and women differently is a fantastic one especially in companies where we operate around the world and sort of often as an organization, you have to take a stance that you may not operate in certain markets or you may not sort of align with the practices of certain cultures if your ethics are different from it. And so I think that understanding your own business incredibly well. Understanding where there might be sort of disagreements with established legal structures, established societal norms, established societal ethics and having internal conversations about it and better and even better writing internally about it. I think it's very important. And you know, I'm very proud of the work Uber did globally as a company and I joined an organization when we were 9,000 people, I left when we were 27,000 people. But I think as an organization, we could have done a better job in being much more transparent and clear internally as to what was okay and what was not okay. And it is also absolutely true that in that environment, there were things done by individuals that were not okay. And that's because we were not as clear or as black and white as we should have been. I want to dig into this a little bit because again, you're referring to a negative example where people, specific individuals, it sounds like and the way that the collective team worked together, the structure of the team in addition to individual personalities led the company to do things in way that perhaps was not entirely ethical and certainly not productive for the product from that company. So I guess the question here is how does a team's organization and its structure play a role in the product's development, particularly in its ability to embed and follow ethical principles? And if you could answer that question maybe with a case or a specific example, it would really help us to crystallize how exactly that role of the team's organization and structure plays in a product's development and its ethical principles or its ability to adhere to ethical principles broadly speaking. I'll give an example and it's an example that I was… involved in the decision making, but it was a very interesting and difficult ethical dilemma for us as a company. So in Brazil, where Uber operates a very large business, we had several cases where drivers were getting either stabbed or robbed or even in some cases killed when they were nicking up riders in certain dangerous neighborhoods. However, for those riders in those dangerous neighborhoods, there was no access to any other transportation method other than Uber. So on the one hand, we were able to provide a very important utility to people that did not have access to that before, but by providing access to it, we were springing certain lives at risk. And we had to find an approach that felt, I think, really aligned to what we should be doing as a company and overall we believed caused the least harm. The solution that the companies sort of went with and we actually had massive investments in trust and safety at Uber, overtime and trust and safety really got embedded into how we thought about product design and sort of the design of our services and how they're done. And I think design is an area where calling out ethics very clearly in the product design process is probably an area where embedding it is one of the most, I would say, meaningful places to do it. But the decision that we benefit as a company is that we would not allow unidentified riders to be picked up especially in certain neighborhoods. If you were requesting for a ride in a certain dangerous neighborhood, you had to be a rider that was identified by credit card or a national ID. And we would also minimize cash trips taken from dangerous locations. And that was sort of the happy sort of medium pathway we found as a company between on one extreme blocking access transportation for certain people versus bringing lives at risk. And those sort of ethical challenges at Uber came up very, very often. And when I was at the company we were doing something along the tunes of 90 or 100 million trips a week, which is 15 million trips a day. And so we used to say what did a million episodes happen 15 times a day. And it was it was important to to think about a lot of these French cases and think about how to make those decisions. And because the company grew very quickly, often those decisions were decentralized. And so there were individuals making that judgment call in operations teams around the world on how to go about that. And so when you're when you're operating things in the real world, when you're operating software, obviously there are there are big ethical considerations. But even when you're running things in the real world, it becomes even more pregnant on how you deal with it. I want to ask a question about remote work. I know current the context of our current landscape, how remote work may have changed the landscape of operationalizing an ethical culture in a company. What have you seen? What has your experience been over the particularly last four years? I think embedding ethics into a culture is like embedding anything else into a culture whereby if you're able to reinforce it often, if you're able to live, breathe it every day, it becomes part of the DNA whereby people don't have to be reminded of it or people don't have to be sure to handbook everyone's or often to be reminded of how things are done. And I think I think nobody will deny that remote work today, the technologies that we have. So we don't live in a wall yet where we can do immersive virtual reality whereby people might feel like they're working in one location even by being in this bite in different locations. So today the immersiveness of being in one physical location is still significantly better than the immersiveness of being in multiple locations. I don't expect that to be true in a decade or two, but today that is today that is the case. And in that environment, I think it is significantly more challenging to embed culture into every individual in an organization. And therefore in in remote cultures, the chances of things going not in the way people would expect or want is higher and and the need to be intentional and explicit about implementing the right ethics into the organization just just just needs to be higher. And I think most leaders want people back in the office or sort of are lobbying against remote work because they're lazy about doing that intentional work. And if if people apply intentionality and do the hard work, I think remote work can work as well as in person work, but you have to be very, very intentional in how you go about it. I mean, I think that this is really interesting because one of the things that may be connected to the rise of remote work is the increasing integration of automatic intelligence into the workplace. And I say that I think there may be a connection there, not just because I think potentially that are moved to remote work facilitated and pushed people to develop more AI tools for the workplace. I look at some of the generative AI tools, for example, that are being created. And to me, there seems to be an obvious connection between the way that customer service moved increasingly to online context, the way that many of our tasks moved to from person to person interactions to online interactions. It seems to me that while many of the in person interactions probably are not replaceable, many of the interactions once they were made online could be automated. So it seems like not only is the computing power there to put these technologies to use and to put them into place, but also or remote context, maybe facilitated or drove or expedited some of that move toward automatic intelligence. I guess the question here is I'm wondering whether you think that AI has changed or is changing the composition of an ethical workforce or an ethical team. There's how it, I mobilizing or changing the way that teams work together in their capacity to do ethical work if you are on board with my proposition that it may be in fact doing so. Yeah, I think there's a few weirds there. I don't believe that the pace of Gen AI development is caused by the move to remote work. I think that remote work to your point leads to people working digitally versus in person and when things have done digitally, the way to apply efficiency using artificial intelligence is higher and therefore I can see how the use cases for how AI can be used are greater. I think that the more important thing to consider, I think about is from a genitive AI is used to knowledge work, what the industrial revolution was to factory workers. And what it is is that it facilitates the movement of labor to lower cost locations versus higher cost locations. So what remote work will do is I believe move more knowledge worker jobs away from places like the US and Western Europe to places like the Philippines. And actually the Philippines today is the largest exporter of non-technical knowledge work globally and stands to benefit significantly from the use of genitive AI at work. And I think that remote work will therefore accelerate how genitive AI is used because it will change the global allocation of knowledge worker labor and it will be accelerated by genitive tools. That is my belief and that's where I would correlate sort of remote work in genitive AI. Now you're on a question on how does remote work in genitive AI change the ethics of the workplace. I think ethics in the workplace is also a broad topic. So there are ethics related to product development. There are ethics related to how people interact with each other. There are ethics related to sort of how people treat customers. I think I don't think the CM on supplies to also I might require a clarification on are we talking about one of those specific pillars or are we talking about this more generally? Well this is such a great question. I'm so glad you asked that clarification question specifically because I think that in this show we have spent a lot of time on the ethics of product development. We spent a lot of time ethics of users or the ethics of endpoint consumers of buyers I actually haven't spent that much time on the ethics of how teams work together, which I know is or specialty. So maybe we could actually delve in specifically there, given that the show has had a lot of commentary on the first two and very little of that last one. And that's your expertise. So it's a fascinating topic and I think this is where I think it is hard to paint things with a common brush and there's a lot of differences between sort of high cost labor countries and low cost labor countries. So we are seeing a massive movement in the world right now of doing less work. And I think that happens in every generation to some extent. I think the generation above us, I think, called our generation lazy and not desirous of working hard. And I think I'm getting to an age where I feel a generational difference between myself and the next generation. And as we think about things like quiet quitting and people believing that work should be done purely to earn money and then life should be lived. I think there are massive differences on how people relate to work in different parts of the world. How they think about how many hours someone should be working or not be working. And sort of how we should think about that relationship between someone sort of really giving their heart and soul to work versus doing that for income. And as someone who runs a 15 person startup, I think that I definitely don't work a 9 to 6. I think it's very hard for us to survive or thrive in sort of difficult economic conditions. When someone believes that they should be working eight hours a day and no one should be getting in touch with them outside of those hours. And there are countries of the world now passing laws around not being able to get in touch with folks outside of quote unquote working hours. And and sort of creating sort of this adversarial environment between employers and employees, which I think is incredibly detrimental to organizations and individuals. And what that sort of forces organizations like ours to do is to really rethink where we're hiring people. And whether we are able to shape the culture and ethics of the workplace or whether we shape by wherever that person lives and what are the broader societal ethics or our policies to the cost of the workplace. And I think this is where organizations defer a lot. I think if you ask most startup founders, they will tell you that the notion of work like balance or work like differentiation is a is a is a great rethink principle. But it has concept to execute. But if you work at a large tech company, it is much easier to find those work life balances. And I think that especially in terms of how teams collaborate, what is okay, what is not okay, what is what is the culture of an organization. I truly believe that those things are deeply personal. I don't think there is a there is a right or wrong answer. And I think an organization needs to decide, you know, my wife works at a large tech company and has six months of paid mortality leave. I don't think that is affordable for very young companies and they would need to do a shorter period. Now is the right period two months is the right period three months is the right period five months. It's unclear. But I think it's very important for a company to be clear about what their right approaches. And I know for a fact that I've made the mistake of not being as explicit as I should have been and we had a situation where a great employee of ours got pregnant and my expectation was she would be okay with three months of paid maternity and she wanted more. And there was an impasse and my mistake was not to be explicit about it. And I think as leaders, we have to be very, very explicit about sort of what we can do and what we can so that people can then make those choices as to I agree. And therefore I'm happy to work here or be part of this organization or collaborate with these individuals or I don't agree or my ethics are different or my my sort of desires are different and and therefore I won't and we're seeing this right now with Israel Palestine and and protest from the world and people in companies and universities protesting and I think it's happening all around us in terms of what are the right ethics for for a company. Or an organization and if if someone believe they want to stand up for something how should companies respond to this and I I don't believe there's one right answer I think it's deeply subjective. I want to challenge you a little bit on that is what I believe that companies certainly have their within their own rights to make inner decisions internally about what kinds of ethics they want their employees to abide by. And to hold employees to that scrutiny what I worry about is that in a larger context those internal norms get adopted by big companies and those become standards right those become not just internal norms but industry wide standards. So we see for example what happened in 2022 when Twitter decided to cut its entire trust and safety team and many of the employees who were working to make sure that that product was equitable that didn't just stay with Twitter that became a signal to other companies that they could to cut their trust and safety. And that became an adoptable standard by an entire industry I know for a goal that in the context of my own work and academia again this is not the tech industry that there are certain expectations set about professors responding to students and those. And the expectations are now industry wide to the detriment of honest thoughtful responses to students and to the detriment I think frankly of knowledge work writ large so while I read that companies are absolutely free within a certain spectrum to decide their own internal ethics. And we can in general say that not all ethical principles are equal and that there are some that are better than others and that the consequences of a company adopting a certain ethical standard are not simply internal in their impact but can reverberate broadly and so that we should be again to use the word choosy or selective about which kinds of ethics we want to uphold. We can say I think commit ourselves to the idea that some of these standards are better than others even as companies are free to choose whichever one they want so asking you as kind of expert on these things what are some of those best practices what do you think that a company independent of whether or not they can choose whatever ethical structures they want to buy by or ethical pillars that they want to buy by what is some of the best practices about how to a create an ethical context for ideas and product development and teamwork. Collaboration but also how a company can create an ethical landscape within its own culture what are the things that facilitate that internally in your view. Yeah and just just for the record I am no expert on ethics in the workplace so I would I would never consider myself an expert in that topic and all all my views here are opinions and not not expertise and Elon Musk is definitely not an expert on ethics. I think that I do quite a bit of work with leadership teams and CEOs around sort of driving the right alignment and the organizations and well defined values so values done well at a leadership level that is then actually embedded into core operating principles of the world. So I think that the operating principles of how we operate as an organization and then ensuring that those get used for even assessing people's performance or celebrating when those values are operating principles are appelled or actively calling out when they are not upheld is a very good tool and vehicle to drive sort of strong practices. So if you have a value of respect for the individual and you build a operating principle around we seek the truth and we disagree respectfully and you and you sort of walk through how constructive conflict or debate happens in the organization and then you're able to model that out as a CEO as a leadership team. I think you can then make sure that nobody feels disrespected or or nobody feels you know treated badly in the service of the truth and actually Kim Scott wrote a new book called Radical Respect because her book Radical Candor was but sometimes sort of reponized by folks to be to be rude or aggressive towards others. in the service of candor. And I think she's sort of gone out and sort of spoken about how candor does not mean disrespect. And often the seeking of truth or the seeking of the right answer can be weaponized to disrespect folks. And it's important for organizations to be very clear about those things and how they get implemented. But what we've seen with our product and how that often gets used to read for suffix is sort of embedding company values into how feedback is delivered in an organization. And when you look for things that individuals or teams are doing well or not doing well, you just don't think about the domain expertise or the subject matter knowledge, but also the behaviorally what people are doing well or not. And if certain behaviors are going against certain company values, then sort of be able to call that out and build those loops into the organization. I want to talk a little bit about this domain of your expertise. And in particular, delivering employee feedback using AI to do so and how all the amounts to an impact into the ethics of a company or the cohesion of a company team. Can you talk to me a little bit about some of the challenges or problems or consequences of giving bad feedback that do companies and managers and management teams face when they are trying to provide feedback? What goes wrong? What are the consequences of things that do go wrong? And how can AI potentially transform or change that or alleviate some of that damage or negative consequence? Great question. And this topic I do have expertise. And so I can speak to it better. Now I think that feedback and good feedback is one of these things that we have tried very, very hard and spend billions of dollars to get people better at. But I think we failed. Right. Why are spending billions of dollars? What is so tough about providing good, thoughtful, constructive feedback? So first of all, feedback doesn't, so there's a myth about feedback that feedback always has to be constructive and feedback has to be about someone else. There are really four types of feedback and actually celebrate feedback. So telling people very specifically what they did well is probably even more important than constructive feedback. So celebrate feedback and improve feedback are two different types of feedback. And actually self feedback can be incredibly powerful too as a developmental tool or says feedback for others. So there's really four different types of feedback and there are different emotions that come in the way of doing each of them well depending on the context. So one of the biggest issues often when someone senior in the organization is delivering constructive feedback to someone typically more junior in the organization is that they can often load their frustration and emotion around the situation into the feedback. So if someone showed up really late for a meeting unprepared, they're like, you are a complete embarrassment to the organization. And saying something like that is not motivating and not constructive for the other person to hear. But as humans, one of the things that are important to us is our emotions and people really struggle to take that emotion out of the feedback delivery. In order to make that feedback motivational and drive behavior change because ultimately all feedback should be in the service of positive behavioral change. If it doesn't lead to positive and sustained behavioral change, it is not good feedback. And what we believe is that human beings are good at making observations and having conversations. They're not good at structuring and delivering feedback. And this is where technology can be incredibly powerful in taking those observations and then restructuring it in the way that it converts into positive constructive feedback. And there are structures, one of those structures that gets started Stanford is the action impact model. There's another one that's the situation behavior impact model whereby taking a particular scenario, breaking that out very factually in terms of in that meeting with the senior client, when you showed up late, it might have discommunicated disrespect to your audience. And as a result, might lower your credibility, but as an individual and for the organization. And the future for important meetings, it would be very, very beneficial to factor in significant buffers that are not late versus saying you showing up late as an embarrassment. Right. So, but you can convert you showing up as an embarrassment and convert that to really well structured situation behavior impact, feedback using technology, which I think is where large language models shine as generative AI tech because it is ultimately giving people triple PhDs in English to be able to have that eloquence democratized for everyone. I mean, I only have one PhD in English. So, I'm not necessarily looking to build two more because the utility of having one is already dubious. But I take your point, which is that human beings tend to, as you put it, observe things, not necessarily be able to communicate them in a way that leverages that observation into something efficacious. And if we're talking in ethical principles and moral principles, one of the ethical principles is don't do things that do no good, right? So think about what the anticipated endpoint is and then you have to think about how you want to get from where you are right now, given what you have observed to the desired endpoint. Right. And this is what you are envisioning that a generative AI in particular large language models can help us do. My challenge here or my problem in thinking about this is that in the kind of interim space between observation and communication lines in your view, at least I think the premise that I'm uncovering here is a certain formula. How does the large language model know or understand, and I understand that those terms are anthropomorphic that they don't really get at what a large language model does, statistically predicts probably a better term out. And is that outcome statistically predicted by that large language model the same thing as somebody who has an understanding of the company culture and he has things that a large language model does not have like an empathy for the soft skills of understanding that particular human being and maybe what that human being is going through right now in their relationship or with their kids or whatever. How does that large language model intervene into making that efficacious commentary that would drive a morally significant behavior for the better without taking into account all of the things that large language models just simply cannot take into account because they are at best highly productive statistical models capable of making predictions. So that's a great question. And I think this is where I will say something controversial. I think machines can be better at being empathetic than a large number of humans in most situations. Depends on who you are and now we go back to the culture thing. I said most humans, I didn't say all humans. And I think what often comes in the will of empathy is your own noise or your own voice on what's going on for you. And I think what machines are able to do is be an entirely in service of the other human who are sort of being in service to themselves. So one of the things that we often think about is how can we give managers and AI co-pilot to ask the right questions depending on what the situation is. And to your point, I think empathy is sort of leading with curiosity, asking the right questions, knowing what to ask in the right situation. And the body of knowledge that a large language model would have would regards to situations they have seen significantly greater than what many humans might have encountered. Now that delivery of that should always be seen as a co-pilot. And which is why I think that even in our product when we transformed that observation and defeat back, we don't deliver it to the other side. We get it back to the author. And the author is then able to then decide whether they share that as is, whether they use that to teach themselves, oh, oh, this is. This is how I should go deliver it. So actually now I'm going to do it without the machine and do it do it myself or they can edit it and send it out. And I think that where the mistake people think about Genai is they think about this as a binary human or no human. I think that is absolutely the wrong way to think about it. This is purely an augmentation and a co-pilot relationship that exists between the human and the machine. And a very good example of this is I have now become a better parallel parker because of cameras that exist in the car. Because now I know how close I am to the curb. Because the technology now exists to help me do that. And so our hope is that we would love people to not need on loop or a product to be amusing at having tough conversations and delivering feedback. In my experience, I would say that is sub 5% of managers. And I think it is sort of a irrational expectation to expect that 5% to become a very large number overnight without machine assistance. And what we truly believe is that we can grow that number if we are able to show people what good looks like and give them a technology co-pilot to move things in the right direction. I wonder if you could help me think through one of the complexities here in what you're proposing. Because to me, one of the complexities in putting machines or highly superpowered data centers in control of assessing and making decisions in articulating commentaries about human behavior, especially when those commentaries go into things like employee personnel files, is that if a receiver of that feedback then has a dispute with the feedback received, there is an asymmetry oftentimes that I have witnessed between the ability of somebody to provide feedback to the feedback when the feedback is delivered by a machine. So for example, what happens when a human and a machine dispute a certain finding or a certain outcome? We have seen, for example, in the context of something like AI judges that when an AI judge provides an assessment of a situation in the context of something, for example, like a loan or an assessment of social care worker about whether or not a situation is, for example, dangerous for a child, an AI or an algorithmic decision oftentimes over and scribes any human assessment or the human ability to challenge that assessment. So what you're talking about here is a delivering mechanism that I would argue not only exists as a delivering mechanism, but in a sense has the power to ossify and to over-determine certain feedback that a human being may not be able to speak back to or dispute because the machine is considered to be, quote, impartial or reasonable. So I'm wondering if you could comment a little bit about how this technology as perhaps as good as it may be at mitigating bad managers changes the balance of power between an employee and a manager to the point that an employee can dispute an assessment. So you bring up a great point. And I say this often, we today live in a society where we have to blame a human if something goes wrong. And that's why we want to have machines take over because that is a societal construct of how we live, which is why I talk about the corpilot model because ultimately we will make a human accountable instead of the court of law or in the court of public opinion that it's a slightly ingrained to how we operate. Now, if we use machines to rate 15 people in organization on their ability to exhibit attention to detail, I have absolutely no doubt that that is a fairer system than 15 individual managers with their own definition of attention to detail, rendering 50 other people on a scale of 1 to 5, how much attention to detail they have. However, that rating system is still then sort of rolled out by the organization. And that organization has a leader at the end of the day who is effectively accountable for that. What that now does is that it does sort of move that adjudication away from a virtual manager who an organization might have much less control over in them doing that assessment in an accurate manner versus a centralized system that is tested, pushed, understood, challenged to make that decision on a uniform basis. And I absolutely believe that is a fairer system than what happens today where arbitrary managers are deciding whether someone meets expectations or does not meet expectations. And then ultimately boils down to who the manager likes and becomes much more about visibility and politics than actually worked out. And a term that I use a lot is called eloquence bias. And then workplace outcomes are largely determined by how eloquent you are or how eloquent your manager is. And that publicly points to further straight, white, since gender, extroverted nails more than anything else in the workplace. And if we're able to apply uniform analysis using technology to assess and understand how good someone's work is, I absolutely believe we'll live in a better world than Stairless Gov. What should a tech worker look for in terms of signs or signals or what questions should a tech worker ask before joining a company if that person wants to join a team that has a culture that is likely to be ethical and equitable with managers who are reasonable, who think about providing good feedback and genuinely helping people to succeed in their company. And I also think here not just about the worker experience, but a worker who wants to intentionally work on products that they believe will be both ethical and equitable in design and in consequence. What should they look for? What are the positive signs? I think the most important thing is people doing back channel references to understand how people in the organization today feel and also try and find people who've left the organization and why they might have left. Now, the organizations like Mehta, for instance, that higher hundreds of thousands of people at this point where different employees might have very different experiences of how the experience organization. And I know people who love it and have people who have absolutely hated it. And I think it is important to gather several data points to then understand if an organization is right for you. And I will again underscore this, what is right for me will not be right for you. When I look for an organization, I haven't looked for an organization a while. I'm trying to keep it that way. So that I don't go look for an organization. Some people are looking for a challenge. Some people are looking for high salary. Some people are looking for good benefits. Some people are looking for societal positive impact. And I don't think everybody is looking for the same things. And you have to make sure that you understand yourself first and really understand what matters to you and understand the trade-offs that exist between wanting one thing versus not. You can't want to change the world, but not get punched in the face every day. Like those things kind of come hand in hand. If you want what, you sort of want to have to want the other. Some people want to change the world and have it very easy. Unfortunately, that doesn't work. And so I think it starts with the investigation of yourself. And then being very clear of what to ask others in the organization or do you get an interview process to really assert that whether there is a fit there with sort of your ethics and your morals and your understanding of the world. I think we have time for one last question. A lot of students who are around the globe, I think we're in over 80 countries now. Listen to this podcast. I think a lot of them to the show because they are budding technologists and budding humanists who want to understand how those two things fit together and how they might fit together in the future of tech and tech work. What would you want that global cohort of students to know or understand or consider as they navigate their careers? Yeah, that's a great question. And I think one thing that I worry a lot about is the lack of first principles thinking or critical thinking because of us living in a instant ratification, click on headlines, share headlines, sensational world. And I think one thing that is very, very important is to stay incredibly curious and think critically about anything that anybody tells. And to try and go deeper and sort of really push it to the second to the third to the fourth degree to really understand it truly versus really believing what you see on the surface and that is super important for us to keep moving forward as humanity. Thank you so much DJ. Thank you step. [Music]