Rob Seamans: Transforming Jobs in the Age of AI

Hello everyone, welcome to another episode of Re-Shaping Work Podcast Work Repair, where we explore the forces shaping the world of work. My name is Yvanna and I will be your host. The OECD has issued a subring forecast. Automation could erase 14% of today's jobs and transform another 32% all within the next two decades. This means that nearly half of the workforce could see their roles drastically altered, or even vanished entirely. Minions of workers may need to completely reinvent their skills and even their careers. To further unpack this super interesting topic and what it means for companies and workers, I am bringing to you Rob Stevens, professor of the New York University and an expert on technology policy and the shifting job landscape in the age of AI. Welcome. Very happy to have you. Thank you. I'm excited to be here. Great. So, I mean, AI and jobs is such a prominent topic today and I feel like the whole debate is about AI will take over our jobs. So just to kick off, I was wondering, what is something about AI and work that you feel most people still get it wrong? So, I think it's exactly that point that you just made. It's that AI is not going to necessarily replace work, but it's going to change work. And I think there hasn't been enough focus on the change. And for some of our jobs, right, for some of our occupations and some industries in particular, I think that there are a lot of changes ahead. Again, I think the big picture topic is not someone said AI is going to replace work or replace humans, but it's going to change work. It's going to change the way that we all work and interact with each other. And when you say it's going to change the way we work, do you think it's going to change a vast number of jobs or are some jobs safe from this disruption? Well, I like to think about it in three different buckets, right? So let's say bucket one is there are some jobs that will disappear, bucket two is jobs that will change and then bucket three is brand new jobs that will be created. And I think, again, too often, people focus on bucket one jobs that will disappear and bucket three new jobs that will be created and people try to net those out and think about whether they'll be net more jobs or net less jobs. But I think what gets forgotten is bucket number two and I think that's the biggest bucket of all. All of our jobs, we will continue to have work, we will continue to work in our chosen profession or specialty, but the way that we do work will change. And I just, there's not enough focus on that. In part, because maybe it's hard to envision all the different ways in which our jobs will change. This is actually quite positive that we don't need to all change careers all of the sudden, right? That's right. So I think at one layer, the positive spin is all the jobs or, you know, the vast majority of jobs that currently exist will continue to exist, right? That's that's maybe not macro macro level, but like a slightly macro level. But I think at a more micro level, when we focus on individuals in a given profession, there'll be some individuals that don't keep up with the changes and those are the folks that are going to not be promoted and perhaps be fired within that profession. Whereas the folks that adapt quickly to the changes are the ones that will get promoted and the ones that will remain in the profession. So I just want to, you know, distinguish between at the profession level or at the occupation level, that occupation might not disappear, but that doesn't mean that you as an individual are safe in that occupation if you don't adjust the way that you work. I see. And can you elaborate a little bit on this change? Are you referring to change meaning that we're going to interact more with the technology or we're going to have to learn new skills? What is change actually mean? So you asked about what change will be like in an occupation. This is one area that I think I have had a hard time talking about in a way that's generalizable. I think it's hard to talk at a general level about the changes that will occur. I think a lot of this is very occupation specific. And so the changes that we see say in healthcare are going to be very different than the changes that we see in education are going to be very different than the changes that we see in financial services are consulting. So there will be changes across all of these industries and occupations, but it's sort of hard to come up with a precise way to say the changes will be like this, XYZ. And that somehow is general across all those professions. Instead I think it's, you'll see very specific changes within each of these occupations. And it's a little hard to maybe generalize from those. I see. And is there something on the general level that you would advise workers? How can they prepare for this shift? Yeah. I mean, I think that the number one thing that any worker could be doing, but I think this particularly pertains to white collar workers, right? So folks in healthcare and law, finance, consulting, et cetera, is to play around with the new technology, right? If you don't already have at least one, let alone maybe two or three of these LLMs that you're working with, sign up for one, download it, and start playing around with it, and start seeing the ways in which you can use it in your job, start seeing ways in which it might be able to make your job a little bit easier, right? There may be some things that the large language model could help automate in your job. But you know, sort of more important is think about new things that that large language model allows you to do that you weren't able to do before. And the only way to figure this out, you know, it's not sitting on a couch and thinking about it. It's literally getting on your computer, opening up that large language model, and just start playing around with it. I think there's so many ways in which, you know, that this technology can be helpful and can lead to innovation in our jobs. But it's really hard to know what those are ex ante, just sort of figure this out via trial and error. And so, but I would suggest that everybody in their profession is open up a large language model, start playing around with it, and start figuring out what it can do well, what it can't do well, and ways that you can use it to do your job better, and to do your job in a more creative and innovative way. I think it's the folks that do that and sort of really lean into this technology. Those are the folks that are going to advance in their careers and do well. And this makes me think of workers that are entering the labor market. Do you think it would be important to look for companies that may encourage them to use AI in their work, or do you think they should just shy away from companies that are mentioning AI? I don't think that anybody is going to be able to avoid AI. I mean, I think it's a technology that's not yet ubiquitous, but eventually will be ubiquitous in the workplace just in the way that the internet and video meetings and what not are. Right? I mean, it's possible that companies don't explicitly mention AI right now, but within a few years, basically, every company is going to be using AI in some way, shape, or form. So I wouldn't try to shy away from companies that are explicitly asking about it. I see. And before we move on to talking about the job creation and business opportunities, I just talk a little bit about companies. Who are companies that you're seeing from your work writing on this wave? Who are the earlier adopters? I think some obvious winners and losers at the moment. Well, it seems like the large tech companies are doing really well right now. And just to list off some of those names, Microsoft, Google, Apple, Amazon, Meta, these large tech companies have been doing well. And it seems like they'll continue to do well, that they all have a large language model of their own. And then in addition, you have the startup labs that they're all doing well, open AI and philanthropic being too that pop to mind, but there are a number of others as well. I think in the short term, all of these companies will continue to do well. I think that they'll all benefit from this technology. I'm not so sure if you look out five to 10 years, whether these will still be the companies that we're talking about or if they're new or companies that we will be talking about. So when I think about this question, what I like to do is think back to the early days of the internet, right, the early dot com days. And in the late 1990s, people were talking about Microsoft, and Microsoft is still around, of course. But if people weren't talking about Google, they weren't talking about Meta. They were maybe just starting to talk about Amazon, and then you sort of fast forward from the late 1990s to the mid 2000s, and Google had become a household word at that point. So I think we're sort of at a similar time period right now. I would sort of-- like in where we are in the AI cycle to sort of late 90s in the internet cycle. So I think over the next five, five to 10 years, there are going to be some very, very transformative companies that get founded, that will become household words, that we're not even thinking about or talking about right now. So you feel like there could be companies providing services that we cannot even imagine today? That's right. And that's what I get really excited about, right? I mean, AI is a general purpose technology. It already is transforming our world, but it'll continue to transform our world. And what's really exciting is that there are going to be new and innovative ways that we can use this technology, new and innovative products and services that get created. And it's the creative individuals and entrepreneurs that start up companies that take advantage of this technology that will be the companies that we're talking about five to 10 years from now. So no, then I have to ask any fantasies that you have. Anything that you would love to see? Kind of the market. That's an interesting question. Yes. So my biggest complaint about AI right now is it's designed to give me an individual exactly what I want. I log on to whatever video streaming service. I'm signed up for and I get content that is precisely attuned to what I want. So that's great, but only in the short term. What I worry about, and again, this is my biggest sort of worry about where AI is right now, is that as a society, we're not just individuals, we're collections of people. We are couples or we are groups. And I worry that AI is not focused enough, or at least like the way that AI has been developed, is not focused enough on designing what's best for the couple, or designing what's best for the group. The group could be a family or for some other unit of individuals. So what I would love is when I log on to my video streaming device, service, that it doesn't just show me what I want to watch, but it shows me what, for example, my wife and I together might want to watch. And so I think if there's, when I look out at sort of the array of services that are provided by the large tech companies, by the large AI labs, as well as by all the startups are out there, to me it feels like they're just a little too focused on the individual. I mean, I'll give you one example not naming a specific company, but what one example of a category, that there are a lot of health apps out there. You know, it could be on on sleep and sleep cycles or it could be on eating and eating cycles or exercise routines, right? So there are a lot of health apps out there. They are all focused on the individual, they're not focused on the family. And what's best for me as an individual in terms of my health might not be what's best for me as a dad and a husband, a spouse, a partner. And so I'd like to see a little bit more focus on coming up with algorithms that are focused on groups of people as opposed to just individuals. I definitely love that as well as my partner and I can never be able to watch together. But that makes me think. Don't you worry about that though? Like I worry about that as a society. You know, if we're just wait, I worry that the technology will, but by focusing so much on satisfying what I as an individual want, it sort of pulls us apart. And I worry about that. But don't you think for AI to be able to do that, both you and I would need to give AI enough data. So it would have to get us get to know us versus individuals or is that all I'm going to say? Yeah. So, right. And that's what we're currently doing is we're giving it a lot of data about what we as an individual want. And we're not giving it data about what we as a couple might want. And I guess what I would want is some very creative startup to enter this space that's focused. And maybe just focus on a very specific part of what I was saying. Let's just say movies for couples. Right? Or streaming for, you know, streaming video streaming for couples. I think that'd be great. And it seems like I'd sign up for it. I would pay money for that. Absolutely. Anybody out there is listening. Absolutely. I would too. That's all the two uses they have that would take the subscription. I'm wondering, I'm wondering, it seems like you're also suggesting some sort of responsibility, accountability in these companies need to take. Or do you feel like they inside their boardrooms do they need to think about what impact what they're doing has on the society? And do you feel like that these new startups are more attuned to that? I don't know if startups or let's call them maybe sort of legacy or incumbent companies are more, I don't know which of those two would be more or less attuned to, let's say sort of ethical considerations. Right? I mean, on the one hand, you might imagine that a startup could be founded with some sort of ethical guidelines or something like that. But on the other hand, startups are so pressed for, the founder of a startup and that early team, they're so focused on creating the product, getting the product out the door, generating some revenue, satisfying their investors, satisfying their early customers. Is there time to think about what's ethical or sort of what's best for humanity? I'm not sure. So maybe the responsibility falls more on the larger incumbent companies who have a little bit more bandwidth, maybe a little bit more slack, and can be focused on things that are better for humanity, things that are more ethical. I'm not sure. But I think it's, I also think that there's certainly is customer demand for, not by all customers, but there's certainly segments of customers that are interested in products that are, let's say, more ethical or sort of focus more on greater good for humanity. But I'm not sure. Your question is a good one, but it's a hard one. I don't have a good answer to it. I see. And seems like you're suggesting that AI is going to infiltrate everything. When it comes to companies, because they compare them, they should incorporate AI. Let me say, think is AI expensive? What about the companies with limited budgets? How can they stay competitive in AI driven market? I think the decision is a hard one right now, because one thing that we see is that these AI models are continuing to advance very quickly. And so a company right now could either, a budget constrained company, could either invest in the LLM as it is right now, and try to fully integrate it into what it is that that company does into that business product or service. The problem, of course, is that a year from now, the LLM, where LLMs are, might be totally different. And so hence, maybe it's better to wait a little bit before investing. So I think that this is a very hard decision that companies are faced with right now. And I think that that's part of the reason why we actually see a lot lower adoption than I would have thought we would see in 2025. Right, so recent research by the US Census Bureau, surveying in a systematic way, US businesses finds that about 10% of them have adopted AI in a meaningful way. Right, so that's very, very low. I think, right, 10%. If you had asked me, what percent of companies have adopted AI, I would have put it maybe around 50% or something like that. I think that might be just my own biases because I think about the technology a lot and I've adopted it for myself. But at the business level, only 10% have adopted. I'll say it differently. 90% of companies out there have not yet adopted this technology that we're all talking about. Why do you think that is? And I think the reason is, well, I think it's precisely because of what you're getting at in your question. It's expensive to adopt and integrate this technology into your business processes. And so companies are strategically waiting. I think a lot of them are strategically waiting until the technology becomes more mature. And is it just about the budgets or is it perhaps also about skills and knowledge of what this technology can do for them? I think you're exactly right. I think that that's probably the biggest impediment right now is that there aren't enough individuals out there that are well-trained enough in terms of how to use the technology. A, and B, know enough about, for your given industry and your occupation, know enough about that, so that you can integrate the technology in your company and adapt it and use it in a way that is productive, right? So you can imagine that sort of vent diagram between those that have the skills and those that are experts in their profession and that overlap, that vent diagram overlap. is a very, very small sliver right now. And so those individuals that have those skills command a large wage premium. And so hence that makes it expensive for firms to adopt and use this technology 'cause they need to make sure that they have those individuals that human capital on hand. - And how does some skills that you would recommend companies investing or reskill or upskill their employees? - Yeah, I think we'll see a lot more discussion of this in the coming year, right. A lot more discussion about reskilling and upskilling. I don't know so much that it's upskilling, right, 'cause these are just sort of new skills and the skills in particular are AI skills. And so I think it's really around reskilling. This, by the way, I think is a huge opportunity for folks in my industry, right. I'm in the education industry and you could imagine either, you know, it could be startups for profit startups but it also could be incumbent not for profit universities and colleges that create programs that are specifically designed to help either entry level professionals or middle managers or even senior level managers understand much, much better about this technology and how it can be used in their workplace. - So in the recent New York Times article by Robert Gaps, you explained some new roles we may have. - Can you elaborate on those? - Yeah, so I was one of several folks that Robert Gaps had interviewed and the three roles that I described to him that I think will be roles that people will be talking about are AI integrators, AI auditors, and AI explainers. You may tell you a little bit more about each of those. - Yes, please. If you could please explain for us what each one of those means. - Yeah. So an AI integrator, the way I think about this is this could either be somebody within a company or it could be a third party professional services firm and for that matter that there'll probably be AI integrators both within companies and in third party firms. What these AI integrators do is they help legacy businesses identify ways in which AI can be used and integrated into the existing business processes that the firm has. The analogy that I have for this is for manufacturing, right? And in the manufacturing sector, there are what are called robot integrators. And these robot integrators are third party firms that help legacy manufacturers identify what types of robotics and other types of new technologies could be implemented in the manufacturing firm. And this robot integrator, this comes back to that sort of Venn diagram that we were talking about earlier where you need people that are really skilled in the new technology but also really understand the business context, right? So these robot integrators are folks that are right there at that intersection of that Venn diagram. They understand where the technology is, the frontier of the robotics technology as well as the other types of software and other types of physical as well as sort of digital infrastructure that's needed. And they understand the production process in the manufacturing firm really well. So I think that we'll have the equivalent, I think we'll have the equivalent in non-manufacturing settings and I would call these AI integrators. So they understand more or less where the state of the art is in terms of what AI can do. They know how to integrate that into the business processes within the firm. So those are the AI integrators. - And do these people exist or is this one of your fantasies? - So they definitely exist. And I think right now it's a role that's being played in part by professional services firms like Accenture and IBM and firms like that that really sort of specialize in identifying for their customers new types of technologies that the customer can use and can help sort of consult to the customer in terms of how they need to reorganize their business to take advantage of the new technology. I think though that those are large legacy incumbents, right Accenture and IBM and there are other firms like that as well. I think that there's a space though for startups to enter the space and offer their services to legacy incumbent firms as well. So the analogy here would be to, but sometimes we'll call system integrators or eintegrators and names like that in the late 1990s during the early internet slash.com boom, right? There are a bunch of sort of startup integrator firms that understood and some of their names were like Ciant and Viant and Sapient. They understood really well what it is that web technology could do and they understood how to help legacy firms integrate that web technology into all that legacy backend IT infrastructure that the firms had. And so I think that there's a space right now for AI integrators to enter and to help firms adopt, integrate and use this new technology. Again, keep in mind that in the United States I think there's probably representative of the rest of the world as well. But let's speak about the United States. In the United States, 90% of firms don't use AI, right? So that's a huge, huge opportunity. And I would imagine that we'll see startups in this space, the AI integration space. - Absolutely, absolutely. And then we have another new role that you think little murder is already out there and that is AI explainer. - Yes, that's right. So the AI explainer, again, this could be inside the firm or it could be a third party firm that places role or maybe a mix of both. But within a firm, what I imagine an AI explainer would do is they, so at this point, now the AI integrator has already done their job and we have AI integrated into the firm. Could be in an HR process, it could be in a sales and marketing process. Could be in some sort of customer services relationship process, right? But so we have AI integrated into the firm. As we know with AI, there's a little bit of black boxing that happens, right? We don't fully understand how it is that the algorithms do what they do. But we've got the AI now integrated into our firm and you have a decision maker, could be a manager, could be the CEO, who wants to really make sure that things are operating the way that they should be and if they don't, wants to understand why they're not, right? So imagine you've integrated AI into your customer services function and it looks like, you know, due to some internal tracking that you're doing, it looks like customers are not as satisfied as they were before. So why is that? Is there something that the AI system is doing differently than you expected? So the job of the AI explainer is to understand what the technology is doing at a, you know, technical level, but then to be able to sort of translate that and explain it to a decision maker so that the decision maker has a good enough understanding of what's happening such that they can make their decisions. Again, the manager, the CEO, they don't have to understand how to run, you know, set up and run large language models but they do need to be able to understand what's working well, what's not working well and it's the job of the AI explainer to help that decision maker. - So the AI explainer is a technical person. - It's a technical person who does a good job of translating that tech for a decision maker to understand. - And I think that would be very difficult to find, right? It's a technical person with social skills. - There you go, yes, that's right. I'll give you a slightly different analogy here. So you could also imagine, by the way, these AI explainers as being out, you know, they don't have to necessarily work inside the firm. They could work outside the firm. It could be a consulting firm or something like that. One specific way in which I think these AI explainers will show up is in court cases, right? In litigation and legal disputes. You could imagine that they're, you know, because of, for some reason, the AI system that a company was using perhaps caused some harm, right? This might be what's being litigated, perhaps caused some harm to a customer. And it goes in front of a judge and jury. And you're not gonna wanna have the technical person there trying to explain what's happening to the judge and jury. They're just not at all versed in terms of how to do that. So instead you would want someone like what I'm calling an AI explainer to do that. So the analogy for that would be, right now, what we have in litigation is we have these economic experts that maybe do some analysis of, to come up with an estimate of what the payment should be in a litigation case. And they're able to explain in a careful way and an accurate way, but also in an easy to understand way, how it is that they came up with their, there. their figures, how they came up with their estimates, and able to explain that to the judge and the jury. Right? So, I think that this sort of AI explainer role that I have in mind, I mean, it's something that could occur within the firm, it's something that could, you know, maybe a consulting firm could do. I think it's also sort of a more general category where, you know, people that understand the tech really well, but also understand how to talk about it in a way that is understanding and understandable by a much broader audience. I think there's a big role for something like that over the next five, 10, 15 years. I see. And the final role is AI audits, all right? Yes. So, AI auditor, the idea here is you have your, and again, this could be within the firm or it could be a third party company that does this AI auditing. And by the way, that that's why there's another area where I think there's a huge opportunity to sort of this AI auditing. So, what the AI auditor does is, maybe one of a couple of things. So one thing that they do is they, you know, you have an AI system that you've put in place, and you want to make sure that there's, let's say, not discrimination happening, right? But the algorithm is not discriminating against certain protected classes, right? So it could be gender or race or something like that. So the AI auditor, this is one of their functions, is they somewhat regularly audit the AI system to make sure that that's not happening. So that's sort of one role that the AI auditor could play. Another role is, maybe the AI system makes some claim, you know, like we can reduce turnover in your firm by 20%. Well, it's one thing for a company just to say that, right, for a startup, let's say, that has some HR oriented algorithm to say that. But how do we know if that's true or not? Well, this is where the AI auditor comes in, and they can actually analyze the data and say whether it's true or not. So the analogy here, I think, is to just to regular auditing firms. They can either sort of analyze a firm's finances and make sure that the firm is doing what it says it's doing, or they can sort of analyze certain claims that the firm is making, and sort of assure that the firm is, in fact, you know, doing the claims that it's saying it's doing. Do you think we should mandate this role by law? Because a number of companies may discriminate unintentionally, and always valuable for them to have these AI auditors internally and just, you know, to self-check, self-regulate. Do you think we should actually have this mandated by law? Yeah, that's a great question. So I think the big, so perhaps, but there are a bunch of tricky things here. So the first thing that we would need is some sort of set of standards. So when it comes to accounting, we have standards, and then the audits that happen, they take the firm's financial statements and then analyze them relative to whatever standard is in place. So when it comes to things like discrimination, I think the big question is, well, what's the standard there, right? I mean, we could, and how accurate is our, so the audits in these cases are, you know, typically probably some sort of data analysis where you find out that let's say race X receives loans that are Y% less than race A, right? I'm just sort of, I'm making this up and potentially trying to be a little bit vague here. So like, what percent tolerance has allowed, I mean, arguably zero, you know, tolerance, but if you're estimating this, then it's going to be sort of estimated with some error. And so what are the bounds that you're going to allow? Is it sort of a 95% confidence interval, a 99% confidence interval? So I think these things are a little bit tricky to specify axianti. But I think that, I think what we're going to see is lots and lots of litigation on this. I think this is where these, you know, AI auditors come in that they're going to help firms try to make sure that they, you know, don't run a foul of the law. But there will be a lot of stuff that's litigated. I mentioned these AI explainers earlier. This is where the AI explainers come in. They're going to be spending a lot of time in court describing for the judge and jury about what it is that the AI system does and what it doesn't do and things like that. Which of these three rules do you think will be most challenging for companies to fulfill? I think it's, I think, well, there might be a temporal component here. I think right now what's most critical are the AI integrators. I mean, I think there's huge upside for companies from adopting and implementing AI. But companies aren't going to do it unless they have folks either inside the firm or outside the firm that can help them implement this technology. And so I think that that's, that's most important for right now are these AI integrators. I think, you know, so, and right now I mean sort of now for the next, you know, five to 10 years. I think within five years, let's say five to 15 years, we'll see these AI auditors becoming a lot more important and, you know, probably at the same time also these AI explainers. So I mentioned that from a, you know, startup or new business formation point of view, we'll probably see entry by startups that are playing this AI integrator role. I think we'll also see startups playing this AI auditor role, although I could also see, you know, the big accounting firms like PWC and KPMG also playing this role as well. Sounds like that we need a whole bunch of AI baby series people that are big paid, basically not to create, but to make sure AI doesn't mess things up. So I'm wondering, do you think this is probably, this is just all too new and the nobody understands. So we need a bunch of these like I say AI baby series and that eventually we won't need this because AI would just be so integrated in our society. Or do you feel like no, we will always need these AI baby series. First of all, I love that term AI baby sitters. Second, any large organization, be it a for-profit or not-for-profit organization, they have huge IT support teams, right? I mean, really, really large IT support teams. I think it's the same thing. I mean, as we adopt and integrate AI into our organizations, again, be it for-profit or not-for-profit or government organizations, stuff's going to break down, stuff's going to not work great, we'll have new people join the organization, I have to be sort of integrated into it. Right now, a lot of that falls to our internal IT teams. I think these internal IT teams, if they don't already have them, very quickly we'll have this sort of an entire division that'll be focused on AI. And yes, I like your term of AI baby sitter. They will be AI baby sitters. And do you think it's temporary? I don't think it's temporary, no. Again, let's think about that analogy of these large-all large organizations have some sort of internal IT support team. We have had IT for many, many years, right? I mean, computers and the internet have been in organizations for the last two, three or more decades. And I haven't looked at the numbers to be interesting to see, but as a proportion of all employees, I doubt it has dropped over time. It's probably remained, you know, been more or less steady. And I think that that'll be the same with AI, right? Again, to use your term, these AI baby sitters, I think we'll, you know, I don't think the head count, I think we're going to see the head count increase, and then it'll probably plateau. I don't imagine it'll drop. No. As long as the study is AI, I wonder if you actually use it yourself in your profession. I do, right? I mean, so I'm in my field is the education field. I'm a professor at a university, and as a professor, there's sort of three main things that I do. One of those things is teaching. Another of those things is research. And then the third is, you know, various administrative duties. And I definitely use it in the first two buckets, I mean, I definitely use it in teaching, and I definitely use it in the research work that I do. Those two sets of tasks, though, if you will, the teaching and the research are pretty different. So the ways in which I use AI are different. So from a years ago, we asked you a three-shaking work conference, what would you like AI to do for you when you set great assignments? Is AI doing that for you? Sorry, what did I say? I want AI to do what? Great assignments. Oh, interesting. You're coming for receipts. Oh, yes, yes. >> You said that too. >> No. Thank you for bringing that up. No, I do not use AI to grade assignment. I don't know they'll ever be comfortable allowing AI to grade my assignments. I know apparently I said that at one point, but I don't know that I'd be comfortable. >> Actually, we have this video. Maybe we have this video where we ask you, what do you want the AI to do for you? I thought you said that. Maybe you said something else. I have to pull up that video. >> No, no. It was in 2020. >> I know that was in March 2020, right before COVID. >> Yes, exactly. I guess I would love it if AI could be good enough to do grading for me. I just don't think that it is. >> Did you give us some complete examples of how you're using AI in your classroom? >> Yes, I've got some good examples I could share with you. I'll actually the example that I'll give you. I'm going to mention a company's name. They're called Breakout Learning. There are other companies that do things somewhat similarly to Breakout Learning, but they're the one that I've had experience working with and I've really enjoyed working with them. Let me just give you a sense of what it is that they do and then how I use them in the classroom. What Breakout Learning does is you can think of it like a business school case. In the olden days, when we taught business school cases in the classroom, it would be either a PDF document that you'd read on your computer or it would be printed out and you would read it hard copy. As a business school teacher, I would assign it to my students and at night, the night before class, they would read through it and they would think about the case, so then they would come to class and we would discuss the case. What Breakout Learning has done is they have created the equivalent of a case, but it's more a video case that small groups of students log in and they watch the video and discuss amongst themselves and there's a few different points where the video stops and they sort of discuss various questions. There's AI running in the background that then captures what everybody says in those small group discussions that then provides to the professor to me a recap, you know, sort of an aggregated summarized recap of the main points that were discussed and some very specific examples of who said what. So I've used this technology and so the students do this ahead of time before class and then they come to class and I would have seen already ahead of time the sort of read out, if you will, from the AI system, Breakout Learning. And I come to class, I sort of know the things that each group talked about and I know who said what and then it makes it much easier for me to run a class because I know who to call on to make certain points. So that's sort of the system Breakout Learning. This is one of the ways that I've integrated AI into the classroom. Now, you might be thinking, well, hold on Rob, you said that there's a video, you know, the students are sort of meeting online ahead of time. There's AI tracking what they're doing. It sounds like it's not just AI but also video and a little bit of, you know, planning on the part of students, also a little bit of planning on my part in terms of assigning stuff in it, you know, far enough in advance that students can plan all this. And my response to that is like, yes, that's right. Right? So there's, I think there's part of a broader point about integrating AI into what we do, it's not enough just to sort of adopt AI. You have to actually adopt and integrate and change the way that you do things. And so this is a great example of this. I have adopted and used a new technology that's actually sort of a bundle of different things. I've had to change the way that I teach. I've had to change the way that I organize lessons. But it has led to a much better teaching experience in the classroom. So one of the things that I have found is that students end up spending more time. There's sort of forced almost to spend more time before the class thinking about the case and interacting with other students before showing up in the classroom. And then that makes for a better classroom experience. So, Giovanna, you asked me to provide an example. So I know that that was a little bit of an extended example. I'm happy to sort of recap any of that. But sort of the broader point that I'm trying to make here is that there's not some sort of quick and easy way to do this. Well, you know, it's not like pressing a button on my computer and suddenly AI is integrated into the classroom. It requires a lot of sort of thinking and rethinking about the way that the teaching happens. Absolutely. And I think that that's an analogy of sorts for firms adopting AI. Again, you don't just purchase the AI from a third-party provider and then you're good to go. You have to actually spend a lot of time rethinking the business processes and how you want to be using AI. There's a lot of trial and error. Some stuff works well. Some stuff doesn't work that great, but it takes time. And did you have to acquire some new skills to be able to incorporate AI? Oh, that's a good question. I didn't, not in this case, not in that specific example that I was giving you. That was more or less an off-the-shelf, the equivalent of an off-the-shelf product that I licensed from a third-party, that company breakout learning. Other ways that I've tried to integrate AI in the classroom have taken a little bit more on my part in terms of trying to understand and use large language models and things like that. So one of the other things I do is I ask my students to try to program a little bit. So this would be for a game theory class. I asked them to program in a large language model a small game. And so this has required a little bit of of me also understanding how to use the technology. And even in the example you gave me, I being able to replicate yourself basically. So instead of you coming up in the classroom, in the physical setting to every group and trying to go to what they're talking about, you replicated yourself with AI and you were in multiple places at once. But the information that AI gave you is information you were not able to obtain before. So when I asked whether you had to acquire new skills, I was also thinking perhaps now you have all this information to deal with. And maybe you need to come up with a way how to reflect back on this, how to explain students, maybe before you had one or two perspectives that you had to explain, but now you have so many. Because like I said, AI essentially replicated doing multiple places. Yeah, that's a great way of thinking about it. So it maybe isn't me that it replicated in 10 different places if I have 10 groups of four students. It might be like a TA. So imagine there was a TA that was sitting in and listening to each of these 10 groups. And then that TA then wrote up a very brief summary and provided it. To me, now I suddenly have these 10 summaries. And I have to think a little bit, I think you're right. I mean, I get these 10 summaries. I have to think a little bit about how to integrate them and what I want to do during the class meeting period where all 40 students are there. I need to think a little bit ahead of time about who I'm going to call on and what main points I want to hit on. And you're right, that might be a little bit different than how I did it in the past. There you go, you love new skills. That's right. I would like to now take a look at the future, what we may be seeing in the coming years in the next decade. And from what I've heard from you, it sounds like we were talking about really high-skilled jobs or perhaps not only high-skilled jobs, but definitely we were talking about acquiring new skills. And that makes me slightly worried about countries or individuals that are not able acquire those skills. Are you worried perhaps about the gap between the rich and poor widening as a result? And here you mean rich and poor countries, or within a country, if you will, to have and to have not. Both. So let's start within country first. So at one level, yes, I am worried. I am worried that there are a certain set of folks that will have easier access to this new technology and will be able to adopt and use it and train themselves up on it more quickly than others. And the folks that have these skills will be the ones that get jobs, the ones that don't have it, will have a harder time getting a job. So I do worry about that. On the other hand, there is research showing that the use of the technology, so the use of AI, seems to particularly benefit folks that are less experienced, right? So there it is. They're about a handful of studies, maybe half a dozen studies or so. That implement AI in a, you know, RCT, rate random control trial type setting where there's a specific task and they, you know, randomly half of the folks doing the task get access to AI and the other half don't, right, those that get AI or treated, those that don't are control. And on average, you tend to see an increase in productivity for those that have access to the AI but really where you see the big boost or for those that have access to the AI who are less experienced, they tend to really benefit from access to AI. So why do I bring that up? I bring that up because it seems like AI is a type of technology. It could really, really benefit folks that are just starting to learn some new skills. So in other words, you could use AI to help teach people about how to use AI and it's something from an education, from an educator's point of view, I get really, really excited about there is, I do worry about, you know, your leading question was about the gap between those that have these skills and those that don't. On the other hand, I think that there's great promise for AI as a technology, sorry, as an educational tool to help teach people about how to use AI. And so that might help alleviate any differences that we see. So that would be maybe like within a country, like, let's just, I'm based in the US. So, you know, within the US, that would be my big hope is that we get AI as an educational tool everywhere that we, everywhere that we can. You can imagine sort of pushing this into every university and every high school and every elementary school across the country. I think that would be fantastic. But now sort of zooming out to sort of a worldwide level, I do worry about sort of, let's call it maybe like global inequality where there'll be some countries where there's a lot of access to this new technology and there's a lot of wealth that gets created and captured in those countries. And it's sort of, it's hard to see how that would be spread globally unless you have enough infrastructure in all the different countries around the world and it doesn't see, and by infrastructure, I mean, you know, sort of educational infrastructure as well as IT infrastructure. It doesn't seem like we, we have that, so I do worry about that. And so I can definitely see that. And what do you think a typical career in AI, how the economy may look like? I think it'll look a lot like what our careers look like now, except that we will be using AI a lot more. Just like, you know, the typical career, I mean, take my career as a professor. A professor's career in 2025 looks pretty similar to what a professor's career looked like in 2015, pretty similar to what a professor's career looked like in 1995. The big difference is that there's a lot more technology and I think that'll be the big difference going forward. Again, a lot more technology being used, in this case, AI in all of our careers, but I, you know, I think the career cycles will, and sort of career ladders and things like that, will still look pretty similar. You probably recall that 19th century British economy during the minor case was very optimistic about technology and he thought that technology will actually enable us to work less. He was predicting that he's going to be working only a few hours a week, yet many of us work many more, much more than 40 hours a week. Do you think AI is finally one of those technologies that will enable us to work less? Yeah, this is a good question. It could, I think it depends a little bit. It's not just the technology that matters here though. I think there are other things that matter as well and these are things like norms and wages and outside opportunities and things like that. I think many of us probably could work less than we currently do, but work is more pleasant now than it was in 1900, and so I think that's part of the reason why people still work more. Even so, if you look back, at least in the U.S., back to say 1940 to today, so over the course of the last 80 years or so, the sector that has seen some of the largest growth is actually the leisure and entertainment sector, right, it used to account for something like 5 percent of the workforce in 1940 and now it's about 10 percent or higher of the workforce. So that sector of the economy would only grow if people had more time and more money to spend on leisure activities. And so that suggests that we work, I don't know if the number of hours that we work is less or not, but for sure we have more money and it feels like more free time to spend on leisure activities, like going out to restaurants and amusement parks and things like that. Absolutely, actually no matter what people think, by now we work least than ever. Yeah, and moreover, to me it feels like our jobs are better on average than they were 100 years ago. Absolutely. What's the one change you'd like to see in how we think about AI and work before it's too late? Sorry, the one change in terms of how we think about AI or the one change in terms of what AI can do. Indeed, how we think about AI. I think just reminding ourselves that it's up to us how we want to use AI, you know, be it in the workplace or be it when we socialize with each other or be it when we're just sitting on our own at home, playing around with technology. And so, you know, let's not forget that we are agentech, right? We human beings are agentech and we get to decide how to use this technology. And I think we want to make sure that we don't lose sight of that. And if you could leave our listeners with one piece of advice for thriving in the AI, what would that be? My one piece of advice would be to subscribe or download a large language model and start playing around with it. I hope this episode gave you a bit of a hope for the future, but also an encouragement to keep reinventing yourself in the light of the new technology such as AI. Thank you very much for listening and make sure to follow reach shaping work on social media for more on the future work. And until next time, make sure to be curious, be playful and see whether AI has something in store for you.