ETS’s Amit Sevak makes the case for continuous assessment

Whether it's in K-12, especially high school or in the higher-rather world or entering or in the work world, we really believe that there's a new merging set of skills that are going to become much more in demand in the world of work, like AI literacy and communication collaboration, that we don't yet have scaled up yet to really solid rope us ways to measure those skills. Standardized tests have been in and out of favor over the last decade plus. Questions of bias and relevance counterclaims of objectivity and fairness. Some of the arguments have echoes in the current debate over AI. In college admissions, the pendulum has swung back after a wave of COVID period test-optional policies and if generative AI has devalued grades as indicators of achievement and aptitude, standardized tests will likely carry more weight. While generative AI is a tempting academic cheat code, it's also changing the world of work from recruitment and hiring through the design of jobs and automation. Amid these sweeping changes, there's an increased appetite for reliable benchmarks of aptitude and skills. Can a new generation of tests, some using AI, improve the selection process for schools and employers, and can students and workers expect a fair deal as well as constructive feedback? Welcome to the Managing the Future Work Podcast from Harvard Business School. I'm your host Bill Kerr. My guest today is a meet CIVAC, CEO of ETS. The HBS graduate is leading the cornerstone testing firms expansion into workforce skills assessment and credentialing. We'll talk about the implications of continuous assessments from school through career. We'll look at the role of AI in testing and in ETS's operations and we'll consider the distinction between being educated and being employable and how to measure the skills workers will need in the evolving labor market. Amid, welcome to the podcast. Thanks Bill, it's great to be on. Amid, tell us a little bit about your career and what led you to focus on the business of education and skill development? I come from a family that was and has continued to be very focused on public service. Both of my grandfathers were teachers and have a lot of education in the family. For me, I was inspired when I was trying to figure out life about doing something that I felt could also give back from the early days. And so you mentioned I'm an HBS grad. I remember in those days, I was one of a very few group of folks that had that interest in education. I felt like my colleagues were going off to banking and consulting and tech was just getting big back then, but I was always trying to find a way to connect what I was doing to some higher purpose. And so my career, since the before and after the business school days, has been largely in the education sector or in education technology or related areas. And I've been really focused primarily on transformations, doing business transformations, but what led me to this ultimately was a chance to apply some of the skills in the business world, managing, leading, thinking about the various elements of what goes into management, whether it's marketing and sales or strategy or product development, finance, but then also doing it with that sense of mission. And so that's what brought me to ETS. Well, someone who spent most of his life in the academic sector and in education, we know that those business skills are important and necessary. So thank you for bringing them into our sector. Many of our, in fact, every one of our listeners is probably familiar with at least one of your standardized test products, be it the ubiquitous advanced placement tests or the GREs or something that they have encountered, probably on a day that was maybe not their most favorite day of all time, but like they have some familiarity with the test backgrounds. But the business is now bigger. So can you give us like a rundown of just what's the whole business of ETS this moment? Yeah. So ETS has evolved from our early days. We were founded in 1947, primarily back then focused on admissions into universities. And from there, we then expanded into K-12 assessments into other higher-rate assessments, like the GRE. And then we went into international student mobility, like TOEFL. In addition, we also work in the international context with governments. We have, for years, been administering the PISA assessment, PISA is the assessment that the OECD sponsors that it gives a pulse of math and literacy and other assessments across 90 plus countries. Or PIAC, which is a similar assessment before adults for adult literacy. And we work with a lot of governments around the world directly, as well in the Middle East and Asia. We also do quite a bit of work now in workforce over the last three years that have been at ETS. We've made three acquisitions that are moving us more and more into workplace assessments. So broadly today, we really think of ourselves more as an education and talent solution. We now today do over 50 million assessments around the world. We're in over 180 countries. And we evolved our thinking of our mission. Our mission now is we articulated as to advance the science of measurement to power human progress, which can include both educational measurement as well as workplace assessment. We launched recently our new commitment, which is to ready 100 million people for the next generation of jobs. So that again, reinforces this notion that we're moving from our core in education to much more thinking also about the future of work. I mean, it just goes back to your career and personal background to combine both mission and the business side. And I think you very sharply articulated why this broader mandate or the way you want to take ETS into these new spaces is important on the mission side. Talk to me a little bit about the strategy of that, though. What led you from the education sector into going into company's workforce assessments compared to other routes that maybe you could have grown the company towards? Really started with the mission and that that that sense of the north started to advance the science of measurement to power human progress and that we've got this commitment to deliver and ready 100 million for the future of work with those broad, visioning concepts and commitment concepts. So the first thing we did is we said, okay, where are we in the various marketplaces? So we did a diagnostic back in 2022 and we identified that there are areas that are under tremendous pressure for external market reasons and that there are areas of growth and potential. We had these gems of brands and assessments like Praxis, which is in the United States, 43 of the 50 states require Praxis. It's a teacher licensure exam. And we said we have this tremendous shortage in the country of teachers. How can we leverage a test and build an ecosystem around it to serve our mission but also to build a business around that? So we started thinking about Praxis as an engine and then another example is TOIC, which is the test of English for international communications. Over 5 million people every year take TOIC to demonstrate their English competency for the workplace. So countries like Japan, Korea, and many other countries around the world, not just in Asia but in Europe and elsewhere, rely on TOIC as an employer to measure English effectiveness. And so that's another example of an brand high quality assessment with broad global reach that has potential. We also, as you mentioned at the beginning, we have this brand at ETS. It's known, it's recognized for quality. We've been administering in partnership with the college board assessments like the SAT and we continue to administer the AP for the college board. So brands that are really well known here at the United States and in other parts. So we said how can we leverage it products and services that we have and partnerships that we have and reimagine a new ETS building on those strengths. The other thing that we have is our research capabilities. We have an extraordinary legacy of phenomenal research that has pushed the boundaries of how do you measure with high quality, with reliability, with validity, with fairness, the skills and competencies that an individual has so that when you get an assessment and you get a score, there's a confidence that you have. So when we took a look at those three elements, really strong brands of at the product level, plus broad geographic reach through partnerships and we had an incredible research foundation. And then as we looked at the external marketplace and did market research and talked to a lot of stakeholders, we'd recognize that there's a tremendous opportunity for us to go into adjacencies around those. So we started looking at products and services around each of those areas and then we started to strategically think about what countries do we want to play a bigger role in and we started identifying those markets and going big into those. This will always be a classic case study of you can grow within areas, you can move into new geographies with existing products, you have green field operations, you've got a lot of different things that you're pursuing. Let's stay on that research dimension for a second and there's the making sure that the tests are effective and the assessments are up to date. You also published something called the Human Progress Report. What's the intended audience for that work and what's the gives a sense of how you move the needle research wise in those domains. We have identified our three big priorities that we're going to go after in research. The first priority is that we want to push the boundary of creating new paradigms or new methodologies for how do you measure. So we are looking, for example, to move away from a standardized testing paradigm to more of a personalized testing paradigm and that is a big shift because the architecture of how we do testing is largely based on putting individuals to take a test on some standard, some point of reference in relation to each other and in relation to a scoring rubric. We're now looking at how do we do it in a more personalized way with adaptive capabilities and ways that can measure individuals per pass performance and predict future performance in a much more personalized contextual way. We're really pushing the boundaries of what's possible using and leveraging our research and our capabilities and we've added some new team members to complement that. Another big area of focus is policy research. One of my big pain points that I see and I imagine many others feel is, you know, we as a country spend over a trillion dollars on education and if you think about it globally, the world spends roughly five trillion dollars on education but we don't measure it at the level that we could measure it. If you think about putting on the business school hat for a second, if you think of finance or pharmaceuticals or sports, these industries have such sophisticated ways to measure. And yet when it comes to our children in schools and when families look at what education am I getting, we don't have real time sophisticated high quality holistic data. So the opportunity to use policy research as a way to really get out there and help shape and really have more measurement and to ensure that we're getting higher quality outcomes from the education that we do for K-12 or higher rate is a second priority. And then finally, a huge priority for our research is on skills. Whether it's in K-12 especially high school or in the higher rate world or entering or in the work world, we really believe that there's a new emerging set of skills that are going to become much more in demand in the world of work like AI literacy and communication collaboration that we don't yet have scaled up yet really solid robust ways to measure those skills. Artificial intelligence is changing many parts of our business landscape and education and beyond. Tell us a little bit about from the research, but also going into practice. First, how is AI changing what you're doing internally at ETS and also how do you anticipate it affecting or shaping the future of the assessment like landscape? We have several initiatives involving AI at ETS, the first is AI literacy. It is a building up of our internal ETS workforce to be prepared to efficiently and effectively leverage AI. And so we have a new tool called Adapt AI that is a suite of assessments that allows us to effectively measure foundational skills, prompt engineering skills, as well as tailoring AI for particular job uses. The second big area is leveraging AI to drive efficiencies. So in our operation, there's three broad steps. I mentioned those 50 million assessments we do every year. And essentially the factory, if you will, of that starts with first designing and building a test, then it's delivering that test, which could be in a physical site or it could be virtual in an online format. And then third is scoring and analyzing the results. So in that factory or in that process, we're now leveraging AI to much more efficiently and effectively manage each of those steps. And so that's allowing us to drive more cost savings. But it's also allowing us, and what I'm excited about Bill, is it's allowing us to better and faster get new ideas on how to deliver tests out there, tests and iterate and innovate on many, many more skill types and really help countries more quickly measure what they want to measure. And then another one is really to leverage our capabilities around thought leadership. And so we do convenings, we write papers, and we're much more focused on leveraging those. We're very fortunate to have Dr. Matt Johnson on our team, a world-leading expert on leveraging AI in the assessment space, and he's building a team around him in our research category, in our research institute to be able to build into that category. And so those three initiatives that we really started two years ago are building on initiatives that ETS has had for many years. We've been leveraging AI in all kinds of ways that automated score reports and engines that we have. Because when you're building tests, there's a lot of learnings that you can replicate using different kinds of automation. But we've dramatically accelerated that through these three initiatives around literacy, efficiency, and thought leadership. That's great. Let's go now to maybe the second bucket, which was about public policy. If you had the opportunity to kind of suggest for the U.S., and maybe also internationally, given that you have a broad set of products, what are the things that could most help move the needle in terms of the way we are assessing individuals and talent in their development and their opportunities and so forth? Bill, I believe we need a long-term vision for what we are seeking to do in our education and workforce, Arena, a Vision 2040 or a Vision 2050. We've got to think about the implications of what the decisions we make now in the vision we said now can have on really building the workforce for the future, the citizens for the future. In 1983, there was a paper that was written called a Nation at Risk and that paper here in the United States galvanized a conversation about how we can strengthen K-12 and even to stomachs at higher education. I think we're ready for that and I think that's a key role that policymakers can make of setting a vision that's not just for the next election, but really thinking 10, 15, 20 years out because it takes time to move educational institutions towards something and we definitely need something new. The second recommendation is to embrace the data that we have today. ETS is very fortunate to have been the stewards for administering NAPE in the United States. NAPE stands for the National Assessment for Educational Progress. It's the nation's report card. Every other September when the U.S. Department of Education announces where the 50 states are in terms of math and reading scores for fourth and eighth graders, which has been in the press quite a bit, particularly after COVID, as we saw the math and reading scores come down. That was ETS behind the scenes administering that for the Department of Education. We've got to look at that data. We've got to understand that data and we've got to help the population really understand what that means. The fact that, for example, today, two-thirds of American eighth graders are not at standard for math or two-thirds of fourth graders are not at standard for reading. That is significant if you take a moment to think about it that our incredible investment, we as a country invests almost half a trillion dollars in K-12 education across the country, and that most of our assessments are showing in most states that we're not able to get to basic levels of fourth grade math and reading and basic levels of fourth grade and eighth grade, you know, for the eighth grade as well. We've got to elevate the conversation and make this much, much more central and that's where measurement comes in. We have to, as a citizenry, as individuals, whether we're in business, whether we're in nonprofits, whether we're in government, ask for more data, demand more data, demand more measurement. A lot of this can happen at the state level. States with forward-looking governors, with forward-looking state superintendents of education, with chance there's a higher ed, can be encouraging better measurement to see what outcomes are for the learning that they're getting. We use very, we use laggard data like attendance or graduation rates. Those don't really represent the skills that someone has in their mind. These are really old numbers that we're using to measure educational attainment. We've got to go beyond just a 10 class. What does that mean? That's just passing time and getting a piece of paper. Did they actually learn something and we can do better? It seems like you could have the extra assessments and data both speaking to root causes behind the shortcomings, but then also the effectiveness of various solutions, particularly if you could be partnered up with experiments that are being run to try to remedy the challenges that we're facing. Yeah, and policymakers can ask for more like why not a 12th grade name or at the state level, how about we think about skills transcripts that complement an academic transcript where a governor says, hey, I want every high school in my state to have a transcript that actually represents job readiness, work readiness. Well, how about career interest surveys where we're trying to capture students in 13 to 15 years old, which is when research shows students start to demonstrate interest in things that they can imagine doing as an adult. That's when we want to get them and measure it, identify it, explain it to teachers and principals so we can support people in their aspiration to know what they're going to do when they grow up. It's not just a test. It can be used for evaluation, for surveys, for policy making, but I think we just all have to decide, do we care about what our children are going to do in B in 20 years? And if so, let's define the vision, let's demand better assessments and let's galvanize really a movement around measurement as being much more valued. It's going to take us very smoothly into our third bucket, which was about skills and about the future of the credentials in the workplace and how will they be assembled and how will they be kept up to date? I think we could all appreciate how whether or not you had a college degree becomes rather irrelevant when you get to our ages. What do you anticipate being the state of play in a very fast-moving technological space over the next few decades? Over the next few decades, the currency is going to be the skill and really a bundles of skills that are relevant for a job to be done. And so if we break it down to the skills that are going to be required for the jobs of the future, of course, some of them will be across the board, but many will be specific to the industry or the job function. We can start to visualize a taxonomy of skills, some existing, but some entirely new skills that we're going to need thinking of few generations into the future. So for me, what I envision is more and more expectation of credentials. We at ETS, as you mentioned earlier, had done the Human Progress Report, this past report that we did was our second anniversary of that report, is a sentiment analysis over 18,000 respondents over 18 countries. One of the key items that insights that came out of that was that more and more people, particularly Gen Z, are seeing the value of a credential as coming close to that of a degree. What people are feeling and thinking is, gosh, if I get a whole bunch of certifications or non-degree credentials, that that bundle is going to start to matter as much as a degree. That's not today, but you can see what a young person is thinking is going to be relevant for them in 20, 30 years. So what does that mean for high school? What does that mean for higher ed? Well, what I think that means is if I'm a principal of a high school, or I'm a president of a college, or overseas sit on the board or oversee those in some way, I want to be encouraging. I'm going to even use a stronger word of demanding that those institutions are having credentials embedded in order to get that diploma for high school or that degree, literally embedded. And it can be an elite institution, it can be a public institution, it can be a specialized institution, but we have to recognize that the credentials that are going to help transition are really at the skill level. And it's much easier for employers to often say, oh, that individual has a project management certification, or they have a certification in digital marketing, or they have a certification in AI literacy, or they have a firm certification for the Society of Human Resources Management. I don't believe the degree is going to collapse in 10 or 20 years, as sometimes I hear saying, or that the diploma is going away. I think that principals and presidents are going to see more and more this opportunity and really try to do it. We need a creditors to help them so that the accreditation process isn't so rigid, so that we can embed those into the curriculum. But I think that the blending of a degree with these certifications is where things are going. Well, it one often has at this point then the question or the objection or just the what about, you know, this notion of employable skills, being employable versus being educated in a broader sense, and things that may matter over the course of a lifetime, or for a good healthy relationship and living that are not necessarily what you need to get that next job. How do you tend to think about or talk about that? It's a yes and that's also there, but we need to assess the skills, or is there a deeper way to bring those together? It's a great question, and I was recently on a panel where there were several university presidents, if we're having that debate exactly, it was in the context of what's the purpose of higher ed and how much of the purpose is around the economic value of helping prepare people for their future jobs and careers versus the societal role that we play in forming citizens, or the research role that we play as research institutions. I really believe that given how much change is happening in the economy, driven by all kinds of forces, global forces, would geopolitics change technological forces, but even just fundamental economic forces at work, that this is a moment right now where higher ed institutions and high schools need to pay much, much, much more attention to what is happening in the economy right now and asking, is this class actually preparing them for what that might look like? Is this curriculum actually designed and am I measuring the relevant skills that can help them be ready for work? I do take your point about it say yes and because yes, there are other elements of what we are doing in education right, we're forming citizens, we're forming individuals to incorporate into society and play roles there and that involves civics and awareness and media literacy and other elements there, but boy, if you ask most families what they want for their college graduate, it's to have a job that is appropriately paid to help them launch and then of course to extend beyond, so I'm born in the camp of really helping get the ROI of college to be more clear. I get for the elite institutions, students going into institutions, they've got the brand, but for the 2500 or so other four-year institutions and for the technical institutions and the community college institutions, really focusing in on the value that you're creating at the skill level, it's going to be just so critical to survive. I mean, we're going to see in the next 10 years, I believe, another couple hundred colleges that will go bankrupt and I think those that will survive and thrive are those that are going to realize there have to adjust to the new market reality that's out there. Considering that the individuals are making these live investments, this may have also been a topic of conversation on that panel, it's certainly on a lot of panel, is what advice in a world where generative AI is coming up to learning curves so quickly on so many different academic disciplines? What advice can we give to kids in terms of where to make those investments now? We clearly have to prepare ourselves and our children for a dynamic world where they're going to continually need to push their skills up the curve and stay in front of technology, but do you have a viewpoint at the moment of where's the safe harbors in terms of skill investment that at least for 2035, we're pretty sure you're going to hold on to? Number one, basic AI literacy and comfort, right? Just playing with it, using it, getting out there, having fun, and that means having access to different large language models, different ways to whether it's chat, GPT, or Gemini, or others, just having the chance for them to play. Now that's important because I know that sounds obvious to many of us, but if you go back 20, 30 years ago when the internet was first taking place, there was a huge digital divide, right? And those that had access early were able to go up the learning curve of getting comfortable for papers, for research, etc. versus those that didn't have access to internet in the early days, right? Back when broadband was still coming at homes and so forth and mobile phones, and I think my concern bill is that we're in a situation right now of not paying enough attention to the AI divide, of not having enough students have access to play with these kinds of tools, whether it's in school or at home, and I think for all of us, you know, even for adults, and then the other is just being comfortable with the applications in different use cases. I really believe we want to be preparing graduates of high schools and graduates of universities to be AI natives, to be so immersed in it in their educational experience that it's just a part of what they do. We don't yet have an education and an higher-read consistent policies on the ethics and the reliability and the fairness of it, and that's not just for detecting cheating or ensuring certain skills. There's actually more fundamental values we are as a society are still trying to figure out about attribution, you know, what who gives credit for what. I think there is some work we have to do there, but that's where students can play a role and help defend some of those and propose some of those policies as we move forward, but I'm very bullish. I'm all in on leveraging all of these capabilities and giving more access to more people to be able to have the AI to help them be ready. You mentioned earlier a fascinating point about how kids that are coming into the labor market are increasingly viewing this bundle of credentials as skills as being able to sum up to something more than their components and start to look a lot like a degree. And on the opposite side, of course, they need to have the employer look at it the same way. And we've had, you know, over the last, you know, five years and even maybe beyond that, some kind of starts and stops towards employers doing more skills-based taxonomies in their workplace and skills-based hiring. So when coming then on to the employer side of this, what do you see as the the challenges or, you know, are there some companies that are starting to really get it right in terms of how they are approaching the skills of the workplace and their use of these alternative credentials? Yeah, I see some institutions and some companies doing it better. They're investing in the human resources functions. We at ETS, for example, are really starting to develop a talent management strategy that's really tied to the competencies that we want by job function in partnership with our colleagues in the coming months and going into next year. I also believe it's important bills for professional associations to take notice. But if you take the American Nurses Association or the American Electricians Association or other professional associations, they can play an amazingly important role in helping to look across the diverse range of companies and identify consistencies in the kinds of skills that are required today and where they see those skills coming, particularly as new technologies, not just AI, but so many other types of technologies that are being leveraged in those spaces. A big part of professional associations is around professional development. They do conferences, they do webinars. And so I see the professional associations as a big opportunity. We at ETS have hundreds and hundreds of associations as clients through our recent acquisition of PSI and other acquisitions. And so we're starting to see how professional associations are leaning in to identifying how they can upskill their staff with certifications for them. And that can then help companies in turn leverage best practices from other companies. But there's a ton we can do here. And it's a great point. You see the tech companies doing this quite a bit, whether it's a sales force or an Amazon or others, creating certifications for their tools, not just for the world of work, but also for institutions for K-12 and higher ed to start to leverage some of those certifications. And it comes back to the currency across education and work being the skill. And a certification is a bundle of skills that bridge you to that 21st century job. I mean, ETS has clearly come a long way from the paper and pencil test that many of us at some dark moment in the past sat down to. What are you looking to do over the next five or so years? Like what's the kind of objectives that you're bringing with the team that we can be on the lookout for? Well, the first priority I have is to continue to drive our growth in the areas where we seek opportunities to serve, increasingly at that intersection of education to work, skills-based products, certifications and so forth. As I mentioned earlier, we've got companies and businesses like PSI, Praxis, TOEIC, and many others in the corporate world and beyond. The second is really to drive ETS's impact. And that impact for me comes in several categories. One is our research impact in those areas that I mentioned earlier. Another is the impact we do in our policy advocacy, working with governments at the federal state and international levels. Another big area is around thought leadership and really driving that. And then we also are looking to continue to focus on impact in terms of evangelizing a skill-based approach to high school and higher rate, a competency-based approach, and really providing the tools to help institutions do that. Products and services at the intersection of education and work, then the second priority is ETS's impact for the world, a more mission orientation. And then third and finally, culture. We have ETS employees across 30 plus countries who've got offices all over the world. We have over 10,000 places where individuals can take tests on our global test network. We offer tests online. But to have that single culture that says, you know what, we're here to serve, we're here to grow, we're here to have impact to do it with incredible integrity and credibility at a time in which we're hiring people, at a time in which we're expanding into markets. So to really have an ETS way that's consistent that really is going to help us move forward. So those are the big three priorities that I see over the next three to five years driving our businesses, driving our impact, and driving our culture. I think that's a wonderful place for us to end and given how many places you touch the education to workplace opportunities and efficiency that we have. We are all wishing you the best. Thank you for joining us today. Thank you so much for having us on your show, Bill. Thank you. We hope you enjoy the Managing the Future of Work podcast. If you haven't already, please subscribe and rate the show wherever you get your podcasts. You can find out more about the Managing the Future of Work project at our website, hbs.edu/managingthefutureofwork. While you're there, sign up for our newsletter.