4. How AI is reshaping UX and the new role for designers

Welcome to the design of AI podcast. We dive into how AI products are designed and how they're changing the world around us. Please subscribe on Spotify, YouTube, or Apple. If AI is so smart, why is it so hard to use? We have to learn text-based prompts, we have to feed it so much information to make it useful. Why can't it be as easy to use as the mobile apps we've learned to love? To answer these questions, we need to think about where AI started and where it's going. So the first AI, Logic Theorist, was built in 1955. It was massive and it was built exclusively to solve math problems. The first chatbot, Aliza, was built in 1966 by MIT professor Joseph Weisenbaum. It was the first example of natural language processing programming that used pattern-maging techniques to engage in conversation and simulated the psychotherapist. And in that nearly 60 years, we've seen countless examples of chat-based AI interfaces. So why are they still so prevalent? It's not like there haven't been other interfaces that have gotten public attention. In 2005, the first AI-powered autonomous car called Stanley came out of Stanford University. In 2011, Apple launched their voice-assistant Siri, followed soon after by Alexa in 2014. And we had this grand promise of voice-based AI. As of 2023, several university research teams have seen up to 60% accuracy with helmets that translate brainwave activity into written texts using LLMs. And in the fall of 2023, NeuralLink got FDA approval to start human clinical trials for their brain implant and successfully completed their first implant in January. The currently growing investments base seems to be in agent ecosystems. So these are AI assistants or agents which connect several AI tools in the background to proactively complete tasks with disparate parts. Today, open AI, Google Cloud, all-use, text-based, and now image-based prompts. Meanwhile, co-pilots like GitHub and Devon are trying to be companions that proactively collaborate with you. All of these businesses are exploring multimodal interfaces, meaning that the program can input and output images video sound. Where we are today is basically a ton of businesses rush to get the products to market. This is basically been an engineering-led push. And what we're seeing are basically prototypes with the future might look like. So I have all the belief in the world that open AI, three years from now, it's going to be a lot easier to use. I have all the belief that Google is really going to listen to all the pressure that investors have been pushing towards and build something that actually is useful from the standpoint of anybody. Multimodal in particular will be that game changer. We spoke to somebody a couple weeks ago where they really believe that multimodal will be the gateway to make the web more accessible to overcome language barriers, overcome age barriers. So there's a lot yet to come, and it'll be interesting to see where we go. Now from the perspective of designers, a lot of them are waiting on sidelines. They've been struggling. There's been a lot of layoffs, a lot of concerns about where UX goes from here. The reality is we don't know that. And that's really what we want to talk to our guest about today, because the fact is what UX might look like two years from now might look a lot like what it looked like two years in the past, or it might be something all together different. It's going to be very exciting to watch this. We might not be living in the movie here yet, but it seems like AI is starting to explore a range of different design patterns. Today we're going to be talking with Emily Campbell. She's the founder of Shape of.AI's project resources. So she is a product design leader and advisor, currently focused on helping companies establish AI design as a core discipline through shared principles, patterns, and practice. She has a background in economics and research, and she combines a deep understanding of behavioral incentives and systems thinking with 15 years experience honing her craft in digital design. I know before we do jump to her, this is a must listen episode for anyone who is in design or interested in UX. [MUSIC PLAYING] Today we're going to be talking about the current and future state of UX for AI. So you have a project called the Shape of.AI. What is Shape of.AI? And more importantly, why did you feel like it was necessary right now? Thanks so much for having me. I'm so excited to chat. The Shape of.AI right now is a library of patterns that I'm seeing emerge in products that are incorporating AI and particularly generative AI. And the reason I created it was I was starting to see things change really quickly, and I needed to wrap my head around it. And I'm seeing it change on a number of levels, but the most obvious is at the interface level. And the way I process things is just by putting my thoughts out there, trying to create some structure. Part of the reason I'm really excited for this conversation today is, I'm assuming a lot that's in there right now will no longer need to be in there a year for now, either it'll be, we've moved past it, or it's become commodified, or we're onto new patterns. But right now it's just as kind of that, this is what I'm seeing at this moment. Let's start talking. To that point, a year and a half ago, a little less than that, Chad G.P.T. started a huge hype frenzy in terms of AI. It is in the public dialogue. My mom's even talking about it. But we're still living in this cursed text-based prompts interface. Please tell me that's not our future. I hope that's not our future. I don't see that being our future. And we're actually seeing these companies start to evolve away from that pure conversational chatbot type interface even just today. Chad G.P.T. has been rolling out some new patterns that make it easier to not just feel like you're staring at an open canvas. There's really good benefits to that as well though. You know what I'm sure we'll get into? It can help companies learn what people are looking for. And it does get us past this feeling that we can only do what the interface face tells us we can do. We now can go deeper and interact with the product and the data ourselves. But yes, we need to have more structure and more predictability for people who come in and don't know what to ask, don't know what to look for. And more importantly, don't know what to do when they don't get what it is that they're expecting back from the system. What types of different interfaces are you seeing? Like I've got a list in front of me and different ones that I'm seeing. So what are you seeing in terms of interface changes that are most exciting? You're right now. AI as a product functionality is showing up in a few different ways. You have some that are just pure AI products. Then you have some that are, let's call it like augmentative. You say that right? They augment the current user experience from what you would see in a sort of traditional interface, screen interface that we've been used to over the last 10 years. But it allows you to interact with it differently. And then there's this kind of like adaptive aspect to it that you can personalize your experience. Essentially have interface that exists only for you. And that I think we're going to see expand. So let me like run through these really quick and say what I'm seeing at the interface level. The pure AI product, that's where we're seeing this text-based experience. At the product level, it's what you think of if you log into cloud.ai or into Julius or into certainly chat GPT. And we're also seeing that at the feature level in the form of these agents and chatbots and co-pilots that are showing up. So the dominant interface pattern there is certainly conversational. But already we're starting to see things like multi-modality show up. So I can choose do I want to see images, do I want to see links, do I want to see summaries. I can add inputs into that and guide the interface to what I want it to create for me. Mid-journey is a great example of a product that literally started in a chatbot built through Discord and now has a web-based interface. So that's one area where we're seeing a lot of rapid change. For the aspects of AI that are augmenting the current experience, this is where we're seeing a little bit of that ability to take the current product experience like notion and interact with it through that text-based medium. So at a column, here's a prompt, give me this information. Summarize this for me. We see this in like Zoom and in GONG and Grain, the video tools that are giving you summaries back, summaries on PDFs. So it's this layer on top of what exists that's letting you go deeper, summarize, synthesize or direct, but it's very much integrated into the product experience. And then where we're seeing the adaptive aspect, that's really just starting to emerge. That's where you know I might say this is what I want to learn from a learning management tool and it will provide the content, the learning path, the depth, the competencies that relate to me and my needs and no other person in my company might ever see that same collection of content in the same order with the same emphasis. I think we're going to continue to see that use case evolve and certainly I can speculate on where we think we're going, but we're just beginning to scratch the service of what truly personalized experiences look like in these digital ecosystems. You highlighted mid-journey, which was one of the first that I really tinkered around with and you highlighted how it started in Discord. Now Discord has a very distinct audience that loves to use it, you know people love or hate that interface, but now it's looking more and more like getty images. Are we going towards a future where the products are just going to start normalizing back to design patterns that we had pre-AI, where they're going to start branching into like this is an AI product is an AI augmented product is an assistant product. What do you see? I think I can answer this by you saying yes and I don't know because we will see the patterns start to become more more permanent, more commodified, more consistent and a lot of the patterns that we see in our products today exist because they work really well. So where we're able to kind of draw from those best practices, of course, we're going to see designers work through that. Part of the reason we're in the situation we are right now is designers have not been as involved in the process of getting us to where we are as we need to be and as we will be moving forward. So you have a lot of products that have been driven by either engineering first principles or business first principles or copycatism. I mean, one of the reasons why everything is a chapot right now is because chap GPD is a chapot and it's easier to replace your interface with one that already works so you can vet your core product, your particular take on an LLAM or on a tuning of an LLAM and so on. We're also going to see new patterns emerge and start to solidify as well because what's different with AI that we haven't had before is as a user, I'm now an agent in my own experience. I can tell the screen what I want to see. I can tell the algorithm that I think it's wrong. I can direct it in a new direction. I can ask it, can you give this to me in a different modality? Can you give this to me as a video? Can you summarize it? Can you produce something for me? That is a capability that users have never had before. And so it's going to make sense that some of our old patterns aren't going to work for those use cases and we're going to need to invent new ways of letting people interact with the data and with these programs in ways that are completely unfamiliar to the average person. One of the things that you and I talked about when we first had a conversation was about how reminiscent it feels right now in 1995 when people were really just discovering how to create GeoCities websites and people were really just figuring things out for the first time. And we've gotten to a point now where I would almost say that the UX industry, if you want to call it that, is really out of a bit of a precipice because so much work has gone into building beautiful interfaces and interaction models and we've really tested and understood human behavior in a way that we hadn't for so long. That right now we're almost jumping backwards in an excitement to get into the engineering space of it. And I think that's partly why it's so frustrating having such text-based prompts for everything. How do we learn from where we've gotten to from just design systems and human center design and apply that more readily to what's happening in terms of AI design? That is such a good question and it is really the moment we're in where we're in this kind of existential crisis right now as an industry trying to figure out how do we define ourselves. Let me take a step back because I think the comparison to the mid-90s, the comparison to the late OTS is another really interesting moment in time where things changed quite a bit. What was distinct about these moments for design is design was helping to shape the clay. We were very, very close to that place of creation. For example, I got into a digital product design right around 2009, 2010 as responsive design was taking off. At the time, we didn't have the tools to be able to mimic a responsive layout in some beautiful web product. I mean, we were all using Photoshop, which like, let's not go back there. And so myself and many, many designers I knew we'd learned how to work with the medium. We learned basic HTML and CSS, even if we just kind of needed to hack our way through, to be able to convey our intent. And I don't necessarily see designers racing to the engineering side to stop being designers. What I'm seeing is people wanting to get closer to that moment of creation, to that medium, the actual material of what it is we're designing with. In terms of how we think about what this means for design as a discipline and as a craft, design will change. All of our jobs are going to change. That's not a bad thing because our jobs need to change to allow us to do what is central to our jobs, which is conveying some sort of intent, some need to shift behavior, heighten some incentive, whatever it is that we use to help people nudge their way through the maze of these interfaces. It's going to change because the way people manage through those interfaces, what data and information they have, what they're trying to do is shifting. When I think about what designers need to do next, it's two things. The first is start to understand how these tools work. That doesn't mean you have to learn how to build them. But understanding the limits of the various models that we are working with. Understanding how the different tools, and I'm going to put this in quotes, but think how they think, how they process information, how they respond, what their personality is. That's critical. But we also need to remember that we have this human-centered skill set. For a reason, it's not going away. This is still people interacting with data. It's still people interacting in an unknown space. It's still companies trying to drive behaviors and incentives. As designers, it's still going to be our role to say, what is the individual within the system trying to achieve? What are the constraints they are operating under? What is the information they don't have? Then by understanding how these models operate, we're going to be much more informed to guide people to the outcomes that they're seeking in a way that reduces harm, is accessible, is easy to use, and is delightful, and most importantly, centers on their interests. If I were to leave anybody listening to this with any little takeaway already, it's this, go out and use these tools. Use them as much as you can. Use them in situations where maybe it doesn't feel like a natural replacement to how you might do some task. But the more comfortable we get interacting with this new generative AI world, the more empowered we're going to be to guide good outcomes for the people we're designing for. Products shape behaviors. They change the way they we work. When I look back, having worked in design for way too long, every new product really created new capabilities, it changed the way that I could work, but I could deliver. When I look back at Photoshop, every release of Photoshop was monumental. The undo was transformational. The idea of moving around layers and macros, and like I'm really dating myself, you worked at Envision. Envision, obviously changed the game a lot, but it became a bit of an arms race and design, and then Figma came around and just blew everybody away. I wonder if AI will have a moment like that. We're right now, we're still in such a nascent point that the interfaces are still so basic because it's just engineers trying to push product out on deadlines. That something soon will come out that's going to blow us away. Have you seen anything that really makes you feel like there's a bright future coming along that's going to change the game? I'm looking for bright futures. There's definitely things that scare me right now. I'll tell you that. I think that the idea of the agent technology, that's really interesting to me. You could see that becoming harmful as well because if people don't understand the power of these tools that they're using, if we remove so much friction in the experience that people just assume it always has the answers and assume it's one of them, I think that there's some anti-patterns to be concerned about there. But at the same time, what we are now able to do is create an experience for people that meets their needs at an intimate and personal level, which as long as we are doing that in a way that reduces harm, it's incredibly empowering. It removes the power of large technology companies to a large point and puts it in somebody's pocket. Let me tell you exactly what they mean by this. Right now, if you log into Instagram or LinkedIn or Facebook, you don't know why you're seeing what you're seeing. You're being bombarded with information that some programmer and some business executive somewhere working with some ad company has decided you should see this because I want you to take some action. I don't know how to say I don't want to see this. I don't know how to say this is what I want to see instead. And I don't even necessarily know why I'm seeing it so that I can change my own behavior or follow or unfollow people or trends or whatever in order to adjust my experience. When I'm interacting with a tool that lets me talk to it, I now have a lot more command over what I see, why I see it, and what I want instead. And I get to dictate that. Now, there's going to be companies who want to take advantage of this. There's going to be ad tech. There's going to be algorithms and training in these LLMs that the individual can't see. But what I can do is I can say, why are you showing this to me? Why are you doing this? Why am I receiving this? And get an answer back. That to me is a really good future. It comes with a lot of caveats of things we need to be aware of. But I think a world where users are interacting as a player in their experience and not just a consumer. If we do it right, carries some really exciting opportunities for it. And then, you know, back to your comment around what does that mean for design and product and engineering. I'm hoping this means that we will be able to work with our users and for our users more individually and not just the incentives of the large corporation and what they're trying to do. It's not always going to be perfect. There are AI tools right now that are doing not great things in the world. But generally speaking, we are now beholden to what the individuals are trying to do in the system and how that relates to what the system is trying to do with the user, unless the business incentives alone in terms of what we're working on and what our outcomes are. I hope. One of the big differentiators with these AI tools is the ability to respond, react, ask for clarification and such. But when I think about that, I think about those annoying YouTube notifications that are like, was this notification helpful? I think of NPS. I think of follow-up surveys. It feels like, at least in my experience, I've not seen a good pattern, design pattern at least, to make me engage with the idea of interacting with the quality of a response. Are you seeing any specific examples which actually make you feel like, yes, this system is actually going to learn something useful other than only getting the best and worst response rate? - So there are two patterns that I've started to identify. One is called Tuners, and the other is Wayfinders, and then there's sort of smaller patterns within that. What you're describing is another pattern, and that's the thumbs up thumbs down. Yes, this is what I wanted. Whenever I see that, I assume that's the company saying, am I training my data correctly and not did you have a good experience? And I just ignore them. I do not trust them to actually shape my experience. But what Tuners and Wayfinders do, is they help me as a user understand what the information is drawing from and how I can respond and get something different. So let's talk about Tuners first. This is this idea of having the ability to put prompts directly into the system without me having to know exactly how to write them. So maybe I write something, but I say, make it shorter, or use this tone of voice that I've saved instead of this tone of voice, or use this reference when you're trying to create the answer. I'm gonna upload a PDF document of something I wrote, and I'd like you to summarize this in a tone that's similar to the way I've written this in the past. The challenge that people have right now is they don't know what they can change and how to change it or if it's wrong. And so they're not empowered to do that. But the more we get these patterns that expose, the power the individual has, the more the individual is going to get comfortable with that. The other pattern of Wayfinders is nudging the user to take action when they're staring at the blank campus or not sure what to do next. I've spent a lot of time interacting with GitHub's co-pilot, for example, which allows you to not just write code with it, but ask it questions about your code as you're going through the process of constructing some document. I can say to it, why didn't this compile? Why did I get an error? Can you show me what I could be doing differently? Tell me what you would do here. Is there a more efficient way to do it? And because the tool is constantly talking to me and pointing to the code, I'm not just like chatting with a chatbot. I'm interacting with the chatbot in this common goal of writing something and trying to get it compiled. And let me tell you, I'm not an engineer and I've been blown away by what I've been able to achieve just by working with this tool. So I think those are the patterns that we should be relying on as designers, the things that give somebody the confidence to take action or give them the information to know, hey, there's an opportunity here for you to make it better and you have the power to say what to do next. In the research and strategy work that I've done, especially when it comes to emerging technology or AI in particular, when people are asked to sort of conceptualize something that is unknown or foreign to them, they get very apprehensive or they kind of will maybe dabble in the simplest of ways. So things like what you're saying, wayfinders, that makes a lot of sense. But when it comes to something like tuners, how can designers in your opinion overcome that hurdle? Because when you're talking about even the GitHub Copilot, I'm thinking back to interviews I've had with people where it's like, yes, this is immensely helpful. But you are still putting extra work on someone that maybe they wouldn't have been asking those questions in the first place, they would just be expecting that perhaps someone else would do it or these tools would do it for them. Yes, I share that concern. It's one of the biggest critiques I have, the current iteration of AI tools. And I think part of it, again, comes back to, we've been in this just race to the starting line where companies are throwing these products out there, many of which don't need to exist. And certainly most of which are not thoughtfully designed through like an actual user journey. What is somebody trying to achieve? But there are a few things changing very quickly that I think will help here, particularly within the space of companies introducing AI into their products versus the sort of vanilla models of chat GPT and cloud and so on. One of the ways that we're starting already to see this evolve is adaptive AI. An adaptive AI or contextual AI is this sense that I'm not going to wait for you to tell me exactly what to do. I'm going to anticipate what you might want to do and maybe check in and confirm with you. So let's go back to that GitHub example. Right now I can be in my little code editor, I can have my chat, am I asking the right questions, why am I having to tell it to do all these things? That's a ton of friction. But if I highlight code, it automatically pre-populates some questions that I am most likely to ask about that. And then it's just a click of a button to go through and actually ask that question. So I might highlight code and some of the samples might give me, why are you choosing this particular formula? Or how did you shape this formula? It anticipates that for me. It's like a conversation. If I was talking to you and we were going back and forth on some issue, you're already anticipating in your mind what I might be asking next so that you're ready to respond when that comes up. And we're starting to see these systems begin to adapt in that way. Along that is the personalization. So if I always take action B after action A, the system should learn that and it should anticipate that. Now the trade-off is you don't want to assume that that is the action someone wants to take. And so we get these kind of progressive disclosures and systems that learn with you and grow with you over time until you get to that hyper personalized experience. And we are seeing that pop up in both the way to sort of save your tone of voice, save your prompt. Chat GPT is apparently going to be rolling out a feature pretty soon where they will actually show you what it knows about you. So as the system gets to know you better, you'll be able to go, oh, interesting. It knows that I have two kids and it knows that I live in the Southwest and it knows that I like tacos. And if I ask it, what should I have for dinner tonight? It might say, hey, have you thought about tacos? Because it already knows this about me. There's some really interesting indie tools that are playing around with this at an extreme level. One is a tool, kin. So I don't know that it's publicly available yet. But this tool, it gets to know you personally. And it actually builds a map of what it knows about you that you can look at and say, oh, interesting. I want to actually delete this. I don't want you to focus on this aspect of me as we go forward. That's in the consumer space. But then you apply that to like the workspace. If you know my role, you know my level, you know the team I'm on, you know what projects my team are working on, you know the information that's coming in about that. And you know my business goals for the month, third for the quarter, you're going to be able to anticipate what I need to do on a Monday when I log into my CRM. And I look at somebody's name and I have a call with them on Thursday and already anticipate what actions I might want to take. We're just not there yet, but we're going to be there faster than we know it. What you're describing sounds amazing, but it also sounds less and less like UX and more and more like service design. And one of the things historically that's happened is when new tools come online, basically the talent needs to move up to another tier of problem type. We've seen that in every industry, you know even in research, right? Like a lot of the activities are simpler. They can quantify, commoditize. And design, we've been seeing that design systems, design patterns have eliminated the need to do the nitty gritty, the buttons, et cetera. Now, if the problems are moving further up and design is moving to more service design, how do we deal with the fact that we probably won't need as many designers? I know a lot of people listening who want to protect their jobs and I want them to too. So how do they exist in this future? That's a heavy topic. And we're certainly on the tail end of a hype cycle around design that we should be honest with ourselves, that the idea we had in our mind of what design was going to be five years ago might not be exactly that, but it will be valuable. And if anything, it'll be more important and more critical to what the companies that are hiring us or what we're trying to build, what we're trying to do. I want to actually stick a pin in one thought here and come back to it, which is the difference between junior designers and people who are later in their career. Because I do think that that's going to be an area where we do see a bunch of friction. But for a mid-level career, somebody who's maybe been in a design for three to five years or a few years after that, our jobs aren't going to go away. And many of us are still going to be working on these products. Remember, there are these core AI products, but there's also AI that exists within this larger system. And that's still going to be a very dominant part of how we interact with technology. There is an aspect of which doubling down on our craft and tying our craft to strategy and understanding the ways that AI will adjust the strategy and the incentives and the systems that we're designing within is how we can create some security in our jobs. The actual day to day of our work and the core competencies of our work is at least in the short term, next five, six, seven years or so, not going to change dramatically, I think. I hope. But I think another way-- and I get excited about this. Another way of thinking about this is what we've considered the product is maybe not the whole product. Maybe the product, the experience or the interface of just this tool that we can interact with on our phone or on our computer doesn't have the boundaries that we have respected as these boundaries of what we do in the same way. So like you talk about service design, and I think that's a great parallel. But why do we think about service design as separate from product design? Because for a customer, anytime I'm interacting with you as a company, anytime I'm trying to get something done, Whether. Here I'm talking to a human or a computer, whether I am interacting over the phone or interacting at my screen, whether I'm talking to somebody who's on my account team or I'm interacting with something that someone I've never met has created for me. It's still my experience interacting with your brand and using the services and products you produce to achieve some outcome. And that's what I am starting to see and have been seeing for quite some time in this full experience thinking about the customer experience, the user experience, the product experience. And as designers, we have an opportunity to say, are we simply here because we're designing screens or are we here because we're designing for outcomes because we're designing for people and what they're trying to achieve? And if it's the latter and some people love just creating screens and digital experiences and I don't think that fully goes away. But for many of us, particularly those of us in the UX and the product design space, I think this is actually a very liberating moment for us to say, how can we have that seed of the table by maybe remembering the table is a lot bigger than we've been treating it as? And that goes beyond just the customer experience, shaping employee experiences, designing for internal tools, designing for what a lifecycle of a customer looks like over time. How does my interaction with this change as the system gets to know me and my team and as the data improves and as our use case improves? These are all product problems that designers should be at the table to help to resolve. And I mean, I'm personally really excited about what that means for our discipline. There's this term that you had mentioned when you were talking before, which was that you had mentioned that the idea of language as technology. And I think that that is super fascinating on a lot of levels, one of which being that if we think about what design in UX has been right now, it's been a lot of website app, things like that. Through these, you know, through these GBT's, the way that that information gets disseminated will become so different. And I think that's really incredibly fascinating. And so what do you see the role of the designer when it comes to these just fundamental shifts in the way that language is used and broken down and disseminated? So I'll start by suggesting that anybody who has not read the book, Conversational Design by Erica Hall should go and pick it up this afternoon. Because what Erica does is she presents this idea that an interface is not the same thing as a screen. An interface is how you interact with some information that is foreign to you. That's not coming from you. And if we allow ourselves that mindset to answer your question, it's incredibly empowering. Again, as designers to think about how we shape these experiences and shape these outcomes that we're working on. And so when I say language as a technology, a technology is something that allows me to do something more than my innate capabilities. So I cannot push a nail into a wall with my palm. I need a hammer. The hammer becomes a tool that allows me to put this nail in the wall. But a hammer is a really terrible tool if I'm trying to chop potatoes, right? So our tools need to meet our needs. And language is similar. The ways that we talk about what you're trying to achieve and how we interpret that request and how we respond the tone, the amount of information, the technical context of the information, the supportive information to that request. That's going to have an outsize impact on how well my response allows you to actually achieve some outcomes, some goal. And so the ways that we have used this so far as design is to talk about things like a pattern language, use common language for components. So when I say, hey, I'd like you to build a primary button that has a loading state that lags for our variable of short, you know, dot, dot, dot, I could say that to an engineer. And as long as we have a common language and a common system, they can know what I'm saying without me needing to show them. So language becomes a technology of communicating intent. When we think about a user who says, oh my gosh, I'm so frustrated, I have this customer service problem, I need to get solved. I don't know really how to describe it. I don't know who I should be talking to. And I don't necessarily know what better looks like. I know I'm in pain and I know I need help. So our ability to say, this is the baseline information we think we need in order to get you to the right person. This is how we frame the options you have to guide us towards your ability to tell us what you're seeing and what you're experiencing. This is the order of suggestions that we give you based off of which we think are most relevant to actually solving your problem. This is the tone of voice. This is how technical our response is. If I'm talking to an engineer, I'm going to be far more technical than if I'm talking to my mom. Those are design problems. They use language to help us infer and communicate and respond in a way that actually moves somebody forward in trying to achieve some goal. But I think the tenants of conversational design are really critical now, not just because this chatbot interfaces, this dominant interface in this AI world, but more importantly because people now can interact with data and communicate with data directly. They're no longer relying on buttons and forms and screens and clicks to get to where they want to go. We have to get a lot more comfortable with the psychology behind how people interact with data, what people are trying to do, and then how we can guide them towards an output that we don't even necessarily know they're looking for. That's such an amazing point that basically we've moved from interfaces, is basically controls that a user has to interact with rather than using their own voice to one where they actually can use language. That constantly points me back to all of us that work in design were enamored with the movie horror. It pointed to a future of AI and love and agents, but the tabriolid check, it's been over 10 years since Siri came out and bots have been here forever and they failed. It makes me wonder if text-based conversational interfaces are the right interface for B to B and E to E, but that voice will be where the average person will finally get an entry point into this. First I think that Siri is the most disappointing AI tool I've ever interacted with and I could go on to that for hours. It does represent this kind of trough of expectations that we've been in with these tools. I think maybe the question, if I were to reframe it, is less around one specific modality as the right solution and more around the idea that the multi-modality is going to be the best solution because it depends on the situation you're in. If I am driving my car on the way to a meeting, then text-based communication is a terrible option for me. It's inaccessible to me. Another great book that people should read is Miss Match by Cat Holmes at I think she's at IBM. But she and IBM's inclusive principles go into this in great depth. The way we think about accessibility extend to our circumstances and our context. If I am in a foreign country and I can't speak the language, then text-based communication is maybe not the right solution for me because I cannot ask my question in the language of the person that I need to communicate with. In this case, written text that can be quickly translated or even a picture might be the better alternative. So mapping the context that people are going to be interacting with products and information to the right modality to help them solve their problem with a degree of contextual awareness, personalization and preferences, I think is really the solution we need to go to. I'm actually really excited about what's happening in this device world. I'm not on the hype train. I don't necessarily think every single device hitting the market is like, oh my gosh, how did we live without this until today? But when I think about the ways that immersive technology can add added context to our lives, put aside the ethical considerations because there are real ones in terms of privacy and so on. If I'm wearing glasses and I'm in a foreign country and I'm at a train station and I can quickly glance around and see, oh, I actually now know which platform I am on. I know what train I need to go to and I know what time it's leaving. Then that is now turning me into a superhero. I have powers I didn't have before. Now if I need to go and ask a conductor, excuse me, I'm looking for a different train. My glasses don't help me. I can't speak the language that I need to communicate it in. But maybe I can, you know, in two, you know, okay, she's in Japan. So the language you need to translate to is Japan and she's in a train station. I can see that from these visual cues. So I know who she's likely to talk to. And maybe I also have some information saved to her phone that she has a train to catch at 2 p.m. Now when I quickly type in, hey, can you help the conductor tell me if I can catch a later train, this device has all of this information, contextual information. And going back to the earlier question of like, how do we make it so I'm not having to do all the work? This is now taken all of this work I would need to do and turn it into a single request that can be as small as a suggested button that I could press and then have the request pop out onto my phone. So I think we're moving. I mean, we're so early or we're moving into a world where this multi-modality, this contextual data. To me, trusting the system, right, like there's a lot of work that needs to happen there. It's incredibly empowering for the user. And as designers, it's our job to sort through which of these signals matter the most in these situations so that it's useful and not overwhelming. What are the signals that you're seeing? Because I remember back in the late 2000s when augmented reality became really. popularize. I was working in an ad agency at the time and it was like, oh my gosh, we need to start using AR in every campaign. There's all of these applications. And that was also around the time that Google Translate was being able to start using Photo Translate. So you didn't have to take a photo, just translate in real time. Why all of a sudden are we now feeling like we're getting the signal where we're in a position that that has changed? It really comes down to the processor technology of these LLMs and the ability to take millions and millions and millions of tokens and translate them into normal speech. Because even if the computer is never giving that speech back to me, it allows me to convey some ask or some intent in my own language with my own terminology with the way I think of the way I talk and it can intuit my needs. And that is the keystone. That's what we've been missing. And if you look at the progress of these models and how quickly they're improving, it's just, it's mind-boggling. It's astounding. Like from a pure technologist perspective, it is absolutely miraculous. And it does allow us to take these ideas and these technologies and these experiences that we have been simmering for years. And all of a sudden, Ching, it works. All of a sudden, it works in ways we didn't even realize. And that's where I suggest that people should interact with these tools as much as possible. I am constantly trying to trick them. I'm trying to produce hallucinations. I'm trying to understand how it got to some answer. Understanding what happens as a conversation goes on for more than a few interactions. What happens when you give it really short responses versus really long responses or long questions versus short questions? What happens when you give it a suggestion to say, "Hey, this is what I'm looking for, but here's what good looks like. Now go do it for these five other experiences. You get different outcomes." And I'm very mindful not to personify these machines because it's not the movie her and they are devices of corporations and we should be very aware of this. And yet at the same time, it is just, it's mind-boggling to interact with them and have it feel so natural. And when you think about what that means for product design is service design. What that means for augmenting the human experience, it fundamentally changes the game of what is possible and the ways technology can serve us. You know, we've been talking about these like B2B spaces and these consumer spaces, but I'm thinking about healthcare. I'm thinking about a patient who knows that they are at the early stages of a debilitating disease. What information would you store so that you can continue to communicate with your loved ones and have them hear your requests in your tone? What requests for your life would you have that you want to make sure are stored and can be communicated in a way that is authentic to the way you frame them in the first place? How might you want to get information if you've lost access to some bodily functions or to the ways that you can cognitively compute things so that you can still receive them in a way that warms your heart and makes you feel connected? There are so many use cases that we are only scratching the surface of that go far beyond the big tech go and get people to click buttons so I can give them ads and make a lot of money use cases that I think are really what drives my excitement about this. And if we are able to benefit from a network and in our day-to-day lives as well in meaningful ways, I think that that's hopefully, you know, generally a net positive. I'm super excited about the capabilities and possibilities of what the tech can do today. But at the same time, I refer back to some examples that I've had in my own life where earlier this year was hired by a weather company where they're basically, oh, we need to reinvent the category. We need to reimagine what's possible with the technology. And this is sort of question of businesses everywhere asking. They're like, this is our chance to get a competitive edge. This is how we grow our market share. This is how we're going to meet our revenue targets. But what I quickly found in this project is once you start dealing with the internal teams and with consumers themselves, they're resistant to change. You put something new in from them, they don't know what to do with it. And a big part of it is the capabilities are theoretical. And when I think back about this, I always go back to this story from Brian Chesky from Airbnb where his analogy was, you know, you think of a five out of five review and it's very obvious. But the best way to get your team to push that idea forward is to think out a 10 out of five review. You know, think about what that super situational outcome would be. But the average person struggles with that. And I think this is one of those things is going to hold businesses back because either you have leadership that take moon shots without much human-centered design focus or practice, or you take a very incremental approach and you'll find that your consumers are very resistant to change. Have you encountered this? Oh gosh, yes. And on many other places too, I think that it's just common reality for those of us working in design and in technology. This goes back to the question of how does design as a practice evolve? What does it mean for our jobs? As designers, whether we've chosen this or received this, we've been somewhat guarded from the people that we're designing for. You know, even designers I know and on the teams I've managed to go to great lengths to interact with customers. You're often interacting with the customer who bought your software and not their end employee, which is their user, the actual person who's using the software day to day. And when it comes to these types of emerging technologies, that means we're designing in the dark. And so we don't want to suggest a moon shot that is not something that we can achieve. It undermines our credibility and it makes it harder to actually deliver value. But we also don't want to shortchange what we could be doing and understanding the ways that this technology or system or, you know, service that we're working on can really help people. So I think the solution here is to talk to people and this is a business change. It's not something that an individual can implement. But one thing I've found in my career is if you go and talk to somebody who talks to other people, you'd be surprised by how quickly they want to connect you with them. Marketing teams who are out in the market talking to people who are prospects. If you go in and just have a conversation, not trying to test your product, not showing prototypes, maybe there's some privacy issues or security issues that you have just talking to them. Tell me about your problem. Tell me how you've tried to solve this before. Why didn't that work? What else have you tried? Oh, interesting. And then you go back to your team and you say, Hey, I know how some actual data that I can use to assess whether this thing that we're exploring is likely to actually meet their needs. And then start to beta test it, play with the model, get to a prototype as fast as possible with real data using something like GPT. We're going to see tools pop here really fast. We are going to see, I mean, Figma, I love Figma. You guys are great. Love your product. It's not the end solution for what we're building. They are going to evolve, but also new tools are going to show up in the market that let you interact with these LLMs and with the data in real time and design the interfaces around them so that you can test them, pressure test them and vet your thinking. So talk to people, make that a cycle. Talk to people come back to your team, come up with more questions, go and talk to more people, and then take those learnings and put them into the product design. Get your, your thinking, your bets, your assumptions, your constraints out in the sun as fast as possible. And this learning cycle is now going to speed up. I think that's a good thing. Our tools will catch up. That's a very good thing. But in the meantime, the fastest way to break through institutional debt is to just go talk to the people you're building for and the people paying you money. Because if you can bring that insight back that actually helps to break through something, it doesn't matter who's at the table, the business will follow. Emily, I think that's such an amazing point and input. And I think from my own personal experience of going out and talking with end users, especially when it comes to utilizing new technology that their business is sort of forcing upon them is this constant idea of underwhelmingness or not actually delivering on the promise of what it is. And so I think for me personally, I'm really excited to see more of that adaptive technology space. Thanks for having me. This has been a ton of fun. Thank you for listening to the Design of AI Podcast. The show is hosted by Britney Hops and RPTRIP Figueroa. Subscribe on Spotify, YouTube or Apple to get our latest episodes. We speak to leaders at the forefront of AI to learn how great AI products are designed and how they're transforming industries. To contact us, visit our website designof.ai