#280: Dashboards Must Die! Long Live Dashboards! with Andy Cotgreave

[MUSIC] Welcome to the Analytics Power Hour. Analytics topics covered conversationally and sometimes with explicit language. Hi everybody, welcome to the Analytics Power Hour. This is episode 280. Friends, analysts and dashboard builders, lend me your eyes. I come to speak of dashboards, not to praise them. The charts we build may live beyond our hands, yet meanings often tuned within the data. Dashboards are loved, executives who click, marketers who scroll, and analysts who sigh. But comprehension rarely walks these halls and insights haunts the tool tips like a ghost. Oh, judgment, thou has fled to decks of slides and reasons lost. My heart is with the charts, and I must pause until it come again. Okay, well, as the bar might say, "Oh shit, here we go again." We're talking about dashboards. The staple consumable of our little industry, but always a little bit hard to find. And now with AI, are they maybe finished for good? I don't know, we've got a lot to talk about. So let me introduce my co-hosts, Julie Hoyer. You've built the dashboard of 500, I would guess. Oh yeah. Well first. And Tim Wilson, I know this is your first time talking about dashboards, but I think you might be excited. I should have checked what we were talking about before we-- Yeah, yeah. Well, we'll try to keep you up to speed because I know you probably don't have too many opinions about it. And I'm Michael Howling. I'm super excited for our guest today. Andy Kotkrieve is the co-author of the big book of dashboards and former technical evangelist at Tableau. He is the co-host of Chart Chat and writes the How to Speak Data newsletter. He's got more than 15 years of experience in data visualization and business intelligence. First honing his skills and analysts at the University of Oxford, he has inspired and trained thousands of people with technical advice and ideas on how to identify trends in visual analytics and develop their own data discovery skills. And finally, Andy has a new book out this month, dashboards that deliver, welcome to the show Andy. Hello everybody, it is the light to be here. I'm looking forward to a lot of banter and possibly arguments. I like your style. That's not what our show is about or it's just from very far. No, it's awesome to have you, Andy. And I think, you know, obviously, you've got a new book about dashboards that's just coming out in like a week or so. I think maybe you're taking the line that maybe dashboards aren't dead, but let's jump into it. Yeah, let's do it. I'm very excited. If I was listening to one of your recent episodes about AI and I don't know your voices by too well, but I don't know Michael or Tim, one of you said, "that's more or less shit." I think that was the line you said at the AI. I have so much to say. That was my fault. That was my fault. That was my fault. That was my mistake. I'm looking forward to taking that one. Can we start by actually defining a dashboard? Because I feel like that is actually one of those things where people get into raging debates and it's actually the underlying debate is what is a dashboard? In the big book of dashboards, we defined and in dashboards deliver, we define a dashboard as a visual display of data that is used to monitor and/or facilitate understanding. 15 words, when we wrote the book of the dashboards, the definition got longer and longer and longer as we were trying to capture all caveats. In the end, we're like, it's such a vague term. We just reduced it to virtually nothing. Stephen Fee chose to write about 1,500 word blockposts, destroying our definition, even questioning in his blockpost whether myself, Stephen GF, the co-authors of that book had the authority to teach about dashboards. Thank you, Steve. He had very specific things, but things like must fit on a single screen, must be interactive, but it's so easy to demonstrate that that is patently wrong. I wrote a whole chapter in the new book about this challenge of what a definition of a dashboard is. In the end, what is a dashboard? It's a piece of word or leather that used to sit on a stage coach between the horse and the driver to stop water dashing onto the driver's legs. People made cars and they stuck that kind of thing to the front of the car and they put gauges on it and call it a dashboard. It's a term that's borrowed from something else in etymology and is a catch all for somebody's thing. We have a really loose definition of what a dashboard is, but if you're building a chart or a collection of charts or even a collection or even a single text table, very controversially, we can make an argument that it is a dashboard, even though many people might say it isn't, we'll be fine with that. But that is, you've said it's displaying information and promoting understanding. That does stop short, I think, of the kind of interface to an underlying data system for deep exploration. I mean, I'll pick on Adobe a little bit, but they've got analysis workspace and they will send, which is, you can build your building visual displays of information and then you're sharing those. So I think in some cases, I think of anything as a challenge for their product, it's built more to be kind of an interactive digging and drilling and ad hoc, but they also say you can use it to create a workspace for the display and understanding of data to end users, which I think it struggles at because that's not kind of what its core basis is. So do you see A line there between displaying and promoting understanding and exploring shower here deeply? Absolutely. So I mean, you get into that question to be also touches upon the the audiences in that there are data analysts and engineers and there are business people or non-data analysts, not necessarily business people, just people who want to update their analysts, right? As a data analyst, then we have access and use the systems to visually explore data and just ask and answer questions and chase insights to find them. But much as I would love everyone in the world to be as skilled as all of us in exploring data, I've discovered that's not a real option. Is there some people are freaked out by data so they don't have time to learn data? So a dashboard is a window on a pre-designed pre-defined finite set of questions to answer, to allow those people to explore in a relatively narrow way, the answers to those pre-defined questions. I feel like that's some of the copy that was in the larger, the longer definition that they got shortened. Oh yeah, yeah, yeah, yeah. So the definition is short vague and then we've written two books about what that actually means. So then within that broader definition of dashboard, I'm glad that Tim that you had us touch on the difference between what's a dashboard and where is maybe like an analyst or an engineer like digging in because I think that's really helpful. My other question then is within bucket of dashboards, do you talk about them Andy as like different categories of dashboards depending on the audience or like you said, is it a table of text or does it have a lot of visualizations? Is it a page long? Is it being put on a single screen or it's a whole Adobe workspace with lots of panels? Are there other categories for format? Are there categories for audience? Are there categories for any other characteristic? So we don't formally create that categorization in our book. In our book we referred to Nick Debarat who's written practical dashboards. He sort of inherited Stephen Fuse teaching course and then has built it into his own. He's brilliant. He created taxonomy of dashboards. I think it's 13 different types of dashboards. Is it an infographic? Is it used for monitoring ongoing problems? Is it used to monitor a short term project? So he's got a really good taxonomy. We chose not to go down that route but that's a really useful reference point if people want it for types of dashboards and we wholly endorse Nick's approach. Another aspect of the question I was around form factor. Should it be a single page? Should it have interactivity? Should it have multiple views? It depends on the vast infinity of user requirements, what they need, how they're going to consume it, how much time they're going to consume it, and again. So we explore that adapt in the book but we don't categorize it. Just because I'm thinking Tim, when we talk about dashboards a lot, I know Tim has the point of view and I generally agree but one of the best uses of a dashboard we talk about for performance measurement. And so Tim, I know when you talk about that, do you believe that's the only way a dashboard should be used? Because I think that's where we end up getting into debates a little bit. Should it only be used for performance measurement where it's very clear concise, it's against a target, it's a quick snapshot. Because does it make you You're a little bit more. like there's lots of categories and uses of dashboards. Are you okay with that? - I think I take a simplistic view and I say this is, I just live in that one category. And I follow Nick, he is brilliant. He's engaging a lot on LinkedIn. So I will further endorse like Andy, like Steven, multiple, everybody who's gonna get mentioned here are really, really sharp. So I see that as kind of being a narrow view that if we use them for just measuring performance, monitoring how it's going, having targets, it really works. I think some of the stuff that Nick and Andy have done is saying, yeah, there are broader uses. I think I've had an overly simplistic view and they make a, it's been a lot more time thinking about it and have a much more nuanced view. Although I do come to questions like data storytelling. I'll see people, none of the people that I've just mentioned, but we'll talk about how a dashboard should tell a story and it's data storytelling and that's one of the ones that just cringes, it makes me cringe and wanna run back to my, no, no, no, here's this one specific area that I think dashboards can be used. Make, I'll put it this way. I think it's very, very, I can so readily defend a dashboard that is generally one screen with some caveats for monitoring the performance of a project or an initiative of a campaign and doing that well is to me easily, easily defensible. When people head off into other areas and start making broad proclamations, they're kind of setting up things that they can then say, these are, it's bad for this. And I'm like, well, yeah, but you're talking about something that maybe isn't really what should be a dashboard. It's that time of year. Half your teams on vacation, your inbox is quiet and maybe just maybe you're thinking about taking a little time off yourself. Well, with 5TRA and you actually can, they're fully managed data integration platform keeps your analytics humming even when you're out of office. I mean, you could be off hiking, surfing, just sleeping in. 5TRA and is syncing all your sales, marketing, and product data into your warehouse accurately, securely and always on. It's like a reliable teammate that never takes PTO. So relax, unplug, stop worrying about late night slack alerts. Your data's not just covered, it's thriving. That's the 5TRA and difference. Check them out at 5TRA and dot com slash APH. See interactive demos or start a 14 day free trial. That's F-I-V-E-T-R-A-N dot com slash APH. Let's get back to the show. I'm desperate to come back to storytelling, but I want to talk about that. Your definition of what some of the things you've explained that to it, right? One of the things that really frustrates me is I've seen lots of organizations, they go, we need to build a dashboard and the project, or we need to do B-I, right? And then the project is sponsored by the CEO or some of those C-Suite person. And so they define and build a system monitoring process. So let's say it's saved. So they are tracking sales by quarter to target and they can drill down to region and product types, right? Brilliant, that is a high level sales aggregated dashboard. Fitting in that criteria you've set. But the problem with those kind of project is they are aggregated and are useless for the people, for the minions, the men or more people who are actually trying to do granular work at the more public space. So a dashboard we had at Tableau, which was hugely used, was I mean, we called it the Hughes-Hop dashboard and it was used by the sales organization. And it would show individual leads or individual people in the CRM and the activity they've been doing over a timeline on the website or the activity related to Tableau they've been doing. So you could see a dot plot on one row was one person. You could be like, visited the website, visited the website, did some training, visited the website, logged on, right, downloaded. And that dashboard allowed the person to look at their opportunity list and be like, "Ah, Michael has been doing a whole bunch of, "what's all the training videos?" So I'm gonna call Michael up and say, "Hey Michael, you've been doing some training "and you should be talking about Tableau." And that is a dashboard that allows people to monitor desegregated data. And it is a fundamental use of a dashboard. It's like it's the desegregated data visualized in a single screen or in multiple screens in that particular example. But that's useless for the C chief revenue officer 'cause he doesn't get in the weeds with that, right? And so you have that spectrum of dashboards that need to be included inside any data strategy. And I get really frustrated when it's like, "Hey, we've built a sales tracking dashboard. "It's off you go, I can't have that exact." And they're like, "What can I do with that? "Nothing." But I think you nailed it when it's like, there is this idea, we have all this data. And dashboard is kind of the, so therefore if we just build the right dashboards, then that is how as an organization we become data driven. So I think, I mean, I'm feeling I'm trying to square the circle. I do tend to say a performance measurement dashboard, which is what maybe could be considered a scorecard, but I've got some kind of beef with scorecards. I think what you just described, I don't think I'm in the habit of saying, this is a dashboard. I absolutely think building something that is not aggregated that is specific, that is part of a process. What you just described, I would say, that's, there is an operational process by which leads are being monitored. And somebody needs, when you define it, they need information displayed that they understand so that they can follow their individual lead follow-up process, them as an individual having multiple leads. I haven't been in the habit of putting the label dashboard on it, but I think I may start because it makes sense. I'm loving this. Don't do it, Julie. I have a little bit of a question for the group then. It's interesting because I do feel like what happens to Tim's point and the pain I felt a lot of times with clients is they want a dashboard that has big lofty promises and that it's, they feel like if I just surface the data for them that all the interpretation and actual synthesis of what those individual data points mean will just happen and they'll know what to do. And obviously we know that's false. So it's interesting, how much interpretation should a dashboard do for you? In practice, a lot of times when I talk to stakeholders about the dashboard, I have ended up making documents where we actually outline, here's the outcome you're going for. Sometimes Tim, I can get them to a performance measurement type of the dashboard sometimes I can't, but at least I can take that and map that to say like, these are the things you're trying to achieve or you're trying to understand. I can turn that into a list of very concise, specific business questions that actually have answers that are impactful for them to understand and then try to get them to something that is usable. And in the dashboard, then I usually put headers of the question, the data point is answering, but that's baking in a lot, again, of interpretation. So how much interpretation it has to come from an analyst, has to be baked into a dashboard for it to be a good dashboard. That's kind of a-- - I'm always a pleased question, but that's where my brain is going. - I'm so proud of that question you asked. Do we always-- - Okay, good, I was worried I was like, Tim might come to people. - No, no, it's just all over the place and, you know, I'm like-- - It's not just me. - So Tim, so first, I think your statement belies the reason we don't really care about definitions because some people describe what we call as reports, they might be applications, they might be dashboards. It's like, what are you trying to achieve with this data and what the customers need? So that, or users need. And, you know, so I'm just not interested in, you know, I'm very interested in selling the fact that the semantics of what we're doing isn't important. It's the end goal that is. So then Julie, you know, coming on to your question, I think it's, which I've now just figured out, now I've answered tips question, I've just forgotten. - It's too much. - Oh, it's your question. - I had the answer. Sorry, get started. - Who does the interpretation of that? - Right, and then this is brutally hard, right? Because I, all your users, at the same level of data literacy, right? Or data fluency because that, how that will change, you know, the way you design these, the designs some of that. So what we talk about in dashboards, the deliver, we've got this whole framework. And, you know, it's about application design, agile development, it's really about thinking about user stories, right? So go to your stakeholders, work out that, work out, you know, as a account executive, I need to see who's active on my website in order to make a high chance, or call it as a high chance of turning into a sale. Those user stories help define the answer to the question. And there isn't a single answer to the question, Julie. I wish that was, I have no fantasy glenstein. - But so, as Julie, you described it, and I'm flashing back to my early days at Search Discovery and there was a, I think it was a monthly report that was getting built entirely in what was Google Data Studio at the time. It was basically a presentation being built inside a BI platform, kind of a, free and it was really and it was every month it was kind of a what's the story we're going to tell and let's build it with these multiple tabs and I was that was a lot of work to build something that was kind of a one-time delivery and when you're saying what question is being answered I feel like and I think this might get Andy to wear your your user stories is this a question that I'm going to ask every day or every week or on a recurring basis in which case I need a dashboard is a good way to say here is the answer to that question or is this a question that I'm asking one time like hey was there any impact from that you know big traffic accident that happened wherever or some you know the airline strike what was the impact answering that question should be kind of a one time and maybe it's a different type of dashboard I think I've just there are times where I've seen the somebody asked this we might as well build it so it's automated and they can just continually get the answer to that question but they're not going to keep asking that question now we've just created another bit of jumbled clutter that they can go to and have to to wait through to get the answer and that kind of comes back to what we were talking about two of like is this an actual is that something that should live in an actual dashboard that a business non-analyst user is going and looking at and having to look at 10 different data points to come to the conclusion of the one time answer that the analyst did or it are we saying like that's the type of question and the type of work data visualizations whatever that's best left to the analyst and the engineers because they had to go into the interactive tool dig and find it and it's not easily repeatable in some single visual that should live in a dashboard I highly agree and you know I think anybody building out data strategies needs to realize the dashboards are just one tool in a in a toolbox right and I think a great example of this is you know I've done work with a cabinet office which to do you know the arm of the UK government that works under the prime minister and you know they have dashboard teams but they also have a team that builds narrative data narratives so when there are big cabinet meetings or cobra are sort of emergency crisis meetings they don't build dashboards they build out narratives and they will be charged with explanation charts with explanation because they've recognized that in that use case and they're every business has use cases like this a dashboard isn't the right thing it's a narrative some of the things happen in the White House during the COVID years they were dashboards and but also narrative builders in the end the narrative builders sort of were more favored by the administration in the White House at that point and I think it's really important you know and I'm sure we'll go on to AI but AI brings another paradigm of exploring data so dashboards just exist within this collection of options to which the hard job is trying to find the right the right one for the right purpose so is that is that distinction of narrative versus the dashboard is that getting to the back to the storytelling dashboards yeah oh yeah absolutely I think I think generally I mean Steve so co-authors of the book is Steve Waxler, Jeff Schaefer and Amanda McCulloch Steve Waxler chapter in dashboards that deliver about what I think is it dashboards of first story finding not storytelling and I think that's really important distinction you know about 10 years ago data storytelling was the hype in the industry and dashboards can tell data stories it's like well dashboards can answer four questions maybe four maybe five questions but probably four questions is the most and you might be able to design a grid flow that goes over you detail I was even filtered details on demand like bench items thing but it's not really a story is it you know so find the insights and then then there's a whole skill and paradigm of having to then tell a story which we don't address in this book but I might in future books I can't believe you're actually already thinking about the next book when the first one I'm not going to talk to you I'm barely I'm sorry I must not have chewed you up and spit you out like I just like that you've got things to say so it you know before you you touched on AI and I want to spend a good amount of time on that but before jumping off we touched on sort of this process you define in the book of how to build a dashboard I would love to just actually go through that in the more of a step by step fashion because we we touched on it but actually feel like that's actually such a helpful thing that's right there in the book but I want to give people kind of a little taste of that do you mind just kind of stepping through yeah so the the framework is largely the mastermind of Amanda McCulloch and people listen as mine others she was exact director at DataVis society and just an exceptional talent in process and so the framework is largely her brainchild and and it's just like basically what happens when you're building a dashboard first of all there is a spark that spark can come from many different places but at some point somebody goes we need a data asset and that is going to be a dashboard so we talk about when it when it when it it isn't a dashboard after that you've got to go and talk to users so that is about discovering discovery and prototyping we took an inspiration there from the double diamond design approach to go and talk to loads of users get their requirements and then prototype with wire framing or with data oh and there was a debate we had internally but all the way all the way through this framework talk to your users you do your users are part of the development team so you've got your prototype and then it's the next stage of the framework is development and that's a slightly circular because again that's very iterative then you go to user acceptance testing release and adoption how did you train you users and then before you loop back to the start and spark the time you come to redevelop or rehash the dashboard is do you still need it you know can you deprecate it because we don't deprecate enough dashboards and many many dashboards should die I'm not here to say they should all live but that's the framework spark prototyping release no development release maintenance and yeah we were just prototyping prototyping with data versus without data what was that what was that what should you should you wireframe a dashboard in the same way you would wireframe an application a website I would say yes without data because as soon as it starts it is so hard to start putting either dummy data right or yeah so I mostly agree but once I got sent once to a client site and Rotterdam and this was a tabloid sales opportunity that they've worked with Jan Willem Tulp who was a really great data designer and they've prototyped this dashboard and they want you to go and build it in tabloid I might bring it so fleet to Rotterdam and they gave me this beautiful sketch not with my companion dashboard of beautiful line charts and bar charts and sketch points with gorgeous and I built it in tabloid tabloid I say the American accent in front tabloid all right I built it in tabloid so I've 15 years working for this American company tabloid stick to the bridge back so anyway I built it and it looked like garbage and I was like why does it look like garbage and it's because when you prototype you draw a beautiful line chart right maybe a little and when you draw a scatter plot you go dot dot dot dot oh look at perfect regression and bar charts but then when you put real data in it your line charts go like yeah and then scatter plots show chaos and bar charts there's just one outlier so that I do not I why framing is amazing but when you why frame a static website or data application the interface doesn't change but the data drives the way your dashboard will look as well so in the end the custom wasn't very happy but they did learn that that data the dashboard that we're building wasn't actually brought goodies for the trends in that data so but that is getting to such this fundamental how people believe that their business that the data is cleaner things are their trends there are patterns I mean I think that gets back to the that coming from the top down where we were earlier when somebody says we need to build a dashboard and do it's kind of to your point there's this assumption that if we just have the data visualized oh we're going to see the scatter plot with everything in that one outlier and that outlier is going to just magically tell us how to drive the business forward or you know you you name it and then it's like oh actually then it becomes the dashboard is a way for them just to understand that when users actually look at it the first time they look at it they say I've never been able to see this data before this is great and the third time they look at it they're like this is all just looks kind of like noise to me it's what I asked for but I'm not getting anything new from it which I think goes back to your user story. The other thing I was going to ask about two in the the data and design and like do you um design it with data or not when you're prototyping my brain actually jump to making sure that what they assume is possible with their data is because a lot of times like I was saying we go through like what are your most crucial questions when are you trying to achieve and they're like and we know we have all xyz data you know all sitting there waiting for you to go build this for me and I'm like okay well we got to get really specific like what you're actually asking means we would need to be able to have you know this type of rate and this type of data point and this this type of relationship and and behold, it wasn't designed that way. So we either have to go take the time to redesign your data and work with the engineering team to get it to do what we're asking it to do and make sure that it can do that. And I think people are usually shocked by that. So when you were saying prototype it with data, my brain kind of went there, is that something that you commonly run into? Is it kind of baked into the process? Yeah, we talk about that in the book as well because I went to marketing for so long in a tablet with the amount of time. It's like, and we're doing a campaign. And we just can you just build as a dashboard for the campaign. So I build you with dashboard. All right. What do you think that's? A various question. Yeah, I'm a tablet pro. So you must see that's half a day. It's like it's going to take you seven days before the data got down. Right? So I you're dead right, you know, so we talk about that in the book. Yeah. And so so getting to the data early in the process, not at the book, still do you want for me, but getting to the data reveals the trends that are not in the data and whether you've got the dates are in the first place or not. So yeah, I know, I know we're going to get to AI, but I've got to ask you, I using using the dashboard as a way to drive some data collection. Like we're in this maybe I'm sure they're somewhere in the book on this of having the dashboard, but having placeholders saying, this date is not available. You have declared that it's important and you want it and you can't get it. It's not we can't not roll this out, but we're going to go ahead because this is a designed entity. We're going to put a box here and says, data not yet available. Do you have a I I will stay I like doing that because it's gives to me. I see it often as a constant reminder that this is something that you said was important that you said when you needed, but it's not yet available. But I could also be not thinking about something and that's a terrible idea. We don't have any examples in the book of empty boxes, but we do talk about that in the framework section that you know, don't let perfect be the enemy of published. Right? Okay. You know, I had one thought that you know, and you touched on this a little earlier, which is you like sometimes a dashboard needs to be deprecated and a lot of companies struggle with this right for various reasons and end up with hundreds, maybe even thousands of data artifacts, dashboards reports that are almost impossible to sift through and need to be maintained and basically burns out the data team because they're simply like carrying a massive carcass of old dashboards. Yeah. I'm trying to see it. It's really. I'm grizzly. I think your carcass along. Technical death for dashboards is carcass in this. Okay. Yeah. And to refine that into a question is, you know, in your experience and then all the work you've been doing over the years, like how do you advise teams how to go through and step through a process where they do that hygienic step of getting rid of the ones that don't work or refactoring or whatever. So I think that one of the things you can do, we talk about doing is right, right or the start. If you're building a new dashboard, put in an end date, right? So at what point will this be deprecated, right? You can define that. Right. So then you can define that as nobody's looking at it. But we'll come back to that. Or it might just have a final date because it's related to a campaign or it might be we will review the business owner owner will sit down and do a review in a year's time or something like that. So there are various aspects like that. At tabloid, we have we've got a great process because we're allowed to publish dashboards internally, but they all get deprecated after two years, if nobody or after two years, if nobody's looking at it, it just gets deleted. The user gets the builder, the owner gets warned and they can go and refresh it or just delete it, but we just we just flush them out, right? And there is a peril in deleting a dashboard that only one person looks at once a year because it could be that one person looking at that one dashboard once a year makes a $10 million decision, right? And it's hard to capture that. But if they are doing that and the dashboard is gone, they'll be like, what the hell is my goddamn dashboard? You're like, oh, now we realize that user had a use case. So one of a fact actually one of the dashboards in the book by Michael Gefferts, he works at an indie car team. His definition of dashboard success is the amount of support requests he gets because every support request is somebody engaging with the dashboard and frustrated. They can't do what they wanted to do with it, right? So that's a slice of side. Yeah, it's those kind of things you can build into the process early to duplicate a dashboard and then good governance monitor usage and just flushes the one flushes the sandbox published ones away. I will say there's the this if you ask somebody or using this dashboard, that's bad data because though, though, even if they aren't, they'll get a they're like, oh, but I should tell you their think I should be and I'm going to start so they'll say yes or they'll be like, well, I know somebody I had somebody else build it. I never really got that much use out of it, but I don't want to hurt their feelings and I don't want to be, I don't want them telling me that why did I even request it? So if there is not the like ability within the platform to monitor usage, I'm a fan of dropping like the big old red text at the top saying, this will be deleted. This will be on whatever this date and know when you put that up and give them plenty of warning, it doesn't necessarily solve the one year problem, but I had an analyst who worked for me years ago who just refused to she just because the number of people she sent the report out to was vast and included some very high senior people. She was like, this must be important because I am spending hours of my week compiling this and sending it out. I was like, but what are they using it for? She's like, I don't know, look at the size of this distribution list. And so I finally told her she needed to start including in the email, we're going to stop sending this. And I was like, you can you tell me who comes back? And then my point being, instead of trying to solve, think you're solving everything for everybody, there probably are some specific use cases in there. The people who are going to freak out, but they need one thing, once a quarter, you don't need to be sending everything weekly so that once a quarter, they can get it. So, but I love the planning and end date at the outside. Yeah, that was awesome. People will set it two years out and they'll be like, it's two years, two years will be there before they know it. Yeah, if they even release it in two years, I'd probably think that. Yeah. We keep dashboards like we keep gym memberships. Oh, I went to the gym all the way. Michael, you can go to my office and already been there. Hey, everybody go buy my book. There's a lot of for for data people, like there's dashboards are exceptional and useful, but there's a lot of pain buried in there as well for the years. All right, Julie, you were about to say something, I think. Push off our AI conversation one more question. Oh, boy. But I'm afraid I have to. Okay, go ahead. Because we were talking about like that one person that's making a $10 million decision on a dashboard. Have you seen, do you have thoughts on in practice? What are some of the best ways people should be sharing learnings that they're getting from dashboards like across a team? Or I think I had seen like you had part of the book talking about like there again, we've talked about different types of users, right? So they're looking at the data, asking questions with different lenses, different purposes that they need the answers for. Do you have any guidelines for people or seen it in practice done really well? How do you communicate what people on the same team are gaining from a dashboard or people on different teams are getting from a dashboard? And it's hard to say, will it ever get back to the analyst that's like, can I deprecate this thing or not? But I'm interested to hear. So I think one of the one thing I've learned over speaking to thousands of customers in my time at Tableau is that I refrain, I hear so often is we spend the first half of every meeting arguing about the data. We need a solution to that. And I like, fine, you know, that's true. But then I try and spin it. It's like, but what if what if I was actually a positive thing, right? What if you had a good robust data reporting system that told you average sales are going up, but mean and median sales are not doing the right thing or they could have that reliable data that they trusted in, so that they could spend half an hour arguing about the data. Because then they're having a data informed conversation rather than a data argument, right? So I think, you know, organizations that bring data into conversations because data, well, first off, data is never clean. There are so many, there are just so many problems in the data pipeline every step in the way that data is never perfect. So it is never going to be able to be the 100% informer of any decision. There is human intuition. There is just human politics and, you know, ego in making decisions, right? And so we shouldn't embrace that. And then it comes a little bit back into storytelling. I think one of one of our best European leaders is a tablo. He, he always brought in every all hands meeting for Europe and tablo. He would always come to each meeting with a brand new chart. I mean, he added a couple of people he would work on these every month. And I thought that was really good because if you say in a meeting every month than you or head of sales or you head of shows the same dashboard every month, it kind of looks the same every month. So three months into watching those meetings, you just start paying attention because it's like, oh, here's the big word James shows the dashboard. Oh, good God, it's the same stuff as boring. Whereas he would be like, okay, I've looked at the dashboard and I'm gonna visualize one insight I've taken from that. And so you'd be like, oh, here's James, telling it inside visually with data. So I don't have, there's no specific framework, answers to that, Julie, but that just for examples, what I've seen success and what I'm talking about, that data is a great stuff. - I mean, counterpoint, the arguing about the data or the bringing a new, I've seen cases where if the argument is about the messiness of the data, why do these numbers not match? And then everybody can get caught up on what is the right number or if somebody brings a new charge and is surprising, then people's tendency to say, well, did you exclude this? Did you do that? Did you do that? And in every case, there would be a, probably eventually some general consensus that yes, maybe returns should have been filtered out. Even if it doesn't material a change, what that person is showing. And then everybody kind of gets sent off swirling on trying to get to agreement on the definition. And there's a lot of work and discussion and deeper understanding of the data, not of the business. And everybody, there's a sense that, wow, we did something because we had an argument and we resolved it. And like all you've resolved is some definitional shit. You haven't necessarily actually resolved anything that's gonna move the business forward. I mean, I think that's where I struggle. - Sam, I'm gonna force the segue now because if humans can't do it, how the hell would a judge in AI have that context to know those things that even we can't work on? - Nice, well done. This man is a professional. Michael, he's coming for your job. - But yeah, so that's the latest type of course in every space, we all have this mandate to leverage AI, use AI, work at AI, do all this. So in the world of data and dashboards, yeah, what are you seeing in the space as it relates to AI? - Let's talk about thought spot. They were early natural language, approach to agreeing data and thought spot are great. And they, you know, that, that, - Oh, this was hilarious. - This is gonna be a lot of work. - And that, it's gonna show. - Genius marketing campaign, as dashboards are that, right? So come and do natural language. And it's a brilliant campaign, I love it. But you go to thought spot's homepage and watch the demo and says, you can ask any question. And when you've got a chart, you like, you save it to a live board, which is what we call dashboards. I'm like, oh, right. So, you're not gonna language. So on the one hand, Generative AI, LLAMs promise infinite answers to infinite possibilities that are related to data too. And that's fantastic. But I am, I'm lazy. I, if I go ask the same question every week or every month, I don't want to type that question every week. I just want to open my phone, go to say tablet pulse, for example, which is really good at this. And it's just like, oh, my metrics are going, the, my metrics are going down, put my phone down and move on, right? I don't want to type that every time. So, and thoughts want to recognize that with their live board. People want to go back to those things. So, on the one level, people still want at a glance ways to monitor and explore data to facilitate on the standard. Hey, that's a dashboard. Then the second aspect of AI, is this context thing we've just talked about. If we can't solve this as challenge, then Generative AI isn't gonna do that. There's a huge amount of efforts around semantic layers across the industry to try and define business data. We have been trying to do that for about 50 years. We haven't done it yet. So, I mean, there's more of a catalyst to get that shit done and get it done properly, but processes, humans are lazy and under pressure. And then using Generative AI as a data analyst, I mean, it's, when I do a data project, I'm up to the build a dashboard for this. So, okay, fine. Who's got the data? I've got to email that person and get them to give me the password. They've got password, it's a local file. So, they send me the copy of the Google sheet. They're not connected somehow to the core data systems. They don't actually match properly, but you can't really do the matching, going to say, so I have to manually connect the data. They don't have to find another data set, but after broadcaster message, 'cause I who owns this data, right? Hey, I can't get over that stuff. That's human processes, 'cause data is messy as anything, right? So, I'm, right, so so far in this answer to the question, I'd be down on Generative AI. I think it is unreliable for data analysis. It is unreliable. It doesn't understand business context, and it can't access the things away human can. It's not all useless like, there are some things it can do pretty well. But in all my experiences, trying to use AI to do data projects, I've been left wholly underwhelmed and really frustrated. And just thought, I could have just done this quicker myself. There's this idea, if you probe when somebody's, if they said, well, I need on the dashboard, I need to see sales training over time broken down by geography. And you'd say, well, okay, you could ask AI that, and maybe it could produce it, but aren't you gonna always wanna look at it? That's a lot more work for you to continue to ask to your first point, ask that question again and again. That's still not really what they want. So I feel like there's this nebulous, optimistic thing that know the question, I don't know if you can get to ask, I'm just gonna have an agent that is just going to tell me insights, and I'm not gonna have to ask it anything. That's sort of the path that's just trying to completely cut out the human, which I don't feel like is all that different from past iterations of, that's what digital was gonna do. All of a sudden, we're gonna have all of this data, and therefore, this will be just a shortcut. We'll be able to do one-to-one marketing, never worked out. So there's a little bit of everything old is new again. There's this weird thing where people think, this is the technology that if they really probe, they want it to just tell them what to do. And then actually they're like, well, no, just do it for me. It's like, what are you as a human doing? Do you have that low regard of yourself that you think that you should just be entirely replaced? It's somewhere thinking through, what do I wanna look at? Where does that data come from? What does it mean? There is value in the friction, even for non-analysts to understand the nuances of the business. What is a monthly active user? Like, what does that really mean? And that's not a data question. That is a, what do we in our business strategy think matters? And if we just were able to ask a chat, you know, just tell me, Mao's over, you know, tell me Mao anomalies. And I haven't actually taken the time to understand what that really means. So I, yeah, I'm similarly frustrated. You know, I've tried to exercise with custom and slide. If you want the AI to be able to ask and answer any question, you know, write down, and so let's say it's to do with active users. Write down as granular as you can, the list of every single thing that has to be accurately defined or anticipated in order for an AI to know where that it can look across loads of data sources. And that list is so long, right? And then, you know, you talk about this, digital was a thing. So I'm gonna tell you all my favorite quote, right? And I'm asking first, do you kind of agree with this? So millions of dollars are spent every year collecting data, but the assumption that having the data solves the problems being studied. Not yes, is that same reasonable? Millions of dollars being spent yearly collecting? Yes, right. Yeah. So second question. What year is that quote from? 1968. 1968, so Tim, Michael. I mean, I don't have no idea, but I'm with Tim that it's probably a way longer than we even think. Yeah, I was gonna say like, well, it's from this book. I know holding up graphic methods for presenting facts by Willard Cobrinson printed in 1914, 100 and say years ago. Oh, I'm like, he said, I'm like, one. With spending millions of dollars collecting data and it doesn't solve the problems, millions of dollars, right? And this whole book is about saying up a data culture about being data analysts, creating curves and charts for executives and presenting data. And this book gives me an existential crisis because we haven't solved the problem in 100 and 10 years. I was gonna say that doesn't make me feel any better at all. But what I've come to realize is that, yeah, there is a frontier, right? There is, we have a bunch of data and we have technology which can create a frontier of what that data and technology can do. And then since we've printed in time, we've just pushed that, our human ideas go beyond that frontier, right? So we will always be frustrated. You know, whatever happens in the next 20 years, the next technological innovations, they'll be frustrating. But at that point, AI will have walked its way into our lives and we'll just be using it and things will be better. So the problems are not new, right? No, I have to wrestle with that every day since I read this, damn it. I do sort of see some really interesting news cases where people could leverage AI as part of the process. So in the discovery phase, use AI to pull the other common themes and make sure you're getting good visibility to everything you've discovered. in all those conversations with users in the prototyping phase, get AI to help decide, am I using the right kind of visualization here? What other alternatives might be good for this use case or in the development help me extend my coding abilities or my data normalization abilities beyond what I'm capable of? And so that's sort of, I've got a friend, Donal Fips, who commented on some of the recently said augmentation two days ago, augmentation over automation. And I think that's kind of the framework that as analysts, something we can all sort of say, okay, yeah, it's not going to replace what we do. It can't, but there's all these different places within a great model or context from the book, we can leverage AI to accelerate our efforts or augment the efforts that we're putting in. >>Eultimate, I should know, automation. I didn't know some brilliant three-word summary. The challenge is that that doesn't satisfy the business model, these tech companies are working on that because they're waiting for the massive enterprise scale solution. And it isn't that because it can't be trusted. >>But it is funny how it goes that that way you start, is it going to kill dashboards and you're like, no, no, no, you wind up with the defender saying, basically saying, well, it's useful for prototyping the dashboard. It's useful for evaluating the dashboard and thinking about what's going to go on the dashboard. And somehow, you're like, see, I proved to you that GNI is, it's like, no, nobody's saying that it's not useful and valuable, but that's a different conversation about its use in the context of traditional analytic delivery of information. >>I'm curious, do you think that using AI in the interpretation of data would end up helping people? Like, Tim, I'm thinking about how you build a dashboard. It has beautiful visualizations. It has clear trends, lines, things. You can see what's going on. But at the end of the day, we always run into how do you help the business actually, correctly interpret? This is time series data, or here are the caveats, or this is expected variation. So variation, time series, those types of things, do you think that there is an opportunity that AI would be able to help people answer the questions around, like, is this a material change I'm seeing in the data? Or I'm guessing that's kind of far-fetched. I don't even know if people would know to ask that question. >>Yeah, I think that there's huge opportunities and many people across the industry are chasing that. So tabloid pulse is traction metrics, but uses the narrative science to just interpret the data and put it into text. You can paste charts into chat GPT, include, and ask it to explain the visuals. And that's two things. So the narrative based on the data is kind of bland because it's quite hard coded as text. It's like, you're almost pre-defining what the paragraphs are going to say. So as an end user, if you see that 10 times, you probably start stop paying, which attention to it. But then if you just get the AI's to interpret a visual, it's not reliable. I've been through that pain and you just explain this chart and it just makes the thought. It's just, because it's a probabilistic, stochastic approach, it sort of makes things up. And so it's hard to rely on it. So yes, it's got huge potentials, Julie. Both. It's just the pitfalls. The pitfalls are huge. And as data analysts, we spend decades trying to build up trust in our stakeholders. And this is from tabloid 10 from his video. I'm stealing his quite here, but it takes minutes to lose that trust. And if we give them tools that say sales went up when they went down, I mean, that's a terrible, that would be a terrible end diamond on what we do. It's analysts. So opportunity currently high risk. But I see, no, I got it. I got to make my point here. Oh, you got it. I've been through so far, where you're in the human world, like put in front of an analyst and I had this happen years ago when I was at an agency and they had outsourced the client, big CPG. I think it was this was PNG, had outsourced the polling of the reports to offshort. And then they said, and this was going to have us not the the onshore agency was going to get less of the work. And they said, those are just going to send you the reports and you'll look at the data and like interpret them. And so I think there's the opportunity for AI to move upstream with the business users to say, let's help you really formulate questions and ideas. And not I'm looking at a dashboard. I mean, that's just I die a little bit anytime someone's like, we've pulled all the data together. We've built all these dashboards. They're amazing. They follow all the bright principles. Okay. And this has happened to be multiple times in my career. Can you come in and tell us how to look at those dashboards and get insights from them? And in and I've gotten pushback. I'm like, let's not look at those. Let's talk about your business for a while. What's keeping you up? What are your challenges? What are your ideas? And then come in with a much much narrower view of what specific question we need answered. Maybe even helping refine what data would we look at and what should we be concerned about? And then arriving at the chart with a much much more clear purpose. You know, oh, we're going to see if sales went up. Okay. When you say went up, you, are you clear? Are we talking about sales going up a material amount and a amount that doesn't look like it's just gauging his noise? So I think huge potential for AI to patiently sit there and kind of be introduced good friction to the process to say, you're not going to jump to looking at charts and then think AI is going to glean insights from it. It's like given somebody a bag of rocks and saying, find the valuable ore in this rock. Well, they could, they could bust them up. They could look at them. They could, there's no guarantee that there's valuable ore in that rock. So you're starting kind of farther too far downstream. Oh, well, outside that is it sounds like you've read the first section of Dutch blog to deliver, which is over the turn around. We had that. Yeah. Bring it. Yeah. Full circle. You see what it is. Andy, incredible. We do have to start to wrap up and that is the new book, Dashboards that deliver how to design, develop and deploy dashboards that work. This has been so fun to have this conversation and I think very helpful and insightful. So thank you very much, Andy, for coming on the show. And now it's time for something I'm really excited about. We're going to take a quick break to hear from a friend of ours, Michael Kaminsky from ReCast, the MediaMix modeling and G-LIF platform helping teams forecast accurately and make better decisions. Michael sharing some bite size marketing science lessons over the coming months and we're happy to have him on the show to help you measure smarter over to you, Michael. What exactly is power analysis? I get asked this question a lot. Power analysis is the process of trying to understand the limitations of an experiment prior to actually conducting. When we're designing an experiment, we face trade-offs. We want to generate the most useful conclusions as cheaply as possible. Power analyses allow us to understand this trade-off between the conclusions we'll be able to draw and the cost of the experiment. The most generalized form of a power analysis is a simulation where we code up the statistical analysis we're going to do once the experiment concludes and we apply that same statistical test to hundreds or millions of simulated experiments showing us under what conditions will draw the correct conclusions. We might see that with the amount of lift we expect to see, we need a sample of at least 250 participants to draw the correct conclusion 99% of the time. But if we accept some risk and are okay if we only draw the correct conclusion 95% of the time, maybe we only need 100 participants. The main variables that affect our statistical power are the assumed underlying variation in the data, the assumed effect size generated by the intervention being tested and the sample size. So what's the takeaway? Power analyses are simulation exercises that help you understand the limits of your experiments and their expected costs before you run them. All right, well if you enjoyed that many lesson Michael and the team at ReCAS have put together a library of marketing science content specifically for analytics power hour listeners, for everything from building, media mix models and house to communicating uncertainty to your board, head over to www.getrecas.com/aph. All right, well one other thing we love to do on the show is go around the horn and share a last call, something that might be of interest to our users. Andy, your or guests, do you have a last call you'd like to share? Yeah, I'll go let's go to the movies, summer 2025. I recently, I'm not a massive model universe fan but I did go and see first steps or fantastic four first steps. Why is that relevant to your users? Because they have some cool data displays in that film, right? In fact, the the visual design of that movie is fantastic and there are some really exceptional data displays and so go check out the movie and also now I've pointed it out to the audience, you will never not see charts when they appear in movies. It's brilliant, they're everywhere and they're really good awesome awesome. All right, Julie, what about you? What's your last call? Mine's in the entertainment space a little bit too, a little different than my other last calls but honestly, I am a sucker for any kind of sports documentary, honestly. And there was one I recently watched, it's Power Moves, it's about Shaquille O'Neal coming and helping Reebok basketball and make a comeback. And I am really curious because of course, like when those come out on Netflix, it's so delayed. So it was talking about like last year, I think the launch of their first basketball shoe was like earlier in 2025. And I immediately was like, I want to know, is Reebok basketball really gonna make a comeback from this? Do people love the shoes, hate the shoes? What are their dashboards look like? Internally, what's their revenue doing? Like I was just kind of like, did he make the, you know, the comeback that he was trying to do? So I'm kind of invested in how to try to dig in and see if. - All right, so if you're from Reebok, reach out to Julie to give her the update on how these go. - I want to know. - Who's the kid from Maine who got drafted first in the NBA? Because he wears converse, anybody, Cooper flag. What does Cooper flag, his shoes are, - I've heard the name, I don't know what shoes he wears. - New balance. - New balance. - I think. - Yeah. - New balance. - All right, the NBA best dorm. - Him is what is your last dorm? - So mine's like right, totally in like entertainment as well. It's, you know, also pandering to our new sponsor, Recast, unintentionally, what are priors in MMM and why they're difficult to get right, but you need to. It's, I mean, it's, I don't know that it's on prime video just yet, but I'm sure it's gonna be made into a movie. But it's a long piece, Marty Sanchez at Recast wrote it and just like, 'cause I lived in the surface level of how does Bayesian differ from frequentest to what you have priors and then you adjust your priors. But so it's a pretty deep dive that I kind of committed to reading it two or three more times and I'll still be a little bit confused, but it's really all written. That's, has not a knock on Marty, it's a knock on my understanding, but it's a good read on priors and media mix modeling and not the NBA or the Marvel Cinematic Universe. Yeah. - I mean, that was exactly the right one for the right person. - Yeah. - For the 10. - What a universe of fun. - Michael, what's your last call? - Yeah. - We're aware. You know, okay, so over the years, there's been this conference in the US that's gone back a couple different names, Exchange, DA Hub, and it's always been one of my favorites. It's got a huddle-based format, so you're sitting down with and having these conversations with fellow practitioners, people from all these different companies. And so the signal to noise ratio is super good. You're not getting talked at by a bunch of speakers all the time. And it's coming back. It's called the Data Exchange Conference and it's happening October 27th to the 29th and Asheville, North Carolina. So I'm really excited about that. And I'm excited to see that conference emerge again. It was I feel like we need something like that for business users, data users to be able to share insights and knowledge with each other in that conference. - It's going to be at the most. - That's not the built-more estate. - It's not at the built-more, but a good question. It's I think there's another hotel or property that they're hosting again at, but Asheville is a beautiful city. So, and it's also right there, Asheville is coming back from those major hurricanes that hit Western North Carolina. So even going there and being part of it is also a part of bringing that region back from some major disaster. So you can even feel good about that too. And if you go, okay. Obviously, we've been talking about dashboards and design of them and how to do that. And I'm sure you have thoughts and questions. We would love to hear from you. So please reach out to us. The best way to do that is through Leather LinkedIn or the Measure Slack Chat group, or you can email us at contact at analyticsour.io. Andy is going to be out in the world. His new book is available just like a week. So go to Amazon or wherever you buy books and check that out. I'm sure that you will see him and bring your books with you so you can get him to sign them when you see him out and the world. And Andy, thank you so much. This has been such a fun conversation and such a needed one, I think, right at this moment in time. My pleasure, thanks, everyone. And no matter what you do out there with your dashboards, remember, I think I speak for both of my co-hosts when I say, keep analyzing. Thanks for listening. Let's keep the conversation going with your comments, suggestions, and questions on Twitter at at analyticsour on the web at analyticsour.io, our LinkedIn group and the Measure Chat Slack group, Music for the Podcast by Josh Crowe Burst. - Both smart guys want to fit in. So they made up a term called analytics. Analytics don't work. - Do the analytics say go for it no matter who's going for it? So if you and I were in the field, the analytics say go for it. It's the stupidest, lazyest, lamest thing I've ever heard for reasoning in competition. - I was waiting for car cars to come back. - Oh no, that's fun. - Even I'm only gonna get stuck during that. - Here, I start panicking like, as Michael Braddock I'm saying, what? I know there was something. Yeah. - I don't have to write them down during-- - Forget to do that. - That would have been a great-- - Yeah. - Damn it. - With some day we'll figure out how to actually do intros with asynchronous arrival. - Well, you know, I wanted to make sure to get some shop talk about book publishing in there. You know, it does. - Yeah. - All right. - Not one of that cool club. - Last thing, I'm just gonna go and tell my goal is to ensure that we're doing code five. - Yeah, that's good. - You're all good. - Just scream it for more your standing. - Yeah, that's the American way. - I can't write down. - I'll be back in the minute. (laughing) - Nobody respects what I'm trying to do here. So get up. (laughing) - Rock flag and dashboards are knotted! That's a verdict.