Welcome to the Analytics Power Hour. Analytics topics covered conversationally and sometimes with explicit language. Hi everybody, welcome. It's the Analytics Power Hour. This is episode 293. Okay, listen. We draw a hard line of this show. We don't talk about tools. But we never said anything about tool selection. And let's be honest, we have all been there trying to figure out which vendor to go with after putting in tons of effort into our carefully crafted spreadsheet with all the selection criteria, which somehow every vendor says, yes, they can absolutely do all the stuff on there. It's enough to make a person cynical and we analysts don't need to help with that. So take a pause from reading the cold sales emails from the latest analytics AI SaaS vendor. Now let's talk about the ins and outs of selecting a tool. But first, let me introduce my co-hosts, Tim Wilson, or is that like to call you a Tim tool selection Wilson? No, I just, how you doing, Joe? I'm just about to rate it, select a new podcast recording platform. So perfect. That's going to probably trigger a bunch of inbound emails. All right. And Mo Kiss, how you going? I know you do a lot of vendor evaluation and selection of your role. I certainly do. I'm very pumped to talk about this. I think the only thing you missed is the like, oh, don't worry. If we can't do it yet, it's on our roadmap. Oh, yeah. That's our equivalent roadmap. Don't worry. Yeah. That happened yesterday to client. They turned to the vendor and they're like, yeah, we can't do that. And the response from the client, which was a very large company, was like, well, is it on your roadmap? And it's on our account. And plus one roadmap. Yeah. Yep. All right. And I'm Michael Helblings. And we wanted to bring on a guest and we found a great one. Jason Packer is the founder of Quantable Analytics. It's an analytics consultancy focused on analytics engineering and implementation. He's also the author of the book, Google Analytics Alternatives. Now in its second edition, and the genius behind the Measure Music Channel on the Measure Chat Slack group. And now he is our guest. Welcome to the show, Jason. Well, thanks, Michael. I'm really happy to be here. You know, it's bucket list item. We're finally making on the podcast. Well, it's awesome to have you. All right. So maybe to kick things off, Jason, maybe just walk us through sort of what brought up the idea and put us behind the idea of writing the book in the first place. Yeah. So, you know, I've always been really interested in like evaluating software and like knowing what's out there. Even back to my early days as a like, you know, Unix Administrator and software developer. You know, I liked looking at all the different tools and back, you know, in this sort of era when the Google, universally analytic, universal sunset was coming up. There was a lot of people that were asking these questions. There were a lot of people asking me these questions. And so I thought, well, you know, I'm, I may as well start doing this research. Which seems like a fun thing to do. And I started out thinking, well, maybe I'll, you know, write a series of blog posts. And then, you know, someone at Columbus at the time, Web Analytics Wednesday said, well, why don't you just write a book, Jason? And that seemed like a good idea to me. And so I did it. And now a few years later, there's some, you know, things have changed. There's some new tools I wanted to look at. And I thought I would just, you know, make the same mistake again. So that's here. Here we are. Who was it? Who was it at the analytics Wednesday who said that? It was, uh, mod, uh, mod. Oh, okay. Oh, nice. Which I think I could have done for in the first, first book, at least, for the, for the idea. Well, like I read it. I didn't memorize the acknowledgments. Jeez. Come on, Tim. But it sounds Jason, like a big part of your process and like understanding the capabilities of the tool is like really playing with it, right? And I think one of the things that I'm often thinking about is like, I see folks trying to evaluate tools without getting their hands dirty. And so like, do you, do you think that's what everyone should be doing? Or is that just the thing that's always worked for you? Well, I think like everybody loves to have an opinion about a tool. And it's very easy to form an opinion, right? You like get in there. You see like, you know, how it looks and how it feels. And that's fine. I mean, you know, I have opinions about that too. But you really have to like balance that against like really learning what the tool is about. And for me, the way to do that is to use it and to use it with real data. Not to use it, you know, not to watch videos about it, not to like be walked through a demo by somebody, but to like install it on a website. Even if it's like just a trivial website, install it and use it. And you know, that's how I learn best. That's how I learn most quickly. And do you think that's that's the process? Using that with real data. The bit that I'm kind of taking away from that is like, it helps you understand it. But how do you think it changes like the evaluation process itself? I think using real data will show you a lot more about where the like issues are. Like for example, you know, if you're working with a vendor and they walk you through it, they're going to show you the highlights, they're going to show you the things that work well, they're going to show you a tool that's completely perfectly set up. And we all know, I mean, that's not how it is, right? And like, you know, the in the book, everything that I evaluate, I used on real websites with real user data. And so for example, like one of the issues with that real, those real websites is one of them had a terrible bot problem. Like, you know, it was like, it was a site that I bought, you know, like on the secondary market, I didn't make the website, I just bought it. And so, you know, it had some real traffic, but it was just like, you know, littered with bots. And so the traffic looked really weird. And like, there was all kinds of strange hits, the pages that weren't there. But that let me, let me to learn a lot about how, you know, these different tools worked in the cases where there's a bunch of four of fours or there's huge amounts of bot traffic. So like, that's the difference between no vendor, whatever, like, show you a demo where, like 90% of the traffic was bots. That'd be crazy. And in some ways, it's also, it can be challenging to do that because that might not be your use case. So a lot of the things I talk about in the book is use case match. Like, that's the challenge as a tool evaluator is to match your constraints of your use case to the best match of a tool. It's not like, you know, like I said, opinions, everybody's got them. And like, there are in some ways in which some tools are more technically advanced than others or some tools are like faster than others or whatever. But like, it's really about, you know, matching use case to tool through the lens of those constraints. Back to the using the actual data. So the book was kind of digital analytics, product analytics stuff. I would, you know, put B.I. platforms in there, put data warehouse platforms. A lot of those, all of those when it's like, you want to try it with your data. I mean, like a really high bar, a real challenge seems to be, we want to do a bake off or we want to do a proof of concept. We want to try it out. And I've gone through processes where it's like, we're going to the RFPs. We're going to select some finalists. We're then going to do a bake off. And that does mean you're fundamentally doing some sort of mini implementation and trying to draw the line of, you know, and that can include getting through some compliance hurdles to say, yeah, we're using our real data. Or do you say, well, we're going to dummy up. We're going to do an effort to make it's kind of like our data, but it's been a non-limitious to the point that it's not our data, but it's still mimics our data enough that we could actually try it in this platform. Like it does seem like companies, to me, that's what motivates a lot of the not wanting to go through that process. Like, ideally, it would be great to use your actual data and to do, like you say, a real mini implementation, but that's just not feasible in a lot of cases. I mean, Will have you done that? Yeah. Like, I'm not going to bet around the book. Like, I do a lot of, like, analysis of different vendors and different tools and that sort of stuff. I would say I definitely lean towards the, we should do multiple POCs. Like the last major tool selection we did, I think I wanted to do maybe four POCs. And obviously, like that's in negotiation with the business and capacity and things like that. We ended up agreeing on two. But I think the thing that I found really hard is like, often the folks doing the evaluation and the assessment and those sort of things. Like, I don't know if the incentives are always there to do multiple POCs. And I find that like hard to reconcile with because it is, it's really hard to like understand how good a feature is or a particular capability that you're looking for without stress testing it. And yeah, I don't know if I just air too far on the POC side, maybe.
I think folks internally would probably say I do. - I think that's really challenging, right? Because a POC is great, but even before you want to get to that POC, you want to feel like you've narrowed it down to something that's worth the effort there. And for me, part of that can be not even doing a real POC, but doing like a toy test. Like, oh, let me do it with my podcast website. Let me do it with my personal website or whatever. And that's part of the reason also why I'm a big proponent of free tier, even on enterprise tools. And that can be a challenge, right? Like, not everybody can offer that. And sometimes if you're talking about a huge BI platform or something, what would a free tier even mean? If it's even doing a simple example, implementation means putting in 100 hours of work or something, but that, like the ability for to get a little bit into the product before you really start talking about committing company resources to it, I think, because I do love the POC approach and the more the better, but it can be hard to get those resources for sure. - And also just like getting through security is like a really big step. You're basically doing like a procurement process for something that you're running a peer say on. And like, yeah, it takes a lot of time and energy, but I mean, I obviously am very biased here because I lean strongly on the side that that's worth it. Yeah, and like that's my lived experience, but yeah, this. - Well, Bill, have you run into it? 'Cause I can see the doubt. So say you say it's only two. You get down to the two tools and you get in and you've got multiple people who are all kind of trying it, and they all have kind of different things they most care about. And then you get to the end of that, and you're like, all we've done is allowed people to dig their heels in further on their preferred tools, 'cause now they have hard evidence that that other tool doesn't do this thing that I think is really important, and it does this thing, like, do you wind up saying, well, this is supposed, we're hoping that we arrive at a clear winner, but even if you do a POC of four tools, they're still not the one clear winner, and you're still in kind of a negotiating phase, and you're also setting up the people who didn't back the ultimate winner to be able to say, "See, we did the POC, and I told you we shouldn't have that one." Sorry, that's just depressing me. I'm just. I can store a member a few years ago, we were doing a BI tool selection. It must have been five years ago, and all the data analysts got in a room, and this was the absolute worst way to do it. I would never ever do this, but we were like, how important is this thing to you whenever I'm would go to one side of the room or the other side of the room? And almost every time I was on the side of the room on my own, and I think. (laughing) It suffice to say we did not pick the tool that I wanted to get, but it is what it is. I think the thing that I find so difficult about data tools in particular, and I know we had Colin on previously from Omni talking about how like, especially BI tools, you're trying to be many things to many different people, and I think what's so challenging about data tools is, data folks have very strong opinions about the things that they do and don't want to work with, but also their opinions are normally. Like, representing what is best for them, and not always what is best for the business. And that's human nature, right? Like, you think about what's gonna make your own job easier, but. And so I think I often come with this perspective of like, a data tool is actually for our stakeholders. So like, even if it's a little bit tricky, or a little bit harder for like us in our day to day, is it going to help our stakeholders in their relationship with data be better? 'Cause I will up wait that, but I don't think that's the common. I'm not sure that's necessarily like a common view. Michael, what's your relationship status with SQL? Oh, I think you know it's complicated. It keeps cast-lating me with syntax error near from, like, I don't know where from lives. (laughs) Well, here's a healthier relationship, Prism, by Ask Why. You ask in plain English, Prism writes the SQL. Ooh, like, revenue by channel week over week, excluding refunds, and instead of me crafting a 47-line query and a three-line apology, Prism just does it? That's right, the best part. It doesn't forget everything the moment you close the tab. Prism's jam a memory remembers your reality, your definitions, your quirks. I mean, not your personality ones, but, you know, your coding quirks. (laughs) Well, but like, then the BigQuery table is the source of truth, and conversion means this, and not whatever gets decided by somebody like, mid-meeting somewhere. Exactly. So you don't have to re-explain your business context like it's a bedtime story for robots. Yeah, I have to admit, I'm a little tired of starting every new session with previously on analytics. And when Prism generates SQL, you get traceability. You can track changes, see what was created, and follow the logic. I like that, 'cause when somebody asks me where this number come from, I can stop saying, "Well, from the number tree." (laughs) It's like version control for your analytics brain. I like it. A little bit of accountability, but it's convenient. That's right, so do you want in? Go to asky.ai and join the waitlist. That's ask-the-letter-y.ai and use code APH to go to the top of that waitlist. I like the idea of letting AI write some of the SQL. And like your memory, do literally anything else. No, I think it's not. And I think everybody also wants to work with the cool, that's good for them personally. Well, like the side you have, sort of like you're implying that, hey, I want to work with the new tool. I want to work with the cool tool. I want to work with the tool that's good for my career. I want to work with the tool that my LinkedIn posts are going to be, go with, and that, that's not the right fit. That means really about the whole organization, not just the analyst, but a lot of times the analyst isn't even really the one. Flip it around and people want to work with a tool they're familiar with. Like I used this in my last job. So I want to use it here, which was good when GA4 came out and Universal Analytics got sunset. And it was like, well, nobody's familiar with it. So reset. Yeah. Well, that's what I was going to say too, is that a lot of times a tool switches not the right answer. Like we all like to think, hey, there's a tool out there, the perfect tool out there that's going to fix my problems, is going to make my personal life better, my company do better, et cetera, et cetera. But there's no perfect tool. There's no grasses, look screener, but a lot of times the tool you have now just isn't implemented correctly. And then, when you get, isn't going to be implemented correctly either. So that can be a real challenge too. And especially if you're like, hey, I want to do these-- hey, we're going to do two POCs and put in all these resources in the end. We're going to say, oh, well, actually, I think the answer is that we stick with what we got. And we just spend a little more time trying to improve our application. Nobody wants that answer. In your experience, like, told me through when there are trade-offs, right? We've all said, no tool is going to meet the brief perfectly. How have you approached balancing those trade-offs? What's your thinking? And how do you, when you're working with businesses, convince them of the trade-offs they should make versus shouldn't? Yeah, it's really difficult because how I evaluate the tools from the book is a totally different mindset than how I think when I'm talking to an organization. And a lot of times I won't even really be talking about the same things. In the book, I talk about the underlying tracking structure of different tools, the databases, the different tools use, how they work with consent, things like that. And when I'm talking to a particular business, I listen for what their real pain points are. Like, is this an organization that they just need to get off both of GA because of their compliance issues? And that's like-- then I focus their selection on solving those pain points as directly as possible, but also trying to not get into the weeds with them about the details of the tools that the people listening to this might find interesting. Because they're not kind of friendly and interesting. Well, but I think you just kind of mixed it because part of what you did-- and maybe it's worth having you-- what I would love to about both editions, because the structure stayed the same is that the tool-by-tool, blow-by-blow-- and it's not a feature-by-feature, but the tool-by-tool kind of write-ups are the second half of the book. The first half of the book is you've got to have kind of a framework of what matters to you. And so you admitted throughout-- there is no perfect categorization. But you just talked about one of those was the tracking methods. And for--
It's I could see for the right company, they would say, we've been getting burned by our current tracking method and we've got to find something. And you're like, cool, well, let's then think about the philosophical difference from the different tools. If somebody else says, we just need something super cheap. It's like, OK, well then let's talk about the nature of your digital experience in the different pricing models. If somebody says, we just got to get off a GA because it's compliance. We actually love everything about it. It's just our compliance team is said we have to get off of it. And so I would say in the example you just gave, you actually, it was kind of how you approached the book. It's just where you're going deeper in that understanding what attributes truly matter and then going deeper, right? Yeah, I think actually that's fair. Like, you know, and like one of the things I've talked about is how things like, it's all about constraints, right? And how price is a constraint. Price is a real important thing for organizations. You know, it's not the coolest thing to talk about when it comes to tooling. And similarly, it's just a question of like how you're engaging with the decision makers, I guess. So, you know, the things that are in that first half of the book are just like a long list of the things that I think about. And it might not be, I might think about a bunch of those when talking to a particular organization about a tool. I might not be talking to them about all those things, you know. But I'm certainly thinking about a lot of them. And I think it's like important to understand them to a certain degree, you know, like, for example, like in the new edition, there's a chapter on server side. Obviously, you know, you're not going to like, I'm not going to teach someone everything about server side analytics in a chapter, you know, a 3,000 word chapter of my book that's not primarily about that. But understanding at least enough about that to know if you're talking to a vendor when they say, oh, yeah, we support server side. It's easy. This is what you do to be able to like understand interpret what they're saying to know like, oh, well, really kind of like anybody could do server side. It's not really about the tool. It's more about the deployment about, you know, oh, we are using server side GTM to deploy that. And if you are than this and perhaps your the real underlying problem is, you know, tracker blockers or something like that. And then your lens for viewing that is different. So that's why I think that like the, you know, the first half of the book, the, the sort of the guide part of it, rather than the product evaluations is like the lens in which I look at all product evaluations. And I'm trying to like share that viewpoint in the first half. So, you know, Tim liked it at least. So that's a. [LAUGHS] Can I ask, and this is probably also a question for multiple people like you. When you said it in kind of some of the earlier discussion that you explicitly did not talk to the vendors, even though they were, especially after the first edition, they knew you were doing the second edition. And they're like, come on, just let our sales engineer help you out. You know, once you just understand. And I think you did that to say, I want a level playing field. And I need to finish this book at some point. And if doing, if doing 15 POCs is tough, letting their sales teams get their hooks into you would be absolutely impossible. Whereas, yeah. And so whereas Moa feel like if you're down to a couple, where does sales play? So I don't know, maybe you can talk through that. I mean, yeah, that's sort of an unusual choice that I make in the book is to like-- I mean, I definitely have talked. And I know a lot of really great people a lot of these vendors, like especially after the first edition. You know, I've talked to a lot of these people. And there's a lot of them told you what you got wrong. Not as many as some. But it's important to me that I was really, really fair more than I was particularly making any value judgments or anything like that. But the not engaging with them is about leveling the playing field to some degree. It also fits in well with how I learn. Like I describing the learning from doing. The again, like getting a demo account or some kind of account where I can use the product is the fastest way for me to learn rather than like being sales that engineering calls. But I think that's a much-- that was my case for writing the book. That's different than most cases with engaging with vendors from a large org that has specific needs. And I think that a lot like engaging with vendor reps can be really, really helpful. But it also like gives you an idea of the culture fit between the product and your organization, which is a real thing. And something that when I started the first edition of the book, I didn't expect to be so important. But is I think quite important? Do you hear that Tim culture, the very important? I just wanted to reiterate that point really quickly. Sorry. Do you go into a moment? Just to add to that, I have personally found that engaging with sales, engineering, support, whatever is like a really big part of the process. Because I want to make sure that we can learn from the expertise that we're not facing challenges that are very easily fixed. And I think part of then, even in the playing field, is making sure that you get that with all the companies that you're peeing at seeing. It's not a favorite game. And you're so right, Jason. Such a big part of it is about the culture or the ways of working that you then get to explore with that other company. And very transparently, I've talked about our relationship with Snowflake quite a bit. And a big, big part of our success, I will rail on about implementation for years to come. But a big part of it is like, we've had really close relationships with their product teams, with their product managers, their tech leads. We will have calls like testing out new features and new functionality. And being able to influence the roadmap, that is a hugely important thing for us when we're doing vendor selection. Because we want to make sure that in a year's time, we have the kind of relationship where we can push their product if we need to. And so I think that letting those folks in the room so that we can stress test each other is a big part of the evaluation for me. Yeah, I agree with that. And again, it depends on your organization and why you're buying the thing to start with. If you're a tiny startup and you're not really going to-- if the thing that I hate is like you're a tiny startup and you're talking to an enterprise software provider, and you get to the point where like, OK, we're ready to actually talk some real prices. And like, OK, well, start for your volume data. We're starting out with $65,000 a month. And you're like, that's my-- what are you talking about? That's my entire yearly budget for all of my analytics. So I love transparency. I make that pretty clear book. And I think that that's just a great thing to get people on the same page as quickly as possible. Because that's super important. And I think that when you are engaging with the vendors, being transparent with them helps everybody. Nobody wants to seem like-- yeah, nobody wants to seem like a dummy when they're talking to a vendor. But if it's a new tool, I don't know the tool. They know the tool. They know their immediate competitors far better than I will. So I try to be very direct about, hey, the budget is this. And here's my seemingly very stupid question. And when you gave me answer, I didn't understand. I'm going to just ask that stupid question again. Because it's important to everybody that we find the best fit in the most direct way possible. And I do think, obviously, I come from a place of absolute tech privilege. I think a lot about one of the hills. We see that all the time. We're like, well. Well, anyway, I just want to be conscious of other folks have very different budget constraints when it comes to tool selection and things like that. But there are things I will die on a hill for. And one of them is-- I do think, obviously, budget is incredibly important. But if there is a very good tool, and it is not like 10X, like other options, but it is a good fit, I personally think that is a fight worth having with the business, like getting support for that extra budget to make the right tool decision, versus being so constrained by it that you make a really, really shitty choice. And again, everyone's not in that position. But that situation you just described, Jason, having the-- knowing the prices much earlier,
in the process is absolutely something that folks should be doing. Like, you can't wait to, you've done a POC to start like getting an idea of their pricing because if it is like way out of the realm of possibility, you don't want to waste your time and energy on it. Yeah. And like, again, you know, I'm going to, I guess I haven't thrown Google under the bus yet. I'm just, yeah. There's always something. There we go. Here we go. Yeah. One of the things that I think Google really made hard for is that, you know, they made a, with Universal, they made a, you know, pretty darn good product and they made it free to just an incredible degree, you know, like it was free to, technically free to 10 million hits a month and, and in reality, it was quite a bit higher than that. And with GA4, of course, there's no, no hard event limit. You know, the one million per day export limit to BigQuery is probably the thing that people hit first, but that's still, they're giving away so much, you know, for free. And that's really caused people in the industry to think that analytics should be basically free, that the software should be free. And like, that's, it's very distorting. I mean, it makes things really hard for, new tools to come out there and to get a foothold in the market. And it makes like, what Moe you're saying as far as like, hey, you know, I, we need to understand that even if this tool is a little bit more money, think of the, the, the, the, the cost in people, the cost in like, you know, data decisions in the organization, you know, and like, you're like really undervaluing analytics and part of undervaluing analytics started with, you know, VA being free. And, you know, that's still what's happening. Oh, Jason, I feel like we could sit around and like, chat for hours because I think fundamentally one of the biggest mistakes I see is, yes, folks want to work on cool shit to put on their resume or LinkedIn or whatever. But it's also the like, the open source fallacy or the free fallacy, which is like, oh, this is open source, or it's free, it's not going to cost us anything. And I, I will push pretty heavily on like, that does not mean it's free. We need to actually think like, we're talking about a solution here that has five full time engineers supporting it. That is not free to me. That is actually a huge cost to the business. And if we want to do that because we think that's the right decision, that's okay. But that needs to be a line item in our decision as well, not just the like, on paper cost of the tool. Well, that, that also then extend that if we're going to have to support and we have those five engineers and one of those engineers leaves, what's the size of the pool of candidates that we're going to have to replace it, which is one of those where market leaders and whatever tend to have a leg up, even if and it's a legitimate leg up, they've achieved some critical mass. There are, you know, nobody got fired for buying Tableau or Power BI. And part of that is because everybody's kind of been exposed and is familiar, but it's also legitimate saying, well, if I need a Power BI developer, that's a much larger pool to draw from, right? Yeah. When Mo, excitingly, get ready for a new wave of that with AI, because now people are going to be like, it's free. We can just build it with AI, which actually, it's a question to Jason, sort of from your perspective, because obviously, you know, we've all kind of been through vendor selection processes and we kind of touched on how like Google Analytics is free. So obviously, you know, as people are sort of jumping on the AI bandwagon and seeing how easy it is to prototype things, not necessarily build full on products yet, but like we're moving in that direction, I would say. Do you think that's going to be something that will enter the process of sort of like the build versus buy debate certainly changes a lot in the future? Yeah, I think so. I think that's already already happening. I think that it's happened not exactly with AI, but the sort of the simplified realm of these tools like the in the book I call them simplified web analytics tools, including things like plausible and phylum um is also quite popular. Um these tools are extremely simple. Most of them are our concuous. They used like久 and I P plus user agent hash to create sessions Um and like there are so many of those tools out there now. There's like you know every few months a new one comes out. Some of them are quite good. I mean like it's not a hard thing to prototype and it's even easier. You know with AI I mean I could like you know go in and. Especially if you start with the ones that are open source you can be like hey you know build me an umami clown here's the umami GitHub repo. About it and you could you could build yourself something like that. Pretty quickly and some people are doing that. Um. You know I think you run into some of the problems that Tim was talking about as far as like having expertise with the tool. Like if it's you know if it's some internal tool then the internal people are going to be the only ones that have the experience with it. And like also when it comes to some of the more complicated underlying database things. Um. That's I think far beyond the complexity of what AI can do a good job with. Um. And so. You know like and also things like schema. Um. AI doesn't do great job with. I mean it can do okay. Uh but it needs a lot of of like you know human handholding. So I think that like ultimately it's like. A mistake for most organizations to try to think that they can build their own when there are so many like really great. Platforms already out there. Um. I think it's it's maybe fine to think like oh we're going to like extend. We're using whatever we're using post hog. And post hog is an open source tool. They have. It's one of the widest tools in the market. They have 34 different apps built into the tool of you know whether you want like session recording or you know future flags or whatever or you know analytics they've got it probably. And if there was something in there that you know that they're not going to do that. Then using AI on top of that I think to extend it makes sense. But like trying to like build. A fat like a foundation to your analytics. Platform your analytics practice. With without really having that real strong attention to detail that these platforms that have been out there tested have. It doesn't make sense whether it's you know humans are not going to be able to do that. And so I think that is the way AI is building it. But I think it's a big part of our sense to like build on some of the great tools that are already out there. That gave me a cool weekend idea though. Mommy coming out. Yeah. Yeah. Just just because you can. I don't know. But is you is you brought a post hog because that's one that I'm not familiar with but you you mentioned them because they kind of well actually I think they're kind of like who. There is this like who was the tool built. You know buy and for like post hog is like. Of buy and for the developer. You know Google analytics, a universal analytics was kind of intended to be. For the more casual like I mean initially they tried and they tried to stick with it like this is this is easy. This is for the marketer. BI platforms seem like they're similar you know if you're if you're in the power BI or in the Microsoft stack like this is built for the enterprise that wants to have the complete ecosystem and progression of tools. So what I think you've you've said here you definitely say it in the book like what is the the philosophy of what is at the what's in the DNA of the company that's building it like who do they feel is the the user that has primacy right I mean is that is that a fair absolutely yeah you and I talked about that right like that. You know the that philosophy philosophy and outlook is more important than any sort of like future comparison. You know future comparisons. I mean probably some people have heard me complain about this before but you know future comparisons are helpful in some ways but they also are you know already outdated by the time you you you posted your future comparison list it's it's already out of date the feet that one checklist item that it says it does X. There's so much to unpack underneath that that that their version of X might not be what you really think that you're getting when you get that feature and people want more features like I just talked about post on they've got every feature under the sun. They've got you know they've got session capture but maybe you already have session capture you're already running Microsoft clarity or hot jar or something like that so while that looks like a great thing on a future comparison list that's a
not something you need, or that's not something that's going to help you, it's going to add confusion to the product. The more features you add onto a product, the harder it can be to use. That's just the way the way these suites work. But it can be really hard at the same time to understand, peel back that layer of marketing a little bit, and be like, "Well, what is this, what is really this product philosophy? Who is this for?" When you're in that job, when you're an analyst, and you get in front of Adobe Analytics workspace, you're like, "Okay, I get it. This is for me, this was written by people that have been listening to people like me. This makes sense to me, and it's useful to my use case." That is clearer, and a lot of times, once you get in and use the tool, like I was saying before, but when you're just looking at the marketing, it can be like, "Well, both platforms say they have the ability to customize reports like done, like they're the same, when they can be wildly different." But so, talk to me about this philosophy piece, because I find this really interesting, the philosophy of the platforms. And I think maybe I was alluding to a similar idea before, but how do people figure that out? Like, is the philosophy like who the user is, or the direction they want to take it? I think it's very, it's not easy, because in a lot of ways, I think the vendors themselves don't know a lot of times. It's a product of the history of the company. It's the product of what the target market of that tool is, and it's a product of the people that built it and who they're listening to when they built it. So, let's take, we're talking about possible on some of the simplified tools. They have a clear philosophy of this is a simple tool. There's not going to be vast ability to customize reporting. Everything's going to be on one screen, or pretty much everything is going to be on one screen. We're not going to have a drown you in configuration options. This is designed to be simple, and part of that is, in that as privacy as well, is that you're giving up some amount of complexity to make it easier, and to perhaps make it more private as well. That's a philosophy. And I think that philosophy, for them, can be pretty clear, right? When you look at their marketing, you look at the sample, you try out a simple product. It can be pretty easy to understand their philosophy when they communicate it well, and it's not a sprawling platform with 27 different components or whatever. Versus some of the more complicated tools, like I talk about piano in my comprehensive category along with tools like Adobe. If you're looking at either of those tools, they offer so many different features and functionality, and there's a much more complicated onboarding process that it can be really hard to understand what that philosophy is until you get much further along in the process. I do think that talking to the vendor and engaging with them before you get too far along can help you understand that, but it also can confuse the process too. I don't know that I have a real great answer. Well, I like that because you said you have to sort of wear the company, like looking at the roots of the tool, and I don't have a million examples, but I look at in the BI space, you had Tableau, which was one of the second generation of tools that clearly was like, the shit should be drag and drop, and we should be able to customize it to conform to things that Stephen Few would give a 10 out of 10 too. They were coming at it saying it's got to be a drag and drop, wizzy wig interface that you can have highly customized to be very, very clean visuals. So there were a BI tool philosophically, I think it was forward on the quality of the visualization. Contrast that with DOMO comes along a number of years later, and I would say DOMO was saying, no, no, no, it's all about the ease of connecting to all of your data sources, and they kind of led with the connectors. Now they're competing with each other, so over time they're getting their sales teams are saying we're losing DOMO, we're losing deals because our visualizations are shitty, and Tableau is getting pushback of saying we're losing deals because we're not easy to connect all these different things. I think they are not necessarily permanently hand-capped, handicapped, but it is one that I would say both of those tools, like that's where their various strengths versus weaknesses are, which means if I'm looking at a BI platform and those two are in the consideration set, I may be thinking, do I have stuff going pretty tightly into most of my stuff going to go into a data warehouse and occasionally it'll be pretty normalized and I want to hook into it and occasionally I might only hook into something else or are we going to just live in a chaotic world where I'm always going to be needing to hook into a gazillion different data sources that are all going to be messy and I'm going to need to be able to do transformation within it. I think it does take a lot of maturity or wisdom or thought to try to map where is my companies kind of which philosophical or historical underpinnings are most aligned with my needs and then stand up and say and guess what? That means our visualizations will never be as good as what the perfect idea would have because that's a lower priority. Yeah, that's where I think I talk a lot about understanding fundamentals. That can be really helpful to close. Some of that you're like talking about is the gap between the marketing that you see from the vendor and the reality. Part of closing that gap and understanding really what a tool is all about can be understanding the fundamentals of how particular things work. If we're talking about databases, if we're talking about this product uses mySQL and this product uses snowflake and this product uses Postgres and this product uses Clickhouse. Knowing just a little bit about the differences between those tools is going to tell you a lot about the product. If we're talking about say we're comparing Poked Pro and Matomo. Poked Pro uses Clickhouse as the database underlying their product and Matomo uses mySQL. On the surface, they're pretty similar products but they end up working quite differently because of that difference in the underlying database. MySQL is a simpler database. It's something that's easy to self-host. It's something that's easy to see the raw data from. It's something that's not super performant and a lot of more complicated analytical queries and all those things surface in the products. If you know that background and you know that it's not like you need to know how to use those tools but just knowing a little bit. The same is true for tracking methods like cookies, tracking with cookies versus tracking with this IP post user agent method or tracking with browser fingerprinting or whatever. Just knowing a little allows you to see the vendor says X. I think what they mean is this. There's not like as much as a vendor might say it's not like there's a million new things and a million new ways to do things. There's a limited number of ways. Before I wrap up, I'm going to give Mo the opportunity to jump in one last time but we do have to start to wrap up soon. But yeah, go ahead Mo, I know you want to ask one more. Jason, we've talked about a lot of different concepts and things you need to think about in this whole tooling decision space. If I'm sitting at my desk and I just take your one like absolute, this is the thing that should be most top of mind from all the things we've chatted about today. What would be like the one thing that you would see just if you pay attention to this, then you'll probably make us slightly better decision. Oh, that's a tough question. Am I actually saying price? Oh, so I'm disappointed. It is just any. I'm disappointed. I'd like to say something cool like the fundamental database schema or something like that. Price is like, it's a shortcut to a lot of putting you in the right area. I'm limiting to what I don't want to do, but that's where I'll go. Price is one input to a total cost of ownership. I mean, that's again, maybe another one. Have you ever come at it that way, Mo, with any of your that's a bad experience. You see our total cost of ownership. Let's just say that. Say I said total cost of ownership. That's what I'm at. That has to be in version three of your book because I like that framing. Total cost of ownership sounds way better than I think I do use it. I don't know. I mean, I think it makes sense if you're going through it differently philosophically. How much am I I'm gonna have to invest in. added tooling to work around a limitation in their tracking or something like it could be. All right, well we do have to start to wrap up. This is also a conversation and honestly so it's a good conversation because I think everybody deals with this in some capacity in their analyst career. So Jason, thank you so much for coming on the show and being our guest today. One thing we like to do is go around the horn and share a last call. It could be any topic, any thing at all. Just something might be of interest to listeners. Jason, you're our guest. Do you have a last call you'd like to share? So my last call is something that you already mentioned, Michael, which is Music League. So, Michael and I, and I believe your sister is a part of this as well, is Music League is a competition sort of. It's a friendly competition where every week somebody like there's a theme, like this week's theme in the Music League that I'm part of is Beatles Covers. So everybody picks a Beatles cover that they like, then a playlist is made automatically from that, whatever, 20 songs. And everybody votes in the ones that they like and fun as had. It's not complicated. It's fun to do with your peers, your friend group, your work. We've been doing it on the measure slack for what three years now or something like that. It's, I mean, it's a lot of fun. I'm supposed to analyze it in the measure. What? For somebody who's interested, they have to be in the measure slack and then in the measure music channel, they can, yeah, that's where the conversation happens. You don't technically have to be part of that. But anybody can start Music League too. And there's also like free, you know, like yeah, but we like to do stuff around, you know, us don't just get people to go out and do their own thing. They get to be part of the measure slack to do this. So join that first. Obviously top here. Yeah. Yeah. And the group is amazing. Like it's also great. We have tons of cool fun, music-based conversations with all your peers and analytics. And it's a lot of fun. So for my own personal experience, it's a great time. And I've got a great idea, Jason, because you know, we've been growing as we grow and then we get the big power hour bump on this now. We can start like different levels of leagues. So there could be like a premier league with delegation and then a championship league like like British soccer, you know, but I think I would be relegated. I'm not sure I would like that. Well, I probably would be too. I don't often score very well, but I have a lot of fun. Anyways, it's also really cool to get a new playlist every few come weeks or so of songs you might not have ever heard or genrez you not that into. So it's nice. I like. So we do occasionally get comments from people who are like, you guys mentioned the measure slack where it's stuck. If you literally go to measure.chat and then you show aint.measure.chat and we'll also have it on the show notes page. So if anybody's like, you guys keep mentioning it and you, I mean, it's in our outro and we don't have instructions for how to find it. So listen, if you're committed, you'll find your way in. All right. No, thank you. Yeah. That's awesome. And Jason, thank you for kind of being the oomph behind that as I know it's a ton of work on the backend to make sure commissioner. Yeah. Commissioner. The Scott, the Scott loving commissioner of the measure music channel. All right. Well, what about you? What's your last call? Okay. So my husband has been listening to a podcast for a long time that folks will probably be familiar with. I have noticed it indexes highly to men. I know a lot of men that listen to it. I don't know a lot of women. And the pivot pod. Yeah. That's what. Sorry. The pivot podcast. And one of the my husband listens to it on like loudspeaker around the house and it like really like drives me dance. And I have not been the biggest fan of Scott Galway. However, I have had my opinion changed very significantly. I am now a listener of pivot. I have been incredibly impressed with how they've talked about, I mean, AI and like tech over the last few months, but particularly the coverage on the Epstein files is something that I just really like it really impressed me. And that's why I've become a really big listener. Scott also last month did this like resistant unsubscribe initiative, which folks might have seen in the media, which was really cool, which was like encouraging folks to basically use our economic power to let tech companies know that kind of we're not happy with how they're supporting the administration. And so yeah, like I just I felt like they were using their voice to share their perspective on something in a really like meaningful way. And also just for everyone out there, like checking on the women in your life, the last few months have been like check it us to the core. And so just to just check in on your your wives, your moms, your daughters, all the women. Nice. Yeah. Great. Great. Yeah. Tim, what's your last call? If you got well, there was this episode of the Rogan. No. So I'm gonna do two. They'll be they'll be quick. One David Epstein, who I'm a big fan of like his books, like his videos, but he did a 15 minute video called why you should fail 15% of the time. And he talks about desirable difficulties, which is a phrase that don't think I knew, but he kind of breaks down the value of doing hard things the hard way. And like specifically what that does for you, which I mean in the world of vibe being shit. Like there are a lot of people grappling with it, but he's just a well done video and he's delightful to listen to. And then maybe kind of adjacent to that, there was just an article, I don't know, it was the metadata weekly, Mark Dupuy, the the AI analyst hype cycle. And I just, there were some quotes in it that were just I thought were gems like quote, if AI can only answer questions that have been preconfigured by the data team in a semantic layer, what have we actually built an expensive natural language interface to existing dashboards, which any kind of makes the case of like where is this all going like it's narrowing down to where what you actually get is maybe not that great, but they also had the analysts who thrive will be those who can translate business problems into the right questions, validate AI output, build the context systems that make AI useful and provide the judgment and recommendations that AI cannot, which I think a lot of people are saying, but it's like that's kind of like a cheap throw away thing to say when I look what people are then also saying, I did this thing, it often kind of skips, skips those components of it. So the AI analyst hype cycle by Mark Dupuy is my second one. Michael, what's your last call? Well, we did an episode a while back talking about semantic layers with CD House in from Thought Spot, which was awesome. And we also did an episode about AI that I remembered something most said about how letting AI's leverage, how the queries are being used in the organization is also a way of training the AI to do that. And I read an article recently from Jacob Madsen at Mother Duck about rethinking the semantic layer and kind of challenging the idea that a semantic layer is kind of the only way to go. And I just thought it was a cool counterpoint. I don't know that I've got a strong opinion one way or the other. Like I very much respect like the conversation we have with Cindy and I really thought it was really powerful. But I'm there's some interesting research and discovery going on as well on sort of like letting the AI consume all your SQL queries and using that to help it understand more the context behind where and how your data is getting pulled together. So anyway, it's a good it's a good read good to kind of think through those things. I don't think we've solved it for our industry. So I think it's early days on all this. So yeah, oh, and what's this breaking news? I'm getting word now straight from our correspondent. There is a book out there that Jason Packer has written called what's the name of the book again? Hold on, I haven't written down Google Analytics alternatives. And for listeners of the analytics power hour, he's going to give you a 20% discount. So that's pretty sweet. If you haven't already bought the book, that is the incentive to do so discount code APH. So there you go. That we'll put the link to that in the show notes as well. All right. Well, Jason, once again, thank you so much for coming on the show. This has been a lot of fun and a really good conversation. Appreciate all the work you've done. It's a labor of love. I'm sure just to do all this. And so very much appreciate it. On behalf of vendor weary industry, I think you're doing us all a big service. So thank you. You're welcome. Yes. Good time. All right. Well, we'd love to hear from you too, because you've been listening and you probably have questions or you've got thoughts. And so reach out to
us and you can do that on the Measures like Chad Group, which we've spoken about on the show, as well as our LinkedIn page or via email at
[email protected]. And we also love to get your comments and ratings on whatever podcast platform you listen to. Please feel free to do that as well. And I think I speak for both my cohosts. - Hopefully, if you, what a nice gift. - What a nice gift. - A few things that are a little important, if our show prep had it. So one, just know that Michael, you and Jason and I will all be at Measure Camp New York on the 28th of March. - That is true. - So that is true, we will. - But even more important. - I didn't expect by now there'd be tickets left. So I was leaving that out because it's your two late, you probably can't make it. - Well, there's something that's. - If you can get a ticket. - Kind of more important, and I mean, more important from an operational perspective. The Marketing Analytics Summit that will be at April 29th. - Yeah. Okay, now you're. - Yeah, I didn't get that. - You can get that. Okay, I did skip that. - Yeah. More breaking news. I'm getting. Yeah, we're going to be a Marketing Analytics Summit and we need your help. We want your questions. We've got a very cool survey of which there's an Easter egg at the end that I had no part of. And we'll have to take the survey and ask a question to see it. But yeah, go to analyticsour.io/lissiner and submit a question. We'll be recording at Marketing Analytics Summit on April 29th in Santa Barbara, California. And we hope to see you there, but if you can't make it there, we can still ask a question. And we may answer it on the podcast. So please do that if you can. if you want to ask us a question. And even if you don't want to, push yourself a little bit and ask with anyway, I highly preference questions and make Tim feel uncomfortable. So like, you know, ask him emotional questions about, you know, the best manager he ever had or. - Yeah, luckily. - Stuff like that. We have not figured out how we're sharing access to all the questions with all the co-hosts. Oh, yeah. That's the little tricky part of that. All right, well, before I forget anything else about the show wrap up, let me just say thanks once again, Jason. And I think I speak for both of my co-hosts, Mo and Tim, when I say no matter what vendor you need to pick, just keep analyzing. Thanks for listening. Let's keep the conversation going with your comments, suggestions, and questions on Twitter at analyticshour on the web at analyticshour.io, our LinkedIn group, and the Measure Chat Slack group, Music for the Podcast by Josh Crowe Hurst. Well, smart guys, we want to fit in. So they made up a term called analytics. Analytics don't work. Do the analytics say go for it no matter who's going for it? So if you and I were in the field, the analytics say go for it. It's the stupidest, laziest, lamest thing I've ever heard for reasoning in competition. All right. Well, we do have an editor who we've been talking so fondly about. So we can stop and start as needed. Well, without further ado, actually before so just, well, just like Mo, are there any, because I mean, you're kind of often in the midst of vendor selection stuff. So you're comfortable there will be anything you talk about. You can, you'll self-edit for whatever named and unnamed. Yeah. So like we just signed a new BIT tool, which I probably can't say. But I will just say I've been involved in multiple BIT tools, so let's just stuff like that. Okay. Yeah. All right. All right. Let's start clacking the keyboard and record this thing. I got it. You got it? Thanks. I was like, I better do it before you start, because if I do it half a year, that was great timing. Actually, it's really good. Pretty sure that I'm gonna take it. All right. Here we go. P-N-5-4-3. Rock flag and an instrumental rock flag rendition by our guest. Oh my gosh. That's that's the permanent one at the end of every show now. That's incredible. I don't know why he was showing that he was gonna play that one and instead it just played like transition two. So it's good that it crossed that.
PodcastTranscript AI