Go back

After the Reform scandal - does anyone still trust pollsters?

41m 54s

After the Reform scandal - does anyone still trust pollsters?

This transcription explores multiple interconnected themes: consumer technology, political polling ethics, and the reliability of public opinion data. It begins with a promotional segment for wonder.com, showcasing a delivery service combining 20 restaurants. The core focus shifts to a critical examination of political polling, particularly through the case of J.L. Partners, where a trip by Dan Jukes to a luxury resort raised concerns about financial influence and transparency. The discussion highlights how polling can be manipulated, with risks of bias, priming, and inequitable sample representation. Experts stress that polling should not be seen as a final truth but as a starting point for understanding public sentiment. The narrative includes real-world failures, such as the flawed Iowa poll that mispredicted the outcome, and underscores the importance of qualitative methods like focus groups to uncover deeper voter nuances. It also notes the growing threat of AI bots and digital sampling biases. While exit polls are praised for their accuracy, the overall message is that polling has significant limitations and must be used with caution, transparency, and critical thinking—especially as political dynamics grow more complex and fragmented. The discussion concludes with a broader reflection on trust, responsibility, and the evolving role of data in democracy.

Transcription

7648 Words, 41433 Characters

English
Hey, man, you hungry? Let's order delivery. Yeah, I was thinking. Italian burgers? Oh! ♪ Game night just got tense ♪ ♪ Two bros on different sides of the fence ♪ ♪ Burgers were pasta tonight ♪ ♪ Why can't we have food without the pie? ♪ With wonder, you can combine 20 restaurants in one delivery. So no one has to settle. Use code MealTime for 50% off your first wonder order, in terms of pricey, wonder.com/newcustomer. ♪ Here's more of Lopez from the All-Moral Lopez Daily Podcast ♪ All right, quick question. You ever notice how some cars just steal the spotlight? Well, that's the 2026 Toyota RAV4. It's got bold standout style, intuitive tech inside, and now hybrid power standard, which means efficiency without the need to plug in. So whether you care about looks, smarts, or just everyday practicality RAV4 brings all of it. You've been holding out for the right SUV. This is your sign. Check out the 2026 Toyota RAV4 at your local Toyota dealer or visit Toyota.com. See packages and options at Toyota.com for feature availability. This is a global production. Nigel Farage's former aide, Dan Jukes, went on an all-expenses paid trip courtesy of the polling company, J.L. Partners. There's only so much distance that Nigel Farage is able to put between himself and Dan Jukes. Jukes was asked, "What does a reformed government want from a polling company?" And he replied, "Showing that we're winning." Well, if J.L. Partners are up to it, our others up to that sort of thing as well. This will raise questions in people's mind. What polling is for, who it's for, and whether we should trust it. I think there is a little bit of informal rule bending that we did with our last one. So, like, when we published those as always, we should have disclosed to the financiers. That is James Johnson from the polling company, J.L. Partners. And it was taken from a channel for documentary two weeks ago, a scoop in which the pollsters admitting to bending the rules to conceal that the reform party were behind the polling that was then splashed all over the newspaper front pages. Reform denied they were involved, but an inquiry is now underway. This is just one company, one pollster. But today, we want to look more widely at polling and public trust. Polling is everywhere. How much does it tell us? And how much does it decide our own behaviour? Welcome to The News Agents. It's John. It's maintenance. And we're going to start today's episode on the Red Sea Resort of L. Casseia. That's nice. December 2025. And the reason we're starting there is that Nigel Farage's former aide, Dan Jukes, went, we now understand from Politico, on an all-expenses paid trip there, courtesy of the polling company, J.L. Partners. This is an internal email that Politico has seen. Now, why does he go? He went to give a talk for the company, which subsequently released opinion polls without then disclosing they were financed by, we now understand, supposed donors to reform. This is part of that channel for the Baton Scoop that came out a couple of weeks ago. And what we now learn from that trip, Jukes and J.L. Partners, was the extent of the business relationship. Somebody with knowledge of the trip said that the company took a limousine from the airport to the hotel, also J.L. Partners' expense. And during that trip, Jukes was asked, what do you want? What does a reform government want from a polling company? And he replied, showing that we're winning. We know what subsequently happened to Jukes. He was essentially chucked out of the party, and what Farage has said is that was nothing to do with reform. That trip was nothing to do with us. It was, you know, in a different capacity. Well, we know that Nigel Farage thinks that Dan Jukes was just a contractor that was working for a form when, in fact, he was the de facto chief of staff to Nigel Farage and has been at his right hand for absolutely years. So, I mean, so much. But I think one of the secondary question it asks is not just about Dan Jukes or even reform, but about the polling company at the centre of this story, Jail Partners, which was pretty well established. It was, I think, owned or half owned by a whole range. James Johnson. to Theresa May. It had come out of sort of, you know, the political establishment, essentially. There was nothing in what they were doing to suggest that they were taking off members of a party and sort of showfaring them by limousine and then saying, "What do you want your polling to show?" And yet, I think inevitably, this will raise questions in people's mind about what polling is for, who it's for, and whether we should trust it as much as we quite often do. I think this is such an interesting and controversial area, all through the 2016 presidential election in America. Anytime there was a poll that was unfavorable to Donald Trump, he would say, "This is a fix. They're working for the Democrats. They're against me. Donald Trump has sued polling companies, you know, in America for producing polls that he felt were unfavorable. And when, I think it was in Iowa. Des Moines. Des Moines registered. The Des Moines registered. There was a very well-respected pollster there. And she was sued by Donald Trump for producing a poll that suggested that the Republicans were going to lose Iowa by substantial margin. By substantial margin. When, in fact, Trump carried it. And so Donald Trump's charge was always, you can't trust polls. They're fake news. They're part of the same. They've got a political agenda just as in Britain. You could say that the Daily Mirror has a political agenda for Labour or the Daily Mail has an agenda for the Tories. And you can't trust what you read. And I have always taken the view that actually polling is really helpful to us. And think how often we refer to polling as a sort of guide in the dark to where political trends might be moving. And I think what JL partners have done is put a light that is deeply uncomfortable on polling. Because you think, well, if JL partners are up to it, our others up to that sort of thing as well. And you've seen that the Daily Telegraph and the Times have said they're not going to be taking JL partners' polls. Which is now being investigated by the Richard Perlin Council. Yes. It's going on. Because what they've done is they've undermined confidence in a really serious way in what we should believe and what trust we should place in polling. Yeah. And today on the news agents, we're going to try and widen that debate. It is not just about reform and it's not just about JL partners. But I guess it does ask about the place that polling has now in our political conversations. Because we're going into the conference season. We will be seeing a lot of polls coming at us suggesting this is how people felt before the Conservative conference is how people felt after the Labour conference. This is where we think we are with reform. This is why we think we, you know, we're going with a Davies leadership of the Lib Dems or whatever it is. And polling has actually undergone some really interesting revolutions recently. Not least because of the way we ourselves communicate. Very few people have landlines anymore. Very few people pick up the phone to strangers anymore. A lot of polling depends on you being quite politically active, sitting down, filling in a form. You know, how many single mothers with three kids are you going to have that suddenly has 20 minutes to talk to a pollster or, you know, wants to go on the computer in the middle of the day and start filling in online forms. So there are a lot of shifts now, which are changing the way that data is received. And we want to talk about this more widely, more generally, with our old friend of the podcast, Luke Trill from Maureen Common. Luke, I know you don't want to talk specifically about the reform question and jail partners, but let's kind of start from the beginning because you're in the business of presumably commissioning polls and listening to parties. And if a party wants polling done, how does that conversation start? Well, normally when a party wants polling done, they will approach you and say, you know, we're interested in finding out about X and most of the time, that polling will be to inform their strategy privately. So most of the polling that political parties are doing is not public because, and because of the British polling council rules, whoever commissions you for a poll, if it's a public poll, if it goes into the public domain, you have to declare that client. And so let's assume the Labour Party Commission depolster, if they sold, you know, sold a poll into a newspaper, they would have to say in the data tables that you have to put up, this was commissioned by the Labour Party. So it's actually quite unusual for political parties to do public polling in the UK. Most of it is polling. and behind the scenes. - And would you ever ask a question like what do you want this polling to show? Would you frame it in a sense that said, if the Labour Party comes to you, what do you want us to show for more polling? Is that a normal thing to do? - No, I think the conversation you have is what are you trying to find out? That is the key question in polling. And I often say there are two types of polls. There are polls to know, which is where you are genuinely interested, in either finding out where the public stand on an issue or actually more often, more interesting message testing, what might shift where the public stands? Lots of people, when they criticize polling, will say, look, are we just taking public as baseline? There's no leadership. Actually, a lot of the polling, particularly private polling, is around what messages might make someone more amenable to rethinking the triple lock. How do I land it in a way which goes down well with voters? So that is a big problem. That's polling to know. On the other hand, you've got polling to show, which is very often, 70% of people agree that. - I don't know, I'm trying to think, we should give out. - Immigration. Immigration, no, not necessarily, but we should give a Cadbury bar to every child in a school when they finish school. It's that kind of motherhood and apple pie stuff that no one can really disagree with. And I guess there is always polling to show, but my thing is, what is that telling you? Actually, I often, when I'm talking to my team. - So you say, you mean nothing? - Yeah, anyway, David, it doesn't really, because the other thing is, when we're talking as a team, we're always thinking about, so let's, I don't know, picks me like a wealth tax. If you poll headline, wealth tax, you get very high support for it. More interesting, I think, in polling is, polar wealth tax, what if it causes some rich people to leave? What does that do to support? You know, putting trade-offs in, and that shows it happens on lots of wealth tax so people still support a wealth tax. And I think that's even more interesting that it's robust to that. And I think the challenge with polling is to try and get into trade-offs, to try and avoid a degree disagree statements as much as possible, because we know there are things like agreement bias. So to give you one example, nationalization should water companies be nationalized? One way of phrasing that question would be, do you think water companies should be nationalized, strongly agree, somewhat agree, neither somewhat disagree, strongly disagree? That is quite a bad way of phrasing that question because of agreement bias. The better way to phrase it is to give a paired statement, which is, which comes closest to your view, water companies should be nationalized, water companies should not be nationalized, because they're forcing people to pick between alternatives rather than ending up with agreement bias. So there's a lot of thought. And actually, you know, our work, a lot of the clients and partners that we work for are in civil society, a lot of the time there's quite a robust exchange between a poster and the organization commissioning them with us saying, you can do that, but it's not gonna tell you anything interesting. - Let's get into the hard stuff about what it means in terms of trade off and how you land this. - Okay, stay with that thought. If you are revealing something to a party that they don't like, take me inside those conversations. Would you ever find the party said, I don't wanna pay for this, or we decide we don't wanna publish it, even if it was going to be a public poll, what happens then? - So, look, I mean, we, because we're a cross party, we brief all parties, we don't actually do paid polling for the political parties. We take our public work and then we go and brief them, but I've had a number of conversations with parties where they've got, are you sure about this, Luke? Are you right because we're decommitted to this? And I guess part of it is, you know, again, you're there to communicate with the parties, you're there to help them reach the public better. And where parties and politics go wrong is where you get a bit of your sticky finger in your ear, where right, you know, this is an accurate representation of the public. And, you know, we're always very frank with that. It is obviously up to, and again, this mostly I think would apply to when you're working with an organization rather than a political party, you know, they will get some polling back and decide that they don't want to publish it. That is up to them. They are the client. But if they publish part of it, and this is really important, this is in the rules, the full data tables have to go up. And one of the rules is that anything in the poll which might contradict a finding that you have put out, you also have to publish. And so that's re-enbrought. So you have to keep it entirely private or anything that's relevant has to go in. So to give you another example, let's say there was an organization campaigning for the two-pan bus cap, which is very popular. If you put that in a poll and they run that, in the same poll, you've got a question which is, I don't want to pay for the two-pan bus cap, if taxes have to rise, you have to publish both of those because it might contradict. You also, there's something else which is called priming, which is, you know, you put a series of questions about climate change first, and then you ask people would they vote for the green party. And because, you know, people would be primed in the poll, so. So not for nefarious reasons or, but just as pollsters, there were so many things you have to think about in terms of, do I randomize the question order to avoid that? What question do I put when? You know, do I put, you know, if it's about Ukraine, do I put that at the top? Or do I put it after questions where you've asked about worries about international security and defense? So there's so much work that goes into good survey design. And then I said, you know, those cross the institute constantly challenging, debating, you know, how you get that right. And the public output to the X percent support, why that you see is only a tiny fraction of what's gone into the thinking there. So do you think that we have become too dependent on polling? Do we, as journalists or as public consumers, depend too much on polling? Because there are so much of it about, but I'm guessing that the data itself is much harder to come by now because we don't have landlines. We don't have as much time. We know exactly who fills in an online polling survey as opposed to somebody who you might reach at sort of three o'clock in the afternoon in their home, right? So how has that changed in terms of how you pull in the material that actually forms the basis for your data? So getting your sample, I think is almost the most important part of polling now. And you're absolutely right, that in the past, you could have relied on landlines, you would call people up, you would reach them, and you know, jobs done. Now you have to be much more reflective or you have to be much more careful about how you construct yourself. So a lot of polling, a lot of the polling that we do users a big online panel provider, we have to be sure that, you know, this is working with the power provider that they are doing proper data checks on the people that they are getting into the poll. We have to make sure that they have proper representation of over 75s, for instance, who might be less likely to use an online panel. We've got a real potential issue, which is already here in the set with AI, because to your question, in order to get people who, you know, very much, most people aren't that interested politics to fill out polls, you have to offer an incentive. So that might be working towards a voucher, some cases to get, you know, reach more middle class respondents who might not be incentivized by that. You do things that Avios or things like that in order to get people to work. - Is that right? - No way to say it, so you're doing, so who gets the voucher, who gets the Avios? - So there are different, you can actually pick on some of the platforms, which one you are going for, but there's obviously clearly different things that have been implemented. - As an MR, there's some, there's other posters do it as an online lottery, so you get entered in and then you pick, it's lots of different ways. - So nobody gives their opinion for free. - There are nobody gives their opinion for, because you don't want that, right? What you don't want is the sort of person who would want to sit and fill out a survey for nothing, because they are by their nature, unusual, politically engaged, and so, and so you've got to think about it. - So the harder you have to work, probably the more useful they are to you, right? - Completely, right, and the risk is, and this is a risk which, you know, the whole industry is taking very seriously is, that therefore it could potentially, I don't think the economics are quite there yet, but it could become quite lucrative to have a series of AI bots filling out polls. So you just unleash them, they're doing it, you know, people often assume they're doing it to try and scoop up poll results, I don't know, for a foreign actor, and that might be a risk, but actually the bigger risk is just that people might end up wanting the incentive, and so we constantly have to think one step ahead of how are we stopping these AI bots from filling out polls? So, you know, one thing which has been used, but isn't enough on its own anymore, is you put one question in white text, and so a human wouldn't fill it in, but an AI bot would fill it in, 'cause it can see it. That's why you're constantly having to see it. - That's the way the young are doing their CVs now. - Yeah. - So they put these subliminal messages. through their CV to get past the answer. Exactly, and so that's one thing, and it may be that we end up weirdly at some point having to regress and go back to more Facebook. A phone calls, face-to-face, there's some innovation we're doing it on apps, on phones and online gaming and getting people at the end of that, but it's why I'm such a big believer in not just relying on the numbers. It's why I think the Qual and the Focus groups are so important, and actually I believe in the the base OS Maxim that when the Quant and the Qual disagree. Just talk to Sue, what do you mean, Quant and Quant? So the Quant is the Poll, so that's the 2000 people, nationally representative survey, goes into the field, comes back, X% want to trade welfare for defense. The Qual is the Focus group, this is where you then what you do is you get eight real people that you're sat around, and if polling can tell us what people think, Focus groups usually tell us why people think it in their own words, and I, you know, look, I'm biased, other people in the industry would disagree, I, as I say, I get a lot more out of the Qual because it's real people, and there are times when you're doing the Qual and you realize, actually, people misinterpreted that question in the poll. They just didn't understand it. They didn't understand it. And actually, so we will sometimes before fielding a survey, do a Focus group first to check people, understand what we mean by it. Give me an example of something people get wrong. So like, I mean, a lot of our work, we as well as looking at sort of political attitudes, we look at some social psychology questions, you know, to try and understand more kind of upstream, what people might think, and when we were designing our segmentation, we had a lot of questions about, you know, things about like, would you rather have a more equal pie or a bigger part, you know, or a bigger pie, even if it wasn't cool? Would you like everyone to get richer, or would you like everyone to be more equal? And actually phrasing a question like that is really important, and you have to use a Focus group to kind of understand, to basically make sure you're testing the right thing. But also, I think Focus groups, you know, will throw up those trade-offs more because you're in a dialogue with someone for 90 minutes, 75 minutes, and you know, we've talked before about that Tony Blair kind of, you know, 30 second, three minute, 30 minute conversation. People who might start off in one place, actually when they consider all the nuances end up some else, or you just find out they're in nuances, they're thinking, so immigration, I think levels of immigration. You're very kind and you say nuances, but, and I speak as a voter here, I would say just massive inconsistencies with, I mean, the way human brains work is random. You know, we slug off a politician and then we admit that we voted for them last time and might do so again, right? And so there's something, I mean, the thing that is deeply human is that we're completely inconsistent. Oh, completely, yeah, we are. And, and inconsistency, I actually think is such a wonderful thing. And you only get that from talking to people and finding out those inconsistencies. And it's quite a weird unusual thing. Basically, people in politics have much more of what you call stacked identities. That is, if I know what you think about climate change, I can probably say what you think about immigration, what you think about criminal justice, what you think about becoming, not always perfectly. Voters are not like that at all. You can have someone who, you know, thinks channel crossings are the biggest issue facing the country. And also thinks the climate crisis is a national, you know, emergency, you know, and vice versa. And again, I think you have to do both the quant and the quality in order to get to that and to get to that nuanced picture. I mean, what you're really saying, I think, is it takes an awful lot of just man hours of time to make any poll actually stack up, right? And we don't necessarily get that. In a moment, we're going to talk about what happens when polling goes wrong. Stay with us. Good haloed by the morning sun, an army hung on her every word. My champions, I have sold my chariot on Carvana to a lovely SUV, an inexplicably queenly offer. They're even coming to the castle to collect it. Tonight, we feast. And offer you can feast on. Sell your car today on Carvana. Pick up these mail. Bye. I want to take you back, Luke, to a moment. I mean, we discuss a lot of sort of polling moments in the way obsessives do, I guess. If I take you back to the US presidential election of 2024, and on November the second, just days before the Trump Kamala Harris presidential election, there was a poll released from Iowa from the Des Moines Register by a very, very well-respected pollster, Anne Celser, who got the 2016 readings of Trump right. She got the 2020 readings of Trump right. She kind of understood that people don't always do what they say. And she released a poll which showed Kamala Harris winning by enough of a margin. I think it was 47% of her poll to to Trump on 44 that put Iowa in the hands of the Democrats. And we all went and it felt like a game changer because once you see it on Iowa, you assume that all the swing states would also follow that pattern. And it started to look at that particular moment. I remember us recording a podcast that that morning as if Harris had the whole thing in the bag. It was not just wrong, it was incredibly wrong. In fact, Iowa was won by Trump on 56%, like a 13% margin. And as it turns out, he won all the other swing states as well. And Anne Celser, curiously, I think I mean, pulled back from pony at that point and has never really been able to explain what went on. I mean, just take us inside the mindset of publishing a poll where, I mean, I want to hear your thoughts on that, but be that sense of you're going out to the public with something that your gut tells you feels wrong, but your data tells you you should say out loud. I mean, I honestly get cold sweats just thinking about that Anne Celser poll and the equivalent happening to us because whilst there have been recent UK polling misses, they tended to at least be in the right direction, although some exceptions we can come on to talk about, but it is that chance you get the data back. And we had to a much more minor degree. We obviously worked with the news agents on RMRPs and they run up to the 2024 election. And we were one of the few posters that still had the Tories getting, you know, sort of 120, 130 seats, I think, in our final. But some others had them behind the Liberal Democrats. And I remember being sat with the team being like, we're outliers here on this, you know, are we confident in the numbers? And of course, you go back and you, particularly with those models, you rerun stuff, you check, you've got all of the factors. You manage the shape to push them closer together or yeah, so I think herding is a really bad thing because they had Vantage, you know, people, some people complain there are too many polling companies. The advantage of having so many polling companies is that you can experiment, you can test, but it is your reputation on the line. There is safety in numbers. What you will do is if you have an output output, you will look at it and you will want to make sure that you have stress tested it. And I think the Anne Celser example is just, I feel so bad for her because I mean, not only in general elections, in presidential elections, she was really good at primaries too, which is really, American primaries, as we've seen this season, are really hard to poll. And then one wrong poll and everything goes wrong. I think one of the specifics, actually, for Anne Celser, is that she is a big fan of what is called just getting a really good sample and leaving it and not touching it. Whereas most pollsters now wait quite aggressively. So there's this, there's this kind of friction, isn't there between the sample, which is like the data and the modelling, which is how you extrapolate. Exactly that. And you know, so, you know, I think all UK pollsters use quite a few weights to make sure, because you know, you won't get enough non-voters very often in polling, you know, in the US education has become, they won't get enough graduates, so you would weight some up. There's a danger to weighting up, though, because what if the, you know, white non-college educated camera supporter is unusual, and you're timesing them by seven in the polls, you've got to have weight caps. Anne Celser is the other way. She's a big fan of just getting a good sample, leave it alone. And I think that was actually the undoing in that particular poll. Amazing. So actually, the raw data was what let her down. Yeah, it was, I think it was shaping that. There was also this really interesting emergence of a polling company called median strategies, which came out in August. And this was about the time of the primaries, the Democratic primaries, and there was one in Wisconsin which had hung winning by a huge amount, and median strategies was one of the companies that was pushing it out there, and everyone was saying, "God, they got it wrong. It was terrible." It turns out that median strategies wasn't a genuine burning company at all. It was a 21-year-old college grad who was basically kind of fucking with us. It was having a go to see what happens. He called it a short-term social experiment designed to test where the bogus polling could enter the political information system without any verification. I mean, the human equivalent of your AI bots and sure enough, it did, right? People just sort of lapped it up. I mean, it was extraordinary. And the one thing I would say is that for any criticisms people might have of polling in the UK, in the US, it is much more of a wild west here because we have the British polling council with whatever limitations it might have. You do have that. You have to have published two polls beforehand. You have to put your data tables up. You have to abide to those rules on things like question order. There is nothing like that in the US. There are some arrangements, but nowhere near to the same degree. And that's the fact that someone could put a fake poll up and it threw it to influence people thinking so much. I think it is really, really extraordinary. I think it is really worrying, and it is, I don't want to sound too pious, but one of the things which I am always conscious of when we put out a poll is we are reflecting public opinion, but also people are going to act on the basis of our polls, right? There is a bit of a, to use the sort of Spider-Man quote, yeah, with great power comes great responsibility, in which is why, you know, and I think I do speak for the vast majority of UK pollsters, where I work for like, we're actually friendly, we get on, we stress test, we challenge each other there, but I think we do also realize like what we put out, we think is important. I think all of us think breaking down echo chambers is important. We've seen what happens when people massively misjudge the public mood, but on the flip side, that responsibility to make sure we are being as accurate as we can and constantly improving, constantly doing better is so important because it has real world consequences. Yeah, I mean, one of the questions I guess is how we all use polling. And I think, you know, I was listening to sort of one US commentator say, this is not about a prediction for the future, it's about, this is my polling tomorrow, right? And so the trouble is that you can treat polling as if it's something that will tell you what's going to happen in the future, which it isn't, which I think for a lot of people would say, well, what's the point then? I guess there's actually an election tomorrow, what was the point of knowing that? I couldn't agree with you more in the sense that what frustrates me, I think lots of criticism of polling comes from treating polling as the end point. Right. I think it's the starting point, right? It is, this is where people are now. If we want, if you're a political party, if we want to appeal to them, if you're a campaigner, if we want to move them, you know, let's take it out of politics, Ali, if you're a business, you know, I want to market to them. What is the state of play now? What is the art of the possible? Where can we go? And there are, you know, politicians should use polling in my view to decide, actually, you know, I'm going to fight that battle that goes against what public opinion is. I might message it slightly differently on this one. I'm never going to win immediately, so I might think about the sequencing and the example I give to go back to a past life is if you look at the education reforms that the Tories brought in in the last parliament, they seem, according to international league tables, to be bearing fruit, certainly the Nick Gibb standard side of it, they poll terribly. Actually, you know, lots of the Gibb Gover gender poll terribly, and yet the government still did it. And I think you get yourself into a very dangerous place if you start using polls as a GPS rather than a marker of where people are. So going back to Superman, do you think that pollsters, do you think you have changed the results of an election? I don't think so. I mean, where I worry is that in the coming election, we've been doing some work looking at tactical voting, and you know, regardless of what comes to pass, I think that tactical voting will play an outsized role in the next election. I think that then puts a premium on pollsters, particularly when you're doing constituency level models, you know, getting it right, because if you put out something which says the stock reform candidate in Exit is the Greens, and in fact, it's Labour, that could allow reforms or the stock Labour candidate is reform or the Tories, getting that wrong in a nearer of tactical voting, could be just on the flip side, without it, voters might be left thinking, okay, I really want to stop Labour, I really want to stop reform. I don't know who the best choice is, so I still think it's net good, but it places an even greater premium on us having confidence in the results. And I mean, the exit polls really, in this country, have been extraordinarily accurate, we should say. You were too modest to say this when you were going through your own wobbles, but actually more in common was pretty much on the money. I mean, certainly with the Tory numbers, you were kind of spot on with the last thing. So the exit poll feels different because you're literally talking to people as they walk out. Yes, that was our MRP, our final MRP, because the exit poll, yeah, it's something different because you're literally, and you know, John Curtis and his team are, you know, hey, they've got this down to a tee, right? You know, they know the individual ballot boxes where they go and survey people and even at the last election in the era of much more multipolitics, you know, they got it pretty close to being spot on, it's been a bit of an over exaggeration of reform, but you know, he and his team do a fantastic job there, and you know, we're lucky to have exit polling, which is so good, because of course, we've seen exit polling lots of other countries where it's been wide of the mark, and what does that then do is it often then creates distrust with the losing side because they were like, oh, well, in the exit poll, we were ahead. So let's just sort of end loop by talking about, I mean, you can call it the social circle or you can talk about the shy Trump voter, the shy toy. This is something that we're sort of very conscious of, but I don't know whether it's become any easier, that people think that they should give a response, or they don't admit, you know, I know in your focus groups, you always start by saying, I don't care what you think, I genuinely don't care what you think, but I need you to tell me the truth. Obviously, when you're not eyeballing somebody, then they can say what they want, and it might be, and I think it was probably definitely in 2016, a lot more people who wouldn't admit to voting for Trump because they found his manner pretty offensive, still quite like to his economics, and so they voted for him anyway. What is, are you any closer to understanding what the way round that is if people are actually too shy to say that loud? Yeah, and he's really difficult. So again, on top of, are you getting the right sample? Are you waiting it correctly? Are people telling you the truth? You've got, again, it's another layer that you've got to get right. I think sometimes there are some interesting sort of experimental ways of doing it. One is saying to people, how are your neighbours voting? Yeah. Or who do you think will win? So Martin Boone at Delta Pose, done some stuff on wisdom of the crowds, which is really interesting, shows that that can often be very prejudiced. So am I more likely to say who I'm voting for inadvertently if I say who I'm voting? Yeah, it seems that way, but there is a problem across polling, and I think we are all conscious of it, and it happened at the last election in 2024, certainly happened in 2015. Yeah, obviously go back to 1992, which led to a big. Or actually go back to Brexit, because that was our sort of Trump equivalent, where there were probably two bubbles going on, and they didn't necessarily mix. A cross, and there was a paper, which has come out on this recently, listed as interesting by amongst others, Will Jennings, who is a fantastic academic, who looks at this stuff, which I'm saying, basically one of the biggest challenges is polls tend to understate the right. Right. And that seems to be pretty consistent. Interestingly, I think that. I mean, that is huge. Yeah. That's massive, right? And if we know that, and it is a pattern, what. And then how do you correct for that? And then. But how do you correct for it? What is the right way? Is it that? You know, for instance, one argument that I think some people made is that in 2024, panels didn't have enough rich Tories on them to go on a bed work, because they were incentivised, the incentives, even the avios weren't enough for. So you don't bother. So you miss that? So you don't pick up your vouchers? Yeah. For other, it might be that you don't have enough non-graduates who are going to vote for parties of the right on, and they're not signing up to panels for. I mean, can I ask you something? If you took it. I mean, there's sort of question that we always sort of gravitate towards. If you made everyone vote. If it was actually compulsory to vote as it is in Australia, would it be a much more even. And sort of, do you think it would make your life easier as poll stars? Undoubtedly, because even harder than working out who people will vote for, is working out who will actually turn out and vote because, again, people overestimate, right? They, everyone wants to see virtuous and so they say, they say they're voting, and then, so then you might do to try and counter that, you ask people, do you normally vote in local elections, which can help, except for reform voters, one of the things we found in last year's local elections was reform voters were the most likely to say that they ordinarily didn't vote, but most likely to say they were going to vote this time. And so as poll stars, you know, I'm going to just, you know, maybe we discount that bit, but actually turned out they were, they were growing the extra. So again, you've got to keep on top and keep innovating and, you know, try and get to, you know, the, the right solution. Luke, chill. Brilliant stuff. Thanks, come in. With wonder, you can combine 20 restaurants in one delivery. Use code meal time for 50% off your first wonder order, terms apply, cwunder.com/newcustomer. So we reported the other week on the elections in Germany, which saw the AFD doing astonishingly well in a former kind of part of East Germany. What was East Germany? Now there's going to be another election this weekend. And the pressure on Chancellor Friedrich Merz seems to be ratcheting up and rumors flying that maybe he's not going to last much longer. Apparently he's cancelled his trip to the UN General Assembly in New York next week, which is where all the world leaders gather. And so it does seem like a particular moment of pressure taking place. There is something a tiny bit starmarish actually about what is now going on in Germany. And Germany is a place where they do not change their genres, you know, with the regularity that you have seen here. But on Sunday, we've got two more states going to the polls. Mecklenburg, again, part of the sort of area around Saxony and Berlin. And Merz has basically said, "I don't think I can afford to be out of the country." You know, we all know things happen. When people are out of the country, things happen. Who's applauded, you know? When African presidents don't leave the country. Yeah, don't get on a plane. And so he's cancelled this plan to speak at the UN General Assembly where really all the world leaders like to be seen. Because his presence, according to a government official, is required in Berlin. What does that mean? Does that mean that people are getting up to replace him because they feel that he's not yet cutting through? You know, we covered the general elections a few years back. And what was clear then was that the AFD was performing astonishingly well, particularly amongst young men, many of the young women were going to the party of the left there. But because of this firewall, the AFD is never going to get through the sort of centrist parties who say, "We'll work together rather than allow them in." But if they think that the centre cannot hold, as in Merz, if they think he's not doing enough to kind of pull people back to the centre, then yeah, maybe things are afoot. So we'll be watching the elections quite carefully on Sunday. And we'll bring you news of them on Monday. Have a good weekend. See you. Bye-bye. Bye for now. Queen Carvani, a stood haloed by the morning sun, an army hung on her every word. "My champions, I have sold my chariot on Carvana to a lovely SUV and inexplicably queenly offer. Tonight, we feast!" And offer you can feast on, sell your car today on Carvana. Pick up fees, male five. This has been a global production.

Podcast Summary

Key Points:

  1. The 2026 Toyota RAV4 is highlighted as a practical, stylish, and efficient SUV with hybrid power standard, appealing to consumers seeking balance in performance, design, and sustainability.
  2. A controversy involving polling company J.L. Partners and Nigel Farage’s former aide Dan Jukes raises serious questions about transparency, bias, and the integrity of political polling, especially when funded by parties or donors.
  3. Polling faces criticism for potential manipulation, including selective disclosure, priming effects, and incentives that skew responses—raising doubts about trustworthiness and public confidence.
  4. Polling is increasingly influenced by digital shifts, such as online panels and AI bot risks, making sample representativeness and data authenticity more challenging than in the past.
  5. Experts emphasize that polling should be a starting point, not a definitive predictor, and that qualitative methods like focus groups provide deeper insight into voter behavior and inconsistencies.
  6. The 2024 U.S. Iowa poll by Anne Celser, which wrongly predicted Kamala Harris’s win, illustrates the dangers of over-relying on polling without sufficient stress-testing or sample scrutiny.
  7. Exit polls are considered more reliable than pre-election forecasts because they collect real-time data from voters, reducing misjudgments and echo chambers.
  8. A key challenge in polling is underrepresentation of certain voter groups—such as conservative or non-graduates—leading to skewed results and a need for methodological innovation to correct for bias.

Summary:

This transcription explores multiple interconnected themes: consumer technology, political polling ethics, and the reliability of public opinion data. com, showcasing a delivery service combining 20 restaurants. L.

Partners, where a trip by Dan Jukes to a luxury resort raised concerns about financial influence and transparency. The discussion highlights how polling can be manipulated, with risks of bias, priming, and inequitable sample representation. Experts stress that polling should not be seen as a final truth but as a starting point for understanding public sentiment.

The narrative includes real-world failures, such as the flawed Iowa poll that mispredicted the outcome, and underscores the importance of qualitative methods like focus groups to uncover deeper voter nuances. It also notes the growing threat of AI bots and digital sampling biases. While exit polls are praised for their accuracy, the overall message is that polling has significant limitations and must be used with caution, transparency, and critical thinking—especially as political dynamics grow more complex and fragmented.

The discussion concludes with a broader reflection on trust, responsibility, and the evolving role of data in democracy.

FAQs

Wonder allows you to combine 20 restaurants into one delivery. Use the code 'MealTime' for 50% off your first order.

The 2026 Toyota RAV4 offers bold style, intuitive tech, and standard hybrid power for efficiency without needing to plug in.

A polling company, J.L. Partners, was found to finance a trip for Dan Jukes, a former aide to Nigel Farage, and released polls without disclosing the funding, raising concerns about bias and transparency.

Polling provides useful insights into public opinion, but results can be misleading due to sampling issues, question phrasing, and biases like agreement bias or priming.

With fewer landlines and less phone-based contact, polling now relies heavily on online panels, requiring careful sample selection and incentives to ensure representativeness.

Polling to know aims to understand public opinion on real issues, while polling to show presents favorable results to support a political or campaign narrative, often lacking depth or nuance.

Chat with AI

Loading...

Pro features

Go deeper with this episode

Unlock creator-grade tools that turn any transcript into show notes and subtitle files.