After the Reform scandal - does anyone still trust pollsters?
41m 54s
This transcription explores multiple interconnected themes: consumer technology, political polling ethics, and the reliability of public opinion data. It begins with a promotional segment for wonder.com, showcasing a delivery service combining 20 restaurants. The core focus shifts to a critical examination of political polling, particularly through the case of J.L. Partners, where a trip by Dan Jukes to a luxury resort raised concerns about financial influence and transparency. The discussion highlights how polling can be manipulated, with risks of bias, priming, and inequitable sample representation. Experts stress that polling should not be seen as a final truth but as a starting point for understanding public sentiment. The narrative includes real-world failures, such as the flawed Iowa poll that mispredicted the outcome, and underscores the importance of qualitative methods like focus groups to uncover deeper voter nuances. It also notes the growing threat of AI bots and digital sampling biases. While exit polls are praised for their accuracy, the overall message is that polling has significant limitations and must be used with caution, transparency, and critical thinking—especially as political dynamics grow more complex and fragmented. The discussion concludes with a broader reflection on trust, responsibility, and the evolving role of data in democracy.
Hey, man, you hungry? Let's order delivery.
Yeah, I was thinking. Italian burgers?
Oh!
♪ Game night just got tense ♪
♪ Two bros on different sides of the fence ♪
♪ Burgers were pasta tonight ♪
♪ Why can't we have food without the pie? ♪
With wonder, you can combine 20 restaurants in one delivery.
So no one has to settle.
Use code MealTime for 50% off your first wonder order,
in terms of pricey, wonder.com/newcustomer.
♪ Here's more of Lopez from the All-Moral Lopez Daily Podcast ♪
All right, quick question.
You ever notice how some cars just steal the spotlight?
Well, that's the 2026 Toyota RAV4.
It's got bold standout style, intuitive tech inside,
and now hybrid power standard, which means efficiency
without the need to plug in.
So whether you care about looks, smarts,
or just everyday practicality RAV4 brings all of it.
You've been holding out for the right SUV.
This is your sign.
Check out the 2026 Toyota RAV4 at your local Toyota dealer
or visit Toyota.com.
See packages and options at Toyota.com for feature availability.
This is a global production.
Nigel Farage's former aide, Dan Jukes, went on an all-expenses
paid trip courtesy of the polling company, J.L. Partners.
There's only so much distance that Nigel Farage is able to put
between himself and Dan Jukes.
Jukes was asked, "What does a reformed government want
from a polling company?"
And he replied, "Showing that we're winning."
Well, if J.L. Partners are up to it,
our others up to that sort of thing as well.
This will raise questions in people's mind.
What polling is for, who it's for,
and whether we should trust it.
I think there is a little bit of informal rule bending
that we did with our last one.
So, like, when we published those as always,
we should have disclosed to the financiers.
That is James Johnson from the polling company, J.L. Partners.
And it was taken from a channel for documentary two weeks ago,
a scoop in which the pollsters admitting to bending the rules
to conceal that the reform party were behind the polling
that was then splashed all over the newspaper front pages.
Reform denied they were involved,
but an inquiry is now underway.
This is just one company, one pollster.
But today, we want to look more widely at polling and public trust.
Polling is everywhere.
How much does it tell us?
And how much does it decide our own behaviour?
Welcome to The News Agents.
It's John.
It's maintenance.
And we're going to start today's episode
on the Red Sea Resort of L. Casseia.
That's nice.
December 2025.
And the reason we're starting there
is that Nigel Farage's former aide, Dan Jukes,
went, we now understand from Politico,
on an all-expenses paid trip there,
courtesy of the polling company, J.L. Partners.
This is an internal email that Politico has seen.
Now, why does he go?
He went to give a talk for the company,
which subsequently released opinion polls
without then disclosing they were financed by,
we now understand, supposed donors to reform.
This is part of that channel for the Baton Scoop
that came out a couple of weeks ago.
And what we now learn from that trip,
Jukes and J.L. Partners,
was the extent of the business relationship.
Somebody with knowledge of the trip
said that the company took a limousine
from the airport to the hotel,
also J.L. Partners' expense.
And during that trip, Jukes was asked,
what do you want?
What does a reform government want from a polling company?
And he replied, showing that we're winning.
We know what subsequently happened to Jukes.
He was essentially chucked out of the party,
and what Farage has said is that was nothing to do with reform.
That trip was nothing to do with us.
It was, you know, in a different capacity.
Well, we know that Nigel Farage thinks
that Dan Jukes was just a contractor
that was working for a form when, in fact,
he was the de facto chief of staff to Nigel Farage
and has been at his right hand for absolutely years.
So, I mean, so much. But I think one of the secondary question it asks
is not just about Dan Jukes or even reform,
but about the polling company at the centre of this story,
Jail Partners, which was pretty well established.
It was, I think, owned or half owned by a whole range. James Johnson. to Theresa May.
It had come out of sort of, you know,
the political establishment, essentially.
There was nothing in what they were doing to suggest
that they were taking off members of a party
and sort of showfaring them by limousine
and then saying, "What do you want your polling to show?"
And yet, I think inevitably, this will raise questions
in people's mind about what polling is for,
who it's for, and whether we should trust it
as much as we quite often do.
I think this is such an interesting and controversial area,
all through the 2016 presidential election in America.
Anytime there was a poll that was unfavorable to Donald Trump,
he would say, "This is a fix.
They're working for the Democrats. They're against me.
Donald Trump has sued polling companies, you know,
in America for producing polls that he felt were unfavorable.
And when, I think it was in Iowa.
Des Moines.
Des Moines registered.
The Des Moines registered.
There was a very well-respected pollster there.
And she was sued by Donald Trump for producing a poll
that suggested that the Republicans were going to lose Iowa
by substantial margin.
By substantial margin.
When, in fact, Trump carried it.
And so Donald Trump's charge was always,
you can't trust polls. They're fake news.
They're part of the same.
They've got a political agenda just as in Britain.
You could say that the Daily Mirror has a political agenda
for Labour or the Daily Mail has an agenda for the Tories.
And you can't trust what you read.
And I have always taken the view that actually polling
is really helpful to us.
And think how often we refer to polling as a sort of guide
in the dark to where political trends might be moving.
And I think what JL partners have done is put a light
that is deeply uncomfortable on polling.
Because you think, well, if JL partners are up to it,
our others up to that sort of thing as well.
And you've seen that the Daily Telegraph and the Times
have said they're not going to be taking JL partners' polls.
Which is now being investigated by the Richard Perlin Council.
Yes.
It's going on. Because what they've done is they've undermined confidence
in a really serious way in what we should believe
and what trust we should place in polling.
Yeah. And today on the news agents,
we're going to try and widen that debate.
It is not just about reform and it's not just about JL partners.
But I guess it does ask about the place that polling has now
in our political conversations.
Because we're going into the conference season.
We will be seeing a lot of polls coming at us suggesting
this is how people felt before the Conservative conference
is how people felt after the Labour conference.
This is where we think we are with reform.
This is why we think we, you know,
we're going with a Davies leadership of the Lib Dems
or whatever it is.
And polling has actually undergone
some really interesting revolutions recently.
Not least because of the way we ourselves communicate.
Very few people have landlines anymore.
Very few people pick up the phone to strangers anymore.
A lot of polling depends on you being quite politically active,
sitting down, filling in a form.
You know, how many single mothers with three kids
are you going to have that suddenly has 20 minutes to talk to a pollster
or, you know, wants to go on the computer in the middle of the day
and start filling in online forms.
So there are a lot of shifts now,
which are changing the way that data is received.
And we want to talk about this more widely,
more generally, with our old friend of the podcast,
Luke Trill from Maureen Common.
Luke, I know you don't want to talk specifically about
the reform question and jail partners,
but let's kind of start from the beginning
because you're in the business of presumably commissioning polls
and listening to parties.
And if a party wants polling done,
how does that conversation start?
Well, normally when a party wants polling done,
they will approach you and say, you know,
we're interested in finding out about X
and most of the time,
that polling will be to inform their strategy privately.
So most of the polling that political parties are doing
is not public because,
and because of the British polling council rules,
whoever commissions you for a poll,
if it's a public poll,
if it goes into the public domain,
you have to declare that client.
And so let's assume the Labour Party Commission depolster,
if they sold, you know,
sold a poll into a newspaper,
they would have to say in the data tables
that you have to put up,
this was commissioned by the Labour Party.
So it's actually quite unusual for political parties
to do public polling in the UK.
Most of it is polling.
and behind the scenes.
- And would you ever ask a question
like what do you want this polling to show?
Would you frame it in a sense that said,
if the Labour Party comes to you,
what do you want us to show for more polling?
Is that a normal thing to do?
- No, I think the conversation you have is
what are you trying to find out?
That is the key question in polling.
And I often say there are two types of polls.
There are polls to know,
which is where you are genuinely interested,
in either finding out where the public stand on an issue
or actually more often, more interesting message testing,
what might shift where the public stands?
Lots of people, when they criticize polling,
will say, look, are we just taking public as baseline?
There's no leadership.
Actually, a lot of the polling, particularly private polling,
is around what messages might make someone
more amenable to rethinking the triple lock.
How do I land it in a way which goes down well with voters?
So that is a big problem.
That's polling to know.
On the other hand, you've got polling to show,
which is very often,
70% of people agree that. - I don't know, I'm trying to think,
we should give out. - Immigration.
Immigration, no, not necessarily,
but we should give a Cadbury bar to every child
in a school when they finish school.
It's that kind of motherhood and apple pie stuff
that no one can really disagree with.
And I guess there is always polling to show,
but my thing is, what is that telling you?
Actually, I often, when I'm talking to my team. - So you say, you mean nothing?
- Yeah, anyway, David, it doesn't really,
because the other thing is, when we're talking as a team,
we're always thinking about,
so let's, I don't know, picks me like a wealth tax.
If you poll headline, wealth tax,
you get very high support for it.
More interesting, I think, in polling is, polar wealth tax,
what if it causes some rich people to leave?
What does that do to support?
You know, putting trade-offs in,
and that shows it happens on lots of wealth tax
so people still support a wealth tax.
And I think that's even more interesting
that it's robust to that.
And I think the challenge with polling
is to try and get into trade-offs,
to try and avoid a degree disagree statements
as much as possible,
because we know there are things like agreement bias.
So to give you one example,
nationalization should water companies be nationalized?
One way of phrasing that question would be,
do you think water companies should be nationalized,
strongly agree, somewhat agree,
neither somewhat disagree, strongly disagree?
That is quite a bad way of phrasing that question
because of agreement bias.
The better way to phrase it is to give a paired statement,
which is, which comes closest to your view,
water companies should be nationalized,
water companies should not be nationalized,
because they're forcing people to pick between alternatives
rather than ending up with agreement bias.
So there's a lot of thought.
And actually, you know, our work,
a lot of the clients and partners
that we work for are in civil society,
a lot of the time there's quite a robust exchange
between a poster and the organization commissioning them
with us saying, you can do that,
but it's not gonna tell you anything interesting.
- Let's get into the hard stuff about what it means
in terms of trade off and how you land this.
- Okay, stay with that thought.
If you are revealing something to a party
that they don't like,
take me inside those conversations.
Would you ever find the party said,
I don't wanna pay for this,
or we decide we don't wanna publish it,
even if it was going to be a public poll,
what happens then?
- So, look, I mean, we,
because we're a cross party, we brief all parties,
we don't actually do paid polling for the political parties.
We take our public work and then we go and brief them,
but I've had a number of conversations with parties
where they've got, are you sure about this, Luke?
Are you right because we're decommitted to this?
And I guess part of it is, you know, again,
you're there to communicate with the parties,
you're there to help them reach the public better.
And where parties and politics go wrong
is where you get a bit of your sticky finger in your ear,
where right, you know,
this is an accurate representation of the public.
And, you know, we're always very frank with that.
It is obviously up to, and again,
this mostly I think would apply to when you're working
with an organization rather than a political party,
you know, they will get some polling back
and decide that they don't want to publish it.
That is up to them.
They are the client.
But if they publish part of it,
and this is really important, this is in the rules,
the full data tables have to go up.
And one of the rules is that anything in the poll
which might contradict a finding that you have put out,
you also have to publish.
And so that's re-enbrought.
So you have to keep it entirely private
or anything that's relevant has to go in.
So to give you another example,
let's say there was an organization campaigning
for the two-pan bus cap, which is very popular.
If you put that in a poll and they run that,
in the same poll, you've got a question which is,
I don't want to pay for the two-pan bus cap,
if taxes have to rise, you have to publish both of those
because it might contradict.
You also, there's something else which is called priming,
which is, you know, you put a series of questions
about climate change first,
and then you ask people would they vote for the green party.
And because, you know, people would be primed
in the poll, so.
So not for nefarious reasons or, but just as pollsters,
there were so many things you have to think about
in terms of, do I randomize the question order
to avoid that?
What question do I put when?
You know, do I put, you know, if it's about Ukraine,
do I put that at the top?
Or do I put it after questions where you've asked about
worries about international security and defense?
So there's so much work that goes into good survey design.
And then I said, you know,
those cross the institute constantly challenging, debating,
you know, how you get that right.
And the public output to the X percent support,
why that you see is only a tiny fraction
of what's gone into the thinking there.
So do you think that we have become too dependent on polling?
Do we, as journalists or as public consumers,
depend too much on polling?
Because there are so much of it about,
but I'm guessing that the data itself
is much harder to come by now
because we don't have landlines.
We don't have as much time.
We know exactly who fills in an online polling survey
as opposed to somebody who you might reach
at sort of three o'clock in the afternoon in their home, right?
So how has that changed in terms of how you pull in the material
that actually forms the basis for your data?
So getting your sample, I think is almost the most important part
of polling now.
And you're absolutely right, that in the past,
you could have relied on landlines, you would call people up,
you would reach them, and you know, jobs done.
Now you have to be much more reflective
or you have to be much more careful about how you construct yourself.
So a lot of polling, a lot of the polling that we do
users a big online panel provider,
we have to be sure that, you know,
this is working with the power provider
that they are doing proper data checks
on the people that they are getting into the poll.
We have to make sure that they have proper representation
of over 75s, for instance, who might be less likely
to use an online panel.
We've got a real potential issue, which is already here
in the set with AI, because to your question,
in order to get people who, you know,
very much, most people aren't that interested politics
to fill out polls, you have to offer an incentive.
So that might be working towards a voucher,
some cases to get, you know,
reach more middle class respondents
who might not be incentivized by that.
You do things that Avios or things like that
in order to get people to work.
- Is that right?
- No way to say it, so you're doing, so who gets the voucher,
who gets the Avios?
- So there are different, you can actually pick
on some of the platforms, which one you are going for,
but there's obviously clearly different things
that have been implemented. - As an MR,
there's some, there's other posters do it
as an online lottery, so you get entered in
and then you pick, it's lots of different ways.
- So nobody gives their opinion for free.
- There are nobody gives their opinion for,
because you don't want that, right?
What you don't want is the sort of person
who would want to sit and fill out a survey for nothing,
because they are by their nature,
unusual, politically engaged, and so,
and so you've got to think about it.
- So the harder you have to work,
probably the more useful they are to you, right?
- Completely, right, and the risk is,
and this is a risk which, you know,
the whole industry is taking very seriously is,
that therefore it could potentially,
I don't think the economics are quite there yet,
but it could become quite lucrative
to have a series of AI bots filling out polls.
So you just unleash them, they're doing it,
you know, people often assume they're doing it
to try and scoop up poll results,
I don't know, for a foreign actor,
and that might be a risk,
but actually the bigger risk is just that people
might end up wanting the incentive,
and so we constantly have to think one step ahead
of how are we stopping these AI bots from filling out polls?
So, you know, one thing which has been used,
but isn't enough on its own anymore,
is you put one question in white text,
and so a human wouldn't fill it in,
but an AI bot would fill it in, 'cause it can see it.
That's why you're constantly having to see it.
- That's the way the young are doing their CVs now.
- Yeah.
- So they put these subliminal messages.
through their CV to get past the answer. Exactly, and so that's one thing, and it may be that we end up
weirdly at some point having to regress and go back to more Facebook. A phone calls, face-to-face,
there's some innovation we're doing it on apps, on phones and online gaming and getting people
at the end of that, but it's why I'm such a big believer in not just relying on the numbers.
It's why I think the Qual and the Focus groups are so important, and actually I believe in the
the base OS Maxim that when the Quant and the Qual disagree. Just talk to Sue, what do you mean,
Quant and Quant? So the Quant is the Poll, so that's the 2000 people, nationally representative survey,
goes into the field, comes back, X% want to trade welfare for defense. The Qual is the Focus group,
this is where you then what you do is you get eight real people that you're sat around,
and if polling can tell us what people think, Focus groups usually tell us why people think it
in their own words, and I, you know, look, I'm biased, other people in the industry would disagree,
I, as I say, I get a lot more out of the Qual because it's real people, and there are times
when you're doing the Qual and you realize, actually, people misinterpreted that question in the poll.
They just didn't understand it. They didn't understand it. And actually, so we will sometimes before
fielding a survey, do a Focus group first to check people, understand what we mean by it. Give me
an example of something people get wrong. So like, I mean, a lot of our work, we as well as
looking at sort of political attitudes, we look at some social psychology questions, you know,
to try and understand more kind of upstream, what people might think, and when we were designing
our segmentation, we had a lot of questions about, you know, things about like, would you rather
have a more equal pie or a bigger part, you know, or a bigger pie, even if it wasn't cool?
Would you like everyone to get richer, or would you like everyone to be more equal?
And actually phrasing a question like that is really important, and you have to use a Focus
group to kind of understand, to basically make sure you're testing the right thing. But also,
I think Focus groups, you know, will throw up those trade-offs more because you're in a dialogue
with someone for 90 minutes, 75 minutes, and you know, we've talked before about that Tony Blair
kind of, you know, 30 second, three minute, 30 minute conversation. People who might start off
in one place, actually when they consider all the nuances end up some else, or you just find out
they're in nuances, they're thinking, so immigration, I think levels of immigration. You're very
kind and you say nuances, but, and I speak as a voter here, I would say just massive inconsistencies
with, I mean, the way human brains work is random. You know, we slug off a politician and then we
admit that we voted for them last time and might do so again, right? And so there's something,
I mean, the thing that is deeply human is that we're completely inconsistent.
Oh, completely, yeah, we are. And, and inconsistency, I actually think is such a wonderful thing.
And you only get that from talking to people and finding out those inconsistencies. And it's
quite a weird unusual thing. Basically, people in politics have much more of what you call stacked
identities. That is, if I know what you think about climate change, I can probably say what you
think about immigration, what you think about criminal justice, what you think about becoming, not
always perfectly. Voters are not like that at all. You can have someone who, you know, thinks
channel crossings are the biggest issue facing the country. And also thinks the climate crisis is
a national, you know, emergency, you know, and vice versa. And again, I think you have to do both
the quant and the quality in order to get to that and to get to that nuanced picture.
I mean, what you're really saying, I think, is it takes an awful lot of just man hours of time
to make any poll actually stack up, right? And we don't necessarily get that. In a moment,
we're going to talk about what happens when polling goes wrong. Stay with us.
Good haloed by the morning sun, an army hung on her every word. My champions, I have sold my
chariot on Carvana to a lovely SUV, an inexplicably queenly offer. They're even coming to the castle
to collect it. Tonight, we feast. And offer you can feast on. Sell your car today on Carvana.
Pick up these mail. Bye. I want to take you back, Luke, to a moment. I mean, we discuss a lot of
sort of polling moments in the way obsessives do, I guess. If I take you back to the US presidential
election of 2024, and on November the second, just days before the Trump Kamala Harris presidential
election, there was a poll released from Iowa from the Des Moines Register by a very, very well-respected
pollster, Anne Celser, who got the 2016 readings of Trump right. She got the 2020 readings of
Trump right. She kind of understood that people don't always do what they say. And she released a
poll which showed Kamala Harris winning by enough of a margin. I think it was 47% of her poll to
to Trump on 44 that put Iowa in the hands of the Democrats. And we all went and it felt like a
game changer because once you see it on Iowa, you assume that all the swing states would also
follow that pattern. And it started to look at that particular moment. I remember us recording
a podcast that that morning as if Harris had the whole thing in the bag. It was not just wrong,
it was incredibly wrong. In fact, Iowa was won by Trump on 56%, like a 13% margin. And as it turns
out, he won all the other swing states as well. And Anne Celser, curiously, I think I mean,
pulled back from pony at that point and has never really been able to explain what went on.
I mean, just take us inside the mindset of publishing a poll where, I mean, I want to hear your
thoughts on that, but be that sense of you're going out to the public with something that your gut tells
you feels wrong, but your data tells you you should say out loud. I mean, I honestly get cold
sweats just thinking about that Anne Celser poll and the equivalent happening to us because whilst
there have been recent UK polling misses, they tended to at least be in the right direction,
although some exceptions we can come on to talk about, but it is that chance you get the data back.
And we had to a much more minor degree. We obviously worked with the news agents on RMRPs and
they run up to the 2024 election. And we were one of the few posters that still had the Tories
getting, you know, sort of 120, 130 seats, I think, in our final. But some others had them
behind the Liberal Democrats. And I remember being sat with the team being like, we're outliers here
on this, you know, are we confident in the numbers? And of course, you go back and you,
particularly with those models, you rerun stuff, you check, you've got all of the factors.
You manage the shape to push them closer together or yeah, so I think herding is a really bad
thing because they had Vantage, you know, people, some people complain there are too many polling
companies. The advantage of having so many polling companies is that you can experiment, you can
test, but it is your reputation on the line. There is safety in numbers. What you will do is if you
have an output output, you will look at it and you will want to make sure that you have stress
tested it. And I think the Anne Celser example is just, I feel so bad for her because I mean, not
only in general elections, in presidential elections, she was really good at primaries too,
which is really, American primaries, as we've seen this season, are really hard to poll. And then
one wrong poll and everything goes wrong. I think one of the specifics, actually, for Anne Celser,
is that she is a big fan of what is called just getting a really good sample and leaving it
and not touching it. Whereas most pollsters now wait quite aggressively. So there's this, there's
this kind of friction, isn't there between the sample, which is like the data and the modelling,
which is how you extrapolate. Exactly that. And you know, so, you know, I think all UK pollsters use
quite a few weights to make sure, because you know, you won't get enough non-voters very often
in polling, you know, in the US education has become, they won't get enough graduates, so you would
weight some up. There's a danger to weighting up, though, because what if the, you know, white
non-college educated camera supporter is unusual, and you're timesing them by seven in the polls,
you've got to have weight caps. Anne Celser is the other way. She's a big fan of just getting a
good sample, leave it alone. And I think that was actually the undoing in that particular poll.
Amazing. So actually, the raw data was what let her down. Yeah, it was, I think it was shaping
that. There was also this really interesting emergence of a polling company called median strategies,
which came out in August.
And this was about the time of the primaries, the Democratic primaries, and there was one
in Wisconsin which had hung winning by a huge amount, and median strategies was one of
the companies that was pushing it out there, and everyone was saying, "God, they got
it wrong.
It was terrible."
It turns out that median strategies wasn't a genuine burning company at all.
It was a 21-year-old college grad who was basically kind of fucking with us.
It was having a go to see what happens.
He called it a short-term social experiment designed to test where the bogus polling could
enter the political information system without any verification.
I mean, the human equivalent of your AI bots and sure enough, it did, right?
People just sort of lapped it up.
I mean, it was extraordinary.
And the one thing I would say is that for any criticisms people might have of polling
in the UK, in the US, it is much more of a wild west here because we have the British
polling council with whatever limitations it might have.
You do have that.
You have to have published two polls beforehand.
You have to put your data tables up.
You have to abide to those rules on things like question order.
There is nothing like that in the US.
There are some arrangements, but nowhere near to the same degree.
And that's the fact that someone could put a fake poll up and it threw it to influence
people thinking so much.
I think it is really, really extraordinary.
I think it is really worrying, and it is, I don't want to sound too pious, but one
of the things which I am always conscious of when we put out a poll is we are reflecting
public opinion, but also people are going to act on the basis of our polls, right?
There is a bit of a, to use the sort of Spider-Man quote, yeah, with great power comes great
responsibility, in which is why, you know, and I think I do speak for the vast majority
of UK pollsters, where I work for like, we're actually friendly, we get on, we stress test,
we challenge each other there, but I think we do also realize like what we put out, we
think is important.
I think all of us think breaking down echo chambers is important.
We've seen what happens when people massively misjudge the public mood, but on the flip side,
that responsibility to make sure we are being as accurate as we can and constantly improving,
constantly doing better is so important because it has real world consequences.
Yeah, I mean, one of the questions I guess is how we all use polling.
And I think, you know, I was listening to sort of one US commentator say, this is not
about a prediction for the future, it's about, this is my polling tomorrow, right?
And so the trouble is that you can treat polling as if it's something that will tell you
what's going to happen in the future, which it isn't, which I think for a lot of people
would say, well, what's the point then?
I guess there's actually an election tomorrow, what was the point of knowing that?
I couldn't agree with you more in the sense that what frustrates me, I think lots of criticism
of polling comes from treating polling as the end point.
Right.
I think it's the starting point, right?
It is, this is where people are now.
If we want, if you're a political party, if we want to appeal to them, if you're a
campaigner, if we want to move them, you know, let's take it out of politics, Ali,
if you're a business, you know, I want to market to them.
What is the state of play now?
What is the art of the possible?
Where can we go?
And there are, you know, politicians should use polling in my view to decide, actually, you
know, I'm going to fight that battle that goes against what public opinion is.
I might message it slightly differently on this one.
I'm never going to win immediately, so I might think about the sequencing and the example
I give to go back to a past life is if you look at the education reforms that the Tories
brought in in the last parliament, they seem, according to international league tables,
to be bearing fruit, certainly the Nick Gibb standard side of it, they poll terribly.
Actually, you know, lots of the Gibb Gover gender poll terribly, and yet the government
still did it.
And I think you get yourself into a very dangerous place if you start using polls as a GPS
rather than a marker of where people are.
So going back to Superman, do you think that pollsters, do you think you have changed
the results of an election?
I don't think so.
I mean, where I worry is that in the coming election, we've been doing some work looking
at tactical voting, and you know, regardless of what comes to pass, I think that tactical
voting will play an outsized role in the next election.
I think that then puts a premium on pollsters, particularly when you're doing constituency
level models, you know, getting it right, because if you put out something which says
the stock reform candidate in Exit is the Greens, and in fact, it's Labour, that could allow
reforms or the stock Labour candidate is reform or the Tories, getting that wrong in a
nearer of tactical voting, could be just on the flip side, without it, voters might be
left thinking, okay, I really want to stop Labour, I really want to stop reform.
I don't know who the best choice is, so I still think it's net good, but it places an
even greater premium on us having confidence in the results.
And I mean, the exit polls really, in this country, have been extraordinarily accurate,
we should say.
You were too modest to say this when you were going through your own wobbles, but actually
more in common was pretty much on the money.
I mean, certainly with the Tory numbers, you were kind of spot on with the last thing.
So the exit poll feels different because you're literally talking to people as they walk
out.
Yes, that was our MRP, our final MRP, because the exit poll, yeah, it's something different
because you're literally, and you know, John Curtis and his team are, you know, hey, they've
got this down to a tee, right? You know, they know the individual ballot boxes where they
go and survey people and even at the last election in the era of much more multipolitics,
you know, they got it pretty close to being spot on, it's been a bit of an over exaggeration
of reform, but you know, he and his team do a fantastic job there, and you know, we're
lucky to have exit polling, which is so good, because of course, we've seen exit polling
lots of other countries where it's been wide of the mark, and what does that then do
is it often then creates distrust with the losing side because they were like, oh, well,
in the exit poll, we were ahead.
So let's just sort of end loop by talking about, I mean, you can call it the social circle
or you can talk about the shy Trump voter, the shy toy.
This is something that we're sort of very conscious of, but I don't know whether it's
become any easier, that people think that they should give a response, or they don't admit,
you know, I know in your focus groups, you always start by saying, I don't care what
you think, I genuinely don't care what you think, but I need you to tell me the truth.
Obviously, when you're not eyeballing somebody, then they can say what they want, and it
might be, and I think it was probably definitely in 2016, a lot more people who wouldn't admit
to voting for Trump because they found his manner pretty offensive, still quite like
to his economics, and so they voted for him anyway.
What is, are you any closer to understanding what the way round that is if people are actually
too shy to say that loud?
Yeah, and he's really difficult.
So again, on top of, are you getting the right sample?
Are you waiting it correctly?
Are people telling you the truth?
You've got, again, it's another layer that you've got to get right.
I think sometimes there are some interesting sort of experimental ways of doing it.
One is saying to people, how are your neighbours voting?
Yeah.
Or who do you think will win?
So Martin Boone at Delta Pose, done some stuff on wisdom of the crowds, which is really
interesting, shows that that can often be very prejudiced.
So am I more likely to say who I'm voting for inadvertently if I say who I'm voting?
Yeah, it seems that way, but there is a problem across polling, and I think we are all conscious
of it, and it happened at the last election in 2024, certainly happened in 2015.
Yeah, obviously go back to 1992, which led to a big. Or actually go back to Brexit, because that was our sort of Trump equivalent, where there
were probably two bubbles going on, and they didn't necessarily mix.
A cross, and there was a paper, which has come out on this recently, listed as interesting
by amongst others, Will Jennings, who is a fantastic academic, who looks at this stuff,
which I'm saying, basically one of the biggest challenges is polls tend to understate
the right.
Right.
And that seems to be pretty consistent.
Interestingly, I think that. I mean, that is huge.
Yeah.
That's massive, right?
And if we know that, and it is a pattern, what. And then how do you correct for that?
And then. But how do you correct for it?
What is the right way?
Is it that?
You know, for instance, one argument that I think some people made is that in 2024, panels
didn't have enough rich Tories on them to go on a bed work, because they were incentivised,
the incentives, even the avios weren't enough for. So you don't bother. So you miss that?
So you don't pick up your vouchers?
Yeah.
For other, it might be that you don't have enough non-graduates who are going to vote
for parties of the right on, and they're not signing up to panels for. I mean, can I ask you something?
If you took it. I mean, there's sort of question that we always sort of gravitate towards.
If you made everyone vote. If it was actually compulsory to vote as it is in Australia, would it be a much more
even.
And sort of, do you think it would make your life easier as poll stars?
Undoubtedly, because even harder than working out who people will vote for, is working
out who will actually turn out and vote because, again, people overestimate, right?
They, everyone wants to see virtuous and so they say, they say they're voting, and then,
so then you might do to try and counter that, you ask people, do you normally vote in
local elections, which can help, except for reform voters, one of the things we found
in last year's local elections was reform voters were the most likely to say that they
ordinarily didn't vote, but most likely to say they were going to vote this time.
And so as poll stars, you know, I'm going to just, you know, maybe we discount that bit,
but actually turned out they were, they were growing the extra. So again, you've got to keep
on top and keep innovating and, you know, try and get to, you know, the, the right solution.
Luke, chill. Brilliant stuff. Thanks, come in.
With wonder, you can combine 20 restaurants in one delivery. Use code meal time for 50% off your first wonder order, terms apply, cwunder.com/newcustomer.
So we reported the other week on the elections in Germany, which saw the AFD doing astonishingly
well in a former kind of part of East Germany. What was East Germany? Now there's going to be another
election this weekend. And the pressure on Chancellor Friedrich Merz seems to be ratcheting up
and rumors flying that maybe he's not going to last much longer. Apparently he's cancelled his trip
to the UN General Assembly in New York next week, which is where all the world leaders gather.
And so it does seem like a particular moment of pressure taking place.
There is something a tiny bit starmarish actually about what is now going on in Germany.
And Germany is a place where they do not change their genres, you know, with the regularity
that you have seen here. But on Sunday, we've got two more states going to the polls.
Mecklenburg, again, part of the sort of area around Saxony and Berlin. And Merz has basically said,
"I don't think I can afford to be out of the country." You know, we all know things happen.
When people are out of the country, things happen. Who's applauded, you know?
When African presidents don't leave the country. Yeah, don't get on a plane.
And so he's cancelled this plan to speak at the UN General Assembly where really all the
world leaders like to be seen. Because his presence, according to a government official,
is required in Berlin. What does that mean? Does that mean that people are getting up to replace him
because they feel that he's not yet cutting through? You know, we covered the general elections
a few years back. And what was clear then was that the AFD was performing astonishingly well,
particularly amongst young men, many of the young women were going to the party of the left there.
But because of this firewall, the AFD is never going to get through the sort of centrist parties
who say, "We'll work together rather than allow them in." But if they think that the centre
cannot hold, as in Merz, if they think he's not doing enough to kind of pull people back to the
centre, then yeah, maybe things are afoot. So we'll be watching the elections quite carefully
on Sunday. And we'll bring you news of them on Monday. Have a good weekend. See you. Bye-bye.
Bye for now. Queen Carvani, a stood haloed by the morning sun, an army hung on her every word.
"My champions, I have sold my chariot on Carvana to a lovely SUV and inexplicably queenly offer.
Tonight, we feast!"
And offer you can feast on, sell your car today on Carvana. Pick up fees, male five.
This has been a global production.
Podcast Summary
Key Points:
The 2026 Toyota RAV4 is highlighted as a practical, stylish, and efficient SUV with hybrid power standard, appealing to consumers seeking balance in performance, design, and sustainability.
A controversy involving polling company J.L. Partners and Nigel Farage’s former aide Dan Jukes raises serious questions about transparency, bias, and the integrity of political polling, especially when funded by parties or donors.
Polling faces criticism for potential manipulation, including selective disclosure, priming effects, and incentives that skew responses—raising doubts about trustworthiness and public confidence.
Polling is increasingly influenced by digital shifts, such as online panels and AI bot risks, making sample representativeness and data authenticity more challenging than in the past.
Experts emphasize that polling should be a starting point, not a definitive predictor, and that qualitative methods like focus groups provide deeper insight into voter behavior and inconsistencies.
The 2024 U.S. Iowa poll by Anne Celser, which wrongly predicted Kamala Harris’s win, illustrates the dangers of over-relying on polling without sufficient stress-testing or sample scrutiny.
Exit polls are considered more reliable than pre-election forecasts because they collect real-time data from voters, reducing misjudgments and echo chambers.
A key challenge in polling is underrepresentation of certain voter groups—such as conservative or non-graduates—leading to skewed results and a need for methodological innovation to correct for bias.
Summary:
This transcription explores multiple interconnected themes: consumer technology, political polling ethics, and the reliability of public opinion data. com, showcasing a delivery service combining 20 restaurants. L.
Partners, where a trip by Dan Jukes to a luxury resort raised concerns about financial influence and transparency. The discussion highlights how polling can be manipulated, with risks of bias, priming, and inequitable sample representation. Experts stress that polling should not be seen as a final truth but as a starting point for understanding public sentiment.
The narrative includes real-world failures, such as the flawed Iowa poll that mispredicted the outcome, and underscores the importance of qualitative methods like focus groups to uncover deeper voter nuances. It also notes the growing threat of AI bots and digital sampling biases. While exit polls are praised for their accuracy, the overall message is that polling has significant limitations and must be used with caution, transparency, and critical thinking—especially as political dynamics grow more complex and fragmented.
The discussion concludes with a broader reflection on trust, responsibility, and the evolving role of data in democracy.
FAQs
Wonder allows you to combine 20 restaurants into one delivery. Use the code 'MealTime' for 50% off your first order.
The 2026 Toyota RAV4 offers bold style, intuitive tech, and standard hybrid power for efficiency without needing to plug in.
A polling company, J.L. Partners, was found to finance a trip for Dan Jukes, a former aide to Nigel Farage, and released polls without disclosing the funding, raising concerns about bias and transparency.
Polling provides useful insights into public opinion, but results can be misleading due to sampling issues, question phrasing, and biases like agreement bias or priming.
With fewer landlines and less phone-based contact, polling now relies heavily on online panels, requiring careful sample selection and incentives to ensure representativeness.
Polling to know aims to understand public opinion on real issues, while polling to show presents favorable results to support a political or campaign narrative, often lacking depth or nuance.
Chat with AI
Loading...
Pro features
Go deeper with this episode
Unlock creator-grade tools that turn any transcript into show notes and subtitle files.