Poll shifts after "events" are illusory

After last week's debate that the media and the elites declared a resounding victory for Clinton, several bogus polls were released purporting to show a little bump for her. How can we tell about specific polls being bogus, and what larger lessons can be drawn from polling after supposedly big "events" like a debate, gaffe, leaked documents, etc.?

PPP put out a poll where 2% said they were voting for Evan McMullin, the fake "true conservative" candidate whose campaign exists only so that the failed cuckservative consultants who were supporting Rubio etc. can still rake in some donor money this season, and delay having to get real jobs for another six months. This nobody polls below 1% -- which is what PPP says is Jill Stein's support level, when in reality it is more like 2-3%.

In other words, it over-sampled cuckservative Republicans to cut down Trump's numbers, and under-sampled progressive Democrats to boost Clinton's numbers.

Fox News put out a poll where only 18% are Independents, and the wording did not group Democrat with "Democrat leaning Indies" and Republican with "Republican leaning Indies," which is the only way to get that low a share of Independents. Since Trump wins Indies in every poll, this one under-sampled a key support group of his.

Reuters did a little better, as they should given their superior track record from the 2012 general election. Their daily poll shows Trump improving after the debate, not a bump for Hillary like the other two. After the debate, Clinton leads by 3-4 points (4-way vs. heads-up), about what the Fox and PPP polls showed.

However, Reuters surgically altered their methodology in the middle of the election season in order to move soft Trump supporters from "Trump" into one of the other / neither / unsure categories. The result was an overnight 6-point boost for Hillary. Using their original methodology (which is what their high track record from 2012 is based on), they show Trump up by 2-3 points.

That estimate is closer to what the USC poll has said for the post-debate period, which is 4-5 points for Trump. The Reuters and USC polls are also similar in their directions after the debate -- Trump doing better, although that improvement had already been under way for several days, and was therefore not a response to the debate. That is, the debate appears not to have mattered, judging from USC and Reuters.

To make sense of this, consider a recent journal article by Gelman et al (2016), "The Mythical Swing Voter" (found among Ricky Vaughn's tweets).

They look back at the 2012 election, when Romney had a good first debate, and the polls afterward suggested a 10-point movement in his favor. But who participates in the samples before and after the debate are not the same people -- maybe that 10-point swing was real, but maybe it was just a more pro-Romney crowd that participated in the sample after their team smashed the other team in a public spectacle.

Using a panel of the same individuals over time, the researchers were able to see how likely someone was to change after the debate. There was in fact a movement in Romney's direction after winning the first debate, but it was only 2-3 points instead of 10, after correcting for demographic and partisanship differences in the before and after samples.

Most people had the same preference the whole time, with only 3% changing their minds, indicating low volatility. The major difference in the before and after polls was who chose to participate -- those who did after the debate were more likely to be Romney supporters than those who participated before the debate. Perhaps it's the same effect as when fans of the losing sports team suffer a drop in testosterone and enthusiasm generally, while the fans of the winning team are turbo-charged.

So, when pollsters contact different groups of people with each poll they release, they cannot be sure that they have a representative sample each time. Maybe after some event that demoralizes the fans of one candidate, they are less likely to respond to the pollster, maybe telling them to call back when they're in a better mood -- while the fans of the other candidate are now energized at their enemy's misfortune, and are only too eager to participate in a poll and let their support for the winner of the event be known far and wide. They are probably getting a kick just from imagining the other sides' long faces when they read the poll results in a few days.

Gelman et al discuss this in the context of party affiliation, but it's broader than that. It's not only that after an event that damages the Democrat, the polls will under-sample Democrats because they're demoralized. They will under-sample anyone who was for the Democrat -- including partisan Democrats (the bulk of support), but also cross-over voters who are normally Republican, and Independents who are inclined toward the Democrat.

Differential willingness to participate also screws up our ability to generalize about Democrats, Republicans, and Independents, from such a sample, as though we were talking about the same populations every time a poll comes out.

The Democrats who do participate after their team suffers a loss are probably not the most rabid and loyal fans, who are more sensitive to public losses and more likely to be sulking. The Democrat participants are therefore less likely to qualify as "likely voters", and are less in favor of the Democrat candidate, compared to the rabid fans who are sitting things out until some good news comes along to cheer them up and make them feel like participating in the polls again. And the Republicans will not include the cross-over voters, making them even more against the Democrat candidate than is true. And the Independents will also be those more inclined toward the Republican.

The only way to keep track of these things is to track the same individuals over time in a panel. That's what the RAND poll did in 2012, and it out-performed just about all others, particularly when it suggested only a minor slump for Obama after he bombed the first debate, while the others suggested that Romney was not only doing better than before but now ahead of Obama.

The USC poll is the RAND poll under new branding, and that's why it's worth giving greater weight to than the other ones, which are going to be affected by swings in willingness to participate among Trump supporters vs. Clinton supporters. In fact, given how roller coaster-y the emotions have been this season, the non-panel polls will probably do worse than in 2012.

In particular, I've noticed PPP and Quinnipiac, which were among the best last time, have slipped a tier down in their accuracy. For example, Quinnipiac's heads-up poll from 9/8 to 9/13 had Clinton up 5, while USC had Trump up half a point over that period. Before that, PPP's heads-up poll from 8/26 to 8/28 had Clinton up 5, while USC had her up only half a point.

In general, it seems like the non-panel polls are biased against Trump, as their deviation from the more accurate panel polls is always in the pro-Clinton direction, never a more pro-Trump result than USC. Some of that is certainly due to the anti-Trump agenda of the pollsters and their corporate sponsors, which is far fiercer than whatever anti-Romney bias there was last time. Now it's the people and Trump vs. all arms of the Establishment.

But it could also be due to a stronger unwillingness among Trump voters to participate in the non-panel polls this time, compared to Romney voters last time. That's not necessarily because Trump voters are a crankier group of people, but they are more subjected to a 24/7 gauntlet of attempted demoralization by the media, compared to what Romney voters had to put up with last time.

That is evidently having an effect on their willingness to share their views with pollsters, who they might feel are about to engage them in another tedious "gotcha!" debate about whatever the Establishment hitjob du jour is. However, the demoralization campaign is clearly not having an impact on their willingness to support Trump in all ways -- to tune into the debates starring him, to follow him on social media, to attend his rallies or watch them from home, to wear Trump gear or put up Trump signs, and ultimately to vote Trump at the polling station -- first in the primaries, and soon in the general.

In the future, I'd like to see heavy restrictions on what kind of stuff goes on during campaign season. We all know how bad it is that unlimited big money gets involved. But the endless roller coaster of events is worse -- none of them end up changing people's minds or affecting the outcome. They're just a bunch of annoying shit that we're forced by the media to pay attention to. And the media make a fortune during election season -- they're the only ones who benefit from all this crap.

Somehow we elected good presidents like FDR and Eisenhower without any of today's non-stop campaigning (for those with the energy to do so), round-the-clock coverage, and roller coaster of attention and emotion.