The unbearable lightness of caving

You just have to laugh.

PolitiFact, after the latest fact-check clamor from the left, has again revised one of its Truth-O-Meter ratings, and this one may top the last one for the nonsensical justification accompanying the new ruling. PolitiFact decided to change its ruling of Sen. Marco Rubio's (R-Fla.) statement that most Americans are conservative from "Mostly True" to "Half True."

The rationale supplies the evening's entertainment.

Let's start with the editor's note:

EDITOR'S NOTE: An analysis of this claim by U.S. Sen. Marco Rubio, R-Fla., was published on Feb. 14, 2012. After it appeared, we heard from many readers who argued that our rating of Mostly True was the wrong call. The debate centered on whether to judge Rubio on his literal statement or the underlying point. We try to balance that question in many of our rulings. Upon further discussion, and bolstered by more reporting, we have decided to change this ruling to Half True. (The original item is archived here.)

The further discussion was "bolstered by more reporting."  Like what? We placed the stories side-by-side for comparison.

Here are the significant differences in reporting:

We found three sets of reputable polls that were comprehensive and addressed voter ideology.

The first version of the story used only a series of Gallup polls.

Before we get to those polls, we should note two important precautions about considering the data. First, it matters how questions get asked. In this case, it particularly matters which options a pollster offers for answers to a given question. Second, political ideology is not the same as party affiliation.These days, people split Republican and Democrat pretty evenly, with the answer "independent" garnering significant support. But here, we’re looking at political ideology on a liberal to conservative scale.

The above is an absolutely excellent addition to the story.  Too bad it doesn't really make any difference except to make clear that the part in the earlier version of the story about the Republican leaning of Independents was completely irrelevant. The summary of the Gallup poll data remains pretty much unchanged.  There were two more polls considered.

Here are the summaries.

 American National Election Studies

In 2008, 32 percent placed themselves on the conservative side; 25 percent said they hadn’t thought much about it; 22 percent chose the liberal side; and 22 percent said they were moderate.

That poll offered two choices that defied categorization into either "conservative" or "liberal," obviously.

POLITICO-George Washington University Battleground Poll

In the most recent poll, conducted in November 2011, 61 percent said they were conservative while 34 percent said liberal.

That poll more or less forced respondents into moving out of the middle to express a preference for either conservative or liberal.  PolitiFact quotes a representative as saying the poll does that by design.

Through this stage we have three relevant new pieces of reporting.  First, the number of answers a poll provides respondents matters.  Second, the ANES poll has results that somewhat agree with the Gallup poll except that more respondents concentrated toward the middle without accepting either the conservative or liberal labels.  Third, we have a poll that unequivocally supports Rubio, albeit by  forcing a choice on those who would trend toward the middle. What we have so far only helps Rubio, pending additional sifting of the evidence.

PolitiFact's next section, featuring comments from pollsters, must serve as the key evidence behind the change in the ruling.

 PolitiFact:

Mark Mellman, a pollster for Democrats, said conservatives never cross the 50-percent threshold when moderate is offered as an option, and moderate always should be offered. "Questions need to contain the range of relevant alternatives in order to be ‘valid,’" he said via email.

Mellman's wrong with the claim that moderate should always be offered as an option, but correct with the latter.  Is "moderate" a relevant alternative in this case?  As  Celinda Lake said in explanation of the approach for the Battleground poll, "(T)hey deliberately don’t offer 'moderate' because it becomes a default answer and is less predictive of voting behavior."

PolitiFact:

"The real question is, so what?" he added. "Identification with these broad general terms is only loosely correlated with people’s views on any particular substantive issue … Saying more people are conservative than liberal tells us nothing about their views on taxing the wealthy, protecting Medicare, protecting the environment or any other of a host of issues."

Mellman's right again.  But if we're concerned about context at all, Rubio prefaced his statement about majority conservatism with comments about the U.S. economy and later added belief in the Constitution as a trait of conservatives.  A fact checker might be interested in whether a conservative majority exists on things like economic approaches and maybe protections of religious liberty--things that might have major relevance when it comes to convincing voters to support conservatives, in other words.

 Nah.

 PolitiFact:

Robert Blendon of Harvard University, who specializes in polling on health care policy; said he prefers the Gallup poll numbers when thinking about political ideology, but added, "I’ve never seen a poll that doesn’t show more conservatives than liberals," he said.

Blendon, then, doesn't support finding Rubio literally true that a majority of Americans are conservative, but appears to support a potential underlying argument that conservatives outnumber liberals.  That underlying point would at least partially encompass the context of Rubio's statement.

PolitiFact:

Karlyn Bowman of the conservative American Enterprise Institute distills the results of many polls on similar topics. She said she thought Rubio was on the right track with his comments.  "Ideologically, we are more conservative than liberal, and that’s consistent across the polls," she said.

And that's it.  Mellman said it's important to use polls that have options representing all the relevant choices, but whether moderates were relevant in the context of Rubio's statement isn't clear.  Blendon prefers the Gallup poll (and presumably the inclusion of the "moderate" choice) but does not apply his preference to Rubio's statement.  From Bowman we have essentially the statement that Americans are more conservative than liberal.

It's time to bolster the new rating with the fresh reporting.

PolitiFact:

Rubio said that the majority of Americans are conservative. Two of the three pollls show that while conservatives are a plurality, they are short of a majority. Rubio can't claim majority status because a significant share of the electorate identifies itself as moderate. In the one poll that gives voters two choices -- liberal or conservative -- conservatives have consistently been the majority. So by the two polls, he was incorrect. By one, he was correct and we find support for his underlying point that there are more conservatives than liberals. On balance, we rate this claim Half True.

As is so often the case with PolitiFact, a "Huh?" is warranted.

One of the new polls essentially agreed with the earlier poll.  The second one agreed with Rubio.  How can that justify downgrading the ruling?  The answer is easy:  It can't.  This PolitiFact story is flimflam.  The claim that the new reporting bolsters the new ruling is untrue, particularly as presented in the summary paragraphs above.  The new reporting provides a pretense for changing the ruling; PolitiFact does not clearly communicate any reasonable justification for downgrading Rubio.  The new ruling is easier to justify from the old reporting.

PolitiFact's blown the story twice.

Explaining the updated ruling

As noted above, the new reporting on the Rubio item appears to support Rubio, on balance, more than did the reporting in the original item. What is the best and most charitable explanation for the downgrade in terms of the evidence?

I suggest PolitiFact may have used the following spurious lines of evidence:

1)  Arbitrary reliance on Mellman instead of Lake leaves us with the proposition that a poll of political ideology should always include "moderate" as an option.  Following that dictum, one should prefer the two polls that offered less support to Rubio.

2)  The PolitiFact summary implicitly suggests that two polls are better than one.  Granted that leaves us with the question of whether 2-1 is better than 1-0.

The two charitable attempts above are of such dubious quality that we can't rule out the less charitable explanation:  PolitiFact repented of its original ruling in advance of and regardless of new reporting, refused to admit any error in reaching the original conclusion and subsequently used the new reporting as a deliberate pretense to unveil a new ruling motivated by criticism from the left.

Wrap

The best version of the story would have taken Rubio in context and taken its cue from from the combined testimony of Lake and Mellman:  Look for polling including all the relevant options, and recognize that the conservative/liberal dilemma is the superior predictor of voting behavior.  Lake is saying that "moderate" is relatively irrelevant in predicting voting patterns.  The Battleground poll stands as the best of the three for testing Rubio's claim. The original ruling was probably correct, but PolitiFact never produced adequate reasoning for either of its conclusions.

The likely explanation for the changed ruling occurs early in the editor's note:  "(W)e heard from many readers who argued that our rating of Mostly True was the wrong call."

So PolitiFact caved to external pressure.  That's the story right there.



Feb. 28, 2012:  Eliminated a redundant and misplaced descriptor, and in the "Wrap" changed "The best explanation" to "The likely explanation."