PolitiFact and playing the percentages (Updated)

American Journalism Review's editor and senior vice president Rem Rieder is back at it again, writing about PolitiFact in a way that demands a response.

Last time, Rieder argued in effect that journalists' role as truth-tellers compelled them to call out lies.  Rieder blurred the line between a "lie" as an inaccurate statement and a "lie" as an attempt to deceive and wound up catching himself in his own criticism.  In my response I argued that objective journalism doesn't permit reporting motives based on misstatements of fact.

This time, Rieder broadcasts his disappointment with PolitiFact's ruling of "Mostly True" on Sen. Marco Rubio's claim that a majority of Americans are conservative.

As it assessed the Florida Republican's assertion, PolitiFact turned to the findings of the Gallup Poll, which regularly asks Americans about their political orientation. In 2011, Gallup found that 40 percent considered themselves conservative, 35 percent moderate and 21 percent liberal. It also found that the number calling themselves conservative has never been above 50 percent.

So this one should be easy. According to Merriam-Webster's Collegiate Dictionary, a majority is "a number or percentage equaling more than half of a total." Not even close. The number of conservatives would have to be more than 25 percent higher to be a majority. The verdict on the PolitiFact Truth-O-Meter is "Pants on Fire," right?

Can we blame Rieder for ignoring the context of Rubio's statement if PolitiFact did the same thing?  Of course we can, kids!  As I pointed out in an analysis over at the associated blog "PolitiFact Bias," Rubio spoke in the context of preferring conservative over liberal.  Here's part of the quotation, via PolitiFact:

"You know, somebody asked me: ‘How do you know that? How do you know Americans are majority conservative?’ Here’s why: How come liberals never admit that they are liberals? They never admit it. They’ve now come up with a new word called progressive, which I thought was an insurance company, but apparently it’s a label."

PolitiFact nearly hit on an approach to the fact check that might have tested what Rubio was claiming.  PolitiFact took into consideration the political leaning of Independents, reporting that "45 percent of Americans identified as Republicans or leaned that way."

Adding the conservative lean of moderates to the percentage of self-identified conservatives probably represents the best method of testing Rubio's claim.

A short illustration will help show why.

Suppose we used a self-identification poll with 50 choices.  Conservatives, for the sake of argument, represent only 4 percent of the total, with the other 36 percent who chose "conservative" out of only three choices picking some other label.

Do conservatives make up 40 percent of the total or just 4 percent?

The number of choices matters.

For that reason, limiting the choices to match the nature of Rubio's claim constitutes the best method of testing the claim.  Calculating the conservative lean of moderates added to self-identified conservatives is a fair measurement.  Gallup doesn't ask that question of moderates, so we can't use that stat to test Rubio's claim.  As a result we don't really know the answer.  My guess is that the conservative lean of moderates would exceed 10 percent of the total and put conservatives over the 50 percent bar.  Therefore I think PolitiFact's rating is reasonably accurate but flawed in its reasoning.  But worse still is the reasoning coming from critics on the left.  Can the statement be reasonably termed "False" based on self-identification surveys with three choices?

Rieder clearly thinks so.  And maybe he'd just as willingly accept a self-identification survey with 50 choices in deciding the question.

Oops, did I lump Rieder in with critics on the left?  But he's an objective journalist, isn't he?

I don't think so.  If Rieder was objective he might have taken the opportunity to criticize PolitiFact's "Mostly True" rating for White House representative Cecilia Muñoz's claim that 98 percent of Catholic women have used contraception.  That claim is off by at least the 10 percentage points representing women who have never had sex at all; it simply couldn't be a true claim at all in contrast to Rubio's, and the degree of error is at least similar to that Rieder finds an obvious "Pants on Fire" for Rubio.

So where's Rieder's objection?

I'm not holding my breath.


Update 3:40 p.m.

Kevin Drum of the liberal rag Mother Jones points to survey data supportive of Marco Rubio's statement:

In this poll, 61% of the country identified as conservative. That's a majority!

Now, this is a poll of "likely" voters. It forces a choice between liberal and conservative. And even though we all know that "independent" voters mostly lean left or right pretty reliably, I imagine this poll still understates the number of true centrists.

Nonetheless, it's a poll. And Rubio could quite reasonably point to it as evidence that a majority of the country is conservative.

The Politico/George Washington University survey samples likely voters, so it isn't quite the perfect support for Rubio.  Of significance, however, Democrats and Democrat-leaners constitute a plurality in the sample population.  Drum fails to emphasize that key point while mentioning a few less significant complications.

American Journalism Review and the lying liars who accuse lying liars

American Journalism Review published a story late last year in defense of today's fact check fashion.  It carried the title "Calling a Lie a Lie," but that was deceptive.  The subsequent deck material was more clear:  "When something is clearly false, journalists shouldn’t be bashful about pointing it out."

The deck sentiment is correct. Unfortunately, writer Rem Rieder appears uncertain whether to support the ambiguous sentiment in the title or the more reasonable assertion from the deck.

Rieder strangles his message in ambiguity from the start:

So should journalists brand a lie a lie?

This should not be a tough call: Of course they should.

If we understand "lie" as  a clearly false statement, then Rieder is correct. 

If, on the other hand, we're talking about the much more widely understood definition of "lie" as a statement intended to deceive, then the answer is no, at least in objective reporting.  Objective reporting does not permit the journalist sufficient leeway to judge the motives of a speaker or writer.

So already we've got a problem.  If an "objective" journalist calls a statement a "lie" in the false statement sense and the audience understands "lie" to refer to intentional deceit, then the journalist is, in effect, communicating a false message to the reader.  A lie?

Why would the journalist use ambiguous language if not for the express purpose of misleading the reader?  Assuming the journalist is intelligent enough to distinguish between the different meanings for the term "lie," of course.

Rieder then follows with a nearly perfect illustration of his own problem with ambiguity:

The issue is very much in the air now in the wake of Mitt Romney's completely duplicitous ad about President Barack Obama.

The "Believe in America" ad shows Obama saying, "If we keep talking about the economy, we're going to lose."

What Obama really said was, "Senator [John] McCain's campaign actually said, and I quote, 'if we keep talking about the economy, we're going to lose.' "

This is a completely cynical distortion of the president's remark. It's an incredibly cynical piece of political mischief. It needs to be called out. 

 Just to state the obvious, journalists very often truncate quotations.  They lop material off the front.  They omit material in the middle.  They chop off material at the end. 

It's okay to shorten quotations under one condition:  It doesn't substantially change the meaning of what the person wrote or said.

President Obama's handlers immediately reacted to the Romney ad, alleging it distorted what President Obama said.  Romney's defenders pointed out that the ad provided enough context so that the viewer--even including left-leaning journalists--should be able to tell that Obama was speaking back in 2008 well before Mr. Obama would have thoughts of whether to talk about the economy when seeking reelection.

To amplify the point, Romney's ad carries the same message substituting the full quotation Rieder provides.  The only exception comes from viewers such as Rieder who assume, contrary to the facts presented in the ad, that Obama referred to himself in the context of the 2012 election.

Plainly the ad was intended to portray the irony of Obama condemning the economy in 2008 while holding responsibility for a worse economy leading toward the 2012 election.  And it's the journalist's responsibility to expose a lie in that?

No, it's the journalist's responsibility to explain the ad in terms of the facts and in terms of its message, correcting any obvious errors of fact in accordance with the statement in Rieder's deck.  It is not the journalist's responsibility to make false statements about how the ad is obviously false and in need of rebuke.

This controversy over fact checking helps illustrate time and again how journalists have lost touch with the concept of objective reporting.