Grading PolitiFact: Sen. DeMint and unanimous consent--version 2

PolitiFact caught with pants on fire

PolitiFact owned up to the most obvious of the mistakes in its grading of Sen. Jim DeMint on his statements about the Senate's unanimous consent procedure.

And they doubled down on their rating of "Pants on Fire."

The issue:

The issue remains the same as I described it before.  The headline and deck of the story remain unchanged, including the juxtaposition of "DeMint says 94 percent of bills are passed unanimously" with the animated "Pants on Fire" Truth-O-Meter rating.

The fact checkers:

Louis Jacobson:  writer, researcher
Bill Adair:  editor

New writer/researcher.  Same editor.  Inspired with confidence yet?

Analysis:

New writer Louis Jacobson does a better job of explicitly and clearly identifying the issues than did his predecessor:

DeMint's number seemed high to us, so we decided to investigate. We see two broad areas to analyze. One is whether the percentage DeMint cites is accurate. The other is whether DeMint's percentage says anything valuable about how the Senate handles legislation.

The earlier story might have benefited from drawing that same distinction early on, though that judgment is hard to justify based on the rest of Jacobson's story.  Indeed, though Jacobson does more to identify the issues than did Lukas Pleva, he did not go far enough.

Many PolitiFact fact checks grade the literal truth of a statement along with the underlying argument intended by the statement.  In this case we have two different statements from DeMint and perhaps two underlying arguments as well.  Jacobson, like Pleva before, fails to keep the subjects properly separated.

Jacobson follows the pattern of the earlier story by taking DeMint to task for his use of the term "bills":

This is the source of DeMint's 94 percent figure -- 855 bills passed by unanimous consent out of 911 total measures taken up.

DeMint in his speech referred to all of these measures as "bills." But the CRS report broke them down more precisely. Some were formal bills or joint resolutions, both of which are binding. Others were simple resolutions or concurrent resolutions, which do not have the force of a law and do not require the president's signature.

As I argued in my earlier response, the distinction PolitiFact draws is arbitrary with respect to DeMint's statement.  DeMint is entitled to use any legitimate definition of "bill" he wishes and the proper interpretation of his words takes his intent to account.  It is true that the CRS document distinguishes between bills and various resolutions, but that does not eliminate the legitimacy of the term "bill" as DeMint used it.

Note this from the Government Printing Office:

The following abbreviations stand for types of legislative documents in the Congressional Bills databases. They are included as part of the bill number in the identification code.

H.R.	House Bill
S.	Senate Bill
H.J.Res.	House Joint Resolution
S.J.Res.	Senate Joint Resolution
H.Con.Res.	House Concurrent Resolution
S.Con.Res.	Senate Concurrent Resolution
H.Res.	House Simple Resolution
S.Res.	Senate Simple Resolution

The entire variety of measures discussed in the CRS document comes from the "Congressional Bills" databases, and the abbreviations to the left constitute part of the "bill number" regardless of whether it is "S. Con. Res." or "H.R."  PolitiFact equivocates on DeMint's statement as to its literal meaning.  The statement is literally true, and DeMint appears to make the underlying argument that Congress does a great deal of its business using unanimous consent.  And his underlying argument is probably a bit more than that, spilling into the underlying argument associated with his subsequent statements about the bills passed unanimously.  As Wesley Denton put it on behalf of DeMint (via PolitiFact):

"Nowhere in DeMint’s speech does he ever say that all 94 percent of these bills are controversial, or that all 94 percent need a roll call vote," Denton said. "DeMint is criticizing the process, not every bill passed by that flawed process. So to try to whittle down the number by focusing on only the bills PolitiFact finds controversial is a disingenuous attempt to discredit DeMint’s factual statement."

Jacobson followed the quotation with this response:

We disagree. We think the fairer test is to focus on the bills that include some element of substance that demands debate.

Jacobson confuses the statement with the underlying argument.  He is correct that it would be better in terms of precision (if not in terms of rhetorical effectiveness) to stick with bills of substance where the lack of debate represents a dire public concern.  But Jacobson makes a mistake in trying to shoehorn DeMint's literally true statement about the percentage of bills passed via unanimous consent into what Jacobson sees as the best way to present the argument.  No matter who is right about the best way to present the argument, the literal statement from DeMint is true.  Jacobson should have focused on whether the true statement communicated the underlying argument without misleading the audience.  Failing in that, Jacobson misleads his audience.

From that confusion between the literal statement and its underlying argument, Jacobson proceeds to the math stage flubbed so memorably in the previous version of the story.  Happily, Jacobson perpetrates no major errors. For what it's worth, I counted 86 bills naming postal service facilities compared to Jacobson's 88 and I found 31 other bills naming various government-owned properties compared to Jacobson's 30.

After Jacobson fiddles away on the numbers for a time he delivers the key graphs:

We now have a couple ways to crunch the numbers. One is to say that 81 percent of non-trivial, binding bills (206 out of 254) were passed by unanimous consent. Alternately, you could separate out the 64 binding resolutions that CRS said were passed by unanimous consent but which received some debate. Doing this would mean that 56 percent of non-trivial, binding bills, received absolutely no debate. (DeMint's office argues that the "debate" on these bills cited by CRS was cursory and thus shouldn't be counted as debate.)

Either figure would be lower than what DeMint said on the floor, and one of the percentages is quite a bit lower. They're also higher than the 27.9 percent we had in our previous article, which we now acknowledge is incorrect.

 The two paragraphs above are best taken as a better comparison than the one DeMint offered--nothing more than that.  Jacobson has nothing through this point to justify judging DeMint's claim as "ridiculous."  DeMint's percentage figure was accurate, and he could have chosen a more appropriate measure.  Either way, the bulk of legislation passed by Congress was via unanimous consent and was notably lacking in the type of debate we ought to favor in our legislative body.

In short, neither the claim on its face nor the underlying claim is ridiculous.

Jacobson tries to suggest otherwise:

(W)e think what's wrong with DeMint's statement is actually broader than the arithmetic. We arrived at the percentages above after first excluding as ceremonial or symbolic 657 measures the senator had counted. These largely non-substantive measures that DeMint included accounted for more than 70 percent of the measures he used in his calculation. This significantly skews the picture of what kinds of measures were taken up in the Senate.

It skews it somewhere between 13 and 38 percentage points.  Ridiculous?   PolitiFact's first attempt at the calculation differed from its second attempt by at least 28 percentage points.  DeMint's percentage was exactly accurate, if applied clumsily to the also-solid underlying argument.  President Obama's claim about Model T gas mileage compared to that of modern SUVs once again provides an illuminating case study in shifting standards.  Obama's claim, like DeMint's, was literally true but a poor basis for comparison.  And Obama's underlying argument, unlike DeMint's, had virtually nothing to recommend it.  Yet Obama warranted a "Mostly True" rating from PolitiFact.

That's hardly a model for journalistic objectivity.

PolitiFact obstinately sticks with their earlier rating:

Ultimately, we find that DeMint's goals, laudable though they may be, are undercut -- not strengthened -- by the math he chose to present in his floor speech. We'll own up to our own mathematical error, but we stand by our overall judgment from the initial article: Pants on Fire!

That standard clearly did not apply in President Obama's case.

The grades:

Louis Jacobson:  F
Bill Adair:  F

Jacobson's potentially informative research might have earned him a passing grade if he had bothered to tease out the entire set of issues clearly.

How can Adair preside over this stuff?


Afters:

PolitiFact earned some credit by keeping the old story archived.  Dating the stories and any subsequent updates might also be a good idea.


June 28, 2010:  added the missing adverb "clearly" to the first paragraph in the "Analysis" section.