Grading PolitiFact: Chris Hayes means the median

We interview impartial experts.
--About PolitiFact

After the article is edited, it is reviewed by a panel of at least three editors that determines the Truth-O-Meter ruling.
--Principles of PolitiFact and the Truth-O-Meter

The issue:

The fact checkers:

Louis Jacobson:  writer, researcher
Martha Hamilton:  editor


Analysis:

It's risky exposing journalists to math.  Just sayin'.

This fact check checks whether journalist Chris Hayes accurately reports what journalist David Kay Johnston writes.  Unless it's a fact check of David Kay Johnston.  Or it could be both.  We have a nested propositional truth, one might say.

The story opens with a description of the venue in which Hayes preached the Hayes/Johnston thesis, followed by a suspiciously generous four-paragraph snippet of Hayes' speech.  I judge that we lose little in the way of essential context by limiting ourselves to the final paragraph:

"This week, Pulitzer Prize-winning journalist David Cay Johnston published a great article called Nine Things the Rich Don`t Want You to Know About Taxes. In it, I learned for the first time that every year, the IRS conducts this study of the 400 wealthiest tax filers. And what the data revealed recently is remarkable. According to Johnston, the average American pays about 22 percent of their income to federal taxes. The richest 400 Americans, 16 percent, pay about 16 percent of their income to federal taxes."

Does anybody doubt that a one-sentence paraphrase could make clear that Hayes and Johnston are making the implicit argument that the richest Americans aren't paying their fair share in taxes?  Great.  Let's move on, following PolitiFact:

We wondered whether Hayes’ description of the statistics was accurate. So we first took a look at the document he referenced.

There's the description of the fact check:  verifying whether Hayes described the statistics accurately.  PolitiFact's plan to look at the document Hayes cites couldn't be better.  The analysis could be better:

The document shows that the 400 top-earning tax returns pulled down a total of $137.9 billion during the year, and that those taxpayers shelled out $22.9 billion in federal income taxes. That works out to a 16.6 percent tax rate. So by this measure -- which, we’ll note, only addresses federal income tax, not federal payroll taxes -- Hayes is close.

It doesn't address corporate or capital gains taxes, either.  So it doesn't much matter if Hayes is close by this measure.  It isn't a good measure.

PolitiFact then used the same flawed method for the rest of U.S. tax filers.  That exercise was mostly a waste of time, except it provided a back-of-the-envelope method of showing that the average American carried a lighter tax burden by percentage than the very wealthy.  But we can't have that.  It would make the two journalists, Hayes and Johnston, wrong.

Hayes told PolitiFact he got his info from Johnston.  So PolitiFact went to Johnston.

Johnston did his own calculations to figure the tax rate paid by the top 400 income earners.  Rather than formulate my own criticism of his procedure, I'll refer my readers to an evaluation of effective tax rates as performed by the CBO at the request of radical conservative Democrat Max Baucus.  That report goes through 2005 (the IRS data Johnston uses goes through 2007), but apparently fails to jibe with Johnston's findings.  For example, the effective tax rate (all federal taxes as well as imputed taxes) for the top 0.01 percentile came in at 31.5 percent (Table 1).  That leaves quite a few percentage points for Johnston to make up on the back of payroll taxes.

Worse for Johnston and Hayes, the CBO report reflects a progressive pattern with the lone exception of the highest earners, who paid a slightly lower effective tax rate than those in the next highest earning category.

PolitiFact, however, concludes "Hayes’ statement would be essentially correct" based on Johnston's numbers.

So, are we checking Johnston's accuracy or is it only important whether Hayes gives an accurate report of Johnston's findings?  It's hard to tell, as PolitiFact starts faulting Johnston's process because it isn't the only way to crunch the numbers.  After providing a few example caveats, none of which seem to approach the force of the objection implicit in the CBO report I cited above, PolitiFact provides the real howler:

In an e-mail conversation, Johnston said Hayes made a reasonable statement.

"To compare the average of the top 400 for whom we have actual data against the model example of a single worker at the median is entirely reasonable," he said. "To compare the top 400 to ‘America as a whole’ is ludicrous. That includes some people whose incomes are negative and some who make almost a million dollars a day. Since Hayes wrote about the average and I wrote about median, then the reasonable comparisons would be to compare to the median income ($33,000) or the median household income from Census (approaching twice that) or the median wage worker, as in my example."

Ladies and gentlemen, that doesn't make sense.

It makes no sense to paraphrase Johnston as saying that Hayes "made a reasonable statement" based on the quotations of Johnston that follow.  Johnston was working with the median figure.  So Johnston could pick a single filer as his example of the median and it would fit his argument perfectly.  That's what he says is "entirely reasonable" and it isn't what Hayes did.   Hayes came close to doing what Johnston said was "ludicrous."  "Ludicrous" is pretty much an antonym of "reasonable."  Hayes talked about "the average American," justifiably taken as a mean income figure including even Americans not required to pay income taxes.  And that's exactly how PolitiFact interpreted his claim.

So Hayes reported Johnston's findings in terms of the mean rather than as the median.  That is a substantial error, as Johnston points out ("ludicrous").  Yet PolitiFact takes Johnston to mean that Hayes is pretty close to the mark and magically grants him a "Half True" rating on the "Truth-O-Meter."

Half True – The statement is accurate but leaves out important details or takes things out of context.

(About PolitiFact)

The statement isn't accurate.  How can "Half True" apply according to this definition?

The grades:

Louis Jacobson:  F
Martha Hamilton:  F

Why not use the CBO's numbers for progressive taxation instead of sticking with Johnston and a pair of sources from left-leaning institutions, with the latter two failing to receive mention in the story?

Epigraph epitaph:  Three PolitiFact editors reviewed this story in order to help fix the final "Truth-O-Meter" rating.  Apparently not one of the three noticed that the way Johnston was quoted condemned Hayes' presentation of the data.  The reviewing editors can share the flunking grades given to the credited staffers.

I'm feeling generous today.


Afters:

I'm a bit late in detecting a significant discrepancy in PolitiFact's self-described principles.  Note the definition of "Half True" from the original and current "About PolitiFact" page:

Half True – The statement is accurate but leaves out important details or takes things out of context.

Now take a look at the definition of "Half True" from the more recently added "Principles of PolitiFact and the Truth-O-Meter":

HALF TRUE – The statement is partially accurate but leaves out important details or takes things out of context.

Now half-truths are partially accurate in addition to being accurate.

Looks like a job for the Flip-O-Meter!