PolitiFact, promises and pioneers

I write often about PolitiFact's problem with selection bias, but that's just one of the most obvious bias problems for the Poynter Institute's fact-checking baby.

In addition to selecting statements to check, PolitiFact chooses what aspect of the statement to check.  This adds another layer of bias distortion to the statistics resulting from PolitiFact's fact checking.

PolitiFact recently supplied two stories that provide an outstanding example of bias caused by the selection of the story focus.

On July 11 PolitiFact rated an ad by the American Commitment PAC.  On July 13 PolitiFact rated an Obama campaign ad.

American Commitment:

The [health care] bill could cost up to $2 trillion, double what we were promised.

The Obama campaign:

Mitt Romney's companies were pioneers in outsourcing U.S. jobs to low-wage countries.

Let's put the primary clause of each statement in bold.

American Commitment:

The [health care] bill could cost up to $2 trillion, double what we were promised.

The Obama campaign:

Mitt Romney's companies were pioneers in outsourcing U.S. jobs to low-wage countries.

The portions without bold emphasis serve as subordinate clauses.

For the American Commitment ad, PolitiFact places its focus mainly on the subordinate clause.  PolitiFact found the primary clause close though supposedly lacking valuable context:

The ad upgrades that $1,762 billion, or $1.762 trillion to $2 trillion. In this case, we won’t quibble about $240 billion, because the $1.76 trillion number itself is extreme cherry-picking. It doesn’t account for the law’s tax increases, spending cuts or other cost-saving measures.

Note to PolitiFact:  Tax increases don't decrease spending and the spending cuts aren't certain, hence the "up to $2 trillion" phrasing.  Back to the point, the fact check focuses on the claim the amount doubled from what was promised--the subordinate clause--and finds that since the new CBO report does not represent a doubling of the cost compared to the old CBO report therefore the statement is "False." 

Supposedly the new figure isn't double what was promised (see "Afters" section at the bottom of the post). 

So the primary clause was at least somewhat true but since the subordinate clause was supposedly false the entire statement was ruled false.

In contrast to its treatment of the American Commitment ad, PolitiFact placed equal focus on the primary and subordinate clauses in the Obama campaign's ad and for good measure rates whether the adjective "Romney's" serves as a reasonable modifier for "companies" for purposes of blaming Romney for what the companies do.  The logic of the ruling is a tad difficult to unravel because the single rating tries to tackle three issues (bold emphasis added):

We find reasonable grounds for labeling the companies as "Romney’s." He was the founder of Bain and assembled a team that looked to make high returns. One strategy was to invest in companies that played off the trend in outsourcing. We make no judgment on whether outsourcing is good or bad. It was widely seen as profitable, and Bain selected companies that would succeed.

If picking a company makes it yours, then these were Romney’s companies and in a general sense, they did what he expected them to do. The one caveat is there is a gray area of direct accountability, because no one has reported that he was personally involved in managing those firms.

We find little evidence that the particular firms were "pioneers in outsourcing." The Obama campaign took the word from the Washington Post but used it as its own. Outsourcing was well established by the early 1990s, and firms were applying it in a variety of industrial areas. The Bain companies were among that group.

The Obama campaign's statement would have voters believe that Romney played a key role in driving the outsourcing phenomenon. We find that an exaggeration.

We rule the statement to be Half True.

So PolitiFact appears to say it's true that the Bain companies engaged in outsourcing.  And it's somewhat true that Romney is responsible for what the Bain companies do.  How much?  PolitiFact doesn't really say.  Nor does PolitiFact describe what degree of exaggeration occurs when the Obama campaign calls Romney's companies outsourcing pioneers.  Of course, there's not much of an ad without Romney's responsibility and the alleged pioneering.

Somehow PolitiFact concocts a "Half True" rating out of the above despite the fact that the primary clause is essentially false.

For review, PolitiFact held that the American Commitment ad contained a mostly true primary clause and a false subordinate clause.  The American Commitment ad received a "False" rating.

So it matters quite a bit which clause receives PolitiFact's focus.


Afters:

Is the current CBO estimate of 10 years of the ACA twice the original estimate?

Yes.  But the claim involves a partially unfair comparison.

Obama proposed the health care reform bill based on a $900 billion price tag over the first 10 years.  Because the first few years institute relatively few of the reforms, the first few years do not accurately represent the long-term cost of the bill.  Republicans have criticized the use of that distortion by those selling the plan, since it hides the real cost of the plan when all of its provisions take effect.

Does that make it okay for Republicans to ignore that the estimates cover different 10-year periods?  Perhaps one could cobble together a complicated ethical justification for doing it.  But in terms of fact-checking, both Obama and his critics deserve the same treatment.  Neither told the whole story.

PolitiFact consistently leaves one of those distortions largely intact.


Correction:  Fixed a number of instances in the latter half of the post referring to the American Commitment ad as the "Romney ad."