Grading PolitiFact: Stephanie Cutter and the case of the missing fact check

It gets a little tiresome seeing PolitiFact repeatedly engage in partial reporting on its stories.  The following may represent the supreme example.

The issue:

(clipped from PolitiFact.com)

The fact checkers:

Angie Drobnic Holan:  writer, researcher
Bill Adair:  editor


Analysis

PolitiFact sets the stage:

The Republican response to attacks on the Ryan plan has been to attack back, saying President Barack Obama has cut "$700 billion" out of Medicare. And the Democratic response to that: Well, Paul Ryan cuts that amount, too!

PolitiFact selects the claim to check:

"You know, I heard Mitt Romney deride the $700 billion cuts in Medicare that the president achieved through health care reform," Cutter said. "You know what those cuts are? It’s taking subsidies away from insurance companies, taking rebates away from prescription drug company. Is that what Mitt Romney wants to protect? And interestingly enough Paul Ryan protected those cuts in his budget."

PolitiFact avows that it will focus on "whether Cutter is correct that Ryan relies on those same reductions in his budget."

PolitiFact uses the next nine grafs to outline the nature of the $700 billion reduction in Medicare expenditures projected by the CBO.

PolitiFact's source, a CBO report, communicates the nature of the reduction a bit more clearly than does PolitiFact (yellow highlights added):

Changes to Payment Rates in Medicare
In February 2011, CBO estimated that the permanent reductions in the annual updates to Medicare’s payment rates for most services in the fee-for-service sector (other than physicians’ services) and the new mechanism for setting payment rates in the Medicare Advantage program will reduce Medicare outlays by $507 billion during the 2012–2021 period. That figure excludes interactions between those provisions and others—namely, the effects of the changes in the fee-for-service portion of Medicare on payments to Medicare Advantage plans and the effects of changes in both the fee-for-service portion of the program and in the Medicare Advantage program on collections of premiums for Part B (Supplementary Medical Insurance).

The bulk of the reduction, then, occurs as the result of the two reductions the CBO identifies.  Therefore, we should expect to see both of those features in the Ryan budget plan at minimum to rate Cutter's statement true.

Having more-or-less identified the nature of the projected Medicare savings, PolitiFact proceeds to the next phase of its fact check:

Now onto [sic] our second question: Does Ryan’s budget keep the reductions in Medicare spending? The short answer is yes.

Here’s what Ryan said in an interview with George Stephanopolous of ABC News in June, before his selection as Romney’s running mate:

Stephanopoulos: "You know, several independent fact-checkers have taken a look at that claim, the $500 billion in Medicare cuts, and said that it's misleading. And in fact, by that accounting, your budget, your own budget, which Gov. Romney has endorsed, would also have $500 billion in Medicare cuts.

Ryan: "Well, our budget keeps that money for Medicare to extend its solvency. What Obamacare does is it takes that money from Medicare to spend on Obamacare. ..." (Read the full exchange.)

Do we know from that exchange that the cost reductions come from the same source?  I don't see how, and I invite any reader who sees it to explain it with a comment below.

Slate's Dave Weigel claimed that Ryan uses the same cap on Medicare spending as Obama.  But his explanation does not appear to help PolitiFact's argument.

Weigel (bold emphasis added):

Remember, Obamacare is supposed to save $700 billion by capping the rise in Medicare spending from GDP growth plus 0.5 percent. The Ryan budgets in 2012 and 2013 don’t alter Medicare for anyone entering it before 2022—a buffer that lets current retirees breathe easy. After 2022, it turns all of Medicare into a premium support plan like Medicare Advantage. At that point, “an annual competitive bidding process” is supposed to push providers to provide lower rates. “The per capita cost of this reformed program for seniors reaching eligibility after 2023,” explains Ryan in his budget guide, “could not exceed nominal GDP growth plus 0.5 percent.” So, if it works, it’s got the exact same Medicare cap as the Obama plan.

Weigel is talking about two different means of obtaining the same future rate of growth on Medicare spending.

As for PolitiFact, it's sticking with Paul Ryan's supposed confession:

So Ryan has confirmed his budget includes the Medicare savings.

"The" Medicare savings?  The same exact ones from the ACA and not just the future rate of growth pegged at the same percentage?  How do we know that?  Where is the fact check?

PolitiFact:

Still, Ryan himself said his plan did include the reductions in future spending that were part of the federal health care law.

Sorry, but that's not a fact check and it's very misleading.  PolitiFact is seizing on an ambiguity from Ryan and insisting that it perfectly dovetails with Cutter's claim.  A real fact check would verify from the text of Ryan's budget that the savings have the same origin as those projected by the CBO for the health care reform law.  This fact check doesn't do that at all.  Ryan's budget is neither listed among the sources on the sidebar nor linked in the text of the story.

Another of PolitiFact's sources helps confirm that PolitiFact simply blew this fact check. The CBO did attempt to score Ryan's budget proposal. The CBO did a baseline scenario using the assumption that the health care reform bill would remain in effect:

The baseline scenario incorporates policies restraining Medicare spending that are embedded in current law. Such policies include the sustainable growth rate mechanism, which determines the payment rates for physicians; payments to other providers in the fee-for-service portion of Medicare that would grow more slowly over roughly the next two decades than the cost of their inputs; and the  Independent Payment Advisory Board (established by the Affordable Care Act), which is required to make changes to the Medicare program to reduce spending if the growth in such spending is projected to exceed certain targets.

And the CBO created an alternate scenario where Medicare savings were much less:

The alternative fiscal scenario incorporates less restraint on Medicare spending.  Specifically, payments for physicians would not be reduced as they would be under the sustainable growth rate mechanism, and payments to other providers  would grow more rapidly than under the baseline scenario after roughly the next decade. The remaining restraints on Medicare spending could also have the potential consequences noted for the baseline scenario, but presumably to a much lesser extent because the restraints would be much less tight.

If the cost reductions are "protected" in the Ryan budget, then why does the CBO run an alternative scenario where the supposed protected cost reductions do not occur?

By all appearances, the PolitiFact team mailed it in on this fact check.  The evidence strongly suggests that the Ryan budget plan only relies on savings through ObamaCare to the extent that the CBO assumes that existing law will remain in effect--its standard procedure--while projecting the effects of Ryan's budget.

Cutter gets a "True" for that?


The grades:

Angie Drobnic Holan:  F
Bill Adair:  F

Seriously:  Where's the fact check?


Afters:

Here's one of those statements from the CBO that seems to have a tough time finding its way into PolitiFact's fact checks (bold emphasis added):

CBO’s cost estimate for the legislation noted that it will put into effect a number of policies that might be difficult to sustain over a long period of time. The combination of those policies, prior law regarding payment rates for physicians’ services in Medicare, and other information has led CBO to project that the growth rate of Medicare spending (per beneficiary, adjusted for overall inflation) will drop from about 4 percent per year, which it has averaged for the past two decades, to about 2 percent per year on average for the next two decades. It is unclear whether such a reduction can be achieved through greater efficiencies in the delivery of health care or will instead reduce access to care or the quality of care (relative to the situation under prior law). Also, the legislation includes a provision that makes it likely that exchange subsidies will grow at a slower rate after 2018, so the shares of income that enrollees have to pay will increase more rapidly at that point, and the shares of the premiums that the subsidies cover will decline.

PolitiFact leaves misleading fact check of Gerard Robinson mostly intact

Updating a past Grading PolitiFact post, PolitiFact Florida made minor changes to its fact check of Florida's ex-education commissioner, Gerard Robinson.

The major errors inexplicably remain.

What changed

PolitiFact Florida altered its quotation of Robinson, adding a pair of ellipses.  The original version drew quotations from three separate paragraphs and, using no ellipses, stuck three of Robinson's sentences together to give the appearance of a single paragraph.

Here's how the new version reads (bold emphasis added):

Robinson penned a June 15 response, which included these comments: "The FCAT neither drives the curriculum nor narrows the educational experience of Florida students. ... These assessments average two to three per student per school year and account for less than 1 percent of the instructional time provided during the year. ... It is worth noting that local school boards require students to take many more assessments than those required by the state."

The changes still do not, so far as I can tell, conform to AP style for quotations.  Statements from separate paragraphs should show as separate paragraphs.

More importantly, what are we missing?


What didn't change

I spoke to PolitiFact editor Aaron Sharockman by phone last week.  I pointed out to him that Robinson had an entire paragraph dedicated to answering the charge that students spend too much time preparing for the FCAT. 

Here's that paragraph (bold emphasis added):

Florida's Next Generation Sunshine State Standards are the foundation for what we expect our students to learn. Subjects covered by Florida standards include English language arts, math, science, social studies, physical and health education, world languages, and fine arts along with other content areas specific to colleges and careers. Contrary to the claim of the FSBA resolution, the FCAT neither drives the curriculum nor narrows the educational experience of Florida students. In fact, at the middle school level, student enrollment in courses such as dance, drama, and world languages has increased more than student enrollment in the subject areas assessed on the FCAT. At the high school level, enrollment in dance, world languages and the humanities has outpaced the growth in student enrollment.

Robinson obviously stresses the point that it is appropriate for students to spend a great deal of time preparing for the FCAT since the FCAT measures "what we expect our students to learn."

PolitiFact omitted Robinson's point from its story.  No quotation.  No paraphrase.  Nothing.

I spoke to Sharockman again today.  I said I received the impression that he agreed with me that Robinson had stressed that it was appropriate to spend a great deal of time preparing for the FCAT.  I asked whether I had received a false impression.  Sharockman said that he did not recall agreeing about Robinson's point and said he thought I had received a false impression.

How does one miss Robinson's point in the above paragraph and then omit it in a fact check story about that same subject?

How does one miss it even after a critical review?


Beyond incompetence?

The update notice provided with the story after the ellipses were added doesn't quite ring true:

Update: During the editing process, ellipses were inadvertently left out of Robinson's comments of June 15. The ellipses are now included.

I alerted the writer and editor by email about the problem with the quotation before 1 p.m. the day after the story published.  The story published on July 30 at 11:53 a.m.  By Friday, Aug. 3, the problem still stems from an inadvertent omission during the editing process?

Something's broken at PolitiFact.

Grading PolitiFact (Florida): Gerard Robinson and the FCAT (Updated x2)

Context matters -- We examine the claim in the full context, the comments made before and after it, the question that prompted it, and the point the person was trying to make.
--Principles of PolitiFact and the Truth-O-Meter

The issue:

(clipped from PolitiFact.com)

Readers should already find themselves wondering whether "FCAT tests" represents the time spent taking the test itself or the time spent preparing for the test.  PolitiFact uses ambiguous wording.  Yet why would we expect the test itself to take more than 1 percent of a year's instructional time?


The fact checkers:

Amy Sherman:  writer, researcher
Angie Drobnic Holan:  editor

Analysis:

Put bluntly, this item isn't fact checking.  It is a disservice to Gerard Robinson and PolitiFact's readers.

PolitiFact:

Reflecting a backlash against testing, more than a dozen individual school boards in the state, including Broward and Palm Beach, have passed a resolution against the FCAT. The Florida School Boards Association passed its own version of a resolution criticizing the FCAT in June.

Robinson penned a June 15 response, which included these comments: "The FCAT neither drives the curriculum nor narrows the educational experience of Florida students. These assessments average two to three per student per school year and account for less than 1 percent of the instructional time provided during the year. It is worth noting that local school boards require students to take many more assessments than those required by the state."

If PolitiFact follows AP style for quotations, the above represents a breach of style guidelines.  Here's how the quotation would look according to AP style:

"The FCAT neither drives the curriculum nor narrows the educational experience of Florida students. 

. . . These assessments average two to three per student per school year and account for less than 1 percent of the instructional time provided during the year.

. . . It is worth noting that local school boards require students to take many more assessments than those required by the state."

PolitiFact:

There are a few interesting claims in Robinson’s statement, but the one that caught our eye was that tiny figure: The FCAT accounts for "less than 1 percent of instructional time." Heck, we wonder if lunch or recess could add up to more than 1 percent. So we decided to research whether Robinson’s 1 percent claim was correct.

PolitiFact reports that Robinson's office backed up his claim with numbers showing that the amount of time spent taking the test amounts to .26 to .90 of the state-mandated minimum of 900 hours of instructional time.

So Robinson's claim was true?  PolitiFact doesn't see it that way:

But Robinson used the phrase "instructional time" in his claim, which could fairly be interpreted to mean classroom time spent preparing for the test.

Yes, Robinson used the phrase "instructional time" in his claim, but it isn't fair to interpret it to mean classroom time spent preparing for the test.

"Instructional time" has a particular meaning in education.  It means the time students spending doing learning activities under a teacher's direction. 

Robinson addressed three specific points with his press release.  The FCAT program is allegedly too expensive, dominates the curriculum and hinders student success.

Robinson addresses the first point by framing FCAT expense in terms of its percentage of state and local investment in the schools.  He deals with the second point in the next paragraph:

Florida's Next Generation Sunshine State Standards are the foundation for what we expect our students to learn. Subjects covered by Florida standards include English language arts, math, science, social studies, physical and health education, world languages, and fine arts along with other content areas specific to colleges and careers. Contrary to the claim of the FSBA resolution, the FCAT neither drives the curriculum nor narrows the educational experience of Florida students. In fact, at the middle school level, student enrollment in courses such as dance, drama, and world languages has increased more than student enrollment in the subject areas assessed on the FCAT. At the high school level, enrollment in dance, world languages and the humanities has outpaced the growth in student enrollment.

PolitiFact ignores this context and tries to make it look like Robinson is saying that the schools spend very little time trying to prepare students for the FCAT.  The above paragraph from Robinson unequivocally puts the lie to the journalists' frame.  He addressed the issue by saying that the test measures things that students ought to spend much of their time learning.

Obviously, then, Robinson was not trying to say that students spend less than 1 percent of their instructional time preparing for the FCAT.  That would argue against his preceding paragraph. 

PolitiFact just ignored the context.

In fact, Robinson's 1 percent figure is more accurate the more time students spend preparing for the FCAT, if one insists on suggesting that he used "instructional time" to refer to test preparation.  For review (bold emphasis added):  "These assessments average two to three per student per school year and account for less than 1 percent of the instructional time provided during the year."  If students spend as little as half of their instructional time on the FCAT then the estimate for time spent on the assessments rises to a range between .52 and 1.8 percent--still in the neighborhood of 1 percent if we willfully ignore Robinson's clear explanation.

PolitiFact appears to conflate the "assessments" with the "instructional time" somehow.  As one is used as a percentage of the other, that path leads to a spectacularly failed fact check.

PolitiFact devotes considerable space to quotations complaining about the FCAT, but none of it addresses Robinson's claim.  We can skip on to the wondrous conclusion consisting of four short paragraphs plus a "False" rating.

One:

Robinson said that the FCAT tests "account for less than 1 percent of the instructional time provided during the year." This was a prepared statement, based on research done by his staff, in response to FCAT critics who say that schools devote too much time to the tests.

Right, but Robinson addressed the critics' point in his preceding paragraph.

Two:

Readers could assume that by "instructional time" Robinson was including regular lesson time in the classroom preparing for the FCAT. He wasn’t. His office says that referred to the number of minutes taking the test out of the total minutes of instruction per year. But he didn’t provide that explanation in his statement.

Robinson may have assumed that those reading his statement have the ability to read.  PolitiFact doesn't explain why people would be seriously misled if they believed taking the FCAT test constituted less than 1 percent of the preparation time.  The smaller the percentage the greater the preparation time by proportion.

It is doubtful PolitiFact could make a coherent case that Robinson was minimizing the preparation time for the test, assuming it would even make the attempt.

Three:

In reality, there is no clear way to quantify how much time teachers spend preparing students for the test. Some teachers say they spend practically all their time on the FCAT.

If all of the teachers spend all of their time on the FCAT then Robinson's statement is perfectly accurate and students spend less than 1 percent of their instructional time taking the assessment.  There's nothing here to use against Robinson's accuracy or veracity.

Four:

Robinson’s goal was to deflect criticism that too much time is spent "teaching to the test." He is suggesting that the FCAT eats up only a smidgen of a school year. But for students, parents and teachers who spend months preparing for those tests, Robinson’s words are misleading.

PolitiFact completely missed Robinson's point by ignoring the context.  The fact check doesn't make a lick of sense.  Robinson's words are misleading if one ignores context, grammar, syntax and logic.


The grades:

Amy Sherman:  F
Angie Drobnic Holan:  F

This case perhaps makes up PolitiFact's penultimate train-wreck to date.  PolitiFact owes Robinson an apology and a front page correction notice.


Update Aug. 1, 2012:

 Gerard Robinson responds to PolitiFact Florida:

In this article, PolitiFact agreed that evidence provided to them by the Department of Education verified the accuracy of my statement when taken in this context.  However, PolitiFact claims my statement was made to deflect criticism that too much time is spent “teaching to the test.”  If this were true, I would agree with PolitiFact’s conclusion.  However, since PolitiFact misrepresents the context of my statement, I rate their finding as False.

Update 2, August 3, 2012:

A Band-Aid for the severed head:  PolitiFact modifies the quotation with the insertion of two ellipses--and no correction notice (at least as of now).

Robinson penned a June 15 response, which included these comments: "The FCAT neither drives the curriculum nor narrows the educational experience of Florida students. ... These assessments average two to three per student per school year and account for less than 1 percent of the instructional time provided during the year. ... It is worth noting that local school boards require students to take many more assessments than those required by the state."

The correction is not quite compliant with AP style by my reading, as the AP Stylebook's example shows a line of space between the sentences prior to the insertion of the ellipse when one pulls sentences from different paragraphs..

That's a minor point, of course.  The story remains incoherent.  The head is still severed and the body is bleeding out through the neck.

Grading PolitiFact: More unprecedented Obama

Words matter -- We pay close attention to the specific wording of a claim. Is it a precise statement? Does it contain mitigating words or phrases?
--Principles of PolitiFact and the Truth-O-Meter

The issue:

(clipped from PolitiFact.com)

The fact checkers:

Amy Sherman:  writer, researcher
Angie Drobnic Holan:  editor


Analysis:

One would think that persons paying close attention to the specific wording of a claim would accurately note the difference between "outspent" and "out-raised."

PolitiFact give an accurate enough account of the statement from President Obama and its context in a fundraising letter.

Then comes the odd turn:

We researched whether Obama’s team could be outraised by Romney and concluded that there are too many unknowns to determine if his prediction will come true. (Read our extended analysis of the issue.)

It's okay to research whether Mitt Romney would outraise Mr. Obama so long as one maintains the distinction between raising money and spending it.  Certainly it's far easier to outspend an opponent if one has raised more money.

The fact check proceeds to focus on fundraising and concludes that presidential challengers Ronald Reagan and Bob Dole outraised the incumbent in 1980 and 1996 respectively.

The issue of spending gets somehow misplaced along the way and we end up with the following conclusion (bold emphasis added):

Obama was wrong to suggest that there hadn’t been other presidents in modern history to be outraised in their re-elections. Looking at numbers alone, Carter and Clinton fall into that category. But what we are comparing is  primary races then with total fundraising for the general election now.

We think 1980 and 1996 certainly fall under "modern history," but the campaign finance picture has changed dramatically since then, and there are important caveats to those races. Those earlier races involved public financing -- this time Obama and Romney are forgoing those dollars.

PolitiFact rates the statement "Mostly False," defined as "The statement contains an element of truth but ignores critical facts that would give a different impression."

Yes, the fundraising letter ignores the critical fact that the modern era for incumbent presidents begins this year with Mr. Obama's campaign.  Both campaigns eschewed public financing for this election.  But what was the element of truth in the statement?  It was false that Mr. Obama would be the first incumbent outspent by his rival.

The fact check doesn't tell you what's true about the statement, if anything.

We are left to guess whether Mr. Obama receives credit for the underlying argument, minus the unprecedented second-place fundraising, that the Romney campaign will outspend the President's campaign.  And the fact check never gets around to checking on spending after looking at fundraising.

In the end, we get another example of PolitiFact applying its standards unequally.  PolitiFact checks the wrong fact--fundraising instead of spending--and after that ignores its own "burden of proof" standard that assumes a claim false if PolitiFact does not discover evidence to support the claim.

The grades:

Amy Sherman:  F
Angie Drobnic Holan:  F

The PolitiFact team checked the wrong fact, so readers will not find out from this story whether past incumbent presidents were outspent by their challengers.  The PolitiFact team gave Mr. Obama a "Mostly False" rating without explaining what aspect of the claim it found true.  That rating appears to contradict PolitiFact's statement of principles.

PolitiFlub: Fact-checking Limbaugh on the Bane/Bain connection

PolitiFact is a joke.

One of PolitiFact's latest fact checks purports to find the truth about Rush Limbaugh's supposed claim that the new Batman movie's villain, "Bane," was chosen by design to damage Republican presidential candidate Mitt Romney through the latter's association with Bain Capital.

Setting aside the strong probability that Limbaugh was making a joke, this fact check is complete nonsense.  It's easy to see in PolitiFact's conclusion:

In politics, conspiracies are everywhere if you look hard enough. But Limbaugh’s superpowers of persuasion can’t make this theory stand up. The villainous Bane first appeared in Batman comic books in 1993, long before Romney entered presidential politics. Even the character’s creator called a suggested link "ridiculous." We rate this statement Pants on Fire!

It should be absolutely obvious that it doesn't matter when the Bane character was created.  I could use a cartoon character named "Boosh" from 200 years ago and make a movie out of it today to mock George W. Bush.  It isn't relevant whether the character was created without any knowledge that George W. Bush would serve as president.

If one takes Limbaugh's claim as a serious one, which is doubtful on its face, one fact checks it by looking at those who decided to make Bane the villain in this movie and when that decision was made, for starters.

The fact check is completely illogical, and that tells us something about PolitiFact.

PolitiFact's use of experts

I'll keep this PolitiFact criticism short and sweet, using Monday's PolitiFact Florida rating of Sen. Marco Rubio as an example of a recurrent phenomenon.

PolitiFact:

We wondered where [Rubio] got his $800 billion tax total, as well as the validity of the idea that the law does not "discriminate between rich and poor." So we decided to check it out.

Time for some experts.  PolitiFact:

We consulted several experts, who disagreed on the best way to account for the law’s tax increases.

A couple experts said Rubio’s $800 billion figure is valid, even if it is out of date and does not account for billions in tax credits.

The $800 billion figure is valid according to two experts.  Therefore it makes perfect sense for PolitiFact to rule the statement "False."

PolitiFact arbitrarily dismissed the testimony of two experts.  Now that's fact checking.

What does expert testimony mean if one obtains varying testimony and then picks one view while dismissing the others?  Not much.  It's just another layer of bias--editorial bias--that helps show that PolitiFact can't be trusted.

PolitiFact and the unimportance of accuracy in fact checking

If you disagree with a ruling, we encourage you to e-mail the writer or editor with your comments about our ruling.
--Principles of PolitiFact and the Truth-O-Meter

We strive to make our work completely accurate. When we think we make a mistake, we correct it if we feel like it and note it on the original item.
--The (annotated) Principles of PolitiFact

Is accuracy important in fact checking?

Most would say it is, I suspect.  My many criticisms of PolitiFact are peppered with the message that using the fact check label raises the necessary journalistic standard.

PolitiFact often seems strangely ambivalent about its standard of accuracy.  I occasionally send a message to the writer and/or editor of a PolitiFact story to point out problems.  Occasionally I get a warm reply.  Somewhat more often I get a defensive reply that does not address the issue.  And most commonly I receive no reply and the problem in the fact check goes unresolved.  Sometimes I post about my attempts to alert PolitiFact about its errors under the tag "Piquing PolitiFact."

And here's the latest example, in response to a PolitiFact story titled "Nancy Pelosi says 'everybody' will get more and pay less under the health care law":

Dear Jon Greenberg,
cc Angie Drobnic Holan

Perhaps my favorite thing about PolitiFact is the way it asks readers to point out problems with its stories while it proceeds to ignore the problems readers point out at an apparently very high rate.

But enough about me.  Let's talk about you.

You wrote:

PolitiFact has looked at Republican allegations that the law is bad because it will drive up premiums for most people and found those statements False.

Are you sure about that?  I found one example (rated "Pants on Fire," which I'd count as false to your benefit).  Perhaps you should include the citations in the sidebar if you're going to write such things.

You also wrote:

But in its study, the CBO number crunchers actually predict that a large number of people will want to buy better insurance. They would opt for a lower deductible, lower co-pays and a wider range of benefits. That personal choice would cause their total premiums to rise some 27 to 30 percent. For a policy covering just one person, the difference would be about $600 a year, before accounting for any of the subsidies the law provides; over half the people in the individual market would qualify for those subsidies.

The above obscures the meaning of the explanation you later quote from  Ed Haislmaier of the Heritage Foundation and runs largely counter to the explanation occurring in the CBO report (note the recurring term "requirement"):

The main elements of the legislation that would affect the amount of coverage purchased are the requirement that all new policies in the nongroup and small group markets cover at least a minimum specified set of benefits; the requirement that such policies have a certain minimum actuarial value; and the design of the federal subsidies, which would encourage many enrollees in the exchanges to join plans with an actuarial value above the required minimum. (The excise tax on high-premium plans would also affect the amount of coverage purchased; the impact of that tax is discussed in a separate section of this analysis.) Those provisions would have a much greater effect on premiums in the nongroup market than in the small group market, and they would have no measurable effect on premiums in the large group market.

Specifically, because of the greater actuarial value and broader scope of benefits that would be covered by new nongroup policies sold under the legislation, the average premium per person for those policies would be an estimated 27 percent to 30 percent higher than the average premium for nongroup policies under current law (with other factors held constant). The increase in actuarial value would push the average premium per person about 18 percent to 21 percent above its level under current law, before the increase in enrollees’ use of medical care resulting from lower cost sharing is considered; that induced increase, along with the greater scope of benefits, would account for the remainder of the overall difference.

The CBO counts two requirements and one personal choice.  You only mention personal choice and obscure the reason for it, that being people's well-understood tendency to spend more if they perceive that they're getting a good deal. The good deal comes primarily from subsidies.

Who cares?

You?  Angie Holan?

We'll see, won't we?

Cheers,
Bryan White

I sent the message on July 8.  Five full days later the Pelosi fact check continues to contain the same two apparent errors.

This type of result, in my experience, is typical of PolitiFact.


Correction 7/14/2012:  Supplied missing text just prior to the quoted email text.

PolitiFlub: PolitiFact puts a price tag on ObamaCare

Hilarious.

In fact, the CBO has said that overall the health care bill actually reduces government spending by about $124 billion over 10 years.

PolitiFact makes this claim in the context of a story about spending under the Accessible Care Act, also known as ObamaCare.

PolitiFact doesn't use "spending" normally, here.

People use "spending" to refer to outlays.  If you make $2000 per month and spend $1500 on rent and essentials while saving $500, your spending for the month is $1500.  PolitiFact would have you count your income against your expenses and find that you spent -$500 for the month.  But that's a budget surplus, not a reduction in spending.

It's misleading to present a budget surplus or deficit as a spending figure.

No serious person thinks the ACA reduces spending.

The closest to that we can get comes from crediting the ACA with reducing the increase in Medicare spending.  We'll ignore the cautions from Medicare's chief actuary that the projections are unrealistic.  Here's what the left-leaning Center for American Progress says about Medicare savings: 

$424.4 billion: The amount of federal deficit reduction from 2010 to 2019 due to changes in Medicare coverage and payment rules under ACA. These include limits on annual rate increases for hospitals and other health care providers.

Nothing else in the ACA suggests a decrease in federal spending.  It spends more on subsidies and more on Medicaid.  The federal government will spend more with the ACA than without.

This CBO report may represent the source of PolitiFact's claim:

In March 2010, CBO and JCT estimated that PPACA and the provisions of the Reconciliation Act related to health care would produce a net reduction in federal deficits of $124 billion over the 2010–2019 period as a result of changes in direct spending and revenues.

Drop the part about "revenues" and it makes for an ObamaCare-friendly soundbite.  The CBO figures the ACA as a deficit reducer not because of decreased federal spending but because of tax increases combined with the accounting trick (built into the CBO's standard methodology) of not foreseeing changes to statutorily mandated decreases in health care provider reimbursements under Medicare (see the "doc fix").  The CBO has to pretend that the "doc fix" will not occur even though it very probably will occur regularly.

PolitiFact's misstatement is the stuff of misleading political ads.  But they call it fact checking.

Grading PolitiFact: Nancy Pelosi exempts ObamaCare from basic economics

“Even if you disagree, our articles will make you smarter.”
--attributed to PolitiFact editor Bill Adair by the Nashua Telegraph

The issue:

(clipped from PolitiFact.com)

The fact checkers:

Jon Greenberg:  writer, researcher
Angie Drobnic Holan:  editor

Analysis:

Why review PolitiFact's grading of Nancy Pelosi's truth-challenged statement when PolitiFact gives it a "False" rating and I don't think any "Pants on Fire" rating is fair?  It has to do with the epigraph above and a very recent PolitiFact Florida fact check of Gov. Rick Scott.

In short, PolitiFact misinforms.

First let's hit the transcript from "Meet The Press" (MSNBC, bold emphasis added):

DAVID GREGORY: Well, Republicans have said they won't waste any time to try to repeal this.  Is that fantasy from your point of view?

NANCY PELOSI: It's being the mouthpiece of the health insurance industry.  And we're saying let's not have them be in charge anymore.  Let the people be in charge of how they receive coverage and health care.  It's -- they'll bring it up, and when they bring it up they will ask for repeal, repeal of all the things I said that help children, help young adults, help seniors, help men or women who may have prostate cancer, breast cancer, whatever it is, any precondition.  And everybody will have lower rates, better quality care and better access.  So that's what they want to repeal, we're happy to have that debate.

Note to scouts:  Pelosi has trouble handling a softball down the center of the strike zone.

PolitiFact:

That last line is a bold statement, because "everybody" covers a lot of people. PolitiFact has looked at Republican allegations that the law is bad because it will drive up premiums for most people and found those statements False.

PolitiFact apparently neglected to publish its findings on those supposed Republican allegations.  Except maybe one.

But back to Pelosi and PolitiFact:

Pelosi’s statement allows us to make an assessment of the opposite point -- the law is good because everyone will save money.

Woohoo.

PolitiFact:

The Congressional Budget Office is designed to be the neutral analytic arm of Congress, and its study on how the ACA will affect insurance premiums is widely seen as the gold standard when it comes to projections. We called Pelosi’s office for background on her claim, and they pointed us to a CBO analysis of the law.

Pelosi’s staff highlighted part of the report that states that for people who buy insurance on their own -- the individual market as it’s called -- rates will be 7 to 10 percent lower. Rates for those in the "small group" market will be 1 to 4 percent lower; and there will be little appreciable change for those in the "large group" market.

PolitiFact points out that the explanation from Pelosi's office misinterprets the report.  An average decrease doesn't necessarily indicate that all share the savings. PolitiFact then explains a separate  problem inaccurately (bold emphasis added):

Second, the CBO did this section of its analysis assuming that people would buy the same kind of insurance they do today, without the new law. So if they have a $5,000 deductible and limited coverage today, the CBO analysts compared that to the rate they would pay in 2016 to get that same coverage under the law.

Because the ACA changes minimum standards for insurance plans in exchanges, the report calculates according to what people would pay today if they had the type of coverage the CBO expects them to purchase in 2016.  Some of today's insurance plans won't qualify for an exchange offering, so it makes no sense to calculate according to plans that will not exist on the exchange.

PolitiFact immediately follows up with another flawed explanation (bold emphasis added):

(I)n its study, the CBO number crunchers actually predict that a large number of people will want to buy better insurance. They would opt for a lower deductible, lower co-pays and a wider range of benefits. That personal choice would cause their total premiums to rise some 27 to 30 percent. For a policy covering just one person, the difference would be about $600 a year, before accounting for any of the subsidies the law provides; over half the people in the individual market would qualify for those subsidies.

The above explanation is contrary to the one offered in the CBO report (bold emphasis added):

The main elements of the legislation that would affect the amount of coverage purchased are the requirement that all new policies in the nongroup and small group markets cover at least a minimum specified set of benefits; the requirement that such policies have a certain minimum actuarial value; and the design of the federal subsidies, which would encourage many enrollees in the exchanges to join plans with an actuarial value above the required minimum.

While PolitiFact sells the resulting higher premiums entirely as a personal choice, the CBO explains them as two parts requirement and one part personal choice.   The personal choice, as it happens, reflects the opportunity to use other people's money (the subsidy) to obtain a better relative value on one's own expense.  A New York Times blog explains it well:

The premium subsidies will be based on the price of the second-lowest-cost silver plan in a geographic area. If you sign up for that particular silver plan, you will pay no more than 9.5 percent of your income in premiums. But if you decide you want lower cost-sharing and want to spring for a pricier gold or platinum plan, you will be responsible for paying the difference between that silver plan premium and those of the more expensive plans, pushing your share of the premium over that 9.5 percent cap, said Jennifer Tolbert, a health policy expert at the Kaiser Family Foundation.

Going with a more expensive plan may make sense, depending on your health, family medical history or other factors. “If they’re paying 20 percent of their income now for insurance, it may be worth it to pay 11 or 12 percent for a more generous plan on the exchange, because they’d get much better coverage,” Ms. Tolbert said.

And of course somebody has to pay for the subsidy in any case.

PolitiFact quoted Heritage Foundation's Ed Haislmaier regarding the tendency of subsidy recipients to upgrade their level of insurance but probably provided insufficient context for readers to properly understand what he was saying.

PolitiFact had Pelosi dead to rights on her trifecta of falsehood but ignored everything for the sake of the fact check except her claim about lower rates.  And then PolitiFact botched the reporting on that issue, making the ACA look better than it is.

Smarter.  Right.


The grades:

Jon Greenberg:  F
Angie Drobnic Holan:  F

Afters:

Pelosi completely misrepresents CBO data and gets a "False" rating from PolitiFact.  Rick Scott accurately represents CBO data and gets a "Mostly False" rating because he supposedly left out a huge amount of critical context.

The ratings make perfect sense in a liberally biased kind of way.

Grading PolitiFact (Florida): Rick Scott and rising insurance premiums under ObamaCare

PolitiFact regularly engages in prodigious spin on behalf of the health care reform bill.  The following case featuring Gov. Rick Scott of Florida and PolitFact's state operation in Florida serves as just one example among many.

The issue:

(clipped from PolitiFact.com)

The fact checkers:

Katie Sanders:  writer, researcher
Angie Drobnic Holan:  editor


Analysis:

PolitiFact quotes Rick Scott:

"(W)e know the Congressional Budget Office said if you’re going to buy your own policy with these exchanges you’ll be paying 10 percent more, or a family will. So about $2,100 more for a family. So you’re going to pay more with these exchanges."

PolitiFact finds that Scott spoke accurately:

As Scott said, CBO expected the average premium per person in new individual policies would rise 10 percent to 13 percent in 2016 compared with where it was before the law took effect.

In this market, average premiums per policy in this market would be about $5,800 for single policies (a $300 increase) and $15,200 for families (a $2,100 increase -- just like Scott said), according to CBO (pages 5 and 6).

If only the report ended there.

The third paragraph hints at significant caveats, especially since we note from the graphic at the top of the story that Scott receives a "Mostly False" rating.  Before we move on, however, note that Scott used the conservative end of the CBO estimate to back his statement.  He used 10 percent instead of 13 percent.  Sometimes PolitiFact uses such factors to award extra credit.  Apparently not this time.

Now for some mighty PolitiFact spin.

PolitiFact explains that using an "apples to apples" comparison in the same CBO report leads to the CBO's conclusion that the provisions of the PPACA lead to a net savings of 7 to 10 percent on the average premium.  The "apples to apples" comparison shows that the exchange system does provide some features that reduce insurance premiums.  The CBO report estimates that for equivalent plans the exchange would save 7-10 percent compared to current law.  Of course the exchanges require much higher levels of insurance, than the current average, and this spikes the cost of premiums.

The CBO's estimate of  the ACA's effect on nongroup insurance premiums

The CBO's chart makes clear that the increase in premiums stems from increases in insurance coverage.  Curiously, PolitiFact calls this a benefit:

People will pay more, but it will be for a bigger swath of benefits. The law requires insurance companies to offer an "essential health benefits" package that would mirror benefits people get through employer plans.

People receive benefits from insurance when their insurance pays for something.  A healthy person who goes to the doctor once per year for 10 years regardless of insurance coverage is not receiving any additional benefit from insurance coverage.  The person is paying much more for the same benefits.

PolitiFact actually refers to coverage when it refers to benefits.  The expanded scope of health coverage primarily enables the government to expand the pool of people paying for insurance so that healthy people who do not have an immediate need of insurance will help pay the benefits of others--a much wider swath of benefits.  In short, overall medical costs go way up while the premium costs per individual go down.  This from a bill that was sold as a means of controlling rising medical costs.  It doesn't do much at all to control costs.  Instead, it coerces the people into paying for higher overall costs.  And the bill contains measures that shift  costs from those who present the biggest risk to those with thicker wallets.

Where does that leave us on this fact check?  PolitiFact charges Scott with leaving things out, and we'll address those charges individually.

Many (CBO estimates 57 percent) seeking nongroup insurance through state exchanges would receive subsidies.

Scott specifically referred to persons paying for their own insurance through the exchanges, which implicitly acknowledges subsidies.  The availability of subsidies through the exchanges should be common knowledge.  It's hard to see why PolitiFact should fault Scott for this omission, especially when the subsidies drive overall costs up instead of bringing them down.  Insurance subsidies represent progressive partial free riding.

Though the insured pay more, they receive a "bigger swath of benefits"

PolitiFact tries to make this seem like a huge benefit, but it's kind of like getting an expensive Buick when all you really need is a Kia.  There's no more Kia.  The Buick is the new entry-level automobile.  And maybe the Buick has leather upholstery, but then again maybe you're a vegan (celibates paying for contraception and pregnancy insurance).  Again,. there's no reason why Scott should need to provide this detail.  He's giving reasons why he's not setting up an exchange.  His objections have to do with overall cost and the priorities of Floridians.  When everyone's driving a Buick, people are paying more for transportation even if Buick drops its prices down to wholesale.

Exchanges do help lower costs in an "apples to apples" comparison

To the extent that Gov. Scott's statement suggests that exchanges do nothing to encourage cost reductions, PolitiFact has a point.  Buick dealers can compete against each other to provide the lowest-cost Buick and that competition does have an effect, not to mention economies of scale.  But Scott emphasizes overall health care costs, so he is justified in keeping the emphasis on the overall increase in premiums.

"People purchasing their own health insurance comprise less than one-fifth of the market"

People who are not purchasing their own health insurance are not purchasing it through an exchange, so this point is irrelevant.  There's no reason for Scott to mention it, since he's giving reasons for not setting up a state exchange.  In fact, the smaller the share of the market, arguably the greater justification Scott has for not setting up an exchange.

PolitiFact routinely asserts that Scott leaves out important information.  PolitiFact routinely leaves out the justification for calling the information important.  Scott's omissions were of borderline relevance at most.

PolitiFact:

Scott said that the Congressional Budget Office said people would pay 10 percent more for policies on the exchange, "so about $2,100 more for a family." What he doesn’t say is that these policies will have to offer comprehensive coverage. So people will pay more, but they’re also get more benefits. Additionally, the federal government will offer subsidies to many of these people to cut the cost.

It’s also important to remember the CBO’s "apples-to-apples" comparison. According to the agency, people in the individual market will actually pay less for the required amount of benefits under the Affordable Care Act than they would for those same benefits under old policies.

We rate Scott’s statement Mostly False.

Note that while Scott emphasized overall medical costs to the state of Florida and its people, PolitiFact's objections all center around costs to the individual.  PolitiFact ignored Scott's central point and graded him according to a standard that failed to respect the context of his remarks.

The "Truth-O-Meter" ruling, as is so often the case, represents an absurdity.  PolitiFact defines "Mostly False" as a statement that "contains an element of truth but ignores critical facts that would give a different impression."  But Scott's statement does not simply contain an element of truth.  His statement is perfectly accurate and even takes the lower premium increase estimate from the CBO report.  None of PolitiFact's caveats alter the impression that a family paying for its own insurance faces a 10 percent hike, unless it shows that the CBO said the hike might end up at 13 percent instead.

This case points up again that despite PolitiFact's continued assertions that "words matter" it makes a great big exception for itself when it comes to defining the "Truth-O-Meter" grades.

Scott's statements were more truthful than PolitiFact's, using the same measure.


The grades:

Katie Sanders:  F
Angie Drobnic Holan:  F

This story reads like a PPACA apologetic, not like a fact check.  The PolitiFact team substituted its own point for Gov. Scott's point and graded Scott according to the result.  That's a wrong approach for fact checking, as PolitiFact admits in its statement of principles:

Context matters -- We examine the claim in the full context, the comments made before and after it, the question that prompted it, and the point the person was trying to make.

PolitiFact may think it grand that everyone in the exchange gets a good deal on a Buick, but that doesn't undermine Scott's point that getting everyone a Buick imposes a heavy burden in terms of cost.

Buick.com

Grading PolitiFact: Obama's "Julia" and Medicare

Context matters -- We examine the claim in the full context, the comments made before and after it, the question that prompted it, and the point the person was trying to make.
--Principles of PolitiFact and the Truth-O-Meter

I've written quite a bit about PolitiFact's problem with selection bias.  This fact check serves as an excellent example of a related and perhaps even more pernicious form of bias, that arising from the selection of a story's focus.  Two stories dealing with the same topic, such as the Medicare claim in the Obama campaign's "Julia" ad, can turn out very differently.  And the ideology of the writer or editor can easily alter the story focus.


The issue:

(clipped from PolitiFact.com)

The fact checkers:

Becky Bowers:  writer, researcher
Angie Drobnic Holan:  editor


Analysis:

As I pointed out in an earlier review of a "Julia" ad fact check, the context of the Obama campaign's claims about Mitt Romney has to do with hypothetical comparisons between her life under Obama's policies and Romney's policies.  Here's a closeup of the relevant part of the ad:

(click image for larger view)

For Obama we see the best-case scenario.  For Romney we get the worst-case scenario.  Medicare "could" end as we know it under either president's policies extrapolated into the future.  In Obama's case Medicare could end through a rationing regime.  In Romney's case it would end by adding expenses for recipients of the benefits to help mitigate the third-party payment dynamic that makes Medicare spending unsustainable.  Obama's argument is cherry-picked.

PolitiFact ignores Obama's point of comparison, narrowing the focus specifically to the claim regarding Romney's plan for Medicare:

We wondered, under Romney’s policies, is it true that, "Medicare could end as we know it, leaving Julia with nothing but a voucher to buy insurance, which means $6,350 extra per year for a similar plan"?

PolitiFact, at least for purposes of this fact check, does not wonder whether it is fair and proper for the Obama campaign to compare the cherry-picked rosy picture of Medicare under Obama with the cherry-picked bleak prospects for Medicare under Romney's supposed plan.  Julia's prospects under Romney could have read "Julia enrolls in Medicare, helping her to afford preventive care and the prescription drugs she needs."

Unfortunately for the Obama campaign, PolitiFact did find a few things wrong with the ad.

1)  It bases its statement of Romney's position on the older Paul Ryan budget proposal.  Romney has said he agrees in principle with the bipartisan Wyden/Ryan plan which is similar in some ways to Ryan's old plan and different in others.  Plus Romney has a plan of his own about which we know few specifics.

2)  Because of the outdated analysis, the ad has no foundation for its claim about the $6,350 voucher figure it cites.

3)  Various features in each of the plans undercut the claim that "Julia" would end up with "nothing but a voucher to buy insurance."

But the news for the Obama campaign wasn't all bad.

PolitiFact originally rated the claim "False," but after "readers" pointed out an error in the story, PolitiFact reconsidered and upgraded the rating to "Mostly False."  Woohoo.

What was the error?

PolitiFact: 

(R)eaders pointed out we had incorrectly said that the Wyden-Ryan and Romney plans for Medicare offered traditional fee-for-service Medicare alongside a premium-support or "voucher" system for purchasing private insurance. Instead, both plans offer traditional Medicare inside of a new premium-support system, as another plan that competes with private insurance.

One might well wonder why that fact makes a difference.  Unfortunately, PolitiFact made the original story unavailable.  That makes finding all the specific changes difficult.  But the summary paragraph likely includes most or all of the justification for the higher rating.

PolitiFact:

The Obama campaign ties Romney to an outdated plan with less generous spending growth and no traditional Medicare option. Romney does support a voucher-like system, but the graphic ignores critical facts that would give a different impression about his plan. We rate the claim Mostly False.

See, the Romney plan is "voucher-like" so saying his plan could only give "Julia" a voucher is true enough to qualify as "Mostly False."

By narrowing the focus to a fine point PolitiFact locates a portion with sufficient truth to avoid a "False" rating.  The problem with the fact check isn't the rating on this narrow point.  The problem comes from PolitiFact failing to provide any mention of the way the particle of truth powers a dynamo of deception.  The ad's argument is a stinker.  PolitiFact's ruling gives it more cover than it deserves.


The grades: 

Becky Bowers: D
Angie Drobnic Holan:  D

I've seen so many fact checks much worse than this one from PolitiFact that perhaps I'm being overly generous.  Bottom line:  A serious fact check takes into account the underlying argument and the context.  The overall argument of the Medicare portion of the "Julia" ad was deceptive and designed to scare voters with its manipulated comparison.  PolitiFact did not allow its focus to fall on the main issue.

PolitiFlub: PolitiFact Florida and the loss of existing insurance

PolitiFact Florida published a fact check yesterday parallel to one I blasted just last week concerning the loss of existing insurance plans with the implementation of the health care reform bill.

The PolitiFact Florida conclusion:

Our ruling

The U.S. Chamber said, "Obamacare could cause 20 million people to lose their current coverage." It's a claim that is oft-repeated and much exaggerated.

The chamber employs a worst-case projection by a nonpartisan research agency. The agency’s other forecasts are lower, but you wouldn’t know that from watching the ad. And unlike Priebus, the group does not specify that the type of insurance potentially affected is a specific type of insurance -- "employer-based" insurance.

Plus, this 20 million estimate only counts people who receive coverage from their employer -- and not those who might receive better coverage elsewhere.

Most importantly, this figure does not represent uninsured people who will get coverage because of the law.

We rate it Mostly False.

How doth PolitiFact mislead thee?  Let me count the ways ...

1)  "The chamber employs a worst-case projection by a nonpartisan research agency"  Following as it does on the heels of "oft-repeated and much exaggerated," this sounds bad.  But the worst case scenario is exactly how one ought to support a claim of potential damage ("could cause 20 million people to lose their current coverage").  PolitiFact's observation is a positive posing as a negative.

2)  "The agency’s other forecasts are lower, but you wouldn’t know that from watching the ad." Hidden context?  One wouldn't know that "could cause 20 million" calls for a high-end estimate from reading PolitiFact's fact check.  Basic English interpretation tells the viewer that "could cause" implies a high-end estimate and a worst-case scenario.  But the 20 million figure is neither of those in the context of the ad.

3)  "(T)he group does not specify that the type of insurance potentially affected is a specific type of insurance--'employer-based' insurance."  PolitiFact doesn't tell you that by using a number specific to one type of insurance it implies that any loss from other types of insurance gets added to the worst-case scenario number.  Consider all types of insurance and the number can only be equal or higher.  So how is it misleading except by underestimating the worst-case scenario?

4)  "Plus, this 20 million estimate only counts people who receive coverage from their employer -- and not those who might receive better coverage elsewhere."  This isn't even a coherent objection.  PolitiFact is probably trying to say that the 20 million figure includes those who opt for "better" coverage apart from an employer-offered plan.  The report on which PolitiFact relied makes no reference to spurning the employer offer for a "better" plan unless by "better" PolitiFact means cheaper.  The report indicates that customers offered more than one option for health insurance strongly prefer less expensive plans.  The report does not mention any alternative motivation, so PolitiFact apparently dreamed up the one it mentions.  It's possible to argue that those who leave employer insurance because of pricing are not forced into cheaper insurance plans, but PolitiFact's implication that consumers freely choose better plans obscures the factors at work and allows readers to assume that the "better plans" idea comes from the CBO/JCT report.  It doesn't.

5)  "Most importantly, this figure does not represent uninsured people who will get coverage because of the law."  A figure for persons losing current coverage should never represent uninsured people who will get coverage because of the law.  Uninsured people have no existing coverage to lose in the first place.  PolitiFact's most important point is irrelevant to the accuracy of the 20 million figure.  At most, it is relevant as context trying to balance the 20 million figure against a benefit of the health care reform law.

6)  A real fact check would have looked for types of existing insurance other than employer insurance likely lost as a result of the health care law.  The same CBO/JCT report lists such losses on the line below losses to employer insurance.  PolitiFact somehow neglects to notice (the figure is an additional 3 million for the baseline scenario and an additional 1 million for the worst-case scenario):

http://www.cbo.gov/sites/default/files/cbofiles/attachments/03-15-ACA_and_Insurance_2.pdf

Nice job of plowing another train into your existing train-wreck, PolitiFact.

We rate it Mostly False.

Trading on what credibility, pray tell?

How did this fact check pass an editor's examination?

PolitiFlub: A static number of federal jobs programs?

What can you say about a fact-checking organization that's often no more accurate than the statements it critiques?

Case in point, PolitiFact did a check on Mitt Romney's claim that the federal government runs 49 jobs programs and oversees those programs using eight different agencies.

PolitiFact used as its key source a Government Accountability Office report that focused on fiscal year 2009.

The current year is 2012, and it is the year in which Romney made his statement.

PolitiFact made a very weak effort to update the GAO numbers with reports of new federal job training programs or changes in the number of agencies overseeing said programs.  Indeed, PolitiFact did not make any mention of the Wall Street Journal column that probably served as Romney's source:

There are no fewer than 49 federal job training programs administered by nine agencies that cost taxpayers some $14.5 billion in 2010. A General Accountability Office performance audit in 2011 looked at fiscal year 2009 and determined that "only 5 of the 47 programs have had impact studies that assess whether the program is responsible for improved employment outcomes."

If the Journal report is correct then Romney is off by one as to the number of agencies overseeing job training programs and did not overestimate the number of federal jobs programs.  PolitiFact, if the Journal report is correct, was accurate regarding the number of agencies but off by at least two regarding the number of programs, with the latter an understatement of the actual number.

It looks very much like PolitiFact simply whiffed on the attempt to check for new information updating the GAO numbers.  Without that information PolitiFact has no business trying to rate the truth of Romney's claim.

(for Glenn Kessler and PolitiFact) How to fact check the job recovery numbers

A valuable media watchdog watchdog post at the new blog "counterirritant" pointed out a problem with Glenn Kessler's fact check of a claim from Republican presidential candidate Mitt Romney.

Kessler writes the fact checker feature for the Washington Post.  Romney claimed that in a normal recovery from recession the country should be adding something like 500,000 jobs per month.

counterirritant:

Kessler decided that the best way to “check” this was determine how frequently 500,000 jobs were created in a month in the last 65 year.

The post goes on to very effectively criticize Kessler's methodology throughout.

By a funny coincidence (general leftward lean of the mainstream media, maybe?), PolitiFact used very similar reasoning on the same claim:

Is 500,000 jobs created per month normal for a recovery?

The short answer is "no."

We arrived at this conclusion by looking at the net monthly change in jobs all the way back to 1970. Since Romney was referring to total jobs, rather than private-sector jobs only, we used total jobs as our measurement. And since Romney was talking about job creation patterns during a recovery, we looked only at job creation figures for non-recessionary periods, as defined by the National Bureau of Economic Research. Finally, we excluded the current recovery.

The Kessler/PolitiFact method is entirely wrongheaded.

Romney didn't claim that 500,000 jobs created per month was a normal figure during a recovery.  I can imagine the furrowed brows of Kessler, Louis Jacobson and other mainstream media fact checkers.  Aren't they the experts?  What am I talking about?

It's actually pretty simple.

The size of the economy changes.  If country A enters a recession losing 1 million jobs  then it takes two months to regain the lost jobs at a rate of 500,000 per month.  If country B experiences a recession losing 10 million jobs then it takes 20 months to regain them at a rate of 500,000 per month.

Not only does the size of the economy vary, but so does the depth of the recession.  The rate of recovery for lost jobs needs  to account for both factors.  Neither Kessler nor PolitiFact gave any apparent consideration to those critical criteria.  It's like comparing prices between now and the 1950s without adjusting for inflation.

The Romney campaign has made a number of statements like the 500,000 jobs claim, and they probably relate to the following chart or one like it from the Bureau of Labor Statistics:

Clipped from nytimes.com

Romney's claim almost certainly derives from the fact that post-war recoveries usually replace lost jobs much faster than the present recovery. 

So how does one check that claim?  It's not that hard.  Take the bottom point of employment, then count the number of months that it takes to get employment back up to the peak level.  Divide the number of jobs lost by the number of months it took to return to return to the employment peak at the start of the recession.  Do the same for each of the post-war recessions, then average the numbers to obtain the average job recovery time.  After all of that, divide the number of jobs lost from the 2007 recession by the averaged job recovery time.

Why didn't the Washington Post or PolitiFact do anything remotely resembling the fact check I just described?  It could be gross incompetence.  It could be ideological bias.  Or it could be both.





Correction 6/7/2012  Removed a redundant division operation from the next-to-last paragraph

Grading PolitiFact: Marco Rubio and taxing small business

It's the old "true but false" trick.

The issue:

clipped from PolitiFact.com

The fact checkers:

Angie Drobnic Holan:  writer, researcher
Bill Adair:  editor


Analysis:

First things first, the context:




PolitiFact provides a partial transcript with its quotation of Rubio:

To explain his vote, U.S. Sen. Marco Rubio, R-Fla., released a statement that day:

"As someone with a student loan and with a state with so many people with student loans, I support a hundred percent making sure that the interest rates on student loans do not go up," Rubio said.

"I cannot support the way the Democrats want to do it, however, because, they want to do it by raising taxes on small businesses, very small businesses. The kinds of small businesses that give jobs to graduates who not only need low interest rates but need jobs in order to pay their student loans."

Rubio went on:

"We have a plan to keep student interest rates low that doesn't raise taxes.  And I hope the Democrats will give us a vote and a chance to pass that."

Rubio tries to make the point that the Republican plan to keep student loan rates from rising does not raise taxes compared to a Democratic Party plan that does raise taxes, and Rubio mentions the entities affected by the tax hike.

What fact shall we check, PolitiFact?

We decided to check out Rubio's claim that Democrats want to raise taxes on "small businesses, very small businesses … the kinds of small businesses that give jobs to graduates."

It's not often one sees an ellipsis used to take the place of a period.  But at least we have the answer to the question.  Let's see how PolitiFact executes the fact check.

PolitiFact:

To fact-check Rubio's statement, we read the text of the legislation the Democrats were proposing, examined a summary of the bill provided by the nonpartisan Congressional Research Service, and ran the proposal by the experts at the Tax Policy Center, a respected independent think tank that focuses on taxes.

Reading the text of the legislation is a good move.  Going to an expert from the Tax Policy Center isn't a bad move, either, if we overlook the fact that it is associated with the center-left Brookings Institution as well as the fact that the consultation with experts stops there.

It doesn't exactly represent a good faith effort to "interview impartial experts."  But let's move on to PolitiFact's reasoning.

PolitiFact reports that the bill limits the tax increases to S-corporation beneficiaries who a) are married and make over $250,000 adjusted gross income, b) are married filing separately while individually making over $125,000 adjusted gross income or c) are individuals making over $200,000.  Democrats promoting the bill say it is targeted at closing a tax loophole that allows some parties to sidestep payroll taxes.

So let's say all that is true.  Is it therefore false that the Democratic proposal raises taxes on small (and very small) businesses?  If it does not affect small businesses then what types of businesses are affected?

PolitiFact doesn't say, at least not explicitly.  Instead we get an implicit answer:
Rubio said he opposed the Democratic bill on student loan interest because it would raise taxes on "small businesses, very small businesses … the kinds of small businesses that give jobs to graduates."

Actually, the bill changed tax rules only for S-corporations, and only on professionals like lawyers and accountants who could be taking advantage of the tax code to avoid paying payroll taxes ...We rate Rubio’s statement False.

PolitiFact just implicitly informed us that small and very small businesses are not taxed more under the Democratic proposal.  We deduce that therefore if any businesses are affected they are medium, very medium, large and/or very large businesses.

How did we end up with that apparently absurd conclusion?  Easy.  PolitiFact interpreted Rubio to say that the Democratic tax change affected small and very small businesses broadly speaking.  In other words, small businesses in general were the target of the tax change rather than a specific subclass of (small) businesses.

Why use that interpretation instead of taking Rubio to mean that the businesses paying for the tax would fall into the smaller business demographic?

That's the $64,000 question.  PolitiFact offers no reason justifying its interpretation other than to claim that people might well take it that way:

Rubio's statement gives the impression that all kinds of mom-and-pop operations might be subject to new, additional taxes, when actually the bill is aimed squarely at high-income professionals who are taking advantage of a loophole.

Is that good enough?

If it's good enough then why do we not equally pay attention to the risk that voters watching Obama's "Julia" cartoon will think a President Romney threatens their right to fight for equal pay?

Obama "Mostly True" and Rubio "False"?  Seriously?

Rubio's claim is literally true, assuming  some "very small" businesses would end up paying for the closed tax loophole.  One could present all types of caveats about the extent to which Rubio fails to present the full picture, but similar complaints could most certainly apply to the Obama campaign's "Julia" Web ad.

We end up with yet another outstanding illustration of the subjective nature of PolitiFact's grading system.  And once again the conservative receives the harm.


The grades:

Angie Drobnic Holan:  F
Bill Adair:  F


Taking PolitiFact's grading system literally, it should not be possible to grade a literally true statement like Rubio's "False."  Yet somehow you did it, PolitiFact.  Amazing.

PolitiFact has yet to learn how to apply the principle of charitable interpretation with anything remotely resembling fairness, assuming anyone on the staff even knows what it is.

Or maybe they're just in the tank for the Democratic Party.  I can't dismiss that hypothesis when PolitiFact produces stories like this one.


Clarification 5/15/2012: Traded out "In other words, such businesses would feel the effects of the law more commonly than not" in favor of  "In other words, small businesses in general were the target of the tax change rather than a specific subclass of (small) businesses)."  Later added sections b) and c) to the description of parties affected by the tax.  My apologies for the temporary omission.