PolitiFact's use of experts

I'll keep this PolitiFact criticism short and sweet, using Monday's PolitiFact Florida rating of Sen. Marco Rubio as an example of a recurrent phenomenon.

PolitiFact:

We wondered where [Rubio] got his $800 billion tax total, as well as the validity of the idea that the law does not "discriminate between rich and poor." So we decided to check it out.

Time for some experts.  PolitiFact:

We consulted several experts, who disagreed on the best way to account for the law’s tax increases.

A couple experts said Rubio’s $800 billion figure is valid, even if it is out of date and does not account for billions in tax credits.

The $800 billion figure is valid according to two experts.  Therefore it makes perfect sense for PolitiFact to rule the statement "False."

PolitiFact arbitrarily dismissed the testimony of two experts.  Now that's fact checking.

What does expert testimony mean if one obtains varying testimony and then picks one view while dismissing the others?  Not much.  It's just another layer of bias--editorial bias--that helps show that PolitiFact can't be trusted.

Grading PolitiFact (Florida): Rick Scott and rising insurance premiums under ObamaCare

PolitiFact regularly engages in prodigious spin on behalf of the health care reform bill.  The following case featuring Gov. Rick Scott of Florida and PolitFact's state operation in Florida serves as just one example among many.

The issue:

(clipped from PolitiFact.com)

The fact checkers:

Katie Sanders:  writer, researcher
Angie Drobnic Holan:  editor


Analysis:

PolitiFact quotes Rick Scott:

"(W)e know the Congressional Budget Office said if you’re going to buy your own policy with these exchanges you’ll be paying 10 percent more, or a family will. So about $2,100 more for a family. So you’re going to pay more with these exchanges."

PolitiFact finds that Scott spoke accurately:

As Scott said, CBO expected the average premium per person in new individual policies would rise 10 percent to 13 percent in 2016 compared with where it was before the law took effect.

In this market, average premiums per policy in this market would be about $5,800 for single policies (a $300 increase) and $15,200 for families (a $2,100 increase -- just like Scott said), according to CBO (pages 5 and 6).

If only the report ended there.

The third paragraph hints at significant caveats, especially since we note from the graphic at the top of the story that Scott receives a "Mostly False" rating.  Before we move on, however, note that Scott used the conservative end of the CBO estimate to back his statement.  He used 10 percent instead of 13 percent.  Sometimes PolitiFact uses such factors to award extra credit.  Apparently not this time.

Now for some mighty PolitiFact spin.

PolitiFact explains that using an "apples to apples" comparison in the same CBO report leads to the CBO's conclusion that the provisions of the PPACA lead to a net savings of 7 to 10 percent on the average premium.  The "apples to apples" comparison shows that the exchange system does provide some features that reduce insurance premiums.  The CBO report estimates that for equivalent plans the exchange would save 7-10 percent compared to current law.  Of course the exchanges require much higher levels of insurance, than the current average, and this spikes the cost of premiums.

The CBO's estimate of  the ACA's effect on nongroup insurance premiums

The CBO's chart makes clear that the increase in premiums stems from increases in insurance coverage.  Curiously, PolitiFact calls this a benefit:

People will pay more, but it will be for a bigger swath of benefits. The law requires insurance companies to offer an "essential health benefits" package that would mirror benefits people get through employer plans.

People receive benefits from insurance when their insurance pays for something.  A healthy person who goes to the doctor once per year for 10 years regardless of insurance coverage is not receiving any additional benefit from insurance coverage.  The person is paying much more for the same benefits.

PolitiFact actually refers to coverage when it refers to benefits.  The expanded scope of health coverage primarily enables the government to expand the pool of people paying for insurance so that healthy people who do not have an immediate need of insurance will help pay the benefits of others--a much wider swath of benefits.  In short, overall medical costs go way up while the premium costs per individual go down.  This from a bill that was sold as a means of controlling rising medical costs.  It doesn't do much at all to control costs.  Instead, it coerces the people into paying for higher overall costs.  And the bill contains measures that shift  costs from those who present the biggest risk to those with thicker wallets.

Where does that leave us on this fact check?  PolitiFact charges Scott with leaving things out, and we'll address those charges individually.

Many (CBO estimates 57 percent) seeking nongroup insurance through state exchanges would receive subsidies.

Scott specifically referred to persons paying for their own insurance through the exchanges, which implicitly acknowledges subsidies.  The availability of subsidies through the exchanges should be common knowledge.  It's hard to see why PolitiFact should fault Scott for this omission, especially when the subsidies drive overall costs up instead of bringing them down.  Insurance subsidies represent progressive partial free riding.

Though the insured pay more, they receive a "bigger swath of benefits"

PolitiFact tries to make this seem like a huge benefit, but it's kind of like getting an expensive Buick when all you really need is a Kia.  There's no more Kia.  The Buick is the new entry-level automobile.  And maybe the Buick has leather upholstery, but then again maybe you're a vegan (celibates paying for contraception and pregnancy insurance).  Again,. there's no reason why Scott should need to provide this detail.  He's giving reasons why he's not setting up an exchange.  His objections have to do with overall cost and the priorities of Floridians.  When everyone's driving a Buick, people are paying more for transportation even if Buick drops its prices down to wholesale.

Exchanges do help lower costs in an "apples to apples" comparison

To the extent that Gov. Scott's statement suggests that exchanges do nothing to encourage cost reductions, PolitiFact has a point.  Buick dealers can compete against each other to provide the lowest-cost Buick and that competition does have an effect, not to mention economies of scale.  But Scott emphasizes overall health care costs, so he is justified in keeping the emphasis on the overall increase in premiums.

"People purchasing their own health insurance comprise less than one-fifth of the market"

People who are not purchasing their own health insurance are not purchasing it through an exchange, so this point is irrelevant.  There's no reason for Scott to mention it, since he's giving reasons for not setting up a state exchange.  In fact, the smaller the share of the market, arguably the greater justification Scott has for not setting up an exchange.

PolitiFact routinely asserts that Scott leaves out important information.  PolitiFact routinely leaves out the justification for calling the information important.  Scott's omissions were of borderline relevance at most.

PolitiFact:

Scott said that the Congressional Budget Office said people would pay 10 percent more for policies on the exchange, "so about $2,100 more for a family." What he doesn’t say is that these policies will have to offer comprehensive coverage. So people will pay more, but they’re also get more benefits. Additionally, the federal government will offer subsidies to many of these people to cut the cost.

It’s also important to remember the CBO’s "apples-to-apples" comparison. According to the agency, people in the individual market will actually pay less for the required amount of benefits under the Affordable Care Act than they would for those same benefits under old policies.

We rate Scott’s statement Mostly False.

Note that while Scott emphasized overall medical costs to the state of Florida and its people, PolitiFact's objections all center around costs to the individual.  PolitiFact ignored Scott's central point and graded him according to a standard that failed to respect the context of his remarks.

The "Truth-O-Meter" ruling, as is so often the case, represents an absurdity.  PolitiFact defines "Mostly False" as a statement that "contains an element of truth but ignores critical facts that would give a different impression."  But Scott's statement does not simply contain an element of truth.  His statement is perfectly accurate and even takes the lower premium increase estimate from the CBO report.  None of PolitiFact's caveats alter the impression that a family paying for its own insurance faces a 10 percent hike, unless it shows that the CBO said the hike might end up at 13 percent instead.

This case points up again that despite PolitiFact's continued assertions that "words matter" it makes a great big exception for itself when it comes to defining the "Truth-O-Meter" grades.

Scott's statements were more truthful than PolitiFact's, using the same measure.


The grades:

Katie Sanders:  F
Angie Drobnic Holan:  F

This story reads like a PPACA apologetic, not like a fact check.  The PolitiFact team substituted its own point for Gov. Scott's point and graded Scott according to the result.  That's a wrong approach for fact checking, as PolitiFact admits in its statement of principles:

Context matters -- We examine the claim in the full context, the comments made before and after it, the question that prompted it, and the point the person was trying to make.

PolitiFact may think it grand that everyone in the exchange gets a good deal on a Buick, but that doesn't undermine Scott's point that getting everyone a Buick imposes a heavy burden in terms of cost.

Buick.com

PolitiFlub: PolitiFact Florida and the loss of existing insurance

PolitiFact Florida published a fact check yesterday parallel to one I blasted just last week concerning the loss of existing insurance plans with the implementation of the health care reform bill.

The PolitiFact Florida conclusion:

Our ruling

The U.S. Chamber said, "Obamacare could cause 20 million people to lose their current coverage." It's a claim that is oft-repeated and much exaggerated.

The chamber employs a worst-case projection by a nonpartisan research agency. The agency’s other forecasts are lower, but you wouldn’t know that from watching the ad. And unlike Priebus, the group does not specify that the type of insurance potentially affected is a specific type of insurance -- "employer-based" insurance.

Plus, this 20 million estimate only counts people who receive coverage from their employer -- and not those who might receive better coverage elsewhere.

Most importantly, this figure does not represent uninsured people who will get coverage because of the law.

We rate it Mostly False.

How doth PolitiFact mislead thee?  Let me count the ways ...

1)  "The chamber employs a worst-case projection by a nonpartisan research agency"  Following as it does on the heels of "oft-repeated and much exaggerated," this sounds bad.  But the worst case scenario is exactly how one ought to support a claim of potential damage ("could cause 20 million people to lose their current coverage").  PolitiFact's observation is a positive posing as a negative.

2)  "The agency’s other forecasts are lower, but you wouldn’t know that from watching the ad." Hidden context?  One wouldn't know that "could cause 20 million" calls for a high-end estimate from reading PolitiFact's fact check.  Basic English interpretation tells the viewer that "could cause" implies a high-end estimate and a worst-case scenario.  But the 20 million figure is neither of those in the context of the ad.

3)  "(T)he group does not specify that the type of insurance potentially affected is a specific type of insurance--'employer-based' insurance."  PolitiFact doesn't tell you that by using a number specific to one type of insurance it implies that any loss from other types of insurance gets added to the worst-case scenario number.  Consider all types of insurance and the number can only be equal or higher.  So how is it misleading except by underestimating the worst-case scenario?

4)  "Plus, this 20 million estimate only counts people who receive coverage from their employer -- and not those who might receive better coverage elsewhere."  This isn't even a coherent objection.  PolitiFact is probably trying to say that the 20 million figure includes those who opt for "better" coverage apart from an employer-offered plan.  The report on which PolitiFact relied makes no reference to spurning the employer offer for a "better" plan unless by "better" PolitiFact means cheaper.  The report indicates that customers offered more than one option for health insurance strongly prefer less expensive plans.  The report does not mention any alternative motivation, so PolitiFact apparently dreamed up the one it mentions.  It's possible to argue that those who leave employer insurance because of pricing are not forced into cheaper insurance plans, but PolitiFact's implication that consumers freely choose better plans obscures the factors at work and allows readers to assume that the "better plans" idea comes from the CBO/JCT report.  It doesn't.

5)  "Most importantly, this figure does not represent uninsured people who will get coverage because of the law."  A figure for persons losing current coverage should never represent uninsured people who will get coverage because of the law.  Uninsured people have no existing coverage to lose in the first place.  PolitiFact's most important point is irrelevant to the accuracy of the 20 million figure.  At most, it is relevant as context trying to balance the 20 million figure against a benefit of the health care reform law.

6)  A real fact check would have looked for types of existing insurance other than employer insurance likely lost as a result of the health care law.  The same CBO/JCT report lists such losses on the line below losses to employer insurance.  PolitiFact somehow neglects to notice (the figure is an additional 3 million for the baseline scenario and an additional 1 million for the worst-case scenario):

http://www.cbo.gov/sites/default/files/cbofiles/attachments/03-15-ACA_and_Insurance_2.pdf

Nice job of plowing another train into your existing train-wreck, PolitiFact.

We rate it Mostly False.

Trading on what credibility, pray tell?

How did this fact check pass an editor's examination?

Grading PolitiFact (Florida): the ACLU, sharks and voter fraud (Updated)

Something about this one made me think of the old joke about the lawyer forced to swim through shark-infested waters.  PolitiFact Florida impressively confirms one premise of an argument while failing to note that the argument is fallaciously equivocal.  Indeed, the ACLU spokesman only utters the words.  PolitiFact provides the equivocal definitions.

Another great moment in the history of journalism.


The issue:

(clipped from PolitiFact.com)

The fact checkers:

Katie Sanders:  writer, researcher
Angie Drobnic Holan:  editor


Analysis:

Is voter fraud a problem?

That's the central and underlying issue in this fact check, which stems from a case in Florida where a schoolteacher ran afoul of the law by not properly following new legislation spearheaded by Florida's Republican-dominated legislature.

The Colbert Report produced a non-serious treatment of the issue.  An ACLU spokesperson contributed in kind.

The Colbert Report
Get More: Colbert Report Full Episodes,Political Humor & Satire Blog,Video Archive

The video commits any number of distortions of the issue, but we'll focus on just one. 

PolitiFact:

One of the people Colbert interviewed for his sarcastic report is Howard Simon, executive director of the American Civil Liberties Union of Florida. Florida officials claimed they needed to pass the law to prevent voter fraud, but these cases are actually pretty rare, he said.

"There are probably a larger number of shark attacks in Florida than there are cases of voter fraud," he said.

Simon says there are probably more shark attacks in Florida than cases of voter fraud.  His statement is intended to minimize the problem of voter fraud, using the perception that few shark attacks occur in Florida to power his point.  Take a guess at how many shark attacks occurred in Florida last year before encountering the stats.

Simon doesn't merely rely on the potential for inaccurate perceptions about the rate of shark attack.  Better than that, he says shark attacks "probably" outnumber the number of cases of voter fraud.

What is a "case" of voter fraud?

The relevant understanding of "case" in this context is an instance or occurrence.  A case of mistaken identity, for example, is any instance of mistaken identity.

That understanding is not for PolitiFact, however.  PolitiFact takes "cases" and interprets it, at least in the practical sense, as the number of state investigations for voter fraud.  A "case" in this fact check is a  "case" in the sense of a designation by a state agency.

Does that equivocal drift in definitions affect the outcome of the fact check?  Almost certainly.

PolitiFact (bold emphasis added):

The shark attack figures include documented instances of sharks attacking human victims. The voter fraud cases indicate the number of cases deemed legally sufficient for an investigation by the Florida Department of Law Enforcement.

Summing up the table PolitiFact provides, shark attacks outnumber the "cases" of voter fraud 72-49 over the past four years.  For only one of those years (2011) did the number of shark attacks fall short of the number of "cases" of voter fraud.

The fallacy in PolitiFact's reasoning comes easily to light.

For most of Florida's history as part of the United States it has prohibited voting by felons.  Every time a felon votes illegally, it constitutes a case of voter fraud--albeit not the phony representation of "case" settled on by the fact checkers.

Most often nothing can be done to prosecute a case of voter fraud against a felon who votes illegally even though it happens with some regularity.  The Miami Herald, for example, reported the following in the wake of its investigations of the 2000 presidential election:

At least 445 Florida felons voted illegally on Nov. 7, casting another cloud over a disputed presidential election already mired in legal challenges, a Herald investigation has found.

The tainted votes -- found in a review of nearly half a million votes cast in 12 Florida counties -- provide evidence that the presidential race was influenced by thousands of ineligible voters. Nearly six million voters in Florida's 67 counties cast ballots.

Got that?  The Herald found evidence that thousands of ineligible votes were cast in just one election.  But let's treat the numbers conservatively.  The Herald says at least 445 felons voted illegally--fraudulently--in Florida.  Add that 445 to the 49 "cases" of voter fraud listed on PolitiFact's chart and we end with a total of 494 cases of voter fraud.  The total documented number of shark attacks in Florida is 637 over its entire history as recorded by the Florida Museum of Natural History.

Who thinks felons failed to vote illegally at least enough to make up the remaining 143 vote difference?

Obviously the number of cases of voter fraud in the relevant sense are more than enough to outpace the number of shark attacks.  It is likely that ideology accounts for PolitiFact's eventual "Mostly True" ruling--not the facts.

The ACLU’s claim is true on its face, but we’re knocking it down a peg with consideration of a few things. One, the state’s count doesn’t represent a complete set of possible fraud being prosecuted in the state. Two, a "case" does not always include just one instance of fraud.

Three, the stat count artificially limits the number of cases of voter fraud in Florida by ignoring a number of sources of fraudulent votes.  Thus the ACLU's claim is not true on its face.  In addition, the ACLU argument misses the point that Republicans try to make in supporting strengthening of voter identification laws:  The law as it stands provides few tools to identify cases of voter fraud.  Nobody knows the full extent of voter fraud because of that lack.

About all we really know is that the cases of voter fraud are overpoweringly likely to outnumber the documented cases of shark attack.

Too bad we can't rely on PolitiFact to make the easy determination.

The grades:

Katie Sanders:  F
Angie Drobnic Holan:  F

This item receives the tag "journalists reporting badly."


Afters:

PolitiFact could have used this fact check to educate readers about the difficulty of uncovering voter fraud.  Instead, it trivialized the issue to almost the degree achieved by the Colbert Report.

A few select comments from the Florida Department of Law Enforcement's 1998 report "Florida Voter Fraud Issues":

FDLE's experience in recent years, including cases predating the 1993 and 1994 " Motor Voter" changes, suggests the areas that are "ripe" for potential fraud fall primarily into these categories:

• Voter Registration Fraud: — Minimal identification and citizenship proof requirements provide ample opportunity for voter registration fraud. This includes specialized "changes of address" done solely to allow a vote in a particular election, when in fact, no actual change of address has occurred.

• Absentee Ballot Fraud: — The desire to facilitate the opportunity for each person to vote has resulted in increased opportunity to use absentee ballots improperly. (Once one has registered fraudulently, he or she can obtain an absentee ballot for every election thereafter if he or she wishes. The lack of "in-person, at-the-polls" accountability makes absentee ballots the "tool of choice" for those inclined to commit voter fraud.)

• Illegally or Improperly "Assisting" Others To Vote Their Absentee Ballot: — Those inclined to do so can capitalize on others' access to an absentee ballot by voting their ballot for them, often with the actual voter not knowing what has occurred. This offers tremendous opportunity for vote fraud, particularly to those who have access to the ill or infirm or those who do not have the ability to resist the influence of another as they are urged to vote in a "required" manner. It also encourages those inclined to commit voter fraud to seek to utilize absentee ballots provided to those whose interest in voting is marginal or non-existent.

• Vote-Buying: — Securing votes by payment or other "rewards" or the "selling of one's vote" — is an age-old problem that still exists.

Once registered to vote, any person may request and utilize an absentee ballot without ever having to appear in person to vote. If the voter registration process does not require significant proof of citizenship, address, and identity, then those inclined to commit fraud will capitalize on the process by successfully registering those who have no right to vote, and then "facilitate" their (illegal) vote by absentee ballot.

To re-emphasize the key issue, these vulnerabilities in the voting system remain very easy to exploit and very difficult to detect once they occur.  Assuming that the number of cases deemed worthy of investigation by the Florida Department of State adequately represents the number of cases of actual voter fraud ranks as epic on the scale of folly (aka the Folly-O-Meter).


Update April 6, 2012:

A Florida television station airs a story documenting approximately 100 non-citizens registered to vote in Florida, at least some of whom voted in past elections. 





March 4, 2012:  Replaced "explain" with "exploit" in the final paragraph.