The role of fact-checking

Politico published a nice little story on the fact-checking game.

I took this as a key graph:

"The one thing I worry about is by doing it the way we do it, putting up articles one after another, we reinforce the notion that all politicians are liars and it doesn’t matter who gets elected," said Brooks Jackson, director of Factcheck.org, which launched in 2003 and was the sole independent site keeping an eye on campaign dictum in the last presidential election.

"In 2004, Factcheck.org was considered [the] authority," said a senior McCain aide, who was provided by the campaign to speak about fact-checking on a not-for-attribution basis. He said he would "love to see it get back" to "fewer" fact-checking arms.

Annenberg Fact Check (Factcheck.org) is currently the gold standard in fact-checking, not that they do a perfect job by any stretch of the imagination. They just do a pretty consistent job of being fair, particuarly in contrast to outfits like Media Matters or PolitiFact.

And speaking of PolitiFact, I'm always interested in what Bill Adair has to say (from the Politico story):

"For too long we were timid about fact-checking because we felt that calling something false would open us up to a charge of bias," said Bill Adair, the St. Petersburg Times Washington bureau chief and editor of PolitiFact.com.

It's not calling something "false" that leads to the charge of bias. It is the way one goes about calling something "false." If, for example, you call one statement "false" based on the underlying argument even if the statement is technically true and another statement "false" because it is technically false despite the underlying argument being true then you have a situation where subjective judgment rather than objective judgment came into play.

And yes of course that type of system is biased. Adair is "Pants-on-Fire" wrong if he believes otherwise.


In the fact-checking world, Annenberg continues to have a legitimate reputation for fair judgment. I think The Washington Post usually does a fair job, though I have not evaluated much of their work. PolitiFact has built-in problems even apart from the bias of its writers and editors. If you can judge whether a statement is true or false with a needle graphic even while claiming that truth is not black and white, then you should be able to make some kind of judgment as to which of your ratings are politically significant and which qualify as campaign ephemera.

Annenberg has the right model. Evaluate campaign claims as fairly as possible. Mistakes will be made, but an attempt at fair evaluation will always have value. PolitiFact brought in the idea of presenting the claims as a collection of data. One can look at all the claims PolitiFact has made for a given candidate and consider graphic trends. Unfortunately, their approach was wrong-headed because it puts no extra weight on serious attempts to mislead compared to slips of the tongue or inartful language.

A system achieving the goal at which PolitiFact aimed is possible, but very difficult to effect without either real bias or at least the appearance of bias.