WaPo's Glenn Kessler offers cold comfort with fact check summary

Criticism of the fact checkers at PolitiFact represents the dominant theme of this blog, but Glenn Kessler's summary of the Washington Post's year in fact checking at least shows that PolitiFact isn't alone in its reckless approach to fact check reporting.

Quantifying opinion then reporting the aggregated results as if they reflect on the political figures involved?  Check:

Democrats fared slightly better than Republicans in terms of Pinocchios. They have an average of 2.32 Pinocchios per statement, compared to 2.49 Pinocchios for Republicans. As we noted at the six-month point, some of the GOP presidential contenders (We are looking at you, Rep. Bachmann) were largely responsible for inflating the Republican score.

The average Pinocchio rating of the political candidates is constantly updated by our Pinocchio tracker. Obama, with a vast White House staff to vet his statements, at the moment is basically tied for the lowest average with his nearest rival, former Massachusetts Gov. Mitt Romney. Romney also has a large campaign staff who help vet his statements, so it should not be a surprise that these two politicians have the best Pinocchio ratings.

Misunderstanding the reason why such aggregated results mean virtually nothing apart from representing a measure of selection bias?  Check:

Some readers have complained that the “average Pinocchio rating” is a bit absurd because it is completely dependent on a random selection of statements that we vet. This is true; it is inherently arbitrary. But we would also argue that over time one can get a broad sense of how accurate a politician is, and so that makes the average ranking is [sic] a rough but imperfect guide.

D'oh!

No, a random selection is exactly what you want if the average Pinocchio rating is used to establish politicians' trends for veracity.  And that's exactly what we don't get from either Kessler or PolitiFact.  Both choose items of editorial interest.  Thus the selection of items is not "arbitrary" in the random sense but rather "arbitrary" because the editors serve as arbiters of what facts to check.

And Kessler's last sentence in the same paragraph is a howler, plain and simple.

The logic of the sentence construction appears to suggest that because WaPo's fact checker argues for the utility of the results therefore the results possess that utility.  Any truly reasonable argument supporting Kessler's conclusion somehow failed to make it into the story.

Kessler helps illustrate the peril of relying on journalists for fact checks.