Saturday, November 27, 2010

The polygraph that tells more about the tester than the testee

If I deliberately provide information that I know will probably mislead my readers, does it make me a liar?

I'd say "yes," but what do I know?

PolitiFact's system tracks how many of a political figure's statements fall into each of its rating categories such as "True," "Half True" and "Pants On Fire."  As media professionals I expect them to possess substantial knowledge regarding the use and abuse of statistics--especially since PolitiFact wears the "fact checkers" label with pride.

With that in mind, I have approved of PolitiFact's reluctance to actively promote the value of its collected ratings with respect to judging an individual's truthfulness.

Unfortunately, that reluctance doesn't so much resemble following the dictates of logic as it does an insincere method of granting the benefit of the doubt.  It's kind of like, "We're too polite to call this person a chronic liar, but isn't it obvious from these statistics?"

As I've pointed out before, PolitiFact's selection bias makes its collected statistics all but useless for grading the individuals whose statements receive scrutiny.  The primary value of such information--skimpy as it is--lies in the indication it provides of selection bias by PolitiFact.

Though that's a fact as much as any other, PolitiFact does not inform its readers as to that point.  Instead, we get stuff like this:

Disclaimer:  These ratings should not be taken as a reliable guide to the subject's truthfulness.

Only there is no such disclaimer.  And by its lack, PolitiFact encourages its readers to draw false inferences.  Many are tempted to look at the collected data and draw ill-founded conclusions, perhaps like "Kay Bailey Hutchison is just as likely to lie as to tell the truth!"

Vic Pilkington drew that type of conclusion in a recent post at PolitiFact's FaceBook page:
Did you know that 90% of the Politifact Pants-on-Fire and False statements come from the Right. It’s true, count them!
Pilkington's other comments discourage offering him the charitable interpretation that he feels he has discovered an indication of PolitiFact's selection bias.

Mounting anecdotal and other evidences compound the criticism by showing a pattern of bias in PolitiFact's stories.

No comments:

Post a Comment

Please remain on topic and keep coarse language to an absolute minimum. Comments in a language other than English will be assumed off topic.