Thursday, December 31, 2009

PolitiFact 2009: a review

After critiquing a few PolitiFact findings back in 2008, I summed up the problems I saw with the new fact-check kid on the block.  With over 60 individual items graded since then, I have much more to draw from in assessing the PolitiFact animal.

I may have given as many as three passing grades thus far.  Why is my grading so harsh?  The easy answer is that PolitiFact deserves harsh grading.  But I am not content offering the easy answer because I continue to believe that journalists tend to do their work sincerely and that failings of objectivity almost never arise as a planned feature of news reporting.

That still leaves me looking rather harsh for grading these sincere truth-excavators with strings of "F" grades.  I grade harshly because of my respect for objective reporting.  It is serious enough when literature bears the implicit label of "objective reporting."  Putting opinion under that objective label is a form of lying.  And producing literature with the label "fact-check" while under that objective label at least doubles the responsibility to get the reporting right.  For any given week, chances are I can find an egregious failing in a PolitiFact fact-check claim.

Since my previous general assessment of PolitiFact, written back in 2008, one distinct improvement seems to have occurred.  PolitiFact no longer lists the left-leaning fact-check site Media Matters on its list of sources.  Though I can imagine ways that Media Matters could be used without sacrificing objectivity, the vast majority of fact-checks should not require any material at all from that source.  Good riddance.

What problems linger?

PolitiFact's insistence on using the "Truth-O-Meter" graphic often traps the operation into unnecessarily interjecting opinion into its fact checking.  The best example occurs when PolitiFact fails to collect enough information to establish a definitive rating but proceeds to produce a rating anyway.  Media Matters appropriately called out PolitiFact on one such example involving Nancy Pelosi.  When readers view the totals by individual or organization the ratings all look the same in terms of weight.

Individual entries often fail to properly offer a statement the benefit of doubt.  For example, PolitiFact tends to take hyperbole literally, displays a tin ear when confronted with humor, and fails to undertake due diligence in otherwise respecting the context of various statements.  PolitiFact's "Lie of the Year" for 2009 exemplifies this, as Sarah Palin's statement about Obama's "death panel" was taken literally and every effort was made to shoehorn the reference in with reference to specific provisions in the proposed health care legislation.  These matters of interpretation allow the latent political bias in the newsroom to manifest itself.  Writers and editors naturally exhibit sympathy for things they find agreeable, and correspondingly offer less sympathy for things they find disagreeable.  The best journalists largely check their sympathies at the door, but newsrooms trend so markedly left that the workplace climate serves as a sort of "echo chamber."  Drive past the Fenholloway River near Perry, Fla. and the visitor will smell the wood pulp.  The locals won't notice it.

I mentioned above the tendency to include opinion under the banner of objective journalism.  PolitiFact's "Lie of the Year" story again provides a clear example.  When PolitiFact published a piece soliciting reader votes, it was freely admitted that the candidates for the title were picked by the editors (a write-in feature was apparently botched).  But even something that obvious failed to warrant the alternate labeling expected in the American news paradigm, "opinion" or at least "news analysis."

Sometimes PolitiFact just makes unaccountable mistakes.  A recent favorite came from a PolitiFact rating of Louisiana governor Bobby Jindal.  Jindal said that the New Orleans Saints would set a national record if they went undefeated for the season and won the Super Bowl and called that feat "unprecedented."  PolitiFact rightly pointed out that the Miami Dolphins went undefeated in 1972 and won the Super Bowl.  But the story omitted all mention of the fact that the Dolphins accomplished the feat during the era when the regular season consisted of 14 games.  One simply cannot omit that information on that particular fact check and call it fair.

Most perniciously, PolitiFact apparently continues to let its readership have a hand in story selection.  The more skewed the coverage, the less likely the offended group will bother with PolitiFact.  Thus any story selection bias coming from readers (minus some sort of organized effort by the minority) will tend to reinforce and perpetuate story selection bias.



The issue of bias

Is PolitiFact biased?

Yes, of course PolitiFact is biased.  But probably not the way people expect.

The writers and editors almost certainly do not have plotting sessions during which they plan ways to favor liberal causes and persons.  The bias comes from the fact that journalists tend to lean left (though most label themselves as "moderates").

The dominance of liberal thinking in the newsroom manifests itself when the newsroom determines which stories to cover.  Stories reinforcing liberal politics will have an edge.

In like manner, the writing process will tend to forgive liberal legends more often than those distinctive of conservative origin.

The end product is flawed because it was produced by humans, and likely flawed with a tendency to favor the left.

PolitiFact's year-end stories helped provide evidence of its bias.  One story highlighted the ten most popular PolitiFact stories as measured by page loads.  All of the top ten stories were critical of conservatives or conservatism.  Those numbers indicate that PolitiFact readers very probably skew to the ideological left.  So let those readers influence story selection?  Great idea!

Other than selection bias, what can possibly explain the fact that Nancy "Loose Cannon" Pelosi, she of the highest rank in the House of Representatives, managed to have just four statements rated for all of 2009?  House Minority Leader John Boehner, for comparison, had 13 statements rated in 2009.  Likewise in the Senate, Sen. Harry Reid had just one statement subjected to PolitiFact's scrutiny.  His Republican counterpart, Sen. Mitch McConnell, was judged on six occasions.  You do the math.


Conclusion

PolitiFact continues to fall far short of its promise.  The idea is good.  The reporters are reasonably talented.  But the whole operation repeatedly fails to meet the basic standards for objective fact checking.

No comments:

Post a Comment

Please remain on topic and keep coarse language to an absolute minimum. Comments in a language other than English will be assumed off topic.