The misperceptions misperception mushrooms (Updated x2)

PolitiFact, it seems, has mired itself in the misperceptions marsh.

The problem started when the fact checkers went to test a recent claim made by comedian/Daily Show host Jon Stewart during an interview with Fox News' Chris Wallace (yellow highlights added):

WALLACE: I don't think our viewers are the least bit disappointed with us. I think our viewers think, finally, they're getting somebody who tells the other side of the story.
STEWART: Right.
(CROSSTALK)
WALLACE: And in -- no, no, no. One more example.
STEWART: Who are the most consistently misinformed media viewers? The most consistently misinformed? Fox, Fox viewers, consistently, every poll.
WALLACE: Can we talk about your network? Can we talk about Comedy Central?

PolitiFact's story ended up rating Stewart's statement "False," noting that the bulk of the available polling showed that Fox viewers were better informed than those who follow some other news sources.

Then things got interesting.
PolitiFact's audience and assorted liberals erupted in protest, resulting in one of those generally unsatisfying updates:

When we gave a False rating to a statement by Daily Show host Jon Stewart, the response from readers was swift and virtually unanimous. They said we were wrong.

Having claimed that PolitiFact will play to its primarily liberal audience--note the "virtually unanimous" response hostile to Fox News--I started following the aftermath of the story with interest.  In its initial followup story PolitiFact offered a somewhat unusual concession to its detractors:

Ultimately, we ruled that for Stewart’s comment to qualify as accurate, the data had to be "consistent" -- a term he used not once but three times. Since not "every poll" showed that result, we rated his claim False.

The striking thing about the comments we received -- almost 100 e-mails plus countless tweets and Facebook postings during the first 24 hours -- is how quickly readers zeroed in on a distinction our piece did not draw. They said we erred by failing to distinguish between a viewer being "misinformed" and a viewer beinig (sic) "uninformed" or "ill-informed."

Typical of a PolitiFact response, the update contains no real attempt to justify the rating against the criticism.  Given the dismay of PolitiFact's political base (liberals), however, I would not register much surprise if PolitiFact updated the item to upgrade the rating.

But PolitiFact probably should leave the rating as it is even though the criticism contains a shred of validity.

I should explain.

The adjective "misinformed" could apply in at least two distinct ways.  First, it could mean that Fox Viewers hold particular incorrect views because they were given false information (apparently by Fox News).  Alternatively, the term can serve as a synonym for "ignorant," as in simply believing untrue things.

Stewart's would-be defenders, in an exercise of futility, try to parse the term in his favor.  PolitiFact's secondary story provided an example from Jane Hamsher (FireDogLake):

The three Pew polls measure how informed viewers are.  They don’t even belong in the discussion, because they don’t go to Stewart’s point.   (...)

On the other hand, the two PIPA studies measure how misinformed viewers are. That’s a very different yardstick.  Listening to Dana Bash may freeze a few neurons in the “off” position, and I may not get the news value that I should out of the segment, but unless she says something that is manifestly untrue I can’t claim to have been misinformed.

Hamsher somewhat botched the parsing job by confusing the adjective form of "misinformed" with the verb form (a person can, after all, qualify as ignorant even if never misinformed about anything), but we can at least see that she favors taking Stewart to mean that Fox News imparted to its viewers false information, a possible interpretation I just covered.  But her interpretation can't bail out Stewart.  Neither meaning can rehabilitate Stewart's statement.

If we take Stewart to mean that polls consistently show Fox News viewers the most ignorant (misinformed) then we're stuck with PolitiFact's analysis using the PEW polling data.  That's exactly what the complaining liberals try to avoid with the alternate interpretation.

The alternate interpretation fails because it leaves Stewart without any supporting data.

As Hamsher notes, Stewart ends up relying on two studies by the Program on International Policy Attitudes.  Both PIPA's 2003 study and its 2010 study suffer from massive problems, such as a failure to control for researcher bias in choosing which points of knowledge to count as misperceptions.  But Hamsher's attempt to help out Stewart fails because neither study makes a scientific attempt to discover the source of the supposed misperceptions.  They simply asked a set of questions, some dealing with the level of belief in a series of statements and others asking respondents about their sources for news (2003, 2010).

The design of the PIPA studies precludes them from scientifically supporting the claim that Fox News viewers received their mistaken beliefs from Fox News reporting.

That leaves Stewart with no defense.  But I'll be interested to see if the liberal outcry convinces PolitiFact otherwise.


Update:

Near the time I first published this post, Jon Stewart's response to PolitiFact hit the Web.

Comedian, sore loser or both?

Sure, Jon Stewart's primarily a comedian.  But his act tonight appears indistinguishable from sore loser behavior as he tries to lessen the severity of his error by presenting even more inaccuracies.

Stewart presents a set of supposed PolitiFact fact checks of Fox News that turned out false or thereabouts.  But he simply takes claims made on the Fox News channel (or presumably so) and attributes them to Fox News.  Take, for example, PolitiFact's "Lie of the Year" for 2009--Sarah Palin's "death panel" statement.  The statement was made on Palin's FaceBook page.  And PolitiFact's "Lie of the Year" story attributed the statement solely to Palin.

Overlooking the fact that PolitiFact blew the "Lie of the Year" analysis (as well as for its 2010 pick), Stewart is covering for himself with lies.

Classy.


Update 2

Quite a few on the Web have tried to go beyond Jane Hamsher's attempt to find substantiation for Stewart's claim, with Chris Mooney's list of studies often presented in support.  I've posted a rebuttal to that use of Mooney's work here.