Grading PolitiFact: Anne Hathaway and the gay marriage majority

Timing – Our rulings are based on when a statement was made and on the information available at that time.
--Principles of PolitiFact and the Truth-O-Meter

The issue:

The fact checkers:

Robert Farley:  writer, researcher
Bill Adair:  editor

Analysis:

Here's how my selection bias works with respect to PolitiFact:  If I see a fact check involving a claim with which journalistic liberals will feel sympathy, there's a good chance I'll inspect it.

Away we go:

Actor Anne Hathaway and other Hollywood celebrities recently wrote to President Barack Obama urging him to support gay marriage. The letter said a majority of Americans now support it.

Indeed, public opinion on gay marriage is shifting quickly. How quickly? Let's just say we're glad we waited a day to publish our item.

I suppose we'll eventually receive an explanation of the cryptic ending to the second paragraph.

The item continues:

The letter came on the heels of the White House announcement that it would no longer defend the Defense of Marriage Act in court. The act defines marriage as a legal union between one man and one woman. The letter, sponsored by Freedom to Marry, a group that advocates for marriage for same-sex couples, called the DOMA decision "a powerful statement about the law" and an "important step toward equal protection under the law for all Americans."

The letter urged the president "to complete your journey and join us, and the majority of Americans, who support the freedom to marry."

The context:

Dear Mr. President:

In February, you made a powerful statement about the law.

Invoking guarantees enshrined in the Constitution, you and the Attorney General determined that federal discrimination against gay and lesbian couples in marriage is unconstitutional. Thank you for taking a principled stance and an important step toward equal protection under the law for all Americans.

Today we ask you to take the next step and join the majority of Americans who support allowing loving and committed gay and lesbian couples to legally marry.

Mr. President, marriage matters. In law, in love, in life, marriage says "we are family" in a way that nothing else does. Marriage is the coming together of two lives, marked by a public promise of love and responsibility in front of friends and family. And marriage brings not only public respect and personal significance, but also a safety net of legal protections, rights, and responsibilities for which there is no substitute.

Like so many Americans, you have spoken of your personal journey toward support for the freedom to marry. You have talked about the gay and lesbian people in your life, their commitment to each other and care for their kids, and their families that aren't so different from any other family.

We ask you now for your leadership on ending the exclusion of same-sex couples from marriage, an exclusion that harms millions of Americans each day. Whether to end discrimination in marriage is a question America has faced before, and faces again today. With so many Americans talking it through in heartfelt conversations, it is a question that calls for clarity from the President.

You can offer hope to millions of young gay and lesbian Americans who are facing discrimination. You can tell them that their future is bright, that they, too, will be able to grow up and marry the person that they love, that the pursuit of happiness truly belongs to all of us. You can put government on the side of those seeking to care for their loved ones, instead of those standing in their way. You can affirm that for all of us, gay or non-gay, love is love and commitment counts— and that we Americans should treat others as we all want to be treated.

Mr. President, the time to end exclusion from marriage is now. We ask you to complete your journey and join us and the majority of Americans who support the freedom to marry.

The latter highlighted statement certainly leaves plenty of room for the literalist.  But in the context of the letter, "the freedom to marry" must be understood in terms of two-person marriages irrespective of the sex of either party.  The fact check, indeed, proceeds with that understanding:

The American public has long been deeply divided over the question of same-sex marriage, and so we wondered if the letter is accurate that a majority of Americans now support it.

More or less in keeping with its tradition of placing a burden of proof on the claimant, PolitiFact obtained two supporting evidences from  "Freedom to Marry" (Hathaway unavailable?).  That organization pointed to two recent polls:

The first was a poll conducted by the Associated Press in August 2010 that asked the question, "Should the Federal Government give legal recognition to marriages between couples of the same sex, or not?"  It was supported by 52 percent and opposed by 46 percent. The percentages were nearly the reverse in a poll the year before.

The margin of error on the AP poll was +/-4.5 percentage points.  That means that the group supporting the proposition could range anywhere from 47.5 percent to 56.5 percent.  The group opposed could range from 41.5 percent to 50.5 percent.  When a poll's results show an overlap when figuring in the margin for error the results of the poll are equivocal--in other words not particularly reliable.

Next:

The second was a CNN poll, also in August 2010, in which 52 percent responded "yes" to the question, "Do you think gays and lesbians should have a constitutional right to get married and have their marriage recognized by law as valid?" But the results differed slightly depending how the question was asked. Half got a question that read "should have" right to marry and the other half was asked if gay people "have" a right to marry. The "should" version produced a 52-46 majority in favor of gay marriage, while the "have a right" question produced a 49-51 majority against.

The margin of error in the CNN poll is +/- 3 percentage points, which again results in an overlap.

PolitiFact:

Other polls show growing support for gay marriage, but less than a majority.

The one example PolitiFact shared also had results within the margin of error, for what it's worth.  And PolitiFact's experts said things like "it is not a majority sentiment yet."

That means that Hathaway's statement has no unequivocal support.  PolitiFact often deals harshly with parties unable to prove the the accuracy of their statements ... but wait!  This just in:

Then, on March 18, 2011, a Washington Post-ABC News poll found that a slim majority now supports gay marriage.

The results were outside the margin of error on this poll.  And suddenly the light comes on.

Let's just say we're glad we waited a day to publish our item.

Right!  Glad because if PolitiFact had published earlier then the "Mostly True" rating would have been false!  Er, no.  Let's look for further explanation in the conclusion:

Bottom line, most of the polls taken in 2010 and 2011 show that while there is a clear trend in recent years toward support for gay marriage, but it is not quite yet a majority. But there are some polls that have found the tipping point has been reached, and that a slim majority now support gay marriage. And that includes the newest poll, from the Washington Post-ABC News. And so we rate the claim Mostly True.

Let's review the meaning of "Mostly True":

True – The statement is accurate and there’s nothing significant missing.
Mostly True – The statement is accurate but needs clarification or additional information.
Half True – The statement is accurate but leaves out important details or takes things out of context.
Barely True – The statement contains some element of truth but ignores critical facts that would give a different impression.

The statement Hathaway signed had no unequivocal basis when it was published, going by the sources PolitiFact investigated.  That means the only way PolitiFact can rate her "Mostly True" is by bending its principles.

Additionally, polling is best understood by keeping on top of issues like sampling bias, questions that influence the response and the margin of error.  The PolitiFact story doesn't even mention the margin of error.  That means that any rating with a definition of "The statement is accurate ..." is hard to justify without relaxing other fact-checking principles.  And the lone illicitly late support for Hathaway used controversial wording that was pointed out in the story but perhaps ignored for purposes of the final "Truth-O-Meter" rating.

What a mess.


The grades:

Robert Farley:  F
Bill Adair:  F

Afters:

Oh, and one more thing.

Let's just say we're glad we waited a day to publish our item.

Doesn't that statement, given the details of the fact check, strongly imply a lack of objectivity, perhaps shedding light on PolitiFact's decision to ignore its principles regarding timing?