Grading PolitiFact: Alan Grayson announces the rankings

The issue:

The fact checkers:

Lukas Pleva:  writer, researcher
John Bartosek:  editor


Analysis

Rep. Alan Grayson (D-Fla) appeared on The Ed Show on MSNBC.  Grayson drew PolitiFact's attention with this:

UYGUR: I swear to you that my next question was if they accused him of being a Martian, do you have to come out every once in a while and say the president is not Martian?

(LAUGHTER)

GRAYSON: It‘s sad. Look, we really are cheating ourselves.  This can be a heaven on Earth. This could be an outstanding place to live, the very top in the entire world.

Instead, we‘re 50^th in the world -- 50^th in life expectancy, just below Albania. We are dead last in math test scores. And believe me, the math is the same in Seoul, Korea, as it is in here. It‘s the same math.

We have to work harder to solve our problems. And we can‘t get distracted all the time by the people who want to distract us for their own selfish purposes.

(yellow highlights added to emphasize portion quoted by PolitiFact)

Grayson made many dubious statements during his television appearance.  Just in the quoted portion he made the claim the U.S. was ranked No. 50 worldwide in life expectancy, ranked below Albania for life expectancy and "dead last" in math test scores.  PolitiFact apparently found the latter two claims uninteresting.

Picking a focus from among those three statements represents a case of framing.  Such framing may make a story potentially difficult to distinguish from political spin.

Take it away, Lukas Pleva:

Is the U.S. really that low on the life expectancy scale? We decided to find out.

Todd Jurkowski, Grayson's press secretary, directed us to the 2009 life expectancy estimates in the CIA World Factbook. The United States is, indeed, No. 50.

But the CIA has since updated those estimates. In 2010, Macau had the highest life expectancy at birth -- 84.36 years. The United States comes in at No. 49, with an average life expectancy of 78.11 years. Still, that's only slightly off from the figure that Grayson cited.

Putting aside the closed eyes brought to bear on Albania and U.S. math rankings, Pleva starts in the right direction to fact check the life expectancy ranking.  A site dedicated to archiving data from past years for the CIA Factbook differs from the Wikipedia source Pleva used, but since updates occur regularly the U.S. might have temporarily slipped to No. 50.  Also note that the Wikipedia page includes two lists, one for recognized independent countries and the other for "territory."  The table lists two separate rankings:

The "34" to the left of the "50" indicates the United States' rank compared to other sovereign nations without competing against itself (as with Puerto Rico).

Pleva went on to note that World Health Organization lists put the U.S. at about No. 31 :

But we quickly found that other sources paint a different picture.

The World Health Organization maintains its own life expectancy data. In 2008, the latest year for which the numbers are available, people in the U.S. had an average life expectancy of 78 years. That put us in a tie with the United Arab Emirates, Kuwait, Costa Rica, and Chile. The figures are all rounded to the nearest whole number, which means that numerous countries were tied. There are 30 countries that had higher life expectancy than the United States.

Then there is the U.S. Census Bureau Statistical Abstract. In 2010, the United States came in at No. 50 with an average life expectancy of 78.2 years. Macau, a Special Administrative Region (SAR) of the People's Republic of China (PRC), came in first, with 84.4 years.

In addition to the CIA data, Pleva found a ranking of about 31 from WHO, another of 50 from the U.S. Census Bureau, and one of 41 from the Population Reference Bureau.  A curious reporter should have looked for the reason behind the surface dissimilarity of the WHO and Population Reference Bureau rankings as well as the similarity of the Census Bureau ranking.

This blog entry almost entirely explains the discrepancy between Grayson's figure and the WHO figure:  The comparison is apples to oranges, or at least apples in general to red apples.

What about the Census Bureau figure?

Once again the list in question includes territories ordinarily not counted as sovereign nations, this time 17 instead of 16.

All of the lists examined so far have approximately the same ranking for the United States once territories are removed.  The accuracy of Grayson's claimed number rests squarely on the inclusion of non-sovereign territories such as Anguilla, Bermuda, Cayman Islands and Guernsey (find a full list in the "Afters" section).

That leaves the ranking from the Population Reference Bureau.  As with the CIA Factbook and the Census Bureau, the list includes nations (eight) not ordinarily counted as sovereign nations.  Subtracting eight from 41 leaves 33, very much in the same neighborhood as the other rankings of sovereign nations and not particularly close to Grayson's number.

Taken at face value, then, Grayson was off.

As for Pleva, he did notice a fishy thing or two about the data:

(T)hat brings us to an important caveat with virtually all of the rankings. While it is technically true that the United States ranked 49th in the latest CIA World Factbook, consider some of the countries that had a higher ranking. Macau, for example ranked first, but it's less than one-sixth the size of Washington, D.C. San Marino, a state nestled inside Italy, came in at No. 11, but it has a population of only slightly more than 30,000 people. You get the picture -- some of the "countries" that ranked higher than the United States are, well, countries in name only.

Pleva is on the right track, but some on the list are not even countries by name.  French Guiana, for example, remains a French territory with a status akin to that of Puerto Rico.  Puerto Rico gets its own Olympic team, but it isn't typically mistaken for a sovereign nation.

Pleva, consistent at last with PolitiFact's criticism of a ranking cited by Sarah Palin, also delves into rankings restricted to industrialized democracies.  The U.S. ranks No. 25 out of 31, within 1.2 years of the mean (79.13 years).  Maybe comparing the United States with the OECD nations is the fairest comparison in some ways, but the comparison is relatively useless without identifying the reasons behind differences in longevity (see Afters).

After that brief digression, Pleva unveils the Truth-O-Meter rating:

Grayson claimed that the United States ranks 50th in the world for life expectancy. We found a few sources that show that we're actually a bit higher. The Population Reference Bureau put us at No. 41, while data from the World Health Organization indicate that we may be in the low 30's. Still, we think that Grayson's underlying point that the U.S. isn't as high in life expectancy as one would think is valid, so we rate this Mostly True.

Considering sovereign nations, as most probably do when they hear about world rankings in the context of nations like the United States and Albania, the U.S. ranked in the low 30's for each of the sources Pleva investigated.  Pleva recognizes that Grayson is off, but he doesn't realize how far off.  Then, out of the blue, Pleva rescues Grayson with an "underlying argument" (a term common in PolitiFact stories) defense.  Grayson's underlying point, Pleva says, is that the U.S. does not rank as high as one would think.

There's at least one problem with taking that point as Grayson's underlying argument:  There is no textual support for the claim.  In the parallel example featuring Sarah Palin, she offered the data in the form of a rhetorical question of the general form "Did you know that ... ?"  That form would help support the claim that the underlying argument had to do with the surprising nature of the truth.  But Grayson offered nothing of the kind.  He offered a trio of factoids as simple matters of fact without giving any indication that the data should be found surprising.  Rather, he simply argued that the data should motivate change for the better.

Pleva invented Grayson's underlying argument for him, perhaps knitting it out of his own surprise.

Since Grayson offered no shred of evidence in favor of Pleva's theory, it is fair to discount the excuse and conclude that PolitiFact gave Grayson a better Truth-O-Meter rating than he deserved.

The grades:

Lukas Pleva:  F
John Bartosek:  F

Pleva's lack of curiosity as he evaluated the data put him on the F border.  Inventing an excuse to go easy on Grayson ensured that he had no shot at a D.


Afters:

"(J)ust below Albania"

I kept Grayson's other claims in mind while I did the research.  The United States was ahead of Albania on every one of the lists discussed.  The CIA numbers for 2009 and 2010 had Albania trailing the U.S. by one spot.  Maybe Grayson meant to say "Just above Albania."  On the other hand, the same WHO list that placed the U.S. at No. 31 put Albania at No. 79.


"We are dead last in math test scores."

I have no idea where Grayson gets that one.  On second thought, maybe I do.  I ran across a graphic that supposedly appeared originally in the Washington Post:

A-ha! The United States is ranked last! Grayson is vindicated!

Actually, no. The Post's graphics folks did that little thing that's all too common in newspaper graphics. They deleted a bunch of stuff to improve the visual impact of their graphic.  And they clued in the alert reader with the description "Top performers in eight-grade math and science tests." (sic)  The most recent TIMSS survey was conducted in 2007:

In 2007, the average mathematics scores of both U.S. fourth- and eighth-grade students were higher than the TIMSS scale averages.

• U.S. fourth-graders scored 529, on average, in mathematics, which was higher than the TIMSS scale average of 500 (table 1).
• The average mathematics score of U.S. fourth-graders was higher than those in 23 of the 35 other countries, lower than in 8 countries (all 8 were in Asia or Europe), and not measurably different from the average scores of students in the remaining 4 countries.
• U.S. eighth-graders scored 508, on average, in mathematics, which was higher than the TIMSS scale average of 500 (table 1).
• The average mathematics score of U.S. eighth-graders was higher than those in 37 of the 47 other countries, lower than in 5 countries (all of them in Asia), and not measurably different from the average scores of students in the remaining 5 countries.

(blue highlights added)

Looks like Grayson was wrong again.


Life expectancy lists:  The extra countries

CIA Factbook 2010

1) Macau

5)  Hong Kong

14)  Anguilla

15)  Cayman Islands

16)  Bermuda

22)  Guernsey

25)  Jersey

28)  Faroe Islands

34)  Saint Pierre and Miguelon

35)  Virgin Islands

38)  Isle of Man

39)  European Union

40)  Gibraltar

42)  Puerto Rico

44)  St. Helena, Ascension, and Tristan da Cunha

48)  Wallis and Fortuna

Census Bureau

1)  Macau S.A.R.

5)  Hong Kong S.A.R.

14)  Anguilla

15)  Bermuda

16)  Cayman Islands

22)  Guernsey

25)  Jersey

29)  Faroe Islands

34)  St. Pierre and Miguelon

35)  Virgin Islands, U.S.

38)  Isle of Man

39)  Wallis and Fortuna

41)  Saint Martin

42)  Saint Barthelemy

44)  Saint Helena

45)  Puerto Rico

46)  Gibraltar

Population Reference Bureau

2)  China, Hong Kong SAR (e)

5)  China, Macao SAR (e)

21)  Martinique

31)  Channel Islands

33)  Guam

39)  Puerto Rico

41)  Reunion

42)  French Guiana

It's worth noting that the Census Bureau rankings are of sufficient similarity to the CIA FactBook rankings that a common source is likely.

Many factors influence life expectancy, including diet, standard of living and availability of medical care. Plus not all of the data are collected with the same methods. If increased longevity means outlawing fried chicken, for example, is that the direction we ought to take?