Grading PolitiFact: Rick Santorum and the aborted third (Updated)

To assess the truth for a numbers claim, the biggest factor is the underlying message.
--Bill Adair

The issue:

The fact checkers:

Louis Jacobson:  writer, researcher
Bill Adair:  editor


Analysis:

The PolitiFact version of Rick Santorum's statement appears to provide adequate context:

"The Social Security system in my opinion is a flawed design, period," Santorum said in response to a question from a caller. "But having said that, the design would work a lot better if we had stable demographic trends. The reason Social Security is in big trouble is we don’t have enough workers to support the retirees. Well, a third of all the young people in America are not in America today because of abortion, because one in three pregnancies end in abortion."

Judging by the context, Santorum is making the point that abortion has a substantial effect on national demographics to the point where it exposes a weakness of the Social Security system.  Despite the fact that the underlying message is the important thing about a numbers claim, once again we have an instance where PolitiFact simply ignores the most obvious underlying message in favor of something else.

In this case the numbers claim itself is the point along with a later nitpick about the extent to which abortions would affect population dynamics.

PolitiFact:

Live births: 4,090,007
Fetal deaths: 25,653
Abortions: 1,250,000

Total known pregnancies: 5,365,660

So abortions account for 23.2 percent of all known pregnancies. That’s less than a quarter of all pregnancies, rather than the one-third Santorum said.

One minor quibble with PolitiFact's equation:  Given Santorum's point that abortions affect population dynamics it makes sense to think he would discount fetal deaths since those occur regardless of elective abortion.  Subtracting fetal deaths boosts the percentage slightly to 23.4 percent.  The adjustment ends up moot since PolitiFact eventually uses a Guttmacher Institute publication's estimate of 22.4 percent.

So what of the "Truth-O-Meter"?

PolitiFact:

(W)e find Santorum is significantly overstating the frequency of abortions when he claims that "one in three pregnancies end in abortion" -- it’s actually in the 22 percent to 23 percent range.  That's less than one in four. And the effect on population would be even lower if women who had abortions had children later in life. We can't know exactly how that would affect the numbers, but it would send the estimate even lower. We rate Santorum's statement False.

The fact check indicates that Santorum inflated the abortion rate by about 50 percent.  That makes this fact check useful to me in graphing the way numerical inaccuracy affects the needle of the "Truth-O-Meter, except that PolitiFact uses two of the sentences in the concluding paragraph to comment on Santorum's inference regarding demographic effects.  Assuming that commentary has no effect on the objectivity of the Truth-O-Meter respecting the stated goal of the fact check, a figure inflated by approximately 50 percent may result in a "False" rating.

PolitiFact's nit-picking got me thinking.

If we're going to pay attention to stats like women having kids later in life, then why don't we pay attention to a few others?  Such as the fact that the abortion ratio for 2003 (as calculated by PolitiFact) and the abortion ratio for 2008 (as calculated by Guttmacher Institute researchers) do not determine the percentage of children missing because of abortion.  If 2003 is relevant then so is 2004.  Not to mention 2005.  In fact, Santorum might have some justification based on the expanded history of the abortion ratio.  The Guttmacher Institute found the ratio as high as 30.4 percent in 1983.  If even one third of each aborted group avoids abortion annually, a multiplier effect takes place over time.  The women who would have been born in 1980 have an opportunity to give birth later down the line, perhaps even to the point where Santorum underestimates the demographic effects of elective abortion.

I'm not going to argue that Santorum is entirely correct, nor will I suggest that Santorum had the factors I mentioned above in mind when he made his claim.  Performing a calculation reliably establishing that notion is beyond my ability, not to mention time-intensive.  In this context, it is sufficient to point out that PolitiFact entirely overlooked a fairly obvious part of the math that would positively affect Santorum's claim while giving emphasis to a far less significant point that negatively affects his claim.


April Fools?

By the time I reached this item for review, PolitiFact had given it an update:

UPDATE, April 1, 2011: After we published this item, a reader pointed out that the statement we quoted from Santorum’s office was incorrect. While a spokeswoman had told us that "the prolife community has been astounded at the increasing percentages in the pregnancies that result in abortion," the reader pointed us to statistics from the Guttmacher Institute that show the rate of abortions declining over time. From a 1981 peak of 29.3 abortions per 1,000 women aged 15-44, the rate has fallen, gradually but steadily, to 19.6 in 2008 -- a decline of about one-third. According to Census Bureau data, the raw number of abortions has also fallen consistently, from a peak of 1,609,000 in 1990 to 1,242,000 in 2006.

PolitiFact is either pretty careless or pulling an April Fools' Day joke.

It is true that the abortion ratio is on the decrease, making the comment from Santorum's office a bit perplexing.  But the number of abortions per 1,000 women (the rate of abortion) is not the same thing and that number is independent of the abortion ratio.  The same goes for the raw number of abortions.  A higher number of pregnancies can allow a higher raw number of abortions even if the abortion ratio drops.


The proper numbers to bring the comment from Santorum's office into question were there in the report. They were just ignored in favor of numbers that failed to support the objection.  Big oops.

The grades:

Louis Jacobson:  F
Bill Adair:  F

The failures to focus reasonably on Santorum's underlying point and to take into account a major factor in support of Santorum's demographic inference justify the failing grades.  If I discover that the team was responsible for the botched update (and not kidding around with us) then the "journalists reporting badly" tag will apply.


Update (Fresh afters) April 4:

Though it remains true that reliably calculating the effect of abortion on a demographic over time is a complicated process best attempted by experts in the field, I programmed my own estimate just to illustrate the point about the cumulative effects of abortion done year after year.  It's an effect like compound interest but in reverse.

I used a fertility rate near the bottom of those recorded since the early 1970s (65 pregnancies in a year per 1,000 women aged 15-44).  I measured the effects over a 40 year period, roughly the length of time since Roe v. Wade made abortion generally legal in the United States.

Plugging an abortion ratio of 20 (200 abortions per 1,000 pregnancies annually) into my model, the control demographic (kids aged 0-10) fell by over 30 percent compared to zero abortions.

My reliance on conservative figures for purposes of my estimate suggests that Santorum would be very close and quite possibly low with his estimate over the time period since Roe v. Wade.  It would not make right his claim that one third of pregnancies end in abortion, of course.



April 3, 2011:  In the paragraph just prior to taking on PolitiFact's "nit-picking," corrected "100 percent" figure to match the previously used (correct) figure of 50 percent. 
April 4, 2011:  Restored the explanation for the difference between abortion ratios, rates and raw abortions as used in the Guttmacher report.  Not sure how it got cut out in the first place, but there you have it. Also fixed a redundancy in my opening paragraph.