Lies and Damn Lies with Statistics: The Roger Clemens Story

I spent this morning in Somerville Hospital’s ER
waiting room watching Brian McNamee and Roger Clemens
testifying before a congressional committee on Capital
Hill over Clemens’ alleged use of performance
enhancement drugs. This is the most important stop on
the Clemens tour to save his reputation, his
credibility, and his butt from going to jail.

In response to being mentioned 82 times in the
Mitchell Report for using federally banned performance
enhancement substances, Roger Clemens and his PR team
have embarked on a massive campaign to convince us
otherwise. One of the primary tools employed in this
campaign involved the publication of a statistical
report that through its extensive number-crunching is
to provide “robust” evidence of how it was feasible
for a pitcher in his late 30s/early 40s to be
conceivably more effective than he was when athletes
are believed to be at their athletic prime—their 20s
and early 30s. After all, numbers don’t lie, or do
they...?

The report purports to “evaluate the career of Roger
Clemens, one of the most successful pitchers in major
league history. It will examine both the quality and
quantity of his pitching over the course of his
24-year career.” The report’s omission of an executive
summary; its absence of a framework; lack of explicitness in comparison group selection; and nonexistent rational methodology all make the report less palatable for public consumption, which raises the question of who exactly is the intended audience for the report. Furthermore, it is hard to extrapolate the research question(s) from the paper's multiple data tables, its redundant text, and its didactic tone. This is why I am grateful an article dissecting the Clemen’s report appeared in
this Sunday’s New York Times.

Given the shortcomings of the Clemens Report highlighted in the Times article, perhaps Mr. Clemens would have best been served by arguing that his stellar performance during the past 8 years was, in fact, freakish. If anything, in every data set there are always a few points that are disregarded because they deviate from the established norm.