CDC: Vaccine Study Design "Uninformative and Potentially Misleading"

David Kirby

CDC: Vaccine Study Design "Uninformative and Potentially Misleading"

Posted June 20, 2008 | 06:43 PM (EST)

CDC Director Dr. Julie Gerberding has delivered a potentially explosive report to the powerful House Appropriations Committee, in which she admits to a startling string of errors in the design and methods used in the CDC's landmark 2003 study that found no link between mercury in vaccines and autism, ADHD, speech delay or tics.

Gerberding was responding to a 2006 report from the National Institute of Environmental Health Sciences (NIEHS), which concluded that the CDC's flagship thimerosal safety study was riddled with "several areas of weaknesses" that combined to "reduce the usefulness" of the study.

"CDC concurs," Dr. Gerberding wrote in an undated mea culpa to Congress, (provided to me through a Capital Hill staffer) adding that her agency "does not plan to use" the database in question, the Vaccine Safety Datalink, (VSD) for any future "ecological studies" of autism.

In fact, Gerberding's report said, any continued use of the VSD for similar ecological studies of vaccines and autism "would be uninformative and potentially misleading."

Ecological vaccine studies are large, epidemiological analyses of risks and trends using computerized data from large populations -- in this case children enrolled at several big HMOs -- without ever examining a single patient in person.

CDC officials conducted at least five separate analyses of the data over a four-year period from 1999-2003. The first analysis showed that children exposed to the most thimerosal by one month of age had extremely high relative risks for a number of outcomes, compared with children who got little or no mercury: The relative risk for ADHD was 8.29 times higher, for autism, it was 7.62 times higher, ADD, 6.38 times higher, tics, 5.65 times, and speech and language delays were 2.09 more likely among kids who got the most mercury.

Over time, however, all of these risks declined into statistical insignificance, statistical inconsistency or else outright oblivion: The relative risk for autism plummeted from 7.62 in the first analysis, to 2.48 in the second version, to 1.69 in the third round, to 1.52 in the fourth, and down to nothing at all in the fifth, final, and published analysis printed in the Journal Pediatrics in November of 2003.

Vaccine officials attributed the steady drop to the elimination of "statistical noise" from the data through due diligence and the endeavor for excellence in governmental statistical analysis.

Indeed, the VSD study was the main pillar of a hugely influential 2004 report by the Institute of Medicine, which also concluded that there was no evidence of link between mercury, vaccines and autism.

To this day, public health officials routinely point to five "large epidemiological studies" representing the "highest quality science," none of which found any link to thimerosal.

In fact, the American VSD study has long been held up as the best and brightest of them all (the others were in Sweden, the UK, and two in Denmark). And this reputation has stuck in the minds of medicine and the media.

Curiously though, even the study's lead author -- Dr. Thomas Verstraeten, an employee of vaccine maker GlaxoSmithKline -- protested that the VSD study "found no evidence against an association, as a negative study would. In fact, he said that additional study was needed, which "is the conclusion to which a neutral study must come."

That's when Congress stepped in.

In 2005, a group of Senators and Representatives headed by Sen. Joe Lieberman wrote to the NIEHS (an agency of the National Institutes of Health) saying that many parents no longer trusted the CDC to conduct independent minded studies of its own vaccine program. Lieberman et al asked NIEHS to review the CDC's work on the vaccine database and report back with critiques and suggestions.

The final NIEHS report was a serious and thoughtful critique of where the CDC went wrong in its design, conduct and analysis of the study. The NIEHS panel "identified several serious problems," with the CDC's effort, criticism to which the agency had not responded -- until now.

In her letter to the House Appropriations Committee, the CDC Director responded directly to many -- though not all -- of the most important criticisms and recommendations contained in the NIEHS panel report.

For example, the NIEHS had criticized CDC for failing to account for other mercury exposures, including maternal sources from flu shots and immune globulin, as well as mercury in food and the environment.

"CDC acknowledges this concern and recognizes this limitation," the Gerberding reply says.

The NIEHS also took CDC to task for eliminating 25% of the study population for a variety of reasons, even though this represented, "a susceptible population whose removal from the analysis might unintentionally reduce the ability to detect an effect of thimerosal." This strict entry criteria likely led to an "under-ascertainment" of autism cases, the NIEHS reported.

"CDC concurs," Gerberding wrote, again noting that its study design was "not appropriate for studying this vaccine safety topic. The data are intended for administrative purposes and may not be predictive of the outcomes studied."

Another serious problem was that the HMOs changed the way they tracked and recorded autism diagnoses over time, including during the period when vaccine mercury levels were in decline. Such changes could "affect the observed rate of autism and could confound or distort trends in autism rates," the NIEHS warned.

"CDC concurs," Dr. Gerberding wrote again, "that conducting an ecologic analysis using VSD administrative data to address potential associations between thimerosal exposure and risk of ASD is not useful."

Read that sentence one more time. The head of the CDC is saying that its most powerful and convincing piece of exonerating evidence for thimerosal is, in effect, "useless."

I hope everyone will read it, including the recommendations to make the VSD better, and the CDC's agreement with all of the suggestions.

As questionable at the US thimerosal study was, "it was an improvement on other studies, including the two in Denmark, both of which had serious weaknesses in their designs," Dr. Irva Hertz-Picciotto, Professor of Public Health at UC Davis Medical School and Chair of the NIEHS panel, told reporter Dan Olmsted at UPI.

That leaves very little for the CDC to go on in terms of proving that thimerosal and autism are not associated in any way.

Yes, there is always the study of disability services data from California -- which seem to be rising among the youngest cohorts of kids, who presumably received little or no mercury because thimerosal was largely removed from childhood shots.

But California is an "ecological study" with problems of its own.

"Although (this) information is often used by media and research entities to develop statistics and draw conclusions, some of these findings may misrepresent the quarterly figures," cautions the website of the California Department of Developmental Services (DDS). "Increases in the number of persons reported from one quarter to the next do not necessarily represent persons who are new to the DDS system."

Even the CDC admits that "there are several limitations" with linking a VSD study design with the California data, Gerberding wrote to Congress, because, among other things, California only counts "persons who were referred to and/or voluntarily entered" the disability system."

It will be interesting to see how the House Committee -- and the mainstream media -- react to this rather breathtaking confession by the CDC, which does seem to want to conduct the best vaccine-autism science possible (see Gerberding's replies to NIEHS recommendations for improving the VSD: CDC officials are currently conducting in- depth follow up studies with VSD patients).

As the waning months of the Bush administration get underway, I can't help but wonder if a little housecleaning might be going on at some of our top health agencies.

(And on that note, please see my blog about the upcoming HHS/FDA/CDC/NIH workshop on autism triggers in mitochondrial disorders (ie, Hannah Poling and her five vaccines).