Vaccines, Diagnosis, and Databases
June 21, 2008 by Kristina Chew, PhD
Filed under Health
After my son was diagnosed with autism in July of 1999 and we had started him in a home ABA program in September of 1999, and as we found ourselves spending more time with families with autistic children, and as we read more and more (in books, on the web) about autism, I started to feel that I could spot an autistic child immediately, across however crowded a room. I knew the things that made Charlie different and had started to see flashes of these in other children and was somehow reassured: We weren’t alone.
Over time, though, I’ve become less sure, or a lot more cautious, at saying that someone has autism or not. My understanding of Charlie and of autism has deepened over the years, but so has my knowledge of the factors involved in giving a child an autism diagnosis. Here in New Jersey, a diagnosis of “autism” can provide a child with more services than one of “PDD-NOS” and, often, more than one of “Asperger’s Syndrome.” Studies like that by Washington University professor Paul Shattuck—which found that, at the same time as autism diagnoses increased, the number of diagnoses of mental retardation and learning disabilities decreased—suggest that diagnosing autism has a bit more art in it than one might at first think. People seem, too, to regularly under-emphasize how the significant changes in the diagnostic criteria for autism and the steady rise in public awareness about autism have contributed to more diagnoses, and even to more people seeking out a diagnosis, as anthropologist Roy Richard Grinker argues in his book Unstrange Minds: Remapping the World of Autism.
It’s helpful and even necessary to keep in mind how our evolving understanding of autism— and a “peculiar fascination” with autism in the contemporary media, to cite a recently published book—has contributed to more children and, too, adults, being diagnosed with autism. This growing understanding of autism has, it has been argued, contributed to the rise in the prevalence rate of autism (now 1 in 150, according to figures released by the CDC in February of 2007) and this expanding understanding of autism needs to be taken in account when reviewing studies about autism in the past several years.
Some proportion of those studies have looked at possible environmental causes of autism, such as vaccines or something in vaccines. The numbers are the thing here: If it can be proven that there is a definite increase in autism in children who received (for instance) vaccines containing the mercury-based preservative thimerosal, then it would seem that a culprit for the increase in autism has been found. A few months ago, much was made of the case of Hannah Poling, after the government conceded that the 9-year-old Georgia girl’s underlying mitochondrial disorder had been “aggravated” by vaccines and led to symptoms of autism. The question has been, was what happened to Hannah unique to her only, or is there some “hidden horde of autistic children with underlying mitochondrial disorders? (So far, the answer seems to be “no.”)
Researchers need to study large populations of subjects; epidemiological studies look at “how disease is distributed in a population and of the factors that influence or determine that distribution” (Do Vaccines Cause That?, p. 53). Consequently, databases of cases—from HMOs, for instance—can provide sufficiently large sample populations for scientists to study. One such database that researchers have used is the Vaccine Safety Datalink (VSD), which was a project created in 1990 by the CDC and which, “as of mid-2007, involves partnerships with eight large health maintenance organizations (HMOs) that continually monitor vaccine safety” (Do Vaccines Cause That?, p. 62). The VSD contains information from more than 5.5 million people in Washington, Oregon, California, Colorado, Minnesota, and Massachusetts.
The VSD has both its strengths and limitations. It is a large database and allows researchers to access data from all medical visits, types of vaccine, date of vaccination, concurrent vaccination, the manufacturer, lot number, injection site, and potential adverse events (Do Vaccines Cause That?, p. 62). Its limitations include the following (see Do Vaccines Cause That?, p. 65-6, for fuller explanations):
- “the population in the participating HMOs is not wholly representative of the United States in terms of geography, ethnicity, or socioeconimic status”;
- “vaccine coverage rates for most vaccines are very high in the participating HMOs, and thus there are usually few non-vaccinated people available for comprehensive comparisons”;
- “the project is not large enough to examine the risk of extremely rare events (one in a million vaccinees) such as Guillain-Barré syndrome after influenza vaccine”;
- “because the database contains patient records, it can only be accessed under circumstances that maintain the confidentiality of the individuals and requires prior approval of an Institutional Review Board.”
Some of these caveats would seem to behind a recent report by CDC director Dr. Julie Geberding about the limitations of VSD data in studying vaccines and autism. The report follows a 2006 panel convened by the National Institute of Environmental Health Sciences (NIEHS) and considers the “feasibility of using VSD data in an ecological study to compare rates of autism disorder (AD) or autism spectrum disorders (ASD) before and after the removal of thimerosal from most childhood vaccinations.” The report has been uploaded as a PDH file on journalist David Kirby’s EvidenceofHarm website and he indicates that he was given the report by a “Capital Hill staffer” (similarly to how “someone” gave him documents about the case of Hannah Poling, which Kirby made available prior to protocol). Kirby’s assessment in that the report is a sort of “mea culpa” on the part of Dr. Geberding and that it suggests that the design of studies about vaccines under the CDC are “‘uninformative and potentially misleading.’” There are concerns about the VSD, but not in the sense that Kirby might wish (a “startling string” of goofs and mistakes by CDC researchers). Rather, the report highlights what EpiWonk refers to as the “ecological fallacy,” in which
Given the increase in frequency of autism (and other neurodevelopmental disabilities) during time time period, you could do an ecological regression analysis of almost any factor that varied over time and you would find an an association with autism. I would bet that you could enter number of sushi bars per capita into an ecological regression and you’d find an association with autism rates.
The report itself notes:
Ecological studies are based on aggregate-group level data collected over time, rather than individual data. There are many limitations of ecologic analysis, such as differences of exposure levels and covariate levels within the study group, challenges in controlling for confounders [this problem is specifically addressed by EpiWonk], and within-group misclassifications that lead to potentially severe biases in the interpretation of the results. These problems severely limit the interpretation of causality, particularly biologic causality, from a study that relies solely on ecologic analysis. For example, trends over time may coincidentally appear to be related even if there is no cause-and-effect relation.
It goes without saying that “vaccines and autism” is a hot topic and it’s no wonder that the CDC is taking some pains to be careful about the sort of data that it uses and the design of its studies. For all of the greater awareness and understanding about autism out there, people are still wary of an autism diagnosis and of an autistic person in a public place acting “different,” but that’s another topic.
There are plenty of autistic children, teenagers, adults, out there. There’s also plenty more to learn about vaccines but that, too, is a separate topic.
And be sure not to miss Epi Wonk posts two cogent critiques of David Kirby’s flawed reporting.
“Given the increase in frequency of autism (and other neurodevelopmental disabilities) during time time period, you could do an ecological regression analysis of almost any factor that varied over time and you would find an an association with autism. I would bet that you could enter number of sushi bars per capita into an ecological regression and you’d find an association with autism rates.”
I just realized that, if they were trying to make their argument to be effective, this perhaps was not the example to use…
Cliff
Hi Kristina
There have been plenty of criticisms of the epidemiology in ASd used to discard causality.Even more, given the heterogeneity of ASD, the studies on vaccines done with epi in ASD- without the clinical aspects and the biochemical/metabolic analysis or the proper analysis of all aspects (accumulation of vaccines more than individual ones to mention one)
a- are not useful at an individual level-
b-they do not consider properly confounders and the nature of numbers involved ( in terms of vaccination schedules and ASD diagnosis) AND the analysis of bias?
Talking with one author of an epi study on thimerosal he was very clear about the lack of uselness of his study to discard contribution in symptomatology in ASD.
There are more and more pathways involved that may be related to the potential inflammation and other cascade of events metabolic and biochemical triggered by vaccination that are not triggered by other environmental aspects. Therefore the sushi example is not applicable
About mitochondrial dysfunction, the open literature present many many reports on different population of autistic children with findings that impact mitochondria functioning (problems with beta oxidation of fatty acids, with transport of Long chain fatty acids-carnitine, with the Krebs cycle, with the urea cycle, etc). The mitochondrial disorder is not the same than the mitochondrial dysfunction.
Now, where are the analysis of the potential situations ab,c,d, ESPECIALLY d based on biological studies-done on ecological epidemiology-considering
< a href=http://faculty.washington.edu/jonno/papers/salway.pdf??
Interesting. But where does that leave us? The abyss?
You know this is going to get a lot of press, with lazy headlines that include the word “misleading.”
Hi Laura
I do not think that this situation is unexpected or the abyss. Perhaps and only perhaps it opens the door for the kind of reflexive and equilibrated analysis – properly done related to the proper high level and deep scientific analysis of the genetics/epigenetics/biochemistry/metabolic/immunologic and other aspects in ASD that have been continuously lacking on the topic of vaccines in ASD.
@Cliff, yes, the “sushi” reference points to a certain “m” word in the eyes of some….
@María, The authors of the book I’ve been reading about vaccines address the concerns you raise about epidemiological studies directly —- am thinking there’s some hyperbole going on in Kirby’s article.
Hi Kristina
Beyond any differences in interpretation, epidemiology does not take into account biochemical/metabolic findings and individual results therefore there is no possibility to an epidemiologist to address the concerns I have.
Epiwonk could probably substitute postal rates for the sushi and continue to draw a similar correlation.
I’m a little confused.
The CDC allows that the VDS database may not be suitable for determinants of causation under the current method of data collection, which makes them bad guys, but the Geiers were attempting to use the same database to determine causation and that makes them good guys?
What is the big Deep Throat moment here–to ad mit that a particular database has analysis limitations, and that those need to be addressed in order to have more individual than ecologic analysis?
?
@María, thanks for noting that—
@Regan, I guess we should be grateful that David Kirby uploaded Dr. Geberding’s report. Reading it suggests that he’s been a little……extreme in his assessment.
Left Brain/Right Brain points this out too.
“Epiwonk could probably substitute postal rates for the sushi and continue to draw a similar correlation.”
Uh huh, which is the point (I actually don’t know postal rates, though).
Cliff
They’ve gone up as exponentially as autism diagnoses……. Epi Wonk does it again with a post that makes it very clear that, in his latest HuffPo post, Kirby himself “shows a startling string of misunderstandings and complete lack of knowledge of basic epidemiologic design and methods.”
I agree that it was very nice of Mr. Kirby to post the CDC document.
Based on the information in it, I think the CDC has tipped its hand as to the results of the yet-to-be-published study on autism and thimerosal.
I go into the discussion here:
http://leftbrainrightbrain.co.uk/?p=891
But, to make a long story short: no effect.
CDC knows the results of the study and are still supporting the 2004 IOM conclusions. They have not started any new autism/thimerosal studies.
Perhaps someone tipped the PSC (plaintiff’s lawyers in the autism vaccine trial) and that’s why they abandoned the ‘thimerosal induced epidemic’ theory.
I actually kind of doubt it. They pulled Saunder Greenland (their epidemiologist) pretty early. In the end, it was just obvious that that there was no thimerosal induced epidemic.
Still, if true, this goes beyond putting yet another stake in the heart of the ‘epidemic’. The actions of the CDC would indicate that there is no increased risk due to thimerosal in vaccines.
Somehow I think Kirby et al will manage not to se that conclusion…..