REVIEW AND ANALYSIS OF 276 ADVERSE EVENT REPORTS IN FDA DOCKET

My colleague (Dr. Norbert P. Page) and I have recently evaluated a group of 276 adverse event reports (AERs) submitted to Food and Drug Administration (FDA) pertaining to dietary supplements containing Ma Huang. These reports were released by FDA on April 3, 2000. The AERs were provided in two sets: one group of 142 received during the period June 1, 1997 to March 31, 1999 (new case series), and another group of 134 (referred to as the pre-case and post-case series) received after April 1, 1999 (except for 14 cases received before May 31, 1997). The majority of AERs appear to have been submitted as a result of the establishment of an FDA hot-line as a reaction to a media alarm on the safety of these products. Of the 276 AERs, most are anecdotal accounts of adverse effects reported by a lay public. Many of these were secondhand reports rather than reports submitted by the primary or affected persons.

1. It is interesting to note that during the time period that these AERs were gathered by FDA, the incidence rate of reporting of adverse effects has not increased although the number of persons consuming ephedrine-containing dietary supplements (ECDSs) has dramatically increased. If indeed there was a relationship between consumption of the ECDSs and reportable health effects, it is reasonable to expect that there would be an increase in AERs consistent with the increased use of ECDSs, which has not occurred.
2. Our review of the AERs shows that ECDSs, when consumed according to current national standards, are not causally associated with serious adverse events, including heart attack, stroke, and seizure. Therefore, FDA�s AER database, to the degree that it is useful at all, supports the conclusion that ECDSs are safe when marketed and consumed according to current standards to which responsible manufacturers adhere, such as those recommended by the American Herbal Products Association.

Inadequacies and Limitations of AER Data

3. This AER file contains numerous inaccuracies, inconsistencies, and omissions, including the age and sex of the affected parties, identification of the product, identification of ingredients in the product, dosage taken, dosing frequency, and dosage duration. In a few cases, the adverse effects were not documented. It was difficult to evaluate a few cases since the medical records contained illegible writing or they were poorly reproduced. This file of AERs is not a good database on which to conduct a hazard evaluation.
4. In at least 17 AERs, the product alleged to have caused the reported adverse effect is not identified. Moreover, in several AERs where the product is identified, information regarding its constituents is missing. More than 34 AERs list products for which there is no indication that any ephedrine alkaloids were even present in the product.
5. In most cases, incomplete information has been provided or collected by FDA through follow-up investigations. Indeed, for the many of the AERs, there is no indication that a follow-up investigation was conducted. In addition, there are no medical records associated with 159 AERs. With respect to the 117 case reports that do contain at least some medical records, many are so scanty or difficult to read that we were unable to glean any information from them.
6. Another major weakness of the AER files is the inadequacy of the dosage information as indicated below:

• 64 (~ 23%) have no dosage information (dosage amount, frequency, or duration);
• 97 (~ 35%) are missing data on the dose amount of product ingested;
• 103 (~ 37%) are missing dosage frequency data; and
• 96 (~ 35%) lack data on dosage duration.

1. By combining the total number of AERs lacking medical records with the number of AERs containing medical records but lacking information on dose amount, dose frequency, or dose duration, we found that 202 (approximately 73%) of the files are missing information for at least one data parameter considered essential to any legitimate causality analysis. It is neither scientifically nor medically appropriate to make any regulatory decisions based on such an inadequate database.
2. FDA has noted the limitations of the AER system:

The evaluation of data in passive surveillance reporting system[s] such as SN/AEMS is limited by several recognized factors:

Because reporting is voluntary, adverse events may occur which are not reported, and are therefore not in SN/AEMS.

A single case may be reported more than once, inflating the number of reports in the system. Health care professionals are encouraged to report all suspected adverse events. In addition, consumers of these products and other individuals may also report suspected reactions to these products. All confirmed duplicate reports are removed from SN/AEMS.

There is no certainty that an adverse event can be attributed to a particular product, or ingredient in a product.

An event may be related to or modified by an underlying disease or condition, to other products which are taken concurrently, or the event may have occurred by chance at the same time the suspected product was taken.

Accumulated case reports cannot be used to calculate incidence or estimates of product risk. They must be carefully interpreted as reporting rates, and not as occurrence or incidence rates. The length of time that a product has been marketed, the market share, experience and sophistication of the population using the product or evaluating the adverse event, publicity about an adverse reaction, and regulatory actions are all factors that influence the probability that an adverse event will be reported. Comparisons of product safety cannot be directly obtained from these data.

"Background Information on SN/AEMS." Center for Food Safety and Applied Nutrition, FDA (1997).

3. It is accepted that FDA�s passive surveillance system for adverse effects may under-report the true incidence of side effects associated with a product. The extent of this "under reporting" is difficult to judge and is most certainly less than that for other products since there has been an extensive and aggressive media campaign about ECDSs. Major media events have been staged since the publication of the proposed rule in December 1997 (TV programs such as 20/20, Dateline, NBC Today, Good Morning America and newspaper campaigns in the Washington Post and other newspapers). These "media events" have most certainly encouraged the use of FDA�s hot line reporting system.
4. Any estimates of under-reporting must be highly unreliable.
5. Listed below are AER cases that FDA has released to the public and that have serious deficiencies and inadequacies and should not be included in a causation analysis.

• Adverse events occurred when the individual was definitely not using an ephedra product (AER 11919, 12466, 13328);
• No adverse effects were listed (AER 12664, 12671, 13226);
• Duplicate cases (AER 12782-12783, 13507-13508, 13955-13926);
• Many products were being consumed at the same time (AER 11566, 11915, 12488, 12859, 13127, 13332, 13408, 13675, 13676, 13762, 13715, 13796, 13901, 13946);
• Unknown product was used (AER 11912, 11913, 12466, 12536, 12537, 12664, 12671, 12860, 12974, 13226, 13503, 13888);
• Obvious overdosing or improper use of products (AER 13408, 13524, 13651, 13661, 13676, 13764, 13829, 13905, 13946);
• Uncertainty that the product was taken (AER 12921);
• Continuation of adverse event after cessation of use of ephedra product (AER 11915, 11918, 12462, 12598, 12608, 12609, 12630, 12602, 12700, 12880, 12950, 13058, 13346, 13351, 13414, 13499, 13463, 13706, 13793, 13930);
• Medically unrelated to ephedrine ingestion (AER 11915, 12720, 12722, 12859, 13266);
• An underlying condition is a more likely cause of the injury (heart disease-AER 11311, 12452, 12485, 12506, 12508, 12537, 12630, 12713, 12782, 12844, 13009, 13009, 13110, 13127, 13167, 13351, 13405, 13510, 13511, 13532, 13632, 13634, 16343, 13702, 13711, 13717, 13718, 13793, 13796, 13798, 13806, 13815, 13863, 13906, 13945) or hypertension (AER 12452, 12485, 12536, 12713, 12733, 12844, 12978, 13041, 13167, 13335, 13336, 13499, 13507, 13675, 13762, 13769, 13793, 13829, 13906, 13945, 13972, 13993) or CNS or psychiatric problems (AER 12734, 13514, 13526, 13632, 13797, 13809, 13826; 13946);
• The information in the file was totally inadequate (AER 12664, 12782, 12783, 12874, 12785,12842, 12860, 12921, 12927, 12974, 13051, 13165, 13226, 13236, 13510, 13516, 13519, 13545, 13625, 13709, 13772, 13776, 13844, 13845, 13858, 13863, 13867, 13874, 13877, 13895, 13898, 13912, 13926, 13935, 13936, 13994, 13950, 13955, 13956, 13968, 13972, 13973, 13993).

Many of the AER cases allege medical conditions for which there is no scientific or medical foundation based on the known biological activity and mechanism of action of the ephedrine alkaloids. Among the alleged medical conditions that are considered medically implausible or obviously due to some other risk factor are:

• Deaths due to hyperthermia and/or violent exercise and bodily abuse (AER 12720, 12722, 12859);
• Death due to congenital heart defect (AER 12921);
• Death in individual with significant cardiac pathology not associated with product and taking many products (81 pills daily) (AER 13127);
• Death due to heat exhaustion and rhabdomyolysis causing sickle cell anemia (AER 13672);
• Ovarian tumor (AER 13265);
• Abdominal tumor that apparently grew from the size of a pea to the size of a tennis ball in one week (AER 13413);
• Thyroid tumor (AER 13933);
• Acute lymphocytic leukemia and mediastinal lymphoma (AER 13464);
• Rhabdomyolysis due to excessive exercise (AER 12722, 12859, 13266);
• Necrotizing enterocolitis in a premature newborn (AER 12594);
• Probable drug abuse in 19 year old Viagra user (AER 13370);
• Diarrhea followed by constipation in individual consuming 32 dietary supplement pills per day along with two supplement drinks (AER 13332);
• Chest pain in a 15-year-old girl who took six tablets all at once followed by two double café lattes. Caffeine overdose suspected (AER 13229);
• Catatonia in an individual taking several different products including Nitro Gorilla, which contains yohimbine (AER 12488);
• Seizures caused by hypoglycemica with metabolic abnormalities (AER13408);
• Thrombocytopenia after one dose. Allergic reaction more likely the cause (AER 13922);
• Mastitis (AER 113917);
• Fulminating liver disease most probably caused by another supplement containing Larrea tridentata (AER 113812);
• Pancreatitis and hypertriglyceridemia (AER 13836, 13903);
• Vaginal bleeding (AER 13864, 13979);
• Pregnancy due to counteracting the effectiveness of birth control pills (BCP) resulting in pregnancy (AER 13773, 13859, 13892, 13893, 13974). In AER 13974 the complainant claims that two pregnancies were the result of using the product. The second pregnancy occurred after discontinuance of Depo-Provera and no other contraceptive was used.

Causality Analysis and Results

1. We have carefully evaluated all AER files and have prepared a detailed table with important data for each AER. Despite the inadequacy of the AER files, we have attempted to make causality judgments for all cases. We are presenting our evaluations and supporting data as Appendix 1.
2. The basic approach to our causality judgments is similar to those generally used in determining a cause and effect relationship. The outline of our causality decision tree is attached as Appendix 2.
3. There is no universally accepted scheme for making causality judgments or risk assessments of pharmaceuticals based on AERs. Stephens (1988) has described 22 such approaches. Jones (1992) and WHO (1994) have more recently reviewed the approaches used to evaluate causation of suspected drug reactions.
4. Most schemes proposed for causality judgments are derived from the common sense criteria of Koch�s original postulates used to confirm causation of disease by an invading bacterium. Basically, four major criteria are used: was the agent present in the body at the time the event occurred (Temporal Relationship), was the type of event or disease known to be producible by the agent under conditions of the exposure (Biological Plausibility), did the effect go away when the agent was removed from the body (Dechallenge), and was the effect reproducible upon future exposure (Rechallenge)? Our algorithm for causal assessment uses these four fundamental criteria.
5. We used the basic construct of the FDA algorithm as described by Jones (1992) with minor modifications. The FDA algorithm listed four primary decision categories (Highly Probable, Probable, Possible, and Remote). Operationally, we found the category "Possible" to be too broad and felt that it needed subdivided to allow for "Low Possible." In a similar manner, we recognized the need for categories for rejection, namely, "Improbable," "Irrelevant," and "Inadequate Information." Thus, our eight categories for causal degree of certainty are: "Highly Probable"; "Probable"; "Possible"; "Low Possible"; "Remote"; "Improbable"; "Irrelevant" and "Inadequate Information."
6. Specifically, we selected criteria that could be evaluated on a uniform basis with subsequent assignment of an AER to one of eight categories. It is our view that this algorithm and its evaluative criteria serve as a basis for the operational aspects of risk analysis as well as a model for critique and use by peer reviewers of the process and risk assessments.
7. We applied the algorithm to all the new 276 AERs pertaining to dietary supplements containing ephedrine alkaloids. Again, we stress that having reliable data is key to the conduct of a credible risk analysis. In order to have confidence in the data, it is necessary to have unbiased reporting in sufficient detail. As previously described, however, much of the information in the AERs assembled by FDA is anecdotal, incomplete, potentially biased, and has a high potential for inaccuracy. Ideally, such reporting should be done by medically trained personnel who are willing to devote the time to accurately and completely prepare the reports. Only a few of the AERs conform to these standards.
8. The summary of causality ratings of the individual assessments are as follows:

Causality Rating #/% of AERs

Highly Probable 0 (0%)

Probable 21 (7.61%)

Possible 50 (17.75%)

Low Possible 46 (16.67%)

Remote 25 (9.42%)

Improbable 77 (27.9%)

Inadequate Information 45 (16.30%)

Irrelevant 12 (4.35%)

9. We have attempted to apply a sound scientific risk assessment approach to the evaluation of the FDA AER files and associated data. However, on account of inadequate and potentially misleading data, we have limited confidence in not only FDA�s conclusions regarding causality, but also in the results of those few cases under our own conservative analysis where the results suggest that a causal relationship may exist between the adverse effect reported and the product consumed.

Death Cases:

10. Of the 22 AER files that reported deaths, the data were inadequate to assign a rating of "highly probable" or "probable" for any. Eight (8) lacked medical records of any type so that the medical history could not be considered. Three of the deaths were associated with self-induced hyperthermia and or/violent exercise, while one death was directly related to a serious congenital heart defect.
11. We rated two deaths as "possible." One (AER 13762) occurred in an individual who was hypertensive and on antihypertensive medication and who died from an intracerberal hemorrhage while exercising in a gym. This individual was also taking a product that contained yohimbine. The other death rated as "possible" (AER 13914) occurred suddenly during a strenuous physical ability test (a running event). The death may have been caused by a cardiac arrhythmia in an individual that was apparently taking several dietary supplements at the time. The file indicates some uncertainty as to what product or products had actually been consumed. While we have rated these two cases as "possible," we are in agreement with Dr. Hutchins� conclusion that the deaths are more likely the result of other factors than consumption of ephedra containing dietary supplement. We have grouped the death cases in a separate table (Appendix 3).
12. Based on our review of the FDA AERs pertaining to dietary supplements containing ephedrine alkaloids, there is no evidence that such dietary supplements, as currently formulated and when used as recommended, represent a potential death threat.

All Cases:

13. None of the 276 cases were rated as "highly probable."
14. We rated only 21 cases as "probables." Of these 21 AERs, only 5 contained medical records (including one overdose). One of the AERs we credited as having a medical record had very limited useful medical information and pertained to a product no longer in being marketed (Formula One). In another "probable" case in which tachycardia was seen, the consumer did not feel that the product was responsible for the tachycardia and refused to cooperate with FDA.
15. Five cases were rated as probable but consisted of overdoses and/or misuse of the products. These overdosing cases reported nausea and emotional upset; nausea, hyperactivity, and sleepiness; syncope, tachycardia, and elevated blood pressure; dyspnea; and tachycardia, respectively. In another "probable" case, an individual with a history of hypertension had a hypertensive crisis, and the treating physician did not implicate the product. In another "probable" case the individual had personality changes ascribed to a caffeine sensitivity he had since the age of 7 or 8.
16. We rated 50 cases as "possibles," only 19 of which were supported by medical records. Among these 19 "possibles" were the following:

• A stroke after vigorous exercise.
• Tachycardia and hypertension in a person who subsequently underwent cardiac bypass surgery.
• Temporary double vision, fever, chills, headache and hypertension.
• Hypertension, loss of motor skills in right hand and wrist (possibly due to a pinched nerve). Patient was subsequently found to be hypertensive.
• Questionable seizure. Toxicologic screen was negative. Possible abuse of product may have occurred.
• Tachycardia, chest pain, agitation, dyspnea, border-line manic depressive. Possible abuse of product may have occurred.
• Chest pain and jitteriness. Probable for jitteriness. Chest pain still present after discontinuance.

• Chest pain in 15-year-old female who had overdosed on 6 tablets taken all at once and followed by drinking two double café lattes. The chest pain was probably due to a caffeine overdose.
• Tachycardia, mild dyspnea and dizziness in an individual taking Proventil.
• Heart attack with ventricular fibrillation. Coronary artery disease was present.

• Death from intracerebral hemorrhage. Stroke occurred during exercise in a gym. The deceased was hypertensive and was on antihypertensive medication. The deceased also was taking a yohimbine product.
• Myocardial injury secondary to drug overdose/abuse. Toxicology screen positive for marihuana.
• Stroke.
• Cardiomyopathy, dyspnea, tachycardia. History of alcohol abuse and marihuana use. The subject consumed four other weight reduction products.
• Respiratory failure and possible stroke. Not sure of all products that were being taken and could not provide a list. Drug abuse has to be considered.
• Sudden death during a physical ability test (a running event). The death may have been caused by a cardiac arrhythmia. This individual may have been taking several dietary supplements. The file indicates some uncertainty as to what product or products had been consumed.
• Dyspnea, chest pain and stomach upset.
• Syncope, anxiety and hypertension. Medical history of transient hyperglycemia. Also had rhinitis, sinusitis and bronchitis at this time. Also taking a product called Sugarbusters.

1. According to its own caveats and disclaimers, FDA never meant for the AER system to support a causal analysis for any product. Indeed, we do not believe that the AERs compiled by FDA support a causal connection between consumption of ephedra-containing dietary supplements and serious illnesses or injuries. The majority of AERs submitted to FDA are inaccurate, incomplete, and not satisfactory for use in any risk assessment. However, even assuming that these reports were all satisfactory, there have been only 1199 adverse reports (923 reports from the 1997 proposed rule docket and the recently added 276 reports) for an ephedrine herbal product in which there were billions of servings sold since 1994. In particular, when ephedra products are consumed according to national standards, there is no evidence in this AER database that alleged deaths, strokes, heart attacks, psychotic episodes, or any other serious adverse events occur more often in individuals who consume ephedra dietary supplements than in those who do not. Further, the majority of the "serious adverse effects" such as death, strokes, heart attacks and seizures seen in the entire AER database, can be attributable to (1) preexisting medical conditions or (2) physical body abuse through excessive exercise along with the concomitant use of medications and other substances, for which the labels recommended against its use.
2. A showing that a treated group experiences more incidents than a non-treated group is essential to the proof of any causality hypothesis, including the hypothesis that exposure to ephedra dietary supplements produces serious injuries. The available data from controlled studies show just the opposite � consumers of ephedra dietary supplements do not experience an increase in adverse events when compared to a placebo.
3. Finally, it is interesting to note that in an FDA internal memo from M. Chen and C. Karwoski of the Division of Drug Risk Evaluation of the Center for Drug Evaluation and Research to the Center for Food Safety and Applied Nutrition, dated March 10, 2000, these individuals concede that it is possible that the reported serious adverse events in all of the AERs are reflective of the coincidental background of spontaneous occurrence in the population and are not causally related to the use of ephedra products. We agree with the FDA in this finding.
4. FDA performed a causality analysis on 132 of the 142 AERs in the new case group and determined that 72 were so flawed as to be rejected from their analysis on the basis of insufficient information. Another 38 had serious deficiencies so that they could be included only as "supportive" cases. Only 22 cases were considered to be "attributable." The fact that only 1 of 5 AERs are worthy of being classified as attributable speaks for the inadequate quality and severe limitations of the AER database.
5. Evaluating the AERs is a highly judgmental process and open to various levels of confidence. This is demonstrated by the fact that FDA's selected outside reviewers had enormous differences in opinion from FDA in regard to causation. Two outside reviewers examined all of the cases. FDA sent only three of the "attributable" cases to the two other reviewers for evaluation. Similarly, only two of the outside reviewers examined all 38 "supportive" cases while the other two reviewers only commented on 8 and 12 supportive cases respectively.
6. As mentioned, considerable disagreement was evident between FDA internal staff and the outside reviewers. Of the 22 "attributables," the three outside reviewers disagreed with FDA on 10 of the cases (45%). For the "supportives," the outside reviewers disagreed with FDA on 25 of the cases (66%). Therefore, FDA failed to receive support from their own selected outside reviewers for its "attributable" or "supportive" classification on 35 out 60 cases, an unacceptable 57% lack of concurrence.
7. This high lack of concurrence among FDA�s outside consultants, as well as the industry�s consultants asked to do a similar causal analysis, amplifies the reason why this inadequate database should not be used for making causality judgments.

CONCLUSIONS

8. The AER database is inadequate for any meaningful scientific or medical analysis. Only a small subset of the reports have the details and integrity necessary for a causation analysis. It is apparent that FDA attaches great significance to the value of the database even after acknowledging the inadequacy of using the AER database to make a comparison or judgement as to safety of a compound. It is incumbent on FDA to provide a firm basis for expressing public health concerns in sufficient detail to withstand scientific/medical critique. As was the case with FDA�s proposed rule in 1997, FDA has again failed to provide a basis for its concerns.
9. The medical fact, confirmed by on-line medical literature and the absence of injury reports to FDA on the ingestion of 25 mg doses of ephedrine by asthmatics, is that 25 mg of ephedrine per single dose has never shown any serious adverse effect on human beings studied in over two dozen clinical trials. Indeed, Pental et al. (1984) states that "doses of ephedrine up to 60 mg generally do not increase blood pressure and those of 60 or 90 mg produced only small increase in heart rate." This safe level is about three times the single dose of a typical ephedra dietary supplement. FDA�s regulations permit asthma-compromised individuals to ingest six daily doses of 25 mg of ephedrine (150 mg/day) on a chronic basis. Yet, there are almost no serious adverse incident reports from the ingestion of the pure drug form, taken at higher doses than are recommended in dietary supplements. FDA does not require on over-the-counter drugs containing ephedrine warning labels to avoid use of such products with caffeine. FDA has not been able to adequately explain this troublesome inconsistency.
10. After the evaluation of these AERs, it remains our opinion that 25 mg per serving, up to a maximum of 100 mg of ephedra alkaloids in a 24-hour period, administered for 12 weeks or more with or without moderate levels of caffeine, is safe.

 

Respectfully submitted,

 

Theodore M. Farber, Ph..D., DABT

Norbert P. Page, D.V.M., M.S.

 

<< Previous | Next >>