“Failed“ Clinical Trials Using Placebo Brain Surgery Controls

"Square Pegs in Round Holes?"

Poster presented  by the Parkinson Pipeline Project at the
2009 American Society for Experimental Neurotherapeutics (ASENT) Conference

Authors: Perry D. Cohen, Wilson H. DeCamp, Linda Herman, Arnold M. Kuzmack, Stan Planton, Carolyn Stephenson, Peggy Willocks and Paula Wittekind - Parkinson Pipeline Project, Washington, DC

OBJECTIVE

This paper examines three failed clinical trials for promising new Parkinson’s therapies. All have similar designs that require surgical intervention to deliver the treatment and utilize sham brain surgery as a placebo control. It relates common reasons for their failure and gives scrutiny to evidence on the benefits vs. risks of placebo brain surgery. Much of this paper was displayed as a poster at the ASENT annual meeting in March, 2009

Phase II of all three trials were multicenter, randomized, double blind, sham surgery controlled studies All three showed favorable results in the open label Phase I trials, but did not meet their primary endpoints in pivotal Phase II studies.

All three trials were conducted using the dopamine replacement theory as treatment by either viability of available dopamine (GDNF), production of dopamine from transplanted cells (Spheramine), or genetic alteration for the production of dopamine (CERE 120).

Table 1. Summary of Design Characteristics of Failed Clinical Trials

GDNF (Neurotrophic factor) – recombinant GDNF by pump infusion method

Amgen sponsored two open label phase I safety trials of GDNF in 15 patients. The studies were initially for six months, with some patients treated for up to 3 1/2 years. Based on the clinical endpoint of reduction in the UPDRS motor "off" score, the efficacy ranged from 39 to 57%.

Randomized, double blind, placebo controlled, parallel group phase II trials were initiated in 34 patients. The clinical endpoint was the Change in UPDRS motor score in the practically defined off condition at 6 months.

The sponsor (Amgen, Inc.) terminated the phase II trials in September 2004. The rationale given was that, “Six months of treatment with GDNF delivered to the putamen failed to improve UPDRS scores compared to placebo." There was "evidence of alteration of brain function," a likely reference to changes on neuroimaging, but improvement on UPDRS scores did not meet the primary endpoint of the trial. However, a participant from the Bristol (UK) study died of an unrelated cause, and, upon examination of his brain via autopsy, neural sprouting was noted (the first report of its kind).

Open label extension studies began to resolve differing trial results. But in Sept. 2004, Amgen sent letters to clinical investigators halting further clinical studies, due to safety concerns – development of lesions in the cerebellum of 4 test monkeys and "anti-r-metHuGDNF neutralizing antibodies found in two of the study participants to date."

Spheramine [Retinal Pigmented Epithelial (RPE) Cells]

In 2000, Titan Pharmaceuticals in a Phase I open label trial consisting of six participants with advanced disease (3 3.5 or greater on the Hoehn & Yahr scale) received unilateral treatment (for their “worst” side) transplanting RPE cells (without the use of immunosuppressant), using a donor eye from a cadaver. (One eye can be used to treat hundreds or patients.) At 12 months, an average improvement of 48% in the UPDRS M (off) outcome measure was realized, along with improvements seen in other measures of motor function and quality of life. One participant dropped out because of a later diagnosis of Parkinson’s Plus. Participants continued to be followed through 48 months, maintaining a 44% average improvement and continue to be followed. It was reported, "The data also demonstrate a very good preliminary safety profile for Spheramine. There has been no evidence to date of any significant side effects in any of the patients . . . a reduction in dyskinesias for most patients and . . . no ‘off state’ dyskinesias . . . observed." (Titan handout, April 2002)

In 2003, a phase II study was initiated with a randomized, double blind, placebo controlled (sham surgery) trial of 71 patients (78 were actually recruited) receiving treatment bilaterally, and received fast track approval by the FDA. Titan was joined with the U.S. Berlex sponsor, which is also Schering AG (Germany), and was later acquired by Bayer (Bayer Schering/Titan). In July 2008, Titan announced that Phase II did not meet its primary or secondary endpoints and Bayer Schering AG withdrew as a sponsor. The sponsors (Bayer Schering / Titan) announced that they had discontinued development of Spheramine in July 2008.

Titan stated that its "potential cell based treatment for Parkinson's Disease failed to meet its primary and secondary endpoints in a Phase IIb study, and likely won't be continued by partner Bayer Schering Pharma... Initial analysis of results from the 71 patient study of Spheramine designed to test the safety, tolerability and efficacy of the treatment found that it had no significant differences from sham surgery arms after 12 months of follow up." (Company press release dated 7/2/08)

Phase II data have not yet been published, but the phase II study was presented at a conference (13th International Congress of Parkinson’s Disease and Movement Disorders, Paris, France, June 7-11, 2009).

The STEPS trial: A Phase 2b study evaluating Spheramine® in patients with advanced Parkinson’s disease. RL Watts, RE Gross, RA Hauser, RAE Bakay, H Reichmann, Weisner, NP Stover, E Reissig, H Steiner-Schulze, K Fichte . Abstract LB-18

The primary endpoint was change in UPDRS III (motor) off score at 12 months

Conclusions:” There was no statistically significant difference between Spheramine- and sham-implanted patients at 12 months in the off state. There were also no differences in secondary outcomes, including on-state UPDRS III, time spent in off or on state, levodopa reduction, or UPDRS ADL score.

“The study failed to show efficacy of cellular implants of human retinal pigmented epithelial cells beyond a remarkable placebo effect,” the authors concluded. “Preliminary long-term results in part of the study patients suggests that the placebo effect persists even longer than 12 months.”

CERE 120 (neurturin) – Gene therapy

CERE 120, a gene therapy product in development for the treatment of Parkinson’s disease, was administered to the putamen with adeno associated virus carrying the gene for neurturin (NTN), a growth factor related to GDNF and shown in experimental models to protect dopaminergic neurons from degeneration. Six patients received a low dose and six a high dose (1.4 x 10^11 vs. 5.7 vector genomes). Neurturin was well tolerated and appeared to reduce symptoms by approximately 40% (p<0.001), as measured by the Unified Parkinson’s Disease Rating Scale (UPDRS) motor “off” score, in an open label Phase 1 study in 12 patients with advanced disease. (Company press release dated 10/10/2006)

The sponsor (Ceregene) announced the phase II trial failure in Nov. 2008. Analysis of the phase II trial data did not demonstrate an appreciable difference between patients treated with CERE 120 versus those in the control group. Both groups showed an approximate 7 point improvement in the protocol defined primary endpoint (Unified Parkinson’s Disease Rating Scale motor off score at 12 months), relative to a mean at baseline of approximately 39 points. Both groups had a substantial number of patients who demonstrated a meaningful clinical improvement from baseline. CERE 120 appeared to be safe and well tolerated."

A company spokesman stated "...we are stunned by the results of this trial and will continue to analyze the data in order to gain greater insight into the factors that may have contributed to this negative outcome, not only to build upon this insight for our Parkinson’s program, but also to help assure continued successful development of our product candidates for other diseases.” (Company press release dated 11/28/08)

In an in depth interview with the Michael J. Fox Foundation, Raymond T. Bartus, PhD, executive vice president and chief scientific officer of Ceregene, reported that they are attempting to redesign the CERE- 120 trial to expand the delivery target area and increase dosage. Two trial participants died from unrelated causes, providing the opportunity to view the progress of the neurturin through autopsies. Discoveries were made that may enable research to continue in the near future. (http://www.michaeljfox.org/research_viewpoints_newsInContext_article.cfm?ID=11)

In May 2009, Ceregene reported that based on additional analyses of data from "30 subjects who continued to be evaluated under double-blind conditions for up to 18 months, there were increasing effects of CERE-120 over time. There was a "clinically modest but statistically significant treatment effect in the primary efficacy measure (UPDRS motor off; p=0.025), as well as similar effects on several more secondary motor measures (p<0.05), were seen at the 18 month endpoint." (Ceregene press release)
In July 2009, "The Michael J. Fox Foundation agreed to fund a long-term (48 months), open-label analysis of data from Ceregene’s Phase 2 trial of CERE-120...The funding will allow Ceregene to collect and analyze data from trial enrollees for another 48 months. While the study will be unblinded, the goal is to gather as much data on safety and efficacy as possible in an open-label setting, while looking for suggestions of a longer-term neuroprotective effect. " (press release)
In September2009, recruitment began for a new phase I/II trial --

Phase 1/2 Trial Assessing the Safety and Efficacy of Bilateral Intraputaminal and Intranigral Administration of CERE-120 (Adeno-Associated Virus Serotype 2 [AAV2]-Neurturin [NTN]) in Subjects With Idiopathic Parkinson's Disease.

“Approximately sixty patients with Parkinson's disease will participate in this study. The first part of the study is designed to evaluate the safety of two different doses of CERE-120. Six subjects will participate in this part of the study, all of whom will receive CERE-120. The second part of the study will provide more information about the safety of CERE-120 and also evaluate if it is beneficial in the treatment of Parkinson's disease. In this portion of the study, half of the subjects will receive CERE-120 and the other half will undergo a "placebo" surgery (or sham surgery) where no medication will be injected. Participants in both phases of the study will be followed for three years after surgery.”

(see: clinicaltrials.gov record at: http://clinicaltrials.gov/ct2/show/NCT00985517?term=Ceregene+safety+phase&rank=1

Some of the differences between the two trials are:

• The earlier trial targeted delivery of CERE120 to the putamin. The new one adds the nigral area as a target.

• There are differences in outcome measures – The first phase II trial ‘s primary outcome measure was the score on the UPDRS Part III while OFF. The Time Frame: was 12 Months.

• New phase II trial’s Primary Outcome Measures are safety issues, while the Secondary Outcome Measures include:

  • “Changes from baseline in clinical laboratory tests, vital signs, weight, and examination findings and clinically significant changes from baseline in brain imaging results.”

  • The Time Frame has been increased to 36 months

RESEARCH FINDINGS ON PLACEBO BRAIN SURGERY

The Placebo Response is based on conditioned expectations from the social context of the intervention for a reward. It is a well known concept in social science including the Hawthorn effect from industrial engineering in studies of worker motivation showing the power of an experiment, and the Pygmalion effect in education documenting the subtle bias from the expectations of teachers (authority figures). The greater the saliency from the risk and other stimulation the more powerful the effect. No wonder experimental brain surgery produces such a dramatic effect.

The mechanism of the placebo effect is release of endogenous dopamine in the brain using the same channels that are used by humans for movement. This makes the placebo effect indistinguishable from and directly confounded with the most prominent features of PD.

• Dilution of placebo effects in randomized experiments. In the treatment group confounding placebo effects may diluted by the less than 100% likelihood that the patient is on the “real” treatment, and in the control group the chance that the patient is on the “real” thing elevates expectations. In addition, unlike real medical practice where both doctors and patients want patients to improve, the experimental situation tendency to dampen hope for fear of biasing results also may dampen treatment effects. Attempts to mitigate the hopes and expectations of patients whose primary if not only reason for taking the significant risks of experimental brain surgery are those very hopes and expectations, will not succeed and may further bias results due to placebo effects.

• False negative (Type 2 errors) bias. Our observation is that in placebo brain surgery controlled trials that placebo effects are so strong that they overwhelm the power of the study and introduce type 2 errors. The assumption that blinding neutralizes this bias to allow measured improvements to be attributed to the treatment does not fit the findings that both treatment and control groups improve. Instead, by randomizing the very strong placebo effect you dilute treatment group effects that may be masked by placebo response, and increase placebo response in the control group. For Ceregene both treatment and control improved (!!) for 70% of subjects; in other studies treatment groups did better than control groups but both IMPROVED so differences were not statistically significant.

• Triggering effects in pain control. Experiments with pain control show the necessity of letting the patient know s/he is getting pain medicine, and placeboes work well if the patient expects that s/he is receiving the medicine.

DISCUSSION

Based on the design of the three studies described above, we suggest two possible reasons for the unanticipated failures in phase II.

1. Selection bias resulted in different types of PD patients being enrolled.

      Examples of such bias are:

• tremor dominant vs. rigidity dominant symptoms

• responders vs. non responders to standard therapy

• responders vs. non responders to placebo

• optimized on medications vs. non optimized

2. Sham brain surgery as placebo may be so powerful that it overwhelms treatment effects for a time (maybe up to 2 or more years)..

Such an effect could force type 2 errors when the interim study results are analyzed after a shorter time.

CONSEQUENCES OF “Failed” Pivotal Trials.

Development of new therapies by industry sponsors is extraordinarily high risk and high cost. It not only requires great understanding and knowledge to identify targets for intervention, but it also requires flawless execution of complex protocols to get it right.

Dr. Stanley Fahn of the Columbia University Medical Center has stated:  “A negative trial result does not necessarily mean that the compound in question is of no therapeutic value – especially when that compound has demonstrated promise in animal studies and earlier, smaller, human trials. There could have been a problem with the study design or lack of optimum dosage of the experimental compound. A variation in the study design (e.g., different duration, different dosage, different patient selection criteria, and a change in method of drug delivery) may yield different results, and should be explored before any particular approach is abandoned."

Business Decisions. The science, however, is only part of the decision to continue development of a new treatment. The economy, patent life, and competitive factors as well as the capital reserves and cash flow of the company weigh in heavily on what is primarily a business decision. Even when money was readily available enormous capital investments (close to $1B ) to carry the development more that 15 years for neurology before receiving any return, and even then many treatments fail in late stages of development, after most of the money is spent. To make matters worse, most of the innovative therapies are sponsored by small entrepreneurial firms with little revenue and investment capital that are betting the whole company on the outcome of the study. These entrepreneurs are usually committed to their idea, so want to give it every chance to succeed, but once a pivotal trial fails. Decisions about further development pass to the responsibility of a dispassionate large company executive or other investors who are not likely to be very familiar with the promise of the science or with patients that have done well on the treatment. Thus, the real consequence of a failed study is most often a termination of the program, and often the closing of the business, such as Titan Pharmaceutical described earlier. Table 2 lists seven more PD therapies that have terminated in late stages

Table 2. Other Therapies recently terminated in late phases
 

KEY QUESTIONS

Our analysis of the three recent failed trials points to questions that need to be addressed in order to justify what many consider to be unjustified risk to ask patients to take in a blinded, placebo brain surgery controlled clinical trial, even given expectations that even if they do not benefit personally science will advance.

1. What adjustments in the design of statistical controls are necessary to account for the impact of the context of an experimental protocol that alters expectations of participants by blinded randomization into treatment and control groups?

2. What scientific criteria are used to determine efficacy or the lack of efficacy of a treatment?

3. What assumptions are made about the interaction effects between a treatment response and a placebo response?

4. What factors should be considered when selecting samples from a heterogeneous populations as the evidence grows that some endpoints may be achievable only for patients (responders) with certain genetic variants or clinical sub types of the disease or are influenced by other factors including the method of delivery?

This presentation adds urgency to the need for these discussions, because trials are failing, and promising therapies are being shelved in what has been called the “tyranny of the type 2 error". (M. Hutchinson, S. Gurney and R. Newson. GDNF in Parkinson disease: An object lesson in the tyranny of type II. Journal of Neuroscience Methods. 163, 2, July 2007, 190 92)

CONCLUSION

The above failed phase II studies were for therapies that were known to work for some people over extended periods. The members of the Parkinson Pipeline Project have analyzed possible explanations for this poor record of accomplishment. We have suggested hypotheses that fit the pattern of results seen in these studies. Our goal is to present a clear and convincing argument that these are plausible hypotheses that merit further study and such a study is a very high priority.

There is considerable research on pain, depression and the mechanism of the placebo effect. These studies suggest that an experimental protocol that views placebo surgery as a "bias" to be minimized may in fact undermine the validity of the study. Key questions are raised that researchers and regulators need to answer in order to prevent type 2 (false negative) errors. Based on the research literature, alternative design features and methods that are more rigorous are needed to reduce error. Particularly valuable would be acceptance of un-blinding patients (not raters) in comparison to best medical practice as was for DBS (a surgical intervention and the most important new therapy for PD in the 40 years since Levodopa was introduced 40 years ago).

Given the number of new, promising, surgically delivered, treatments in the PD pipeline, policy discussions among FDA officials, scientists and knowledgeable patient advocates (including patients that volunteer for experimental treatments) on both the scientific and ethical issues about what constitutes adequate control in the study design must be a high priority to provide guidance to sponsors. The topic needs to be addressed fully before other promising therapies are shelved based on faulty assumptions about human behavior and the response to medicines.

The authors wish to acknowledge the support of the
Parkinson's Disease Foundation to the work of the Parkinson Pipeline Project.