Gold Standard Clinical Trials - Opportunity Squandered

We left off with the proposition that for psychiatric purposes “randomized, double-blind, placebo-controlled” (RDPC) clinical trials - medicine’s gold standard - actually tell us nothing. One reason is that we know next to nothing about the phenomenon we’re supposed to be investigating (what the hell is depression, anyway?) which leads to a whole domino effect of faulty assumptions (such as testing “depression” with “antidepressants”).

The other reason has to do with the fact the only ones with the resources to run RDPCs - the drug companies - have marketing objectives in mind, which renders all their findings highly suspect from the outset. I could write a book about the manifold abuses (such as statistical manipulation, data-spinning, ghost-writing, payola to researchers and clinicians) associated with these enterprises. Other people have.

In the real world, if these studies actually told us something, our doctors would have something better to say to us than, “Let’s try Prozac (or Abilify, or whatever) and see how it works.” There would be no endless and frustrating rounds of pill roulette. And maybe there would be less mental illness kicking around.

Ironically, in their over-eager pursuit of “one-size-fits-all” treatments for depression and other common DSM illnesses, Pharma may have missed a golden opportunity to investigate specific things that truly go wrong with us - never mind the illness - where targeted meds treatment may indeed make a huge difference.

I came across my first taste of this at the very first American Psychiatric Association annual meeting I attended in 2002. The conference was stuck in an “antidepressants to treat depression” mindset, and so was I. Then I came across a poster of a very small pilot study (maybe ten patients) by Joseph Goldberg of the Mt Sinai School of Medicine involving treating bipolar depressed patients with Mirapex (pramipexole).

Mirapex, a “dopamine agonist,” is FDA-indicated for treating Parkinson’s. We know what dopamine can do for the brain - just ask any crackhead, tweaker, speed-freak, or anyone who craves a good buzz. But we also know that not enough dopamine equates with being a vegetable - just ask anyone overmedicated on an antipsychotic, which acts as a dopamine-blocker.

To vastly oversimplify, when dopamine is firing right, we have a feeling of being alive. We experience pleasure, our thinking is sharp, we are fully engaged, we look forward to our future. In short, we feel the very opposite to feeling “depressed.”

It’s easy to see where Dr Goldberg was coming from: If we could boot up the dopamine system in the brains of those who would appear to benefit from it, with a med that was safe and non-addictive, we might actually witness a number of patients suddenly coming to life. These type of patients were more likely to be found in the bipolar population, whose depressions tend to manifest as more “anergic” (lack of energy) than those with unipolar depression.

In another words, a depression is not just a depression. The unipolar-bipolar distinction is a very crude one (the DSM doesn’t even bother to make the distinction), but at least this is a move in the right direction.

Pharma already had clinical trials up and running for treating bipolar depression when I was at the 2002 APA, including GSK with Lamictal. Eli Lilly would be out in force the next year with its Prozac-Zyprexa combo Symbyax, and later AstraZeneca would produce two eye-popping results with Seroquel. But there was something different about Dr Goldberg’s approach. In a letter published three years earlier in the American Journal of Psychiatry, he and two colleagues observed:

The greater density of [dopamine] D3 receptors in the mesolimbic areas involved in mood regulation may relate to pramipexole's efficacy in psychomotor-retarded conditions (ie, negative symptoms, Parkinson's disease, depression).

Around the same time, the novel wakefulness agent Provigil (modafinil) came on the scene, FDA-indicated for treating narcolepsy. We didn’t exactly know how the stuff worked, but psychiatry viewed it essentially as “speed with brakes.” Not exactly an amphetamine, but with some effect at booting up dopamine (and possibly also hypocretin, which plays a major role in wakefulness). Could the med be recruited to treat certain types of depression?

Unfortunately, its manufacturer, Cephalon, figured it would be far more efficient to make its case directly to doctors rather than go through the trouble and expense of clinical trials. In 2008, Cephalon pled guilty to a criminal misdemeanor for illegally marketing Provigil and two other meds for off-label uses, and paid a $425 million fine.

Meanwhile, various small studies were proving a clear disincentive for Cephalon to gamble on a major trial. The studies tended to focus on Provigil as an augmenting agent to treat depression, but with no attempt made to single out those patients most likely to respond to an energizing agent. In other words, the failures from treating those who didn’t belong in any of these studies (even if they were “depressed”) masked the successes of those who did (but who were they?).

The Aug 2007 American Journal of Psychiatry published an RDPC study that actually provided a clue. The study followed the approach of Dr Goldberg’s 2002 pilot study that tested Mirapex on a depressed bipolar population, but with Provigil as the dopamine med of choice. The study was funded by the Stanley Medical Research Institute. Cephalon’s involvement was peripheral. The company provided the meds and some supplementary funding.

Eighty-five depressed patients already on bipolar meds (including some on antidepressants) were enrolled in the trial (way less than the 250-500 patients typical of industry-sponsored trials). After six weeks, 44 percent responded (ie depression scores reduced by at least half) vs 23 percent in the placebo group. Nothing to write home about, but clearly a finding that that actually tells us something and demonstrates the potential of RDPCs.

Imagine what we would be able to learn if we were to slice and dice depressed populations in far more sophisticated ways, and run full-scale trials on these various subgroups with both established and novel agents.

Alas, virtually no progress has been made since I stumbled upon Dr Goldberg and his poster way back in 2002.

Gold Standard Clinical Trials - Science or Marketing?: Part IV

This is the fifth in a series that takes a hard look at the information that we and our clinicians rely on to give us an insight into our condition and how to treat it. In the first piece, we reviewed three unscientific reader surveys I conducted here at Knowledge is Necessity, rightly at the very bottom of the evidence food chain but that actually told us something. The next three articles (Part I, Part II, Part III) looked at a “randomized, double blind, placebo-controlled” (RDPC) clinical trial that tested J&J's Invega for schizoaffective - regarded as the gold standard of research - but which actually told us nothing.

Moving on ...

In psychiatry, RDPCs imitate science rather than embody science, and therefore need to be regarded with a very high degree of skepticism. Two principles are in play:

1. We know next to nothing about the phenomenon we are investigating.
2. RDPCs are conducted for reasons of marketing.

We Know Next to Nothing

What is depression? Feeling sad? Loss of pleasure? Unmotivated? Psychic numbness? Reaction to stress? Loss of energy? Over-ruminating? Under-thinking? On and on, it goes.

In all likelihood, we are talking many - possibly hundreds - of illnesses involving by far the most complex organ in the body and all its related systems, with myriad possible causes and effects (including genetic vulnerability to environmental stressors) recruiting a host of different pathways. RDPCs do not recognize this. Rather they test “antidepressants” on a “depressed” population.

The equivalent would be to treat all infectious diseases as the same, call every such outbreak “runny nose disease,” and run a clinical trial of the same test med on everyone with a runny nose or high fever. Any reasonable bystander would rightly regard this as insane. Why, then, would anyone apply this approach to testing psychiatric meds? Read on ...

Marketing

Only pharmaceutical companies can afford the $40 million or so (my best guess) it takes to conduct the two clinical trials required for an FDA indication to treat an illness, but the pay-off is huge. For a common chronic illness, such as “depression,” an FDA indication amounts to a license to print money.

The drug companies are abetted by the FDA, which sets a very low success threshold for a clinical trial. In the case of antidepressants, assuming safety and tolerability standards are met, a mere 50 percent of those in the drug group need to get merely 50 percent better. (For antipsychotics to test schizophrenia, the standard is even lower.) You can achieve nearly as good a result with a placebo - literally. But why would a drug company care about that? Once they have succeeded in putting a tiny bit of daylight between their drug and the placebo in two trials, they now have their license to print money.

What RDPCs Do Tell Us

Of all things, despite the best efforts by drug companies to produce results favorable to their test med, the one thing clinical trials involving antidepressants do show us is their spectacular lack of effect on wide populations classified as “depressed.” Commentators such as Robert Whitaker have had a field day with this. Likewise Whitaker et al can point to the indisputable fact that these meds have failed to make a dent in the worldwide “depression” epidemic.

There is no way to “refute” or “repudiate” this evidence, except with equally weighty evidence, which does not exist. We can make a case that antidepressants are likely to work extremely well on a certain small subpopulation of patients, but to prove that we would first need to identify this population and then test them in a randomized, double-blind, placebo-controlled clinical trial.

Ironic, isn’t it?

Gold Standard Clinical Trials - Science or Marketing?: Part III

The story so far ...

At the 12th International Congress on Schizophrenia Research in April 2009, J&J debuted a “randomized, double blind, placebo-controlled” study which found its antipsychotic Invega produced a better result on a population of schizoaffective patients than did the placebo.

Researchers point to studies of this type as representing the gold standard of clinical research, but there is a major fallacy in their reasoning, and this study in particular represents a splendid example: Although the methodology of the study was flawless - even dazzling - the result showed us nothing. We already knew antipsychotics, including Invega, worked (with obvious limitations) for conditions involving psychosis. Psychiatrists already prescribe antipsychotics to patients with schizoaffective.

The result was meaningful to one party, and one party only: J&J (which owns Janssen, which manufactures Invega). Thanks to the success of this trial (and to the decidedly less convincing success of an earlier one), J&J had a bullet-proof brief to take to the FDA. In July, the company announced in a press release:

“The U.S. Food and Drug Administration (FDA) today approved the first and only [my emphasis] antipsychotic for the acute treatment of schizoaffective disorder.”

J&J had received clearance to market its Invega as a treatment for schizoaffective - which, since it was the only player in the field, translated to a golden opportunity to market an illness rather than simply a product. Thus, in its press release:

Among people who frequently use mental health services, schizoaffective disorder may account for approximately 24 percent of cases.

And again:

“Schizoaffective disorder can be challenging to diagnose because of the broad range of symptoms patients experience," said Nina Schooler PhD, SUNY Downstate Medical Center.

And again:

"Schizoaffective disorder is a chronic, disabling condition," said Husseini Manji MD, FRCPC, Global Therapeutic Area Head, Neuroscience, Johnson & Johnson Pharmaceutical Research & Development. "This new indication for Invega further demonstrates Janssen's commitment to helping people with serious mental illnesses."

In the article of the second study, published in the Oct, 2010 Journal of Clinical Psychiatry, we are told that schizoaffective was first identified in 1933, and the first study article in the April JCP we are informed that:

Although schizoaffective disorder occurs less commonly than schizophrenia, it may account for up to one-quarter of admissions to inpatient mental health facilities.

Back in the glory days of Pharma, J&J would have sponsored at least two dinner (or luncheon or breakfast) symposia at the American Psychiatric Association’s annual meeting. Anywhere from five hundred to a thousand psychiatrists would have crowded into a hotel ballroom to hear a panel of experts expound on “Schizoaffective Disorder, the Forgotten Illness” or “Treatment Modalities for Schizoaffective Disorder” or something similar. The psychiatrists (and other clinicians) would have received CME credits for their attendance.

From 2002 to 2009, I must have attended more than 60 symposia of this nature at APAs. Some of them were blatant advertorials, but most were highly educational and a large number of them were outstanding. I actually recall congratulating some Wyeth people for sponsoring a luncheon symposium the year before that had nothing to do with depression or Effexor. They looked at me funny, as if to say, “How did we screw that up?”

But in the eight consecutive APAs I attended, never once did I see in the program an industry-sponsored symposium devoted to schizoaffective disorder. If I had, I’m sure I would have penciled it in.

The APA has done away with industry-sponsored symposia at its annual meetings, but J&J would have had ample opportunity to get out the word in the exhibit hall. I did not attend last year’s APA, but I guarantee any clinician approaching their exhibit could have watched a multimedia display on schizoaffective, helped themselves to the literature, put their names on a contact list, and had their questions answered by a phalanx of J&J eager beavers. If the clinician expressed an interest in knowing more, one of the eager beavers would have escorted him or her to a back area to talk one-on-one with a J&J big cheese. 

Meanwhile, J&J would have been getting out the word in the poster area of the annual meeting. Perhaps attendees there would have viewed the same poster I did at the Schizophrenia Congress. In all likelihood they would have been exposed to many many more, of little significance, something like “Paliperidone ER Taken with Bottled Water vs Tap Water in a Nursing Home Mongolian Population with Schizoaffective Disorder.”

The multiple poster effect would have been wall-to-wall, marketing dressed up as research, with illness and product occupying an attendee’s entire field of vision. Working the posters would be an academic superstar with his or her minions, eager (okay, perhaps not eager) to answer questions.

In the meantime, away from the APA, J&J would be staging its own events with expert speakers, as well as launching its drug reps on special missions of search and sell. Moreover, they would be placing articles in publications. In a recent piece on his Carlat Psychiatry Blog, Daniel Carlat, author of “Unhinged,” had this to say about a feature in Current Psychiatry, which goes out free to psychiatrists:

Another promotional CME Supplement was published in October 2010, and is entitled  "Differential Diagnosis and Therapeutic Management of Schizoaffective Disorder." It is supported by Janssen, the maker of Invega, which was recently the first medication FDA approved for - you guessed it - schizoaffective disorder. I haven't read it yet, but this is a de facto advertisement simply by virtue of the choice of topic. Janssen makes the only approved product for schizoaffective disorder, and in order to advertise it, they paid off Current Psychiatry to write a huge article about the disorder. It doesn't need to be "biased;" it just needs to be focused on a topic that is of inherent commercial benefit to the supporter.

The article doesn’t tell us what we don’t already know about about schizoaffective, but it does hit on all the talking points in the J&J press release and related literature, including the fact that:

Only 2 randomized, double-blind, placebo-controlled studies of an atypical antipsychotic (paliperidone extended-release, now FDA-approved for the treatment of SAD), have been conducted in a well-defined SAD patient population.

Even a squeaky-clean expert in a forum uncorrupted by Pharma would be obliged to mention something to this effect. There is no way of avoiding it, and that is the beauty of J&J’s strategy.

But when all it said and done, psychiatrists will not be changing the way they treat patients with psychotic features, whether you call it schizoaffective or something else. They will still use an antipsychotic, just as they have been doing for the last fifty years. It’s just that a few more of them will be doing it with Invega.

Gold Standard Clinical Trials - Science or Marketing?: Part II

Moving right along ...

In my previous post, I gave an example of a gold-standard clinical trial that told us next to nothing. The study, which debuted as a poster at the 12th International Congress on Schizophrenia Research in April, 2009, found that J&J’s antipsychotic Invega worked better than a sugar pill in reducing PANSS scores in patients with schizoaffective disorder.

PANSS is a 30-point rating scale used to assess patients for schizophrenia. But there is a twist with patients with schizoaffective, as they also experience mood symptoms. Therefore, the study also measured for depression (using the HAM-D) and mania (using the Young Mania Scale). But clinical trials are only allowed to measure for one thing, so reduction in PANSS scores became the “primary outcome," also known as "primary endpoint.”

The way I heard this explained to me: Suppose a study measured for five different results. Suppose there was only a 20 percent chance of each result coming up positive. That would mean a 100 percent chance of one result coming up positive, which the study sponsor could trumpet as evidence for the success of the treatment. There is value in data from “secondary endpoints,” but you could use secondary endpoint logic to argue that the Steelers beat the Packers in the Super Bowl.

You have to select your primary outcome before the study. No picking the most desirable one when it’s all over.

Okay, now that we’ve agreed on the primary outcome, let’s round up our schizoaffective patients and get crackin’. Um, define schizoaffective. From the DSM-IV:

An uninterrupted period of illness during which, at some time, there is either a Major Depressive Episode, a Manic Episode, or a Mixed Episode concurrent with symptoms that meet Criterion A for Schizophrenia.

Clear as day, right? The DSM-5 work group responsible for coming up with something better actually said, “the current DSM-IV-TR diagnosis schizoaffective disorder is unreliable,” then did not come up with something better.

Common sense dictates that an unreliable diagnosis automatically calls into question the credibility of any study of this sort from the very outset, gold standard or not. I would extend this line of reasoning to include all antidepressant trials on "depressed" patients (what the hell is depression, anyway?), as well, but that's me mounting my hobby horse.

But the practical problem remains. Patients with schizoaffective are not going to exactly materialize for a clinical trial, especially if they’re being diagnosed with something else. To compensate, J&J took the highly unusual step of recruiting patients from more than 40 centers worldwide, including India, Russia, the Ukraine, and all across the US. This way, they were able to round up a total of 311 patients who met DSM-IV criteria.

Keep in mind, in any meds trial both the drug group and placebo group need to be as close to an exact match as possible. The “randomization” in “randomized double-blind placebo-controlled” trials means patients are assigned to different groups by chance rather than choice (thereby preventing abuses such as clinicians selecting good prognosis patients for the drug group). It’s not as simple as a coin toss, especially across more than 40 different centers worldwide. Give J&J credit where credit is due.

There was an additional complication to this trial: The patients were also on mood stabilizers and/or antidepressants. Still, antipsychotics have a very good trial track record in the PANSS challenge. Even accounting for all the special challenges of this particular trial, how hard can a glorified counting exercise be, right? Did someone say “discontinuation”?

It turned out about 40 percent of the patients dropped out of the study, about equal between the Invega and placebo groups. What this tells me is that when a doctor prescribes this med for this illness there is a four in ten percent chance of failure, even before the patient walks out the door. But this has never bothered the people putting together these “gold standard” studies.

To compensate, the J&J people employed a standard statistical fiction known as “intent-to-treat analysis,” which includes the drop-outs in the final tally, as if these patients had completed the study. One way of implementing this is with “last observation carried forward” (LOCF). Thus, if a patient drops at at week two of a six-week trial, his or her two-week result is “carried forward” to the final week.

There is a legitimate reason for doing this. Patients who drop out of drug trials are more likely to be bad responders. Thus, if only good responders stay in the study, the results are likely to overstate the benefits of the test drug.

To me, however, a high drop-out rate means that any positive finding is next to useless. But who listens to me? Anyway, now that we’ve jumped through all the various statistical hoops, we’re finally ready to get down to some serious counting. Oops! - did someone say placebo?

Placebos are the bane of all clinical trials, especially for psychiatric meds. In an earlier trial conducted by J&J, the placebo group fared almost as well as the Invega group. The second time out, those on the Invega did no better than before, but those on the placebo did a lot worse (perhaps due to a bad batch of placebos). Thus, the J&J investigators had “clear separation” from the placebo group.

Thus, instead of tearing their hair out, J&J had reason to celebrate. All their hard work had paid off. Between the two trials, they now had an airtight case to take to the FDA. Indeed, six months after the trial results came through in February, three months after J&J debuted its study as a poster in April, in July the company proudly announced in a press release:

“The U.S. Food and Drug Administration (FDA) today approved the first and only [my emphasis] antipsychotic for the acute treatment of schizoaffective disorder.”

Mission accomplished: J&J had received clearance to market its Invega as a treatment for schizoaffective.

To market, to market ... That was the real - and only relevant - endpoint to this gold-standard study. 

More to come ....

Gold Standard Clinical Trials - Science or Marketing?

This is the second in a new series on analyzing information. Credit Robert Whitaker’s “Anatomy of an Epidemic” for getting the ball rolling, as there is no way of critiquing his book without evaluating the evidence he relied upon. Only then can we ask ourselves if his conclusions are justifiable.

This is a different proposition entirely than whether Whitaker is right or wrong. You can decide that for yourself.

In my previous piece, I commented on some findings at the very bottom of the evidence food chain (a series of wholly unscientific reader polls I had conducted here), but that actually told us something. In this piece, I will flip it. I will talk about a study at the very top of the food chain, but that tells us nothing. Let’s get started ...

Two years ago, in my capacity as a journalist, I was at the 12th International Congress on Schizophrenia Research in San Diego, just down the hill from where I live. The top brain scientists in the world were presenting and in attendance, including 2000 Nobel Laureate Arvid Carlsson. I recall going to a table with my morning coffee and greeting the researchers there with, “Hi, I’m the only C student at this table.”

Part of the Congress involved poster sessions, where scientists stand in front of blow-ups of their latest findings. This affords an invaluable opportunity for the researchers at the conference to engage their fellow researchers in one-on-one discussion. It also affords a golden opportunity for people like me to walk up to some future Nobel Laureate and ask, “What’s a neuron?”

The first three or four days of posters involved cutting edge brain research. I was like a kid in a candy store ... a wrestler in a chair factory, a hippie in a drum circle. (Did you see that Super Bowl commercial?) The last day of the conference featured posters on treatment. Instantly, I felt I had been sucked back in time a half century.

Some very enthusiastic Johnson&Johnson people informed me of a new study of theirs that shed some very important new light on schizoaffective disorder.

Really? I thought. Schizoaffective is probably the most confusing diagnostic classification in the entire DSM. Everyone agrees that it has something to do with that undelineated middle ground where bipolar and schizophrenia bleed into one another, but what is it exactly? Schizophrenia lite? Bipolar heavy? Some kind of schizophrenia-bipolar mix? Or a stand-alone illness in its own right?

Naturally, I was interested. I looked at the poster. "A Randomized, Double Blind, Placebo-Controlled Study of Flexible Dose Paliperidone ER in the Treatment of Patients with Schizoaffective Disorder," I read.

I smiled politely, engaged in very short chit-chat, then made my way to other posters. In one glance, I knew, the study told me nothing. Here’s the deal:

Randomized, double blind, placebo-controlled studies are the gold standard of treatment research. Perhaps the most celebrated example of a clinical trial is James Lind’s 1747 experiment involving 12 scurvy-infected sailors. He divided his patients into six pairs. Five of the pairs showed no improvement. Of the pair receiving oranges and lemons, one had fully recovered after five days, the other had almost recovered. (Then they ran out of fruit.)

A more refined study would have pitted a much larger group of orange-and-lemon people against a proportionally large group taking a look-alike, taste-alike placebo (say a citrus pill vs a sugar pill). To keep the study honest, one would have made sure the clinicians handing out the pills had no way of knowing which pill was which (the double-blind).

I knew at a glance the J&J people had all their “i”s dotted and “t”s crossed. Studies of this nature are immensely expensive - my best guess is the $20 million-range - designed by some of the smartest people in the world overseeing an immense and highly complex undertaking where an infinity of things can go wrong (and often do).

The catch? Even though the study was conducted scientifically, to rigorous standards, it was not a scientific study. It was a marketing exercise.

Paliperidone is the generic name for J&J’s recently-approved Invega, which is Son of Risperdal. Risperdal represents the first of the new-generation atypical antipsychotics (discounting Clozaril) introduced in the early 1990s and hyped as superior to old-generation antipsychotics.

We already know that as problematic as antipsychotics may be, they are very effective in knocking out psychosis. The first studies involving Thorazine proved that six decades ago. Since then, antipsychotics have been the treatment of choice for schizophrenia and any condition involving psychosis, including bipolar and schizoaffective, accounting for some $20 billion annually in sales.

So what’s new? Nothing much, really, except that no drug company had ever undertaken a trial on a schizoaffective population. What was the point?

Keep in mind that when James Lind was investigating scurvy, the British Royal Navy was not issuing citrus fruit to its sailors. His findings would give them cause to change their practice, though it would take them another fifty years to get around to it.

Contrast this with the Invega study. Is a clinician, reading this study, really going to change his or her practice by putting all his schizoaffective patients on an antipsychotic? He already has them on an antipsychotic.

But could the study possibly be persuasive in getting a clinician to change her practice from prescribing say Geodon or Zyprexa to Invega? Ah, a marketing question.

More to come ...