Gold Standard Clinical Trials - Science or Marketing?: Part III

The story so far ...

At the 12th International Congress on Schizophrenia Research in April 2009, J&J debuted a “randomized, double blind, placebo-controlled” study which found its antipsychotic Invega produced a better result on a population of schizoaffective patients than did the placebo.

Researchers point to studies of this type as representing the gold standard of clinical research, but there is a major fallacy in their reasoning, and this study in particular represents a splendid example: Although the methodology of the study was flawless - even dazzling - the result showed us nothing. We already knew antipsychotics, including Invega, worked (with obvious limitations) for conditions involving psychosis. Psychiatrists already prescribe antipsychotics to patients with schizoaffective.

The result was meaningful to one party, and one party only: J&J (which owns Janssen, which manufactures Invega). Thanks to the success of this trial (and to the decidedly less convincing success of an earlier one), J&J had a bullet-proof brief to take to the FDA. In July, the company announced in a press release:

“The U.S. Food and Drug Administration (FDA) today approved the first and only [my emphasis] antipsychotic for the acute treatment of schizoaffective disorder.”

J&J had received clearance to market its Invega as a treatment for schizoaffective - which, since it was the only player in the field, translated to a golden opportunity to market an illness rather than simply a product. Thus, in its press release:

Among people who frequently use mental health services, schizoaffective disorder may account for approximately 24 percent of cases.

And again:

“Schizoaffective disorder can be challenging to diagnose because of the broad range of symptoms patients experience," said Nina Schooler PhD, SUNY Downstate Medical Center.

And again:

"Schizoaffective disorder is a chronic, disabling condition," said Husseini Manji MD, FRCPC, Global Therapeutic Area Head, Neuroscience, Johnson & Johnson Pharmaceutical Research & Development. "This new indication for Invega further demonstrates Janssen's commitment to helping people with serious mental illnesses."

In the article of the second study, published in the Oct, 2010 Journal of Clinical Psychiatry, we are told that schizoaffective was first identified in 1933, and the first study article in the April JCP we are informed that:

Although schizoaffective disorder occurs less commonly than schizophrenia, it may account for up to one-quarter of admissions to inpatient mental health facilities.

Back in the glory days of Pharma, J&J would have sponsored at least two dinner (or luncheon or breakfast) symposia at the American Psychiatric Association’s annual meeting. Anywhere from five hundred to a thousand psychiatrists would have crowded into a hotel ballroom to hear a panel of experts expound on “Schizoaffective Disorder, the Forgotten Illness” or “Treatment Modalities for Schizoaffective Disorder” or something similar. The psychiatrists (and other clinicians) would have received CME credits for their attendance.

From 2002 to 2009, I must have attended more than 60 symposia of this nature at APAs. Some of them were blatant advertorials, but most were highly educational and a large number of them were outstanding. I actually recall congratulating some Wyeth people for sponsoring a luncheon symposium the year before that had nothing to do with depression or Effexor. They looked at me funny, as if to say, “How did we screw that up?”

But in the eight consecutive APAs I attended, never once did I see in the program an industry-sponsored symposium devoted to schizoaffective disorder. If I had, I’m sure I would have penciled it in.

The APA has done away with industry-sponsored symposia at its annual meetings, but J&J would have had ample opportunity to get out the word in the exhibit hall. I did not attend last year’s APA, but I guarantee any clinician approaching their exhibit could have watched a multimedia display on schizoaffective, helped themselves to the literature, put their names on a contact list, and had their questions answered by a phalanx of J&J eager beavers. If the clinician expressed an interest in knowing more, one of the eager beavers would have escorted him or her to a back area to talk one-on-one with a J&J big cheese. 

Meanwhile, J&J would have been getting out the word in the poster area of the annual meeting. Perhaps attendees there would have viewed the same poster I did at the Schizophrenia Congress. In all likelihood they would have been exposed to many many more, of little significance, something like “Paliperidone ER Taken with Bottled Water vs Tap Water in a Nursing Home Mongolian Population with Schizoaffective Disorder.”

The multiple poster effect would have been wall-to-wall, marketing dressed up as research, with illness and product occupying an attendee’s entire field of vision. Working the posters would be an academic superstar with his or her minions, eager (okay, perhaps not eager) to answer questions.

In the meantime, away from the APA, J&J would be staging its own events with expert speakers, as well as launching its drug reps on special missions of search and sell. Moreover, they would be placing articles in publications. In a recent piece on his Carlat Psychiatry Blog, Daniel Carlat, author of “Unhinged,” had this to say about a feature in Current Psychiatry, which goes out free to psychiatrists:

Another promotional CME Supplement was published in October 2010, and is entitled  "Differential Diagnosis and Therapeutic Management of Schizoaffective Disorder." It is supported by Janssen, the maker of Invega, which was recently the first medication FDA approved for - you guessed it - schizoaffective disorder. I haven't read it yet, but this is a de facto advertisement simply by virtue of the choice of topic. Janssen makes the only approved product for schizoaffective disorder, and in order to advertise it, they paid off Current Psychiatry to write a huge article about the disorder. It doesn't need to be "biased;" it just needs to be focused on a topic that is of inherent commercial benefit to the supporter.

The article doesn’t tell us what we don’t already know about about schizoaffective, but it does hit on all the talking points in the J&J press release and related literature, including the fact that:

Only 2 randomized, double-blind, placebo-controlled studies of an atypical antipsychotic (paliperidone extended-release, now FDA-approved for the treatment of SAD), have been conducted in a well-defined SAD patient population.

Even a squeaky-clean expert in a forum uncorrupted by Pharma would be obliged to mention something to this effect. There is no way of avoiding it, and that is the beauty of J&J’s strategy.

But when all it said and done, psychiatrists will not be changing the way they treat patients with psychotic features, whether you call it schizoaffective or something else. They will still use an antipsychotic, just as they have been doing for the last fifty years. It’s just that a few more of them will be doing it with Invega.

Gold Standard Clinical Trials - Science or Marketing?: Part II

Moving right along ...

In my previous post, I gave an example of a gold-standard clinical trial that told us next to nothing. The study, which debuted as a poster at the 12th International Congress on Schizophrenia Research in April, 2009, found that J&J’s antipsychotic Invega worked better than a sugar pill in reducing PANSS scores in patients with schizoaffective disorder.

PANSS is a 30-point rating scale used to assess patients for schizophrenia. But there is a twist with patients with schizoaffective, as they also experience mood symptoms. Therefore, the study also measured for depression (using the HAM-D) and mania (using the Young Mania Scale). But clinical trials are only allowed to measure for one thing, so reduction in PANSS scores became the “primary outcome," also known as "primary endpoint.”

The way I heard this explained to me: Suppose a study measured for five different results. Suppose there was only a 20 percent chance of each result coming up positive. That would mean a 100 percent chance of one result coming up positive, which the study sponsor could trumpet as evidence for the success of the treatment. There is value in data from “secondary endpoints,” but you could use secondary endpoint logic to argue that the Steelers beat the Packers in the Super Bowl.

You have to select your primary outcome before the study. No picking the most desirable one when it’s all over.

Okay, now that we’ve agreed on the primary outcome, let’s round up our schizoaffective patients and get crackin’. Um, define schizoaffective. From the DSM-IV:

An uninterrupted period of illness during which, at some time, there is either a Major Depressive Episode, a Manic Episode, or a Mixed Episode concurrent with symptoms that meet Criterion A for Schizophrenia.

Clear as day, right? The DSM-5 work group responsible for coming up with something better actually said, “the current DSM-IV-TR diagnosis schizoaffective disorder is unreliable,” then did not come up with something better.

Common sense dictates that an unreliable diagnosis automatically calls into question the credibility of any study of this sort from the very outset, gold standard or not. I would extend this line of reasoning to include all antidepressant trials on "depressed" patients (what the hell is depression, anyway?), as well, but that's me mounting my hobby horse.

But the practical problem remains. Patients with schizoaffective are not going to exactly materialize for a clinical trial, especially if they’re being diagnosed with something else. To compensate, J&J took the highly unusual step of recruiting patients from more than 40 centers worldwide, including India, Russia, the Ukraine, and all across the US. This way, they were able to round up a total of 311 patients who met DSM-IV criteria.

Keep in mind, in any meds trial both the drug group and placebo group need to be as close to an exact match as possible. The “randomization” in “randomized double-blind placebo-controlled” trials means patients are assigned to different groups by chance rather than choice (thereby preventing abuses such as clinicians selecting good prognosis patients for the drug group). It’s not as simple as a coin toss, especially across more than 40 different centers worldwide. Give J&J credit where credit is due.

There was an additional complication to this trial: The patients were also on mood stabilizers and/or antidepressants. Still, antipsychotics have a very good trial track record in the PANSS challenge. Even accounting for all the special challenges of this particular trial, how hard can a glorified counting exercise be, right? Did someone say “discontinuation”?

It turned out about 40 percent of the patients dropped out of the study, about equal between the Invega and placebo groups. What this tells me is that when a doctor prescribes this med for this illness there is a four in ten percent chance of failure, even before the patient walks out the door. But this has never bothered the people putting together these “gold standard” studies.

To compensate, the J&J people employed a standard statistical fiction known as “intent-to-treat analysis,” which includes the drop-outs in the final tally, as if these patients had completed the study. One way of implementing this is with “last observation carried forward” (LOCF). Thus, if a patient drops at at week two of a six-week trial, his or her two-week result is “carried forward” to the final week.

There is a legitimate reason for doing this. Patients who drop out of drug trials are more likely to be bad responders. Thus, if only good responders stay in the study, the results are likely to overstate the benefits of the test drug.

To me, however, a high drop-out rate means that any positive finding is next to useless. But who listens to me? Anyway, now that we’ve jumped through all the various statistical hoops, we’re finally ready to get down to some serious counting. Oops! - did someone say placebo?

Placebos are the bane of all clinical trials, especially for psychiatric meds. In an earlier trial conducted by J&J, the placebo group fared almost as well as the Invega group. The second time out, those on the Invega did no better than before, but those on the placebo did a lot worse (perhaps due to a bad batch of placebos). Thus, the J&J investigators had “clear separation” from the placebo group.

Thus, instead of tearing their hair out, J&J had reason to celebrate. All their hard work had paid off. Between the two trials, they now had an airtight case to take to the FDA. Indeed, six months after the trial results came through in February, three months after J&J debuted its study as a poster in April, in July the company proudly announced in a press release:

“The U.S. Food and Drug Administration (FDA) today approved the first and only [my emphasis] antipsychotic for the acute treatment of schizoaffective disorder.”

Mission accomplished: J&J had received clearance to market its Invega as a treatment for schizoaffective.

To market, to market ... That was the real - and only relevant - endpoint to this gold-standard study. 

More to come ....

Gold Standard Clinical Trials - Science or Marketing?

This is the second in a new series on analyzing information. Credit Robert Whitaker’s “Anatomy of an Epidemic” for getting the ball rolling, as there is no way of critiquing his book without evaluating the evidence he relied upon. Only then can we ask ourselves if his conclusions are justifiable.

This is a different proposition entirely than whether Whitaker is right or wrong. You can decide that for yourself.

In my previous piece, I commented on some findings at the very bottom of the evidence food chain (a series of wholly unscientific reader polls I had conducted here), but that actually told us something. In this piece, I will flip it. I will talk about a study at the very top of the food chain, but that tells us nothing. Let’s get started ...

Two years ago, in my capacity as a journalist, I was at the 12th International Congress on Schizophrenia Research in San Diego, just down the hill from where I live. The top brain scientists in the world were presenting and in attendance, including 2000 Nobel Laureate Arvid Carlsson. I recall going to a table with my morning coffee and greeting the researchers there with, “Hi, I’m the only C student at this table.”

Part of the Congress involved poster sessions, where scientists stand in front of blow-ups of their latest findings. This affords an invaluable opportunity for the researchers at the conference to engage their fellow researchers in one-on-one discussion. It also affords a golden opportunity for people like me to walk up to some future Nobel Laureate and ask, “What’s a neuron?”

The first three or four days of posters involved cutting edge brain research. I was like a kid in a candy store ... a wrestler in a chair factory, a hippie in a drum circle. (Did you see that Super Bowl commercial?) The last day of the conference featured posters on treatment. Instantly, I felt I had been sucked back in time a half century.

Some very enthusiastic Johnson&Johnson people informed me of a new study of theirs that shed some very important new light on schizoaffective disorder.

Really? I thought. Schizoaffective is probably the most confusing diagnostic classification in the entire DSM. Everyone agrees that it has something to do with that undelineated middle ground where bipolar and schizophrenia bleed into one another, but what is it exactly? Schizophrenia lite? Bipolar heavy? Some kind of schizophrenia-bipolar mix? Or a stand-alone illness in its own right?

Naturally, I was interested. I looked at the poster. "A Randomized, Double Blind, Placebo-Controlled Study of Flexible Dose Paliperidone ER in the Treatment of Patients with Schizoaffective Disorder," I read.

I smiled politely, engaged in very short chit-chat, then made my way to other posters. In one glance, I knew, the study told me nothing. Here’s the deal:

Randomized, double blind, placebo-controlled studies are the gold standard of treatment research. Perhaps the most celebrated example of a clinical trial is James Lind’s 1747 experiment involving 12 scurvy-infected sailors. He divided his patients into six pairs. Five of the pairs showed no improvement. Of the pair receiving oranges and lemons, one had fully recovered after five days, the other had almost recovered. (Then they ran out of fruit.)

A more refined study would have pitted a much larger group of orange-and-lemon people against a proportionally large group taking a look-alike, taste-alike placebo (say a citrus pill vs a sugar pill). To keep the study honest, one would have made sure the clinicians handing out the pills had no way of knowing which pill was which (the double-blind).

I knew at a glance the J&J people had all their “i”s dotted and “t”s crossed. Studies of this nature are immensely expensive - my best guess is the $20 million-range - designed by some of the smartest people in the world overseeing an immense and highly complex undertaking where an infinity of things can go wrong (and often do).

The catch? Even though the study was conducted scientifically, to rigorous standards, it was not a scientific study. It was a marketing exercise.

Paliperidone is the generic name for J&J’s recently-approved Invega, which is Son of Risperdal. Risperdal represents the first of the new-generation atypical antipsychotics (discounting Clozaril) introduced in the early 1990s and hyped as superior to old-generation antipsychotics.

We already know that as problematic as antipsychotics may be, they are very effective in knocking out psychosis. The first studies involving Thorazine proved that six decades ago. Since then, antipsychotics have been the treatment of choice for schizophrenia and any condition involving psychosis, including bipolar and schizoaffective, accounting for some $20 billion annually in sales.

So what’s new? Nothing much, really, except that no drug company had ever undertaken a trial on a schizoaffective population. What was the point?

Keep in mind that when James Lind was investigating scurvy, the British Royal Navy was not issuing citrus fruit to its sailors. His findings would give them cause to change their practice, though it would take them another fifty years to get around to it.

Contrast this with the Invega study. Is a clinician, reading this study, really going to change his or her practice by putting all his schizoaffective patients on an antipsychotic? He already has them on an antipsychotic.

But could the study possibly be persuasive in getting a clinician to change her practice from prescribing say Geodon or Zyprexa to Invega? Ah, a marketing question.

More to come ...