Whitaker vs Torrey: Crunching Numbers

This is the fourth in our series of pieces dealing with Fuller Torrey’s response to Robert Whitaker’s 2010 “Anatomy of an Epidemic.” In his review, Dr Torrey asserts that on matters of schizophrenia and antipsychotic drugs, “Whitaker got it mostly wrong.” The same day that Torrey published his review, Whitaker in a blog post issued an angry rebuttal and his own counter-attack.

In our most recent installment, we discussed Whitaker’s heavy reliance on a 2007 publication of a Harrow-Jobe 15-year longitudinal study which found that about 40 percent of schizophrenia patients did well when taken off their antipsychotic medication. What the authors of the study found of greater significance, however, was the fact that the patients who did well were those with a “good prognosis” to begin with.

In his book, Whitaker made the barest passing reference to Harrow’s real findings, neither reporting on the primary results of the study nor how this information can be used to help patients with schizophrenia achieve better outcomes. Rather, Whitaker interpreted the study to mean that “that the drugs worsened long-term outcomes” (p 118).

Had this been a long-term trial of antipsychotics as Whitaker would have us believe, the authors would have conducted an entirely different study. There would have been at least two evenly-matched groups of patients, one on antipsychotic meds and one not on antipsychotic meds, if not at the very beginning of the study then at a key stage further along.

The catch is long-term studies of this type are impossible to conduct. The costs are prohibitive and no review board would dare permit such an enterprise. But the real world affords untold opportunities to observe the natural course of schizophrenia without meds. As Torrey scathingly notes:

[Whitaker] fails to focus any attention on the fact that on any given day in the United States half of all individuals with schizophrenia, or about one million people, are not being treated. This is a huge natural experiment to test his thesis. Many of these individuals are found in public shelters, sleeping under bridges, in jails, and in prisons. If Whitaker had spent more time in these settings observing the outcome of this natural experiment, instead of delivering lectures on his vision of the impending antipsychotic apocalypse, he would have written a very different book.

In this regard, Whitaker’s highly selective use and non-use of information becomes a major issue, akin to writing “Gone with the Wind” without noting that there is a Civil War going on.

So, is there any legitimacy to Whitaker interpreting the Harrow study his own way? Yes. It’s called “secondary analysis,” a fairly common practice amongst researchers and journalists. Basically, one mines other people’s data in search of new - and often startling - insights. And Whitaker certainly had more than enough data to work with.

The 2007 Harrow-Jobe article tells us that 12 of 64 patients with schizophrenia (19 percent) experienced a period of recovery over 15 years. Of these: Eight of 12 (40 percent) were no longer on any meds and two of five were on meds but no longer on antipsychotics.

In contrast, only two of 39 patients (5 percent) on antipsychotics experienced a period of recovery. Moreover, 19 of 23 patients (83 percent) with uniformly poor outcomes after 15 years were on antipsychotics. Tellingly, 64 percent of these patients had psychotic activity at the 15-year point vs only 28 percent not on antipsychotics.

Harrow’s numbers back up Whitaker, right? Wrong. Recall, the Harrow study involved an apples-to-oranges comparison - good prognosis patients vs bad prognosis patients. The good prognosis patients, identified at the beginning of the study, were the ones most likely to get better in the first place and therefore were in a position to go off their meds.

For Whitaker’s secondary analysis to work, he would have had to show that the “good prognosis” patients who stayed on antipsychotics did worse than the good prognosis patients who went off antipsychotics. (It would have been useful to compare results in the bad prognosis group, as well, but this finding wouldn’t have had the same significance, as we don’t have high expectations for this group.)

So - we’re looking for an apples-to-apples comparison. Easy to show, right? We just pull up the relevant number and ...

No number. Whitaker doesn’t cite one. No problem. We’ll find the number in the 2007 Harrow-Jobe study that Whitaker refers to. Wait, this is weird. The number isn’t in the study, either. The authors slice and dice the data in a multiplicity of ways, but the closest they come to what we’re looking for is a finding that 17 percent of the good prognosis patients were on antipsychotics after 4.5 years and 13 percent after 15 years.

No mention of how these particular patients actually fared. Why? The answer is simple (okay, complicated for me). Let’s assume one-third of the patients in the study were good prognosis patients. Let’s make the number 20. If only 13 percent of these patients were on antipsychotics at the 15-year mark, we are looking at a study sample of two, at most three, patients.

A three-patient sample? Okay, let’s be generous and double it. A six-patient sample?

In his blog, Whitaker insists that “in every subgroup of patients (by prognostic type), those off medication had better long-term outcomes (in the aggregate).” In making his claim, Whitaker relies upon this paragraph from Harrow-Jobe:

In addition, global outcome for the group of patients with schizophrenia who were on antipsychotics was compared with that for the off-medication schizophrenia patients with similar prognostic status. Starting with the 4.5-year follow-ups and extending to the 15-year follow-ups the off medication subgroup tended to show better global outcomes at each follow-up.

Had Whitaker actually asked, “how much? how many?” he would have realized the absurdity of his assertion.

Conclusion: Whitaker’s secondary analysis fails. Totally, absolutely, completely. Case closed, right? Not so fast:

When the dust settles, we still have two key pieces of data that simply will not go away: Eight in 12 patients not on meds (plus two of five on meds but no antipsychotics) experiencing periods of recovery vs only two of 39 on antipsychotics.

Still a very small sample size. Still apples-to-oranges. Still other factors in play such as the type of life experience and personal make-up that separates good prognosis from bad prognosis patients.

But when all is said and done, Whitaker is perfectly justified in saying: “Hey, hold on a minute, have a look at these figures.” And we need to be listening.

Likewise, Torrey is perfectly correct in insisting that Whitaker got it wrong. Again, we need to be listening.

More to come ...

Previous Whitaker vs Torrey pieces:

At Last, a Conversation

Collision Course

Digging Deeper

Whitaker vs Torrey: Digging Deeper

This is the third in our series of pieces dealing with Fuller Torrey’s response to Robert Whitaker’s 2010 “Anatomy of an Epidemic.” In his review, Dr Torrey asserts that on matters of schizophrenia and antipsychotic drugs, “Whitaker got it mostly wrong.” The same day that Torrey published his review, Whitaker in a blog post issued an angry rebuttal and his own counter-attack.

Previously, we examined Whitaker’s use of two studies he cited in support of his thesis that mental illness is on the rise because of psychiatric medications rather than despite them.

In the first instance, Torrey busted Whitaker for blatantly misrepresenting one study’s findings. The study involved a mere passing reference, but was illustrative of the highly selective cherry-picking that Whitaker employs throughout his book. Whitaker counter-attacked, accusing Torrey of “dishonesty,” but failed to answer the charge.

In the second instance, Torrey dismissed two WHO studies that Whitaker heavily relied upon, characterizing them as “discredited.” Essentially, Torrey (citing various researchers) interprets these studies as an apples-to-oranges comparison. The studies (which found better outcomes among the third-world patients surveyed) - serve as a textbook example of why no finding can be taken at face value. Basically, both Torrey and Whitaker were justified in interpreting these studies the way they did. It would have been helpful, however, had Whitaker addressed Torrey’s apples-to-oranges issue. Instead, he issued another gratuitous “dishonesty” charge.

Let’s move on ...

The Harrow-Jobe Study

Whitaker’s Position

This 15 (later 20)-year longitudinal study, published in 2007, served as Exhibit A in Whitaker’s case against antipsychotic medications. In his book, Whitaker interprets the study results to advance his claim that patients with schizophrenia fare worse on meds over the long-term than those not on meds. According to Whitaker (p 115):

...The off-med group began to improve significantly, and by the end of 4.5 years, 39 percent were “in recovery” and more than 60 percent were working. In contrast, outcomes for the medication group worsened during this thirty-month period. ...

Whitaker in his book did note that Harrow in an interview attributed better outcomes to “stronger sense of self” and “better personhood,” but Whitaker did not elaborate why these factors were significant. Instead, he kept his focus exclusively on meds vs non-meds.

Later in his book (p 311), Whitaker accuses both the NIMH and NAMI of failing to promote (and by implication suppressing) the study’s optimistic finding:

... I also searched the NMIH and NAMI websites for some mention of the studies listed above and I found zilch. ... Forty percent of those off medications recovered over the long term! But that finding directly contradicted the message that NAMI has promoted to the public for decades ...

Torrey’s Response

In his review, Dr Torrey points out that the Harrow-Jobe study is not exactly the hot news that Whitaker makes it out to be. Since at least as far back as 1938, Torrey reports, numerous schizophrenia outcome studies have found that “on average one-quarter of the patients recovered completely, one-quarter had a continuous illness, and the other half had intermediate outcomes between these two extremes.”

Of critical importance, citing one researcher: “There are relatively benign and malignant forms of illnesses generally diagnosed as schizophrenia.” This leads to what the study was really all about, with Torrey noting that “Harrow et al. even explicitly state that their study provides no evidence on whether very long-term use of antipsychotic medication produces undesirable effects for some SZ [individuals with schizophrenia].”

Once again, we have an apples-to-oranges issue. As I observed in a number of previous blog posts, rather than comparing a meds group to a non-meds group as we are used to seeing in clinical trials, the study actually compared a “good prognosis” group to a “bad prognosis” group. These patients were identified at the beginning of the study, then followed over 15, then 20 years.

Predictably, the good prognosis group had better results, with more of them in a position to go off their antipsychotics and to function well. But, according to Torrey, Whitaker twisted these findings to serve his own purposes:

Using tortured logic, he asserts that the Harrow et al. study proves that long-term antipsychotic use causes brain damage and is responsible for many of the symptoms of schizophrenia, when in fact the study does nothing of the kind.

Whitaker’s Comeback

In his blog post, Whitaker breaks down those parts of the Harrow-Jobe study that showed how all the subgroups of non-medicated patients fared better without antipsychotic medications. This includes both “good prognosis” patients and “bad prognosis” patients. Says Whitaker:

Although [Harrow] didn’t provide the global data for these two subtypes, he did report this finding: “In addition, global outcome for the group of patients with schizophrenia who were on antipsychotics were compared with the off-medication schizophrenia patients with similar prognostic status. Starting with the 4.5-year followup and extending to the 15-year follow-up, the off-medication subgroup tended to show better global outcomes at each follow-up.”


In other words, in every subgroup of patients (by prognostic type), those off medication had better long-term outcomes (in the aggregate).

Whitaker (predictably) assigns another “dishonesty moment” to Torrey. This is based on the fact that Whitaker did not cite the Harrow-Jobe study for the proposition “that long-term antipsychotic use causes brain damage and is responsible for many of the symptoms of schizophrenia,” as Torrey claimed. Rather, Whitaker kept his interpretation to outcomes.

Nevertheless, Whitaker in his book (p 118-119,) linked the Harrow-Jobe study to a study that did suggest that “drugs made patients more vulnerable to psychosis over the long-term,” and (p 120) he noted that “we can also see how this drug-induced chronicity has  contributed to the rise in the number of disabled mentally ill."

What Harrow et al Actually Say

From a 2005 article (free on PubMed):

More recent data of ours suggest that some of the schizophrenia patients who go off antipsychotics are a different type of patient. They have better premorbid developmental achievements, have more favorable prognostic characteristics, and are more resilient and less vulnerable to psychopathology (or “healthier”), leading to their better functioning.

And from the 2007 article Whitaker refers to (you have to pay $50 for this, the abstract is free):

The results suggest that the subgroup of schizophrenia patients not on medications was different in terms of being a self-selected group having better earlier prognostic and developmental potential.

And the key finding from the wider dataset (including patients with other diagnoses), in a 2009 article (available free on PubMed):

A more external locus of control is related to fewer periods of recovery, to both depressed mood and psychosis, and to various aspects of personality.

Locus of control (LOC) “refers to the extent to which an individual perceives events in his or her life as being a consequence of his or her actions, and thus under his or her perceived control.” Internal LOC is good. External LOC is bad.

Of the schizophrenia patients in recovery over 15 years, according to the 2009 article, 67 percent had internal LOC. In the overall sample, 75 percent had internal LOC.

Basically, Harrow et al are telling us that an individual’s personal make-up is a key predictor to recovery, perhaps THE key. In their study, these were the patients more likely to go off their meds and do well off their meds. In the authors’ own words (2007):

Patients who are internally orientated and have better self-esteem are the types of patients who are more likely, if their functioning improves, to urge that they try functioning without medications ...

This is worth restating: According to Harrow et al, based on their study findings, doctors should be encouraged to consider taking a good prognosis patient who is doing well off his or her meds. This is an entirely different proposition than what Whitaker would have us believe - namely, get EVERYONE off their meds as soon as possible.

Whitaker doesn’t say this in so many words, but the thrust of his interpretation of the study  - namely “that the drugs worsened long-term outcomes” (p 118) - leaves himself open to Torrey’s charge of using “tortured logic.”

What Whitaker is doing is making a “secondary analysis,” a common and legitimate practice that can often yield far more interesting insights than the primary analysis. The catch is that you have to read Whitaker’s book microscopically to know he is making this of kind analysis. He certainly does not trumpet Harrow et al’s real findings, much less explain them, and this is disturbing, to say the least.

This brings us to the crucial question, does Whitaker’s secondary analysis hold up?

To be continued ... 

Collision Course: Whitaker vs Torrey

This is the second in a series of pieces dealing with Fuller Torrey’s response to Robert Whitaker’s 2010 “Anatomy of an Epidemic.” In his review, Dr Torrey asserts that on matters of schizophrenia and antipsychotic drugs, “Whitaker got it mostly wrong.”

My first piece laid out the background to the controversy, namely my view (spread across numerous pieces throughout 2010 and 2011) that Whitaker had not made his case that psychiatric meds have caused a mental illness epidemic. Nevertheless, he made a very strong “case to answer,” one that demands a considered point-by-point response, preferably from a leading psychiatrist. More then two years went by since the publication of “Anatomy,” with no signs of intelligent life from psychiatry. Last week, Torrey broke the silence. Better late than never.

The same day that Torrey published his review, Whitaker in a blog post issued an angry rebuttal that cited Torrey for at least four instances of “dishonesty,” as well as using the occasion to attack Torrey and the Treatment Advocacy Center for its aggressive stance on assisted outpatient treatment (which a good many of us - myself included - also have serious issues with).

Lost in the noise was that on key points Whitaker has failed to respond to Torrey’s criticisms. Let’s get started:

The 1994 Outcome Study

Whitaker’s Position:

In the foreword to his book, Whitaker says that he “encountered two research findings that didn’t make sense.” One of these was a 1994 study conducted by researchers at Harvard, which found that “outcomes for schizophrenia patients in the United States had worsened during the past two decades ...”

He says no more about the study and makes no further mention of it in the rest of his book.

Torrey’s Response:

In his review, Dr Torrey notes that what the study actually said was “quite different,” namely that when a broad definition of schizophrenia was in vogue, outcomes were a lot better. Moreover, “the data showed a clear improvement in outcomes during the 1960s and 1970s following the introduction of antipsychotic drugs.” Outcomes worsened during the 1980s and 1990s, “which the authors attributed to the introduction of a narrow definition of schizophrenia.”

The broad and narrow definitions are no mere diagnostic quibbling. The narrow (DSM) version (introduced in 1980) mandates six months of symptoms.

Torrey also noted that Whitaker “later added that the worsened outcomes were due to the use of antipsychotic drugs.”

Whitaker’s Comeback:

In his blog post, Whitaker claims that he only mentioned the study in passing in the foreword to his book, as something that “piqued my curiosity,” thus implying that his use of the study was not worthy of Torrey’s attention. Surprisingly, in his defense, Whitaker acknowledges that the study authors said exactly what Torrey said they said, namely:

... the researchers reasoned that improved outcomes in the middle part of the century were due to both a change in diagnostic criteria that broadened the definition to include patients who were less ill at disease onset and then to the introduction of neuroleptics.

Then Whitaker served up his own theory of why maybe - sort of - the study supports his thesis anyway. (We won’t get into that here.)

Torrey’s big mistake? Whitaker made no specific reference to this study when he talked about worsened outcomes on antipsychotic medications, as Torrey stated in his review. This brought down Whitaker’s wrath in the form of “dishonesty moment number one" for Torrey.

Actually, though, on page 118 of his book, Whitaker says, “We have followed the trail of documents to a surprising end ...” Why wouldn’t we assume the Harvard study was part of that paper trail?

Verdict:

Dishonesty moment to Whitaker, big time. This is an egregious example of a journalist misciting a study to serve his own ends, then conveniently forgetting about it when the actual facts failed to support his thesis. This is hardly the only example of Whitaker’s highly selective cherry-picking in his book. Torrey was perfectly correct to call out Whitaker.

As for Torrey’s “dishonesty moment,” using a flimsy pretext to brand a critic as dishonest violates all the basic principles of playing well with others.

Finally, Whitaker totally failed to address Torrey’s extremely relevant point concerning broad and narrow diagnostic criteria. More about that, coming right up ...

The WHO Outcome Study

Whitaker’s Position:

Whitaker devotes considerable attention in his book to two World Health Organization studies that found that those with schizophrenia in developing countries had much better outcomes than those in developed nations. As Whitaker reported in his book (p 111):

... the bottom line is clear: In countries where patients hadn’t been regularly maintained on antipsychotics earlier in their illness, the majority had recovered and were doing well fifteen years later.

Whitaker then goes on (p 119) to link this study to other studies to conclude that “evidence for long-term recovery rates are higher for nonmedicated patients appears in studies and investigations of many different types.”

Torrey’s Response:

Dr Torrey in his review evinces far less enthusiasm, noting that the WHO study claim “has continued to be criticized over the years and has now been largely discredited.” At issue, once again, is broad and narrow diagnostic criteria. Citing various sources, Torrey notes that many of those enrolled in the third world centers probably did not have true schizophrenia (some of the patients were referred by religious and traditional healers). More likely, the researchers were dealing with a good number of those suffering from “acute reactive psychosis,” which have much better outcomes than schizophrenia.

Torrey also cites a five-year 2011 study on a cohort of Ethiopian patients with findings that contradict the WHO studies. Finally:

Faced with such criticisms, the authors of the WHO studies have recently modified their claims, stating that “we do not argue that the prognosis of schizophrenia in developing countries is groupwise uniformly milder” and acknowledging that “the proportions of continuous unremitting illness…did not differ significantly across the two types [developed and developing] of setting.”

Whitaker’s Comeback:

Predictably, Whitaker assigns Torrey another “dishonesty moment.” This is based on the fact that far from “modifying their claims,” the authors of the WHO studies in the same paper Torrey cited actually vigorously defended their findings. Says Whitaker:

Dr. Torrey, in his review, was intent on discrediting the findings from this WHO study, which reported superior outcomes in poor countries where only a small percentage of patients were regularly maintained on antipsychotics. To do so, he implied that the WHO investigators now agreed with the critics of the study, when that is not true.

Verdict:

We have a big wet loogie on the table, which Whitaker fails to address, namely: The authors of the WHO studies have explicitly acknowledged that the patients in the third-world countries had a milder prognosis than those in the developed countries. Why is this important? If we are comparing apples to oranges, then the findings of the WHO study are totally meaningless.

More likely, the “apples to oranges” controversy merely raises questions about the study rather than discredits it, as Torrey maintains. If anything, the WHO study is a textbook example of why no finding can be taken at face value. Certainly, we all know this when it comes to clinical trials sponsored by drug companies.

What is particularly disturbing is that Whitaker would have been aware of the “apples to oranges” controversy when he wrote "Anatomy of an Epidemic." Yet he makes only a fleeting reference to it in his book, and only in the context of vindicating the first WHO study (as if there were no reason to question the second study). A straightforward and thorough stating of the controversy would hardly have undermined both studies' findings or Whitaker’s thesis. If anything, preemptively dealing with this concern would have greatly strengthened Whitaker’s argument, along with his credibility.

Instead, we are left with the feeling that Whitaker is hiding something.

Keep in Mind ...

In past blog posts, I have been supportive of Whitaker, but I have also not hesitated to point out numerous examples of where he played fast and loose with the facts, or where - quite frankly - he failed to turn in his homework. I continue to be supportive of Whitaker, but I also support any critic of Whitaker acting in good faith. In my 13 years researching and writing on mental illness, one vital lesson stands out loud and clear: Never - never-ever-ever - take anyone (and I include myself here) at face value. Always maintain a healthy skepticism, even if the party involved claims to be speaking for you - especially if the party involved claims to be speaking for you.

To act otherwise is to place your life on the line. Our illness takes no prisoners.

Much more to come ...  

Torrey Responds to Whitaker: At Last, a Conversation

I have devoted considerable space on this blog to Robert Whitaker’s 2010 “Anatomy of a of an Epidemic,” which posits that mental illness is on the rise because of psychiatric medications, rather than in spite of them. In reviewing Whitaker, I read the same studies he cited in his book and came to the conclusion that Whitaker had not made his case.

Nevertheless, I pointed out that Whitaker had made a very strong “case to answer.” In other words, until someone (presumably a psychiatrist with weighty credentials) made a convincing counter-argument (preferably in a point-by-by rebuttal), Whitaker’s thesis - whatever one’s misgivings - stood as the authority.

I also stated that Whitaker had initiated a conversation that we badly need to have. Whitaker was also very clear that he wanted to have this kind of discussion.  

To my dismay - and to the shame of psychiatry - that conversation never eventuated. Daniel Carlat of Tufts University in two blogs issued what was essentially a collegial light dusting, taking issue with Whitaker’s presentation of the evidence in a friendly sort of way, but hardly knocking any holes in his main arguments.

Andrew Nierenberg of Harvard purported to “rebut” and “refute” Anatomy of an Epidemic in response to a grand rounds Whitaker delivered at Mass General. The so-called rebuttal amounted to an irrational and high-volume hissy fit (one punctuated by totally unprofessional ad hominem attacks) that I could only characterize as “sick, very sick.”

A few commentators quibbled about Whitaker’s interpretation of the term, “Epidemic,” tossing in an ad hominem attack or two for good measure, but otherwise avoiding engagement.

That all changed last week with an article posted on the website of the Treatment Advocacy Center. Anatomy of a Non-Epidemic - A Review by DrTorrey, read the heading. “How Whitaker Got it Wrong,” read the subheading.

E Fuller Torrey (pictured above) has a way of getting a rise out of certain mental health advocates. Dr Torrey is the founder of the Treatment Advocacy Center (TAC), which pushes for aggressive outpatient treatment laws for those with severe mental illness. The issue is a hot-button one, and TAC and Torrey have come under considerable criticism for their position and their tactics (including from this writer).

But Torrey has paid Whitaker the ultimate compliment of intelligently and thoughtfully responding to Whitaker. Too often, in our focus on personalities, we lose sight of the issues. Advocates who should know better have elevated Whitaker to the status of cult hero who can do no wrong. This is a grave disservice to both Whitaker and the people we purport to serve.

Torrey, too, enjoys a certain cult following, particularly among first-generation NAMI parents, as well as villain status from a host of mental health advocates. We will discuss these matters in a future blog. But, for right now, let’s focus on the issues. Essentially, Torrey has shifted the whole discussion. He has convincingly answered Whitaker’s “case to answer.” This hardly means that Torrey is right and Whitaker is wrong. Indeed, a constructive synthesis would move the discussion to a new level, one that Whitaker and Torrey could easily agree upon - the need for some serious research.

In other words, if the scientific evidence is insufficient to either support Whitaker’s case or to rebut it, then let’s put some serious money into unearthing the evidence.

In future blog posts, we will explore point-by-point Dr Torrey’s responses to Whitaker. In the meantime, this disclosure: Dr Torrey wrote a very glowing back-cover blurb for my 2006 book, “Living Well with Depression and Bipolar Disorder.”  The blurb states: “Very helpful for those affected by bipolar disorder and their families ... I recommend this book enthusiastically.”

I have had no other involvement with Dr Torrey and none with the Treatment Advocacy Center.

Stay tuned ... 

I Get Cartoonified

This is my story and I'm sticking to it: During the month of May, humorist Chato Stewart does a mental health heroes cartoon-a-thon, honoring those who go to bat for us. Each day in May, he publishes a new caricature. Last year, I was to be included, but then got excluded. As Chato explained to me, 2011 was a leap year, which meant May only had 27 days.

Made sense to me.

Anyway, Chato said, just send me a photo, and I will be sure to include you this year. Not only that, I will do to you what da Vinci did for that Mona lady. So I sent Chato my best mug shot. Here I am:

And here is what Chato drew:

Eeek!

Many thanks, Chato. Check out Chato's Cartoon-a-thon.

Raccoons Respect My Piss: The Promo Video

Living with Someone Who Has Bipolar; Living With Someone Who Has Normal