Generalizability of patient samples: a more important indicator of credibility of evidence than is quality of clinical trial design
That the samples of published clinical trials of depressed youth may not represent populations of children and adolescents presenting for treatment of depression is corroborated by findings reported last year by researchers at the National Institutes of Health and elsewhere (C. Blanco and colleagues, Pediatrics 140(6); 2016-2017, 2017).
The authors calculated “rejection rates” for enrollment in standard clinical pharmacotherapy and psychotherapy trials among 592 adolescents who had been diagnosed with major depressive disorder pursuant to the rigorous diagnostic criteria used for admission to the NIMH-funded National Comorbidity Study.
The authors found that 61.9 percent of their strictly diagnosed adolescents would have been excluded from a typical antidepressant pharmacotherapy trial (while 42.2 percent would have been excluded from a typical psychotherapy trial). Among the 412 strictly diagnosed patients among them who presented for treatment of depression, 72.7 percent would have been excluded by the standard selection criteria for a clinical trial (compared with 52.2 percent who would have been excluded from a standard psychotherapy trial).
The authors’ findings highlight a critically unappreciated distinction between quality and credibility of clinical trials that constitute medical evidence–one based on generalizability of patient samples. (Quality of studies refers to protections “built in” to their designs to exclude artifacts that make chance differences appear to indicate clinically significant superiority of candidate treatments over placebo. By itself, quality cannot assure that those treatments will prove superior in patients seeking treatment if those patients differ from patients in studies of high quality.
Studies of high quality typically feature random patient selection and assignment to treatment groups, inert or active control groups, double-blindedness of treatment administered and assessment of response, and disclosure of potential conflict of interest. Studies of highest quality also are vetted for publication bias (witholding negative results of similar studies), commercial bias, and full disclosure at the time of publication with respect to participation of the authors in execution and preparation of studies, use of data-appropriate and uniform statistical analyses, and documentation of enough “raw data” to justify exclusion of “outliers” critical to the significance of differences detected.).
Studies of high quality are not necessarily more credible than studies of lesser quality, with their fewer protections against adventitious influences on observation or analysis, when the latter are representative of naturalistic patient populations and the former are not.
In their discussion, Blanco and colleagues (op. cit.) confide that more than seven of ten adolescent patients presenting to naturalistic settings for treatment of major depressive episodes are likely to have been excluded from published clinical trials (typically, because of symptomatic severity and suicide risk), and that their response profiles and predictors of outcome in practice may therefore remain uninformative. Quality represents a dimension of validity–that differences between candidate treatment and placebo (or active comparator) in the sample studied represent the same differences in the “true” population from which it was extracted). The latter, in the study by Blanco and colleagues, appears to be the subgroup of the general population of depressed adolescents whose symptoms of depression are relatively mild and selected for lack of significant suicide risk. Treatments found effective in the latter are unlikely to resemble those of a preponderance of adolescents who present for treatment in naturalistic settings.
Accordingly, the current standard for credibility of evidence, quality of evidence is, in the event, not a credibility index unless patient samples of evidence represent persons for whom evidence is intended. The authors’ study suggests that published reports and studies of high quality currently may be credible with respect to fewer than three persons in ten who present for treatment.
Prescribing preferences associated with areas of specialization
Another of the contributions to variability within evidence are prescribers’ areas of specialization. A study reported two decades ago, by University of Michigan researchers (J.L. Rushton and colleagues, Pediatrics 105(6): E82, 2000) was among the first to report that different specialties wrote different proportions of psychotropic drug prescriptions for children and adolescents: More primary care physicians than pediatricians were prescribing SSRIs (for depressive syndromes) and were also prescribing SSRIs more frequently for off-label indications than pediatricians were prescribing them for depression. Moreover, more primary care physicians (54 percent) had been coadministering psychotropic medications with at least one other medication than had pediatricians (31 percent). Yet, only eight percent of the primary care physicians reported that they had received “adequate training” in the treatment of depression in youth, and only 16 percent of the pediatricians felt “comfortable” prescribing pharmacotherapy for depression.
Inverse relationship between community prevalence of psychiatrists and extent of prescribing for psychiatric disorders by primary care physicians
Researchers at the University of Pennsylvania’s Center for Pediatric Clinical Effectiveness (S.L. Mayne and colleagues, Pediatrics 137(5) May 2016, published online April 1, 2016) broach an inverse relationship in the extents of prescribing psychotropic medications between numbers of psychiatrists and numbers of primary care physicians in the same locales.
Having surveyed a population of 294,748 children and adolescents (four- to 18-years-old) from 43 primary care practices in the U.S., the authors identified 40,932 (15 percent) who had received a psychiatric diagnosis, and, it is assumed but not explicit, a subgroup of 39,695 (14 percent) for whom a psychiatric medication had been prescribed.
A prescription for a psychiatric indication had been prescribed in four to 26 percent of the patients in the practices studied, with one to 12 percent of them having been prescribed regimens comprising two or more psychiatric medications.
The most frequent psychiatric diagnosis conferred upon the children and adolescents of the practices was attention deficit hyperactivity disorder (one to 16 percent, of whom three to 18 percent received stimulants, coadministered in zero to eight percent with an alpha-1 agonist, such as clonidine or guanfacine).
Among children and adolescents with other psychiatric diagnoses, antidepressants were prescribed for one to 12 percent, while atypical neuroleptics were prescribed for zero to 5 percent.
The percentages of patients receiving psychotropic medications in the primary care practices was negatively correlated with availabilities of psychiatrists within the same practice areas. No significant association was observed between percentages of patients receiving prescriptions of psychotropic medications and numbers of non-physician mental health professionals (psychotherapists, social workers, and psychologists) in referral areas absent of psychiatrists. (Percentages of children and adolescents in foster care in the referral areas likewise was unassociated with percentages receiving psychotropic drug prescriptions.)
Interpretation herein of the inverse relationship between numbers of psychiatrists available to see patients and primary care physicians’ prescribing rates may be as unmysterious as Occam’s Razor suggests: when psychiatrists are available for referral, primary care doctors refer their patients to them for pharmacologic treatment of psychiatric disorders.
What may be the cost of reducing antipsychotic prescribing at any cost?
Another study of how the availability of specialty psychiatric services may affect prescription drug utilization (without necessarily changing prescribing practices) in youth with psychiatric disorders, by University of Washington (Seattle) researchers (R.P. Barclay and colleagues, Health Services Research 52(2):561-578, 2017) proceeds from an assumption (as a preponderance of studies pertaining to Medicaid-supported treatment of youth with second-generation, atypical neuroleptics do) that reduced rates of prescribing the latter are, wherever and whenever observed, tantamount to improvement in the quality of care. Factually inaccurate and variably proxied in their assessments of response and outcome, studies cited to corroborate that claim have proved useful as exemplars of cognitive bias variably referenced as “reverse-halo effect,” “negative affect bias,” and (the long-documented, local fallacy) “begging the question.”
Driven by unprecedented cost of the second-generation, atypical neuroleptics, and seemingly experiencing the latter as a threat to financing mental health care generally, states have organized utilization review based on guidelines for prescribing atypical neuroleptics in children, adolescents, adults and geriatric adults. In so doing, individual states contract with physicians affiliated with academic institutions, and thereby conduce to an appearance of scholarly familiarity of evidence intended to support treatment recommendations. Treatment recommendations, however, have so far garnered little consensus, and their authors have been appropriately humble in their admission that rigid enforcement of them (and peremptory rejection of alternate pharmacotherapy or polytherapy) are unjustified for substantial subgroups of the general population (particularly those proving insufficiently responsive to the first-line treatments they recommend).
Ostensibly driven by concern about metabolic adversity and cardiac dysrhythmia, the bestriding and actual motivator evoking review of utilization–the reversed halo and negative affect of the cognitive biases correspondingly designated–has been cost (and whether the latter has been offset by reduced need for interventions for adverse effects and/or lesser rates of or longer latencies to relapse or recurrence of remitted illness). So far, it has not, even as a finding that failure to authorize payment for atypical neuroleptics exacts a cost, having been associated replicably with interruption of or dropping out of treatment (neither responded to nor further studied by agents of Medicaid disbursement).
Seemingly oblivious of confirmatory bias in assuming that what was to be demonstrated is true and then using it to prove that it is, the University of Washington researchers report the latter from analysis of antipsychotic utilization data from Medicaid pharmacy records from July 1, 2006, through December 31, 2013. During that span, the state child psychiatry consultation program implemented “elective consults” with prescribers targeted by virtue of exceeding the state’s (arbitrary) allowed dosages for age or simultaneously prescribing more than one antipsychotic per patient. (Prescribers’ rationales for neither are described, and the authors offer no information about whether potentially extenuating conditions were adduced by prescribers intending to balance risks of adverse effects with those of insufficiently controlled symptoms.)
The authors’ elective consults were associated with an overall reduction of 0.26 percent in antipsychotic prescribing over their 90-month observation period, during which the incidence of antipsychotic use fell by 0.022 per thousand per month after introduction of their consults, and by 0.065 and 0.022 per thousand per month after “mandatory reviews” were made triggerable by exceeding a dosage guideline for age and by prescribing more than one antipsychotic drug, respectively (contemporaneous outcome data unspecified).
(ObiterI: The University of Washington group believes that its findings of reduced antipsychotic prescribing tell a story of unexceptionable success; they depart in their exposition from usual and customary discussion of alternative explanations for their observations or how they interpret them. No discussion of specific drugs or adverse effects do they offer; no analysis of cost-effectiveness do they allude to; and no benefit or advantage of second-genereration antipsychotics do they discuss.
The reason for their unconventional ellipses of discussions of usual and customary issues of benefit and risk becomes clear as they close their report: for them, “good outcome” is not the answer to questions about symptomatic outcome, but rather about how many fewer patients are receiving second-generation antipsychotics! They make no mention of risks of failure to authorize prescriptions of second-generation antipsychotics. If they had, they would have been obliged to cite findings published a decade ago (T.A. Moore and colleagues, Psychiatric Clinics of North America 30(3):401-456, 2007; M.J. Sernyak and R. Rosenheck, Adm Policy Ment Health 34(6):540-547, 2007) reporting that a preponderance of clinicians who prescribe higher than recommended dosages do so for rationales based on risk and benefit (that is, evidence-based reasons applied to patients they knew, whose regimens would have been peremptorily disrupted by appointees to peer review committees who had never seen those patients). The authors of those studies (Moore and colleagues and Sernyak and Rosenheck, ibid.) gave as their rationale for disagreeing with utilization reviewers that they had already prescribed the alternatives suggested by the peer reviewers and had observed relapse and symptomatic worsening while doing so. Ninety-four percent of the prescribing physicians in the study by Sernyak and Rosenheck (op. cit.) reported that they had been able to justify their disagreement with utilization reviewers not only by virtue of an evidence base pertaining to antipsychotic effects in persons with psychotic symptoms, but also on a record of personal experience with their patients (whom the utilization reviewers had never seen). (This appears to have had little influence on the State’s peer reviewers, whose goal (to reduce prescribing of second-generation atypical antipsychotics) differed from that of the well-intentioned colleagues who knew the patients they were treating and had not been prescribing pursuant to a bestriding goal of avoiding second-generation antipsychotics.)
ObiterII: What was the question when there arrives an answer whose outcome proxy is cost alone? What null hypothesis was shown rejectable by the University of Washington group? Utilization reviewers remonstrate that their failures to authorize comport with published guidelines. Yet, they reinterpret the latter, exceed them, and either do not understand them or are attempting to change them, along with the intent of the guideline process: Guidelines are meant to apprise, to encourage deliberation; to suggest: When a guideline becomes a list of punishable transgressions, it is no longer a guideline, but rather a list of compulsories misrepresented as well-documented, extensively corroborated treatment recommendations. Negative affect bias blinds the peer reviewers, who are like drivers who believe that they will reach their destinations safely by looking at windshields painted on the inside with what they wish to see (instead of reacting to the road as it is through unpainted glass).)
A walk through the risks of newer-generation antipsychotics in vulnerable patients
A handful of published reports over the last decade have suggested that foster children are more likely than other children to have met diagnostic criteria for a psychiatric disorder, for which they have been treated with psychiatric pharmacotherapy. Never a highly controversial finding, in view of differences between foster and non-foster children with respect to higher rates of diathetical associations and risk-conferring exogenous exposures specific to fostering (trauma, removal from home, separation anxiety), trends in data, disattached from its content, have been regarded by virtue of extreme prejudice as tantamount to evidential findings warranting change of treatment recommendations. The most perspicuous example among first-generation antipsychotics devolved from the first “wave” of tardive dyskinesia at a time when its observed rates had not yet been replicated, and a possibility of irreversibility and ineluctable worsening from a first generation of observations had been fanned by a bellows of panicky hyperventilation into such a firestorm of “certainty” that few challenged the affect bias of any bucco-lingual adventitious movement in a neuroleptic-treated patient as reasonable cause for prompt intervention.
A second (thankfully more quickly disposed of) professional panic accompanied concern about metabolic adversity of some of the second-generation antipsychotics (quelled by detection of a plateau of severity at about one year of exposure and interdrug differences in the propensities of the drugs of the class with respect to likelihood and worsening of metabolic uncertainty), yet here too were we reminded that affect bias can ignite a firestorm of no less heat and with no less speed when nine-figure firewood threatens to run out before the end of winter. What began as a team at Medicaid again became everyman for himself.)
Concern about costs, and the justifiability of the costs devoted to prevention and treatment of the objects of concern (in our context, drug adverse effects) are just beginning to be investigated (a status that has characterized them for a long time). Two of the most severe concerns associated with neuroleptic exposure are metabolic adversity and tardive motor syndromes (tardive dyskinesia the most cited). (The two other chief concerns apropos of neuroleptics (neuroleptic malignant syndrome and granulocytopenia) are an order of magnitude or more less common (and the latter, the only one reported in association with only one member of its drug class) associated with, and accordingly less commonly reported on.
None of the “big four” risks of neuroleptic exposure are newly discovered, but two of the four (the two most common and least acute) receive a volume of high-pitched attention from without (and exerted inward, that is, guildward) hinting of indignation. Doubtless, the second generation antipsychotics are expensive, and thereby threatening to underwriters of healthcare for all sectors of the public who need them. The first-generation drugs, like the second-generation descendants, were no less likely to induce metabolic adversity and are (still) more extensively the attribution for tardive dyskinesia as a ratio of current to total cases identified (normalized for extent of prescription). So then, is cost driving the “muchness” of reportage concerning those drug-related adversities? The driver does not seem to be originating within guild walls, where concern seems “about right” and less than would inspire multiple separate data-thin reports on the same patient samples.
Metabolic adversity quickly manifested after introduction of the first atypical antipsychotic (clozapine), but was largely neglected under the penumbra of the latter’s granulocytotoxicity (as were bowel obstruction and complications, and atypical presentations of neuroleptic malignant syndrome).
With the “launch” of risperidone in the early Nineties, followed by a dozen others (up to lurasidone (Latuda)), metabolic adversity emerged as obesity, glucose intolerance, and dyslipidemias. Today, we can stratify atypical neuroleptics pursuant to their likelihoods and severities (clozapine and olanzapine most in adults and youth, risperidone mild to moderate in adults and moderate to severe in youth, the least ziprasidone and lurasidone, the rest intermediate, with aripiprazole closer to least and quetiapine to most); we know that new-onset diabetes mellitus, type I is a rare, causal association, diabetes mellitus, type II, uncommon to common, obesity (seven percent over pretreatment body mass index) commonly occurring with biochemical indicators of metabolic adversity; the latter typically absent when patients are not obese; and that both obesity and biochemical metabolic abnormalities plateauing in severity after approximately one year of expsosure to atypical neuroleptics). We believe that biochemical metabolic abnormalities associated with atypical neuroleptics are proxies for cardiovascular disease. With respect to prevention, dietary restriction with exercise is mitigating, as is coadministration of metformin.
Tardive motor syndromes (hereafter “tardive dyskinesia,” consonant with usage) have been obfuscated in their diagnosis by informal recharacterization owing to an esoteric, almost anonymous, past. Defined in the Sixties and characterized in the Sixties and Seventies as a syndrome or oral, buccal, lingual repetitive movements caused by long-term exposure to (perforce) first-generation antipsychotics, its pathogenesis remains obscure, beyond plausible speculation about “dopaminergic supersensitivity” induced by long-term dopaminergic antagonism. Initially thought to be progressive in distribution and severity, it no longer is so considered after long-term study, which withal revealed that up to half of the cases improve and remit after neuroleptic discontinuation. It would seem to follow that continuing exposure to progressively higher neuroleptic dosages of any of the neuroleptics will eventuate in greater severity (as amplitude or frequency of movements), since the latter is positively correlated with cumulative neuroleptic dosage. Since no treatments have proved effective, prevention is critical. The movements of tardive dyskinesia are suppressible by compounds with antidopaminergic activity (including neuroleptics), and typically worsen when neuroleptic dosage is lowered. Catecholamine depleting agents, such as tetrabenazine (and more recently deutetrabenazine and valbenazine) also have been reported ameliorative, and have not so far been reported to cause or worsen tardive dyskinesia. Associated with greater likelihood of developing tardive dyskinesia (although causality of association is in some instances unknown) are cumulative neuroleptic dosage, diagnosis of affective disorder, brain disease or damage, age, female sex, length of illness, early onset of or sensitivity toward development of acute extrapyramidal adverse effects, prescription of antimuscarinic agents, and first-generation neuroleptics (reported ten- or more-fold likely to be associated with tardive dyskinesia per person-duration of neuroleptic exposure).
Curiously absent among the so-called risk factors of manifesting Mtardive dyskinesia is exposure to dopaminergic agonists. Studies by NIMH researchers beginning in the early Seventies, of d-amphetamine challenge doses in neuroleptic-treated patients with schizophrenia, was not reported associated with emergence or worsening of tardive dyskinesia. (D-amphetamine and methylphenidate are releasers of newly synthesized dopamine (and ultimately, in persons exposed to frequent, high dosing, dopamine depleters) from an intraneuronal site different from that of catecholamine depleters, such as tetrabenazine and congeners (which cause release of dopamine and norepinephine from “storage sites” within intraneuronal vesicles that evaginate to become continuous with cell membranes and release catecholamines into the synapse).
One of the reasons for adverting the field about questionably noteworthy findings is not so much to illustrate their questionable noteworthiness as to incise their interpretations and expose some of the conclusions that depend upon them.
For example, a report by Italian researchers (S. Pisano and colleagues, Clinical Drug Investigation 2018 Feb12:. doi: 10.1007/s40261-018-0627-3), which examined genetic abnormalities in 19 non-diabetic youth (mean age = 159 months) with obesity and associated metabolic complications in association with exposure to second-generation neuroleptics, and compared them with nineteen controls matched for age, sex, pubertal status, and body mass index who had not been exposed to any psychotropic drug.
A preponderance of metabolic markers were not different between groups; exceptions in the patients exposed to antipsychotics were C-peptide and glucose-dependent insulinotropic polypeptide (both higher in antipsychotic-exposed patients), and visfatin (lower). Among the preponderantly similar metabolic markers were leptin and ghrelin), of which both were higher relative to non-obese subjects. The authors’ findings do not rule out antipsychotic-related associations between C-peptide and glucose-dependent insulinotropic polypeptide, but do not suggest that the latter were causal. Indeed, given similarities of leading “suspect” markers leptin and ghrelin, the authors’ observations are consistent with the idea that metabolic abnormalities, such as hyperglycemia and hyperlipidemias, in patients who become obese while taking atypical neuroleptics may more likely devolve from obesity rather than from direct, causal action of atypical neuroleptics, which are independently associated with obesity. (None of the authors’ subjects had been exposed to first-generation antipsychotics, such as chlorpromazine, thioridazine, perphenazine, or trifluoperazine, which, like second-generation drugs olanzapine, clozapine, risperidone, quetiapine, and aripiprazole have been implicated with different likelihoods of obesity. (Ziprasidone, lurasidone, and aripiprazole are among the least likely to). Metabolic abnormalities, absent obesity, are uncommon in patients exposed either to second-generation antipsychotics or their predecessors. Commonly cited as rationale for failure to authorize second-generation antipsychotics in patients with income-qualified Medicaid, and strictly monitored by requirement enforced by revocation of authorization (neither the case apropos of first-generation antipsychotics), metabolic abnormalities of second-generation, atypical neuroleptics may (as with first-generation antipsychotics and some psychotropic drugs indicated for treatment of depression, mania, and anxiety) represent a complication of obesity associated with excessive consumption of food and/or insufficient exercise.