James C. COYNE* 1, 2 & Robin N. KOK 3,4
1 Department of Health Psychology, University of Groningen, University Medical Center, Groningen, the Netherlands
2 Institute for Health, Health Care Policy and Aging Research, Rutgers, The State University of New Jersey, USA
3 Department of Clinical Psychology and the EMGO institute for Health and Care Research, Faculty of Psychology and Education, VU University Amsterdam, The Netherlands
4 Centre for Mental Health Research, The Australian National University, Canberra, Australia
Abstract
Recognition of the unreliability of findings in the biomedical literature, and especially pharmaceutical trials has led to a number of reforms. These include reporting preregistration of protocols for clinical trials and meta-analysis, reporting standards and making data available to others for all clinical trials, as well as recognition of the influence of conflicts of interest. These improvements are only partially and inconsistently reflected in the literature evaluating psychotherapies. The psychotherapy literature is currently of too poor quality to provide a reliable guide to clinicians, consumers, and policymakers. The literature is dominated by underpowered trials with high risk of bias producing positive effects at a statistically improbable rate. Meta-analyses that are poorly conducted with undisclosed conflicts of interest compound these problems. A number of reforms are proposed. These include accelerating adoption of those already occurring in the pharmaceutical literature. Additionally, psychotherapy research should parallel the orderly sequence of treatment development seen in the pharmaceutical literature. Phase III trials providing the effect sizes of treatments should not be conducted until the acceptability of treatment and the feasibility of accruing sufficient numbers of patients are established. The role of investigator allegiance as a potential and potent source of conflict of interest needs to be recognized. Yet, enforcement of existing standards could counter many of the deficiencies of the current literature, but such enforcement may only come with pressures emanating from outside the field of psychotherapy.
Keywords: CONSORT, reporting standards, confirmatory bias, investigator allegiance, meta-analysis, conflict of interest.
