ACCURACY AND CONFIDENCE OF TRAINING THERAPISTS’ RECOGNITION OF SESSIONS BEFORE SUDDEN GAINS

Michael F. GREENFIELD* & David A.F. HAAGA

American University, Washington DC, USA

 

Abstract

The critical session hypothesis (Tang & DeRubeis, 1999) posits that patient changes in a single critical therapy session lead to sudden gains in psychotherapy outcome.  This study examines if psychotherapy trainee raters can accurately differentiate critical sessions leading to sudden gains (called pregain sessions) from control sessions using session chart notes, and whether rater confidence is related to accuracy.  The 23 patients experienced sudden gains within a cognitive behavioral therapy (CBT) training clinic staffed by Ph.D. clinical psychology trainees.  Four raters evaluated unidentified pregain and control sessions, classified them as pregain or control, and self-reported confidence.  They also identified therapeutic change processes they perceived in the sessions.  Raters were unable to differentiate pregain from control session case notes and displayed slight overconfidence.  The lack of differentiability between sessions suggests that if pregain in-session patient changes are taking place, they are not so overwhelming as to be apparent in a global rating task.

 

Keywords: sudden gains, Cognitive Behavioral Therapy, psychotherapy training clinic, clinical overconfidence
Introduction

 

Sudden gains are large and abrupt improvements in treatment outcome between consecutive therapy sessions.  The gains are dramatic improvements that are much larger than typical between-session symptom reductions, and they do not appear to reflect transient mood fluctuations (Tang & DeRubeis, 1999).  An influential early study (Tang & DeRubeis, 1999) found that 39% of clinically depressed patients treated by Cognitive Behavioral Therapy (CBT) experience sudden gains, with a fifth of sudden gaining patients experiencing more than one sudden gain; in addition, these gains accounted for 51% of total improvement and predicted better outcome at post-treatment and at 6 and 18 month follow-ups relative to patients not experiencing sudden gains.

Similar results have been found in other CBT clinical trials for depression (Busch, Kanter, Landes, & Kohlenberg, 2006; Hardy, Cahill, Stiles, Ispan, Macaskill, & Barkham, M., 2005; Tang, DeRubeis, Beberman, & Pham, 2005; Tang, DeRubeis, Hollon, Amsterdam, & Shelton, 2007).  Investigators of other approaches (e.g., Interpersonal therapy, Supportive-Expressive therapy) or of different populations (e.g., social phobia and generalized anxiety patients, adolescents) have also found sudden gains (Gaynor, Weersing, Kolko et al., 2003; Hofmann, Schulz, Meuret et al., 2006; Kelly, Cyranowski, & Frank, 2007).  Sudden gains have also occurred under routine clinical conditions with a heterogeneous patient population (Stiles, Leach, Barkham, et al., 2003; Greenfield, Gunthert, & Haaga, 2011).

Since sudden gains are associated with better outcomes, attempts have been made to identify mechanisms that facilitate the conditions for sudden gains within therapy.  Tang and DeRubeis (1999) hypothesized that patients improved in the pregain session, i.e., the session immediately prior to the sudden gain.  This patient breakthrough during this critical pregain session triggers a sudden, drastic improvement in outcome.  This critical session hypothesis has gained more traction than have rival hypotheses about determinants of sudden gains, such as positive life events occurring outside the therapy context.

Testing the critical session hypothesis, Tang and DeRubeis (1999) measured six possible process variables, finding that only patient cognitive change was significantly higher in pregain sessions compared to control sessions, which immediately preceded the pregain session.  They concluded that in-session cognitive changes in the pregain sessions are responsible to some degree for sudden gains.  Finding that patient cognitive change leads to sudden gains resonates with a recent review of cognitive therapy for depression that concludes cognitive changes mediate change, with cognitive improvement preceding and often predicting improvement in depressive symptoms (Garratt, Ingram, Rand, & Sawalani, 2007).

The critical session hypothesis was also tested in a clinical trial of Supportive-Expressive therapy (SET), a psychodynamically-oriented therapy (Andrusyna, Luborsky, Pham, & Tang 2006).  This study found that 15 of the 40 patients experienced sudden gains.  Therapist interpretation accuracy (a SET-specific process) was significantly higher in pregain sessions, with therapeutic alliance trending towards significance and cognitive changes showing no differences.  The process at work in sudden gains may be theoretically linked to the treatment modality being applied: cognitive change in CBT and interpretation accuracy in SET.

An as-yet untested implication of the critical session hypothesis is that raters should be able to distinguish pregain and control sessions.  That is, regardless of whether sudden gain determinants are specific to treatment orientations, or more general processes such as high levels of therapist empathy, or in-session processes not yet identified in the psychotherapy literature, it should still be the case (if sudden gains indeed are caused by what happens in the pregain session) that raters can sense something different and better about the pregain sessions relative to the control sessions.  Since only a subset of specific processes tested have shown differences between pregain and control sessions (cognitive change, Tang & DeRubeis, 1999; interpretation accuracy, Luborsky, Pham, & Tang, 2006), and other processes have shown no differences (e.g., therapeutic alliance, therapist application of CBT techniques, Tang & DeRubeis, 1999), it seems useful to take a step back and understand if on a global rating task, raters can correctly classify pregain and control sessions.  Stiles et al. (2003) raised a similar issue, namely whether therapists notice sudden gains in their patients.

Picking up Stiles et al.’s (2003) question of therapist recognition of sudden gains, Davies, Leach, Lucock, Stiles, Iveson, and Barkham (2006) interviewed therapists about their recollections of sudden gaining and non-sudden gaining patients who had left therapy under routine conditions.  These investigators provided therapists their patient chart notes and asked if they could identify which patients experienced a sudden gain and when the critical session occurred. While therapists could retrospectively identify which patients experienced a sudden gain at better than chance rates, nearly half of the sudden gaining patients were not identified as such.  Moreover, therapists believed a fair number of non-sudden gaining patients (20 of 92) experienced sudden gains.  The critical session was also difficult to identify, with only 30% of therapists pinpointing the exact critical session preceding a sudden gain.  These results show that while clinicians were somewhat aware when sudden gains occurred for their patients, quantitative assessments may aid sudden gain identification.

The primary objective of this research is to test further the critical session hypothesis by exploring therapists’ perceptions of sudden gaining patients.  In this study, psychotherapy trainees attempt to distinguish pregain session from control sessions on the basis of therapists’ chart notes.  We refer to this global distinction of sessions as “basic-level differentiability”.  As indirectly alluded to in the Stiles et al. (2003) study, if therapists are able to notice the changes in patients that lead to sudden gains, they may be able to help patients preserve and build upon that gain.  This may help prevent sudden gain relapse, or the situation when a patient significantly worsens following their sudden gain.  Such relapses (known as “sudden gain reversals,” Tang & DeRubeis, 1999) occur in anywhere from 17% (Tang & DeRubeis, 1999) to 43% of sudden gaining patients (Stiles et al., 2003).

As our raters attempt to differentiate between pregain and control sessions on a basic-level, they self-report confidence levels and state which therapeutic change processes (e.g., patient insight, cognitive shift, problem resolution) they based their classification of pregain sessions upon.  In other words, raters reported which processes they perceived as evidence that a sudden gain may be developing, revealing which processes they think cause a sudden gain.  Identification of what processes in particular distinguished the pregain from the control sessions we refer to as “process-level differentiability”.

Besides testing the basic-level and process-level differentiability of pregain and control sessions, our study addressed whether raters’ confidence in making this distinction tracked their level of accuracy in doing so.  Sudden gains mark an important transition point in therapy in which the new skill underlying the gain needs to be generalized (Hayes, Feldman, Beevers, Laurenceau, Cardiaciotto, & Lewis-Smith, 2007).  Therapists are better positioned to help patients build upon the gain and avoid sudden gain relapse if they assume the appropriate level of confidence in directing therapy for patients following a sudden gain.

 

Method

 

Participants

The stimulus materials evaluated by raters came from the charts reporting on therapy experiences of 23 patients who had experienced a “sudden gain” while in treatment at the James J. Gray Psychotherapy Training Clinic in the Department of Psychology at American University.  These 23 sudden gaining patients were identified from a larger sample of 106 patients obtaining services at American University over a 10 year period.  The criteria used to select these patients are outlined in the Measures section below.  One additional patient experienced a sudden gain in this clinic, but the chart notes were not available.  These patients are routine referrals from the university community or local practitioners and are adults (18 and over).

The therapists were trainees enrolled in a CBT practicum that takes place in the final year of a three-year practica sequence within an APA-accredited clinical psychology Ph.D. program.  The niche of the clinic is provision of CBT with a sliding scale fee structure.  There were various lengths of treatment and protocols used depending on diagnosis and case conceptualization, but treatment in the training clinic is typically manual-guided CBT.  There is no set limit on number of sessions that patients can receive.  Additional detail on these sudden gains, the criteria, and the clinical setting are outlined in Greenfield, Gunthert, and Haaga (2011).

 

Measures

Various standardized measures are used in the clinic at intake and throughout treatment, depending on features of the case.  The common denominator is that the Outcome Questionnaire-45 (OQ-45; Lambert et al., 1996) is administered to all patients at the beginning of each session.  The sudden gain criteria used to identify sudden gainers are essentially Tang and DeRubeis’s criteria (1999) with a modification for identifying first session sudden gainers proposed by Gaynor et al. (2003).  These three criteria require that the gain be:

(1)  large in absolute value, i.e., >= 14 points on the OQ-45, thus exceeding the RCI of the measure (Anderson & Lambert, 2001);

(2)  large relative to pregain severity, i.e., >= 25% of the pregain session’s OQ-45 score.

(3)  large relative to symptom fluctuation before and after the gain; the difference between the mean OQ-45 score of the three sessions before the gain (n-2, n-1, and n) and the three sessions after the gain (n+1, n+2, and n+3) is at least 2.78 times greater than the pooled standard deviations of the these two groups of sessions’ OQ-45 scores.  When gains occur after the second session or on the second to last session, n-2 and n+3 are not used.  When gains occur after the first session, 50% of the gain must be maintained for two sessions.

 

Procedure

Pregain and control therapy session notes were the content of our assessments.  The pregain session is the session immediately prior to the session at which the sudden gain improvement is measured.  The control session refers to the session immediately previous to the pregain session.  This session is chosen as a control (a) because it is close in time to the pregain session, and (b) because both sessions occurred before the sudden gain was measured at the aftergain session, the patient’s initial state in the control session should, on average, more closely resemble his or her initial state in the pregain session than it would in any other session (Tang & DeRubeis, 1999).  For the 3 patients experiencing multiple sudden gains, we evaluated only the first sudden gain.

Sessions were evaluated through case notes, which were thought to provide sufficient information for the rating task.  Case notes were written by the trainee close to the time of the therapy session and typically cover what was discussed in the session, how the patients presented, homework that was assigned, etc.  Some case notes in this clinic followed the format outlined by the acronym, SOAP, which stands for Subjective, Objective, Assessment, and Plan.  Subjective refers to what the patient subjectively reports in the session, while Objective refers to more objective facts associated with a patient’s case.  Assessment involves the clinician’s informal assessment or evaluation of the patient’s symptom levels, occupational situation, or interpersonal relationships.  Lastly, Plan refers to the therapist’s treatment plan moving forward (Borcherding, 2005).  This portion of the case note may have been completed through consultation with a therapist’s faculty supervisor.  His or her clinical supervisor reviewed case notes for completion and signs the notes.  Case notes varied in length from 200-500 words.

The raters conducted two assessments of each case note.  Three of the four raters are advanced Clinical Ph.D. students.  They were either in their third or fourth year in the program, with an average age of 26.6.  Two of the raters reported that their theoretical orientation was Integrative, the third cognitive behavioral.  They averaged 3.6 years experience in therapy or counseling.  The fourth is an MA student who was 38 years old, in her second year of the Psychology Department’s MA program, of eclectic theoretical orientation, with 4 years of experience in therapy, and 2 years of experience in structured anxiety assessments.  Though several of our raters did not self-report being primarily cognitive behavioral in orientation, they operated in a graduate psychology program that is grounded in CBT.  Moreover, they all worked as researchers in CBT-influenced research labs.

Assessment #1: Evaluating a Single Note.  The raters were given a packet of unidentified case notes consisting of either a pregain or control session case note.  They were instructed in the background of sudden gain research and the notion of critical sessions influencing outcome. The first author explained the difference between pregain sessions and control sessions in the context of sudden gain research, and how understanding these session categories was important for their rating task. Raters were then given an overview of both the first and second assessment and the process variables the investigators hoped to test.  Raters were given two weeks to complete the assessment materials.

Upon reviewing a case note, raters were given a forced choice question asking them if this note referred to a pregain or control therapy session.  After selecting their choice, they were then asked the following question, “How certain are you that the case note corresponds to session you chose above (i.e., pregain or control session)?” with a 7-point response scale from – 3 (extremely uncertain) to 3 (extremely certain).  Raters then assessed the case notes on a range of variables in two different categories.  Process variables were based on a review by Orlinsky, Rennestad, and Willutski (2004) and were deemed representative of process variables used across theoretical orientation.  The first category of variables related to session content and asked raters, “Based on your assessment of the case note, to what extent do you think that the therapist focuses on the following patient factors?”  These factors included the patient’s affective processes, cognitive processes, resistance s/he shows in the session, interpersonal relations, problems, and resources.  The rationale for including this session content category was to understand if the mere focus on certain content in a patient’s life has a positive link to sudden gains.

The second category of variables assessed therapists’ perception of patient positive impacts and asked the raters, “Based on your assessment of the case note, to what extent does the note show the following positive impacts for the patient?”  The following areas were considered: cognition, insight, problem resolution, emotional relief, morale, and self-efficacy.  The therapists’ perception of patient positive impacts variables contrasts with the session content variables by asking raters to ascertain if the patient makes forward progress in a certain area within the session.

Assessment #2: Evaluating Pairs of Pregain and Control Notes.  Raters evaluated pairs of unidentified case notes (i.e., pregain session and control session case notes from a given patient), rather than one unidentified case note presented alone as in Assessment #1.  They responded to another forced question asking them which case note appears more likely to be the pregain session case note, followed by the same question that appeared in Assessment #1 regarding subjective certainty.  They then answered a free response question of how the two case notes in the pair differ and the specific relationship between the two case notes (e.g., the patient shows more insight in case note #5).  The second assessment’s free response question allowed us to probe inductively for patterns in the relationship between pregain and control sessions.

In terms of coding the free response answers, we planned first to read through all free responses and develop a list of process variables that the raters use to distinguish pregain from control session.  These process variables may or may not have resembled the variables we deductively selected for the numerical rating questions that appeared in Assessment #1 where raters evaluated a single note.  For example, raters mentioned in several ways that patients achieved cognitive change within a session.  Therefore, we created the category of “Cognitive Shift.”  We then reviewed the free responses again and tabulated the number of times raters cited a process variable to justify labeling a case note as pregain.  The process variables mentioned by raters were rank-ordered by the frequency of times these variables were mentioned in free responses.  For example, we eventually found that raters mentioned “Cognitive Shift” 5 times while evaluating case notes.  We adopted a coding scheme involving rank-ordering to examine which therapeutic mechanisms were perceived as most important for sudden gains.

Calculating Confidence-Accuracy Relationship

            After calculating raters’ accuracy in session differentiation and their confidence levels, we determined if confidence and accuracy are at all related.  Since accuracy is a categorical variable and confidence a continuum, we used the point-biserial correlation coefficient (r_pbi; Brown, 1988) to evaluate their relationship.[1]

 

Results

 

Participants

The sudden gaining patients’ average age was 41.6 (SD = 13.7), and 14 (or 58.3%) were female.  They averaged 28.2 sessions (SD = 21.7).  Twelve had a primary chart diagnosis of an anxiety disorder, 7 mood disorder, 2 impulse control disorder, and 2 were in an “other” category that would include such diagnoses as substance use disorders, adjustment disorders, and sleep disorders.  Seven of the 23 sudden gaining patients also had a comorbid diagnosis, as seen in Table 1.

 

Table 1. Primary and Secondary Diagnostic Categories

 

	Primary Diagnostic Category		Secondary Diagnostic Category
Category	Frequency	Percent (%)	Frequency	Percent (%)
Mood Disorders	7	30.4	1	14.3
Anxiety Disorders	12	52.2	2	28.6
Eating Disorder	0	0	1	14.3
Impulse control disorders	2	8.7	0	0
Other	2	8.7	3	42.9

 

 

The sudden gainers’ intake score on the OQ-45 was 77.8 (SD = 22.0).  The patients were in the clinical norm, as the recommended OQ-45 cutoff for distinguishing between clinical and community samples is >=64 vs. <=63 (Lambert et al., 1996).   Of all sudden gainers, three had at least 2 sudden gains, 1 of whom had 3 sudden gains.  The median magnitude of sudden gains was 24.5 OQ-45 points.  The median session during which sudden gains occur was session 4 (mean = 6).

Assessment #1: Evaluating a Single Note

Raters were only 30.4% accurate in classifying an unidentified case note as referring to a pregain or control session.  They more often responded that a case note referred to a control session than a pregain session.  Of the 46 case notes, 27 (or 58.7%) were labeled as controls and 19 (or 41.3%) as pregain sessions.  Table 2 shows a 2 x 2 table relating actual session type to classified session type, or whether raters labeled the case note control or pregain.

A chi square test was conducted to determine if there was a statistical association between their forced choice (i.e., their classification) and the actual session type.  It showed a relationship between the actual session type (pregain or control) and the classified session type: c² (1, N = 46) = 7.3, p < .007.  The accompanying phi coefficient was f = -.40.  This means that raters had a systematically wrong sense of whether an unidentified case note referred to a pregain or control session and were not accurate at the level of chance.

 

 

Table 2.  Cross-Tabulation of Actual Session Type and Classified Session Type

 

	Actual Session Type
Classified Session Type	Pregain	Control
Pregain	5	14
Control	18	9

 

 

Raters then listed the level of certainty with which they made their forced choice decision on a 7-point scale from -3 (extremely uncertain) to 3 (extremely certain).  The mean across all raters’ case notes was 1.1 (SD = 1.1), which translates approximately to “slightly certain.”  This mean certainty level was significantly different from the mid-point of our 7-point certainty scale, which was denoted as “neither certain nor uncertain”: (t = 6.8, N = 46, p < .05).  Despite being systematically wrong in their forced choice task, raters were slightly certain about their performance.

Having found that raters were slightly overconfident on the whole, we turn to the question of whether their confidence in rating specific session notes was at all related to their judgment accuracy in making those ratings.  There was a positive correlation between certainty and accuracy (r_pbi = .12), though it was not significant.

Process-level Differentiability: Raters’ Perceptions of Change Processes.  Independent of whether raters can globally distinguish between pregain and control case notes on a basic-level, we were interested if process variables were associated with (a) the perception that a case note refers to a pregain session (i.e., a case note classified as referring to a pregain session) and (b) the reality that this note was from an pregain session.  We accomplished this first part by calculating the mean differences in the ratings of certain process variables between sessions classified as pregain or control regardless of whether the classification was accurate.  This analysis reveals raters’ perceptions of the sessions, and what processes they used to justify their forced choices.  Higher levels of all 6 process variables under the category of therapists’ perception of positive patient impacts were endorsed in those case notes labeled as pregain notes according to a t test.  Cognitive shift, insight, and emotion relief were most highly significant, at the confidence level of .001; problem resolution and self-efficacy were significant at the level of .01; lastly, morale at the .05 level.  This suggests that therapists’ perception of positive patient impact variables were associated with raters’ decision to label a case note as referring to a pregain session, whereas the content of the case note was not.  In addition to evaluating classified session notes, we looked at whether ratings of process variables showed differences between actual pregain and control session notes.  No significant differences were found in this series of t tests.  Table 3 displays mean process ratings for both actual session type and classified session type.

 

Table 3.  Mean Process Variable Ratings for Actual and Classified Session Type

 

	Actual Session Type				Classified Session Type
	Pregain		Control		Pregain		Control
Session Content Variables	Mean	SD	Mean	SD	Mean	SD	Mean	SD
Affective process	3.65	1.23	3.09	1.24	3.58	1.22	3.22	1.28
Cognitive process	3.61	0.84	3.87	1.32	4.11	1.10	3.48	1.05
Resistance	2.26	1.39	2.00	1.60	2.47	1.78	1.89	1.22
Interpersonal relations	2.96	1.87	2.48	1.53	3.11	1.70	2.44	1.69
Problems	2.78	1.65	3.13	1.74	3.32	1.45	2.70	1.81
Resources	3.22	1.54	2.61	1.64	3.26	1.73	2.67	1.49
Positive Patient Impact Variables
Cognitive shift	2.39	1.53	2.87	1.39	3.89***	0.99	1.74***	1.02
Insight	2.61	1.31	3.30	1.46	4.16***	0.90	2.11***	1.05
Problem resolution	2.26	1.32	2.83	1.53	3.37**	1.16	1.96**	1.34
Emotional relief	2.13	1.42	2.61	1.41	3.26***	1.28	1.74***	1.16
Morale	2.09	1.47	2.30	1.55	2.84*	1.64	1.74*	1.23
Self-efficacy	2.22	1.51	2.83	1.40	3.21**	1.47	2.04**	1.29

* = the means were significantly different at α = .05 level according to a t test;

** = significant at the α = .01 level;

*** = the means are significantly different at α = .001 level.

 

 

Assessment #2: Evaluating Pairs of Pregain and Control Notes

When raters assessed a pair of unidentified case notes, they were accurate only at the level of chance, making the right choice about which note referred to a pregain session 50% of the time.  There was no relationship between classified session type and actual session type: c² (1, N = 46) = .087, p < .77.  Despite being inaccurate, they were not systematically wrong as was the case in Assessment #1 when evaluating a single note.

Raters’ mean certainty level was .73 (SD = 1.34), which is between the points of “Neither certain nor uncertain” and “Slightly certain.”  Certainty levels were lower than in Assessment #1 when a single case note was evaluated, but this difference in certainty between Assessments did not approach significance (t = 1.40, N = 46, p = .16).  Moreover, as was the case in Assessment #1, the mean certainty level in Assessment #2 was significantly different from the mid-point of our 7-point certainty scale: (t = 3.6, N = 46, p < .00).

We earlier noted that when evaluating a single case note, raters’ certainty and accuracy were weakly positively correlated (r_pbi = .12).  When computing this statistic for the evaluation of a pair of case notes, we find a slight negative correlation between certainty and accuracy (r_pbi =  -.07), which again was not significant.

Tabulation of the free responses showed that raters most often justified their classification of case notes by referencing the following process variables in the order from the greatest to least number of times mentioned: insight, activation of patient resources, emotional relief, cognitive shift, readiness to change, and lastly installation of hope.  Table 4 outlines operational definitions of each of these variables, the number of times raters mentioned them in their free responses, and sample phrases raters used in the free response.  As can be seen from this table, more than half of the times a rater invoked a process variable to justify their choice of case note as pregain, they mentioned insight or activation of patient resources.

 

Discussion and conclusions

 

The critical session hypothesis states that unique, critical sessions engender patient change and cause sudden gains (Tang & DeRubeis, 1999).  While this hypothesis is promising (Tang & DeRubeis, 1999; Andrusyna, Luborsky, Pham, & Tang 2006), our finding that raters could not differentiate between pregain and control sessions on a basic-level does not provide additional support.  Raters in this study found the representation of pregain and control sessions to be roughly equivalent.  Since these sessions consecutively preceded the sudden gain, it could be hypothesized that patient changes were occurring at the same rate in each session and that accumulative change over the course of these sessions led to the sudden gain.  We would then not expect therapists to be able to recognize an approaching sudden gain based on patient changes within one critical session.

That said, these results may also be due to therapists not noticing the patient changes in the critical session (Stiles et al., 2003) and omitting them from their chart notes, thus preserving the validity of critical session hypothesis.  Indeed, the raters’ accuracy depends in part on therapists noticing that the patient is achieving a major therapeutic benefit in the session and describing it in the note.  This possibility dovetails with Davies et al. (2006), which shows that therapists were not able to identify retrospectively nearly half of their sudden gaining patients and likewise believed nearly a quarter of non-sudden gaining patients had a sudden gain.  Moreover, our results were in large part dependent on psychotherapy trainee perception of case notes.  Given their minimal level of experience, their accuracy in session classification may have been impaired.

Along with being inaccurate in session classification, raters displayed slight overconfidence.  Their confidence levels and decision accuracy were weakly correlated and switched sign across the two assessments, resonating with previous research finding that confidence is only a weak predictor of accuracy (Bornstein & Zickafoose, 1999).  Overconfidence with respect to sudden gains could have relevance to real-world therapeutic practice.  As a therapist adjusts to patient changes following a sudden gain, it is important they do not become overconfident so as to rigidly adhere to a prior treatment plan (Smith & Agate, 2004; Dunning, Griffin, Milojkovic, & Ross, 1990).  With respect to sudden gainers, therapists may need to adapt treatment to patients’ status post-gain to enable continued recovery and prevent a sudden gain reversal.

Emotive and cognitive patient factors played an important role in whether raters felt sudden gains occurred.  Raters perceived patient insight, emotional relief, and cognitive shift occurring at a substantially higher rate in pregain sessions in Assessment #1.  Furthermore, in the open-ended response of Assessment #2 where raters evaluated a pair of notes, these factors were also frequently mentioned.  Davies et al. (2006) similarly found that therapists recalling sudden gaining patients believed sudden gains were associated with gaining insight and expressing previously hidden feelings or thoughts.

Patient insight appears to have particular importance for therapists as a factor leading to sudden gains.  In Assessment #1, patient insight was defined as “improved understanding of connection between thoughts, feelings, and behaviors.”  While other variables such as cognitive shift were rated highly in the Assessment #1, raters perceived that sessions classified as pregain revealed more patient insight than any other variable.  Furthermore, in the open-ended response of Assessment #2 where raters evaluated a pair of notes, insight was the most frequently mentioned variable used to justify the classification of pregain sessions.  The therapists in Davies et al. (2006) also most frequently mentioned “gaining insight” as being associated with sudden gains.  The focus on insight reveals that therapists may view sudden gains to be primarily the result of therapy clicking for the patient.

The lack of basic-level differentiability could be due to using case notes that filter session content through the therapist writing the notes.  To stringently test the critical session hypothesis, audio- or video-tapes of sessions would be ideal.  The minimal experience levels of our psychotherapy trainee raters could also be addressed by employing experienced psychotherapists as raters.  In addition, to evaluate whether therapists notice sudden gains, future research could administer structured questionnaires to therapists after every session to gauge their perception of patient response.

It is unclear whether the lack of basic-level differentiability is due to equivalency of pregain and control sessions, or the breadth of disorders and comorbidities in our patient sample.  Sudden gains from a naturalistic sample differ slightly from those observed in controlled clinical trials using only depressed patients.  For example, sudden gains from diverse patients account for a greater proportion of total gain across all of therapy (Stiles et al., 2003; Greenfield, Gunthert, & Haaga, 2011).  Stiles et al. (2003) has also suggested that naturalistic patients may experience wider swings in symptoms intensity than depressed patients in clinical trials.  Therefore, the features of sudden gains from a diverse patient group may make it more difficult to accurately classify pregain and control sessions.

Another avenue for future sudden gain research would be to assess therapists’ evaluation of self-reported patient improvement in the aftergain session, or the session at which the sudden gain is measured and which follows the pregain session.  While therapists may miss the subtle in-session patient change in pregain sessions that spark sudden gains, they may be more likely to detect patient change in the aftergain session.  Whether or not therapists label this improvement a sudden gain, they would still be positioned to help patients sustain the momentum of their gains.

 

 

REFERENCES

 

Andruzyna, T. P., Luborsky, L., Pham, T., & Tang, T. Z.  (2006).  The mechanisms of sudden gains in Supportive-Expressive therapy for depression.  Psychotherapy Research, 16, 526-535.

Borcherding, S.  (2005).  Documentation manual for writing SOAP notes in occupational therapy.  Thorofare, NJ: SLACK Incorporated.

Bornstein, B. H. & Zickafoose, D. J. (1999).  “I know I know it, I know I saw it”: The stability of the confidence-accuracy relationship across domains. Journal of Experimental Psychology: Applied, 5, 76-88.

Brown, J. D. (1988). Understanding research in second language learning: A teacher’s guide to statistics and research design. Cambridge: Cambridge University Press.

Busch, A. M., Kanter, J. W., Landes, S. J., & Kohlenberg, R. J.  (2006).  Sudden gains and outcome: A broader temporal analysis of cognitive therapy for depression.  Behavior Therapy, 37, 61-68.

Davies, L., Leach, C., Lucock, M., Stiles, W. B., Iveson, W., & Barkham, M.  (2006).  Therapists’ recall of early sudden gains in routine clinical practice.  Psychology and Psychotherapy: 79, 107-114.

Garratt, G., Ingram, R. E., Rand, K. L., & Sawalani, G.  (2007).  Cognitive processes in Cognitive Therapy: Evaluation of the mechanisms of change in the treatment of depression.  Clinical Psychology: Science and Practice, 14, 224-239.

Gaynor, S. T., Weersing, R., Kolko, D. J., Birmaher, B., Heo, J., & Brent, D. A.  (2003).  The prevalence and impact of large sudden improvements during adolescent therapy for depression: A comparison across cognitive-behavioral, family, and supportive therapy.  Journal of Consulting and Clinical Psychology, 71, 386-393.

Greenfield, M. F., Gunthert, K. C., & Haaga, D. A. F.  (2011).  Sudden gains vs. gradual gains in a psychotherapy training clinic.  Journal of Clinical Psychology, 67, 17-30.

Haaga, D. A. F. & Stiles, W. B.  (2000).  Randomized clinical trials in psychotherapy research: Methodology, design, and evaluation.  In C.R. Snyder & R.E. Ingram (Eds.), Handbookof psychological change: Psychotherapy processes and practices for the 21^st century (pp. 14-39).  New York, NY: John Wiley & Sons, Inc.

Hardy, G. E., Cahill, J., Stiles, W. B., Ispan, C.,  Macaskill, N., & Barkham, M.  (2005).  Sudden gains in cognitive therapy for depression:  A replication and extension.  Journal of Consulting and Clinical Psychology, 73, 59-67.

Hayes, A. M., Feldman, G. C., Beevers, C. G., Laurenceau, J-P., Cardiociotto, L., & Lewis-Smith, J.  (2007).  Journal of Consulting and Clinical Psychology, 75, 409-421.

Hofmann, S. G., Schulz, S. M., Meuret, A. E., Moscovitch, D. A., & Suvak, M.  (2006).  Sudden gains during therapy of social phobia.  Journal of Consulting and Clinical Psychology, 74, 687-697.

Jacobson, N. S., Schmaling, K. D., Holtzworth-Munroe, A., Katt, J. L., Wood, L. F., & Follette, V. M.  (1989).  Research-structured vs. clinically flexible versions of social learning-based marital therapy.  Behavior Research and Therapy, 27, 173-180.

Kelly, M. A. R., Robert, J. E., & Bottonari, K. A.  (2007).  Non-treatment-related sudden gains in depression: The role of self-evaluation.  Behavior Research and Therapy, 45, 737-747.

Lambert, M. J., Hansen, N., Umphress, V., Lunnen, K., Okiishi, J., Burlingame, G., Huefner, J., & Reisinger, C. (1996). Administration and scoring manual for the Outcome Questionnaire (OQ 45.2). Wilmington, DL: American Professional Credentialing.

Lambert, M. J., Harmon, C., Slade, K., Whipple, J. L., & Hawkins, E.J .  (2005).  Providing feedback to psychotherapists on their patients’ progress: Clinical results and practice suggestions.  Journal of Clinical Psychology, 61, 165-174.

Liberman, V., & Tversky, A. (1993). On the evaluation of probability judgments: Calibration, resolution, and monotonicity. Psychological Bulletin, 114, 162–173.

Liebowitz, M. R.  (1987).  Social phobia.  Modern Problems of Pharmacopsychiatry, 22, 141-173.

Orlinsky, D. E., Rennestad, M. H., & Willutzski, U.  (2004).  Fifty years of psychotherapy process-outcome research: Continuity and change.  In M. J. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (pp. 307-389).  New York: Wiley.

Oskamp, S.  (1965).  Overconfidence in case-study judgments.  Journal of Consulting Psychology, 29, 261-265.

Rabinowitz, J. & Garelik-Wyler, R.  (1999).  Accuracy and confidence in clinical assessment of psychiatric inpatients risk of violence.  International Journal of Law and Psychiatry, 22, 99-106.

Schneider, B. H. & Bryne, B. M.  (1987).  Individualizing social skills training for behavior-disordered children.  Journal of Consulting and Clinical Psychology, 55, 444-445.

Smith, J. D. & Agate, J.  (2004).  Solutions for Overconfidence: Evaluation of an instructional module for counselor overconfidence.  Counselor Education & Supervision, 44, 31-43.

Stiles, W. B., Leach, C., Barkham, M., Lucock, M., Iveson, S., Shapiro, D. A., Iveson, M., & Hardy, G. E.  (2003).  Early sudden gains in psychotherapy under routine clinic conditions: Practice-based evidence.  Journal of Consulting and Clinical Psychology, 71, 14-21.

Tang, T. Z. & DeRubeis, R. J.  (1999).  Sudden gains and critical sessions in cognitive-behavioral therapy for depression.  Journal of Consulting and Clinical Psychology, 67, 894-904.

Tang, T. Z., DeRubeis, R. J., Beberman, R., & Pham, T.  (2005).  Cognitive changes, critical sessions, and sudden gains in cognitive-behavioral therapy for depression.  Journal of Consulting and Clinical Psychology, 73, 168-172.

Tang, T. Z., DeRubeis, R. J, Hollon, S., Amsterdam, J., & Shelton, R.  (2007).  Sudden gains in cognitive therapy of depression and depression relapse/recurrence.  Journal of Consulting and Clinical Psychology, 75, 404-408.

Weber, N. & Brewer, N.  (2004).  Confidence-accuracy calibration in absolute and relative face recognition judgments.  Journal of Experimental Psychology: Applied, 10, 156-172.

Yaniv, I., Yates, J. F., & Smith, J. E. K.  (1991).  Measure of discrimination skill in probabilistic judgment.  Psychological Bulletin, 110, 611-617.