Ramona MOLDOVAN
Romanian Psychological Board, Cluj-Napoca, Romania
Abstract
Apparently, it is neither the type nor the length of education or psychotherapists’ clinical experience that are responsible for their performance. The present study is part of a larger research which aims at identifying the variables (emotions, cognitions, behaviors, personality traits) that predict psychotherapists’ performance in therapy. The main objective of this study is to assess psychotherapists’ personal variables and compare them with those identified in the general population. The study included 126 psychotherapists under supervision (psychologists and psychiatrists) in two Cognitive Behavioral Therapy training programs (Cognitive Therapy and Rational Emotive Behavior Therapy). During the final stage of the training program, the psychotherapists under supervision filled in self-reported check-lists; psychotherapists’ personal variables were analyzed and compared with those of 122 participants from the general population. The possible implications of the results in predicting psychotherapists’ performance and their impact on psychotherapy training programs are also taken into consideration.
Keywords: psychotherapists’ performance, cognitive behavioral psychotherapy, psychotherapists’ personal variables.
Pages: 129-140
INTRODUCTORY REMARKS
Recent studies (Christensen & Jacobson, 1994; Dawes, 1994) indicate that there are no significant differences between professionals and paraprofessionals in psychopathology. On the other hand, there are no significant differences between professionals with long or brief training when the efficiency of treating emotional disorders is taken into consideration. Moreover, clinical experience doesn’t seem to improve psychotherapists’ performance (Garb, 1998). Inevitably, the existing literature leads us to an honest reflection: if, at least not apparently, neither the type nor the length of studies or the clinical experience is responsible for the clinical performance, than what is? Systematic research that explicitly approaches psychotherapists’ performance predictors is not well-articulated yet. Also, there are no systematic studies concerning the relationship between psychotherapists’ psychopathology and their performance. The prevalence and incidence of psychological disorders seem similar among psychopathologists and in the general population. Is this fact relevant? The psychotherapists referred to in several studies (Deacon et al., 2000; Pope & Tabachnick, 1994) indicated marital problems (66%), depressive episodes (45%), anxiety disorders (31%), problems with their own children (35%), divorce (29%), life stress (27%), sexual difficulties (12.6%), countertransference (14.4%) etc. Is psychotherapists’ performance affected by this? Is it important for psychotherapists to be not only skilled but also psychologically healthy in order to assist their clients?
Most of the research concerning psychotherapists’ performance has focused on such matters as demographic variables (e.g., age, sex, ethnicity or religion), psychotherapists’ theoretical orientation, and personality traits or aspects characteristics of their experience (e.g. years of practice). The results are not conclusive; most of the differences identified between psychotherapists are either not significant or significant only under strict circumstances (Beck, 1988; Beutler et al., 1994; Huppert et al., 2001; Greenspan & Kulish, 1985).
Research addressing the questions stated above might allow for the identification of some “key” variables responsible for psychotherapists’ performance, variables that might suggest the need for adapting the psychotherapists training programs, by including or developing training levels that address specific aspects that predict psychotherapists’ performance.
There is an unanimous consensus over the fact that psychotherapists’ performance can be explained by several factors; but the precise factors, their nature and impact is still unknown (see Najavits, 1997).
Most of research indicated that the age of psychotherapists is not significantly related to therapy outcome (Beck, 1988; Greenspan & Kulish, 1985; Beutler et al., 1994). Demographic variables and aspects of psychotherapists’ professional background (such as age, sex, theoretical orientation, and years of practice) are obviously not enough. Such studies indicated that the psychotherapists’ age is not significantly related to therapy outcome (Beck, 1988; Greenspan & Kulish; 1985, Beutler et al., 1994). Research on the impact of psychotherapists’ sex on therapy outcome is not conclusive. A literature review (see Huppert et al., 2001) revealed the existence of three hypotheses: 1. Women psychotherapists are more efficient than man psychotherapists; 2. Psychotherapy outcome is better when psychotherapist and client are of the same sex; 3. Psychotherapy outcome is not influenced by psychotherapists’ sex. Research also concentrates on psychotherapists’ theoretical orientation; however, the theoretical orientation is a very modest predictor of the therapeutic outcome (see Najavits, 1997). Almost stereotypic, psychotherapists’ experience is one of the most frequent variables mentioned when talking about psychotherapy outcome (Huppert et al., 2001). However, clinical experience has been proved to be a very modest predictor (if at all) of therapy outcome. Additionally, there hasn’t been reached a consensus on how should clinical experience be evaluated (e.g., number of years of clinical practice, number of years of psychotherapy practice, number of patients, number of years under supervision etc.) or whether psychotherapists’ experience is mediated by other factors (such as therapeutic alliance), so most research is rather inconsistent (Huppert et al., 2001).
OBJECTIVES
Our general objective, established for a rather broad area of research, aims to investigate the relationship between psychotherapists’ personal variables (on cognitive, emotional, behavioral, personality level) and their performance in therapy. We will concentrate not only on the evaluation of psychotherapists’ personal variables or on their performance, but we will attempt to clarify the relationship between them – to identify those variables which predict, mediate or moderate psychotherapists’ performance.
Our main objective for the present study was to compare psychotherapists and general population on the level of depression, state or trait anxiety, irrationality, self-efficacy, distress, dysfunctional attitudes, unconditional self acceptance, self esteem and optimism are compared.
METHOD
Participants
The study included 126 trainees, psychotherapists under supervision in two Cognitive Behavioral Psychotherapy training programs (CBT), Rational Emotive Behavioral Psychotherapy (REBT) and Cognitive Psychotherapy (CT) offered by The International Institute for the Advanced Studies of Psychotherapy and Applied Mental Health, Cluj-Napoca, Romania in collaboration with Albert Ellis Institute, New York, USA, Aaron Beck Institute, New York and The Department of Psychology, Babes-Bolyai University, Cluj-Napoca, Romania (21 males and 105 females, mean age 31.7).
The research included also 122 participants from the general population (37 males and 85 females, mean age 34.8).
Procedure
In the first part of the study we evaluated psychotherapists’ personal variables, using an assessment battery including measures of cognitions, emotions, behaviors and personality traits. At the beginning of the supervised clinical practice stage, each psychotherapist received all the self-reported instruments used in this study; the scales were handed in after three days, at the end of the last meeting of the training.
After this training module, the psychotherapists start their supervised clinical practice. Each psychotherapist will have 2 supervisors and will have at least 10 clients covering eight different forms of psychological problems (for details, see www.psychotherapy.ro). In the next phase of this study, each therapist will be monitored until the completion of their training and supervision. In order to assess performance as accurate as possible, psychotherapists’ activity will be evaluated after every solved case, from a number of perspectives: the supervisors’ evaluation, carried out with The Evaluation Scale in Cognitive Therapy, (for details, see Macavei, 2002); the clients’ evaluation – the level of satisfaction with therapy and the quality of working alliance from the patients’ perspective, carried out with the Working Alliance Inventory; the psychotherapists’ self-evaluation, carried out with the Evaluation Scale in Cognitive Therapy (Macavei, 2002) and the quality of working alliance form the psychotherapists’ perspective, carried out with the Working Alliance Inventory; the comparison of the patient’s scores in the first and the last session assessed with the adequate scales for the diagnostic and symptomatology identified, and the patients’ relapse rate (measured at six months and one year after treatment).
Measures
Attitudes and Beliefs Scale, the Short form (DiGiuseppe et al., 1988, Macavei, 2002).
The Attitudes and Beliefs Scale, the Short form (ABSs) was used as a measure for irrational beliefs. The ABSs is a self-reported instrument consisting of 8 items measuring the four irrational beliefs central to REBT (demandingness, awfulizing, self-downing and low frustration tolerance) and their rational alternatives. The scale allows for the assessment of the four irrational beliefs central to REBT and of the global score of rationality and irrationality. ABSs has very good psychometric properties for both the American and Romanian population. In this study we used the scores indicating the global score of irrationality.
Beck Depression Inventory (Beck et al., 1971)
The Beck Depression Inventory (BDI) was used as a measure of depressive symptoms. BDI is a 21 items self-reported measure of depression with very good psychometric properties.
Life Orientation Test (Scheier & Carver, 1985)
The Life Orientation Test (LOT) was used as a measure of dispositional optimism and pessimism. Among the 12 items included in the scale, four are positively worded, four are negatively worded and four of them are filler items (for an updated review of the literature, see Scheier & Carver, 1995).
Self Esteem Scale (Rosenberg, 1965)
The Rosenberg Self Esteem Scale (SS) is the most classic measure of self-esteem. The scale is a 10-item Likert scale with items scored on a 4 point scale, from strong agreement to strong disagreement. High scores on the SS indicate a high level of self-esteem.
Self Efficacy Scale (Schwarzer & Jerusalem, 1993)
The Generalized Self-Efficacy Scale has been developed by Jerusalem and Schwarzer in 1981. The scale used in the present study is an adapted form of the original. The original 10 items were rephrased by a group of experts so as to reflect the positive aspects of psychotherapists’ efficacy. A high score indicates a high self-efficacy level. The psychometric properties of the scale (Cronbach coefficient alpha between .75 and .90) allow for very good testing conditions.
Social Desirability Scale (Stoeber, 1999, 2001)
The 17 original items of the Social Desirability Scale consist of descriptive statements of socially desirable behaviors. The present version of the scale (Stoeber, 2001) consists only of 16 items, as the item referring to drug consumption was omitted. There are seven items reversely worded; a high score suggests high level of social desirability.
Dysfunctional Attitudes Scale (Weissman & Beck, 1978)
The instrument was designed to identify the statements, which generally are the core of idiosyncratic thinking associated with depression. Cognitive distortions and perfectionist attitudes related with depression are therefore assessed. Based on a 7-point Likert scale, the Dysfunctional Attitudes Scale (DAS) indicates how characteristic dysfunctional attitudes are for the respondent: a high score points towards a high level of dysfunctional attitudes. The psychometric properties of the scales reveal an alpha Cronbach coefficient of .89.
Unconditional Self Acceptance (Chamberlain & Haaga, 2001)
The Unconditional Self Acceptance scale (USA) consists of statements taken from the unconditional acceptance philosophy, which has strong connections with the theory and philosophy of REBT. The instrument is made up of 20 items rated on a 7-point Likert scale. A high score on this questionnaire indicates a high level of unconditional acceptance.
Profile of Mood States (Shacham, 1983)
The Profile of Mood States (POMS) is aimed at assessing mood states. The scale consists of five scales which assess negative mood states (anxiety, depression, anger, fatigue and confusion) and one scale, which assesses positive mood states (vigor/energy). The 47 items are grouped into six subscales: Tension-Anxiety, Depression-Dejection, Anger-Hostility, Vigor-Activity, Fatigue-Inertia, Confusion-Bewilderment. The scores from each of the above scales can be interpreted separately or they can be summed up in a total distress score. In the present study we used the total score of distress. High scores on POMS reflect a high level of distress. Internal consistency (.90) and test-retest reliability (with values ranging between .65 and .74) guarantee the scale’s good psychometric properties .
State Trait Anxiety Inventory (Spielberger et al., 1983)
The State Trait Anxiety Inventory (STAI) consists of two self-assessment scales which measure two separate concepts related to anxiety: the state of anxiety (STAI X1) and anxiety as trait (STAI X2). STAI X2 is made up of 20 descriptive statements based on which individuals express the way they feel in general. Anxiety as trait refers to relatively stable individual differences manifested in the tendency to react to situations perceived as threatening with an increase in state anxiety. STAI X1 is made up of 20 descriptive statements but the participants are asked to indicate the way they feel at a certain time; in the present study the participants are asked to specify the way they felt in the past few days.
RESULTS
According to the objective formulated, this research was aimed, in this preliminary first phase, to evaluate several personal variables of psychotherapists. As indicated in the previous section, we evaluated the level of depression, anxiety, distress, irrationality, dysfunctional attitudes, unconditional self acceptance, self-efficacy, self-esteem, optimism and social desirability. Table 1 indicates the descriptive statistics (mean values and standard deviation) for each of the variables measured.
The BDI scores indicate the level of depression, more specifically attitudes and symptoms of depression. The mean value of depression for the psychotherapists group is 3.32 (SD=3.87) and for the general population group 7.93 (SD=7.90). STAI X1 scores indicate the level of anxiety symptoms the respondents experienced in the last few days before filling in the scale. The anxiety state mean level for the psychotherapists group is 31.96 (SD=6.57) and for the general population group the mean reaches 38.26 (SD=11.21). STAI X2 indicates the respondents’ level of trait anxiety (stable tendencies to react to perceived threatening situations with an increase in the anxiety level).
Table 1. Descriptive statistics for the dependent measures.
Dependent measures |
Group |
M |
SD |
Depression |
P |
3.32 |
3.87 |
G |
7.92 |
7.90 |
|
State anxiety |
P |
31.96 |
6.57 |
G |
38.26 |
11.21 |
|
Trait anxiety |
P |
35.40 |
7.62 |
G |
41.27 |
8.75 |
|
Irrationality |
P |
4.60 |
3.64 |
G |
11.50 |
3.46 |
|
Self-efficacy |
P |
31.29 |
4.83 |
G |
31.40 |
4.50 |
|
Distress |
P |
57.81 |
16.68 |
G |
67.36 |
24.40 |
|
Dysfunctional attitudes |
P |
91.16 |
25.51 |
G |
131.53 |
30.97 |
|
Unconditional self-acceptance |
P |
98.02 |
14.88 |
G |
84.94 |
13.37 |
|
Self-esteem |
P |
33.67 |
3.95 |
G |
31.00 |
4.85 |
|
Optimism |
P |
32.46 |
4.66 |
G |
28.96 |
5.41 |
|
Social desirability |
P |
6.80 |
3.63 |
G |
8.85 |
3.11 |
P=psychotherapists sample
G=general population sample
M=mean
SD=standard deviation
The anxiety trait mean level for the psychotherapists group is 35.40 (SD=7.62) and 41.27 (SD=8.75) for the general population group. We used the global score to estimate the level of irrationality. The mean for the psychotherapists group is 4.60 (SD=3.64) and for the general population group 11.10 (SD=3.46). Specific self-efficacy mean scores for the psychotherapists group is 31.29 (SD=4.83) and 31.40 (SD=4.50) for the general population group. POMS scores indicate the level of distress (anxiety, depression, fatigue, anger, confusion). The mean level of distress for psychotherapists reaches 57.81 (SD=16.68) and for the general population 67.36 (SD=24.40). The mean levels of dysfunctional attitudes for the psychotherapists and for the general population groups are 91.16 (SD=25.51) and 131.53 (SD=30.97). The SS scores indicate the level of self esteem; the mean scores for the psychotherapists group is 33.67 (SD=3.95) and for the general population group 31.00 (SD=4.85). The level of optimism is indicated by the LOT scores; the mean value for the psychotherapists group is 32.46 (SD=4.66) and for the general population group 28.96 (SD=5.41). The last self-reported measure used in this research evaluates the level of social desirability. SDS mean score for the psychotherapists group is 6.80 (SD=3.63) and 8.85 (SD=3.11) for the general population group.
To evaluate if and the degree in which the two groups differ, we decided to compare the means of the two groups. The most adequate technique to compute the difference between the means of two groups is the t test for independent groups. Table 2 includes the t values, the p values and size effect values for each of the dependent measures.
Table 2. Comparisons between psychotherapists and the general population on the dependent measures.
|
T test |
Sig. |
Size effect |
BDI | T (246) = 5.78 | p<.05 | 0.34 |
STAI X1 | T (245) = 5.37 | p<.05 | 0.32 |
STAI X2 | T (218) = 5.28 | p<.05 | 0.33 |
ABSs | T (244) = 15.22 | p<.05 | 0.69 |
SES | T (238) = .18 | p<.05 | 0.01 |
POMS | T (246) = 3.58 | p<.05 | 0.22 |
DAS | T (245) = 11.16 | p<.05 | 0.58 |
USA | T (246) = 7.28 | p<.05 | 0.42 |
SS | T (243) = 4.71 | p<.05 | 0.28 |
LOT | T (246) = 5.43 | p<.05 | 0.32 |
SDS | T (244) = 4.74 | p<.05 | 0.29 |
Significant differences (p<.05) were identified between mean scores for every dependent measure. More specifically, mean scores are significantly lower for the psychotherapists’ group than for the general population group when comparing the level of depression, state and trait anxiety, irrationality, distress, specific self-efficacy and dysfunctional attitudes. Psychotherapists’ mean scores of unconditional self acceptance, self-esteem and optimism are significantly higher than those of the general population. The social desirability mean scores also indicate significant differences between the two groups: the social desirability level is significantly lower for the psychotherapists group than for the general population group.
Apparently, the results are greatly in favor of psychotherapists – they think more rational, they experience less negative emotions, they are more optimistic, they accept themselves unconditionally and they have a higher self esteem than the general population. Psychotherapists seem to be much happier!
When we assessed the level of specific self-efficacy (regarding profession), we didn’t expect to find any significant differences between the two groups. The results indicate, contrary to what we expected, a lower self efficacy regarding profession for psychotherapists than for individuals from the general population group.
The social desirability scale we included in this research was meant to identify possible biases of desirability in respondents’ answers. Taking into account the counter intuitive results presented above, we would have expected either not to find any significant differences between the two groups, or, if such differences are present, to find that the psychotherapists had a higher level of social desirability. Our hypothesis was not confirmed this time either. There are significant differences between the two groups – individuals from the general population seem to have higher social desirability than psychotherapists.
Although the results indicate significant differences between the two groups, the significance value indicates only that there is a slight possibility to obtain these results if the null hypothesis is true; consequently, it indicates the degree in which the differences identified are similar for the population. From this perspective, it is highly recommended to compute additionally the size effect (Cohen’s d), which indicates how large the differences identified are (see Table 2).
Although all differences between the two groups were statistically significant, most of the differences identified have a small or medium size effect. The largest size effect identified is the one referring to the difference between the mean scores indicating the level of irrationality in the two groups. Other medium size effects identified to support the difference between the mean scores of psychotherapists’ and general populations’ are those for dysfunctional attitudes and unconditional self acceptance.
In other words, it seems that although the dependent variables differ significantly between the two groups, the size effect values suggest a cautious interpretation of these results.
Taking into account the size effect and the p values we can interpret the differences between psychotherapists and general population as significant but with not much practical value. For the situations in which the size effect is medium or high, we can conclude that it is not likely that the effect appeared randomly (Fan, 2001).
CONCLUSIONS AND DISCUSSION
Although all the differences identified between the two groups were statistically significant, the size effect was clinical significant only for few of them. In other words we can conclude that psychotherapists and the participants from the general population do not differ significantly in what concerns the level of depression, state or trait anxiety, specific self-efficacy, distress, self-esteem and optimism (the size effect varies between 0.01 and 0.42). Regarding the level of unconditional self acceptance and dysfunctional attitudes, there are more important differences between the two groups – medium size effect values (0.42 and 0.58) – . The major difference between the two groups concerns the level of irrational beliefs; here, not only that the difference is statistically significant, but the size effect value is more important (0.69).
The results are concordant with the existing literature (David, 2003; Deacon et al., 2000; Pope & Tabachnick, 1994). It is not surprising that there are no major differences between psychotherapists and the general population, especially since they are at the beginning of their supervision period and they haven’t covered yet the module of self-development and self-optimization. It is very likely that during this module cognitions and consequently negative emotions will change, there where some of the scales indicated high scores (e.g., dysfunctional thoughts, dysfunctional emotions).
When the differences between the two groups are statistically significant and the size effect values are larger, there may be at least two possible and pertinent interpretations: either psychotherapists have, for real, a lower level of irrationality and a higher level of unconditional self-acceptance than the general population, or, at least on a declarative basis, their answers were strongly influenced by the principles that fundament CBT practice, principles the trainees know from previous theoretical modules. More to the point, from all the variables assessed, irrationality and unconditional self-acceptance are most related with the theory and practice of CBT. As we anticipated above, the differences noted may be due to the training program where psychotherapists are under supervision.
Obviously, and especially for the psychotherapists included in the study, all results should be interpreted individually. For this matter, the supervisors received the self-reported measures for each psychotherapist they have under supervision and they will, if considered useful, be used during the module of self-development and self-optimization.
Before initiating the research, we expected social desirability to be one of the obstacles in obtaining accurate results. Additionally, knowing that if familiar with several scales, some of the psychotherapists could be psychologically contaminated by the knowledge they have (e.g., what the scales assesses), we tried to avoid possible biases by the instructions the psychotherapists received when given the reported measures to fill in. With the agreement of the president of The International Institute for the Advanced Studies of Psychotherapy and Applied Mental Health offering the two training programs in CBT, psychotherapists were presented the scales as being part of the module of self-development and self-optimization; the instructions also emphasized the fact that the scores will not be considered as evaluation or inclusion criteria for the training program, but will be used in he module of self-development and self-optimization, with their supervisor. The fact that the evaluation took place during a training program with high relevance for them (financial, professional, personal) might have also had an impact on the results, in a way that they might have been highly motivated and might have perceived “unfavorable” results as a high risk for their goals.
Another possible explanation for some of the results may be due to the fact that, although psychotherapists under supervision in CBT, some of them might have had training and experience in other forms of psychotherapy or in psychiatric environment. This might be responsible for the specific self-efficacy scores (highly correlated with the years of practice) or the familiarity with some of the scales. Also, psychotherapists with experience in other training programs might have covered already a module of self-development and self-optimization, which could have had a clear impact on the results.
The existing literature and the results of the present study suggest some very interesting and high impact research areas. Interviews or Q sort methodologies with experts in psychotherapy who should offer explanations, conceptualizations or their own representation of the therapeutic process, the evaluation of psychotherapists’ expectations about performance, the impact (if any) of psychotherapists’ psychopathology on their performance are just some of the very intriguing directions in this research area. The assumption that certain personal variables of psychotherapists could predict their performance is not new. It is a well known fact that the efficiency of psychotherapy has not a unique cause, but it is the result of many contributing factors: psychotherapeutic alliance (30%), placebo effect (15%), patients’ personal factors (40%) and psychotherapeutic techniques (15%) (Lambert, 2003, cited in David, 2003). Identifying the psychotherapists’ variables that influence or interact with these factors (e.g., therapeutic alliance, placebo) and that explain or predict the psychotherapeutical outcome, might contribute to the emergence of validated psychotherapy programs.
Acknowledgement
The author would like to acknowledge Dr. Daniel David, the president of The International Institute for the Advanced Studies of Psychotherapy and Applied Mental Health, for his consent to use some of the Institute’s materials and for his useful comments and assistance during the research.
REFERENCES
- Ackerman, S.J., & Hilsenroth, M.J. (2003). A review of therapists characteristics and techniques positively impacting the therapeutic alliance. Clinical Psychology Review. 23, 1-33.
- Deacon, S.A., Kirkpatrick, D.R, Wetchler, J.L., & Niedner, D. (1999). Marriage and family therapists’ problems and utilization of personal therapy. The American Journal of Family Therapy, 27:73-93.
- David, D., Holdevici, I., Szamoskozi, Ş., & Băban, A. (2000). Psihoterapie şi hipnoterapie cognitiv-comportamentală. Intervenţie cognitiv-comportamentală în tulburări psihice, psihosomatice şi optimizare umană. Editura Risoprint, Cluj-Napca.
- David, D., Schnur, J., & Belloiu, A. (2002). Another Search for the “Hot” Cognitions: Appraisal, Irrational Beliefs, Attributions, and their Relation to Emotion. Journal of Rational-Emotive & Cognitive-Behavior Therapy, Vol. 20, No. 2, pp.93-131.
- David, D. (2003). Castele de nisip. Editura Tritonic, Cluj-Napoca.
- Fan, X. (2001). Statistical significance and effect size in educational research: two sides of a coin. The journal of Educational Research, (94)5, 275-282.
- Guinee, J.P. (2000). The reporting of therapist sample data in the Journal of Counseling Psychology, Journal of Counseling Psychology, vol. 47, nr. 2, 266-270.
- Huppert, J. D., Gorman, J.M., Bufka, L.F., Barlow, D.H., & Shear, M.K. (2000). Therapist, Therapist Variables and Cognitive – Behavioral Therapy in a Multicenter Trial for Panic Disorder, Journal of Consulting and Clinical Psychology, vol. 69, no. 5, 747-755.
- Macavei, B. (2002). Scala de atitudini şi convingeri II (ABSII) – Date preliminare pentru populaţia de limbã romanã. Romanian Journal of Cognitive and Behavioral Psychotherapies, vol.2, no.2, 105-122.
- Najavits, L.M. (1997). Psychotherapists’ Implicit Theories of Therapy. Journal of Psychotherapy Integration, 1997; volume 7, pages 1-16.
- Orlinsky, D.E. (2005). The Master Therapist: Ideal Character or Clinical Fiction? Comments and Questions on Jennings and Skovholt’s “The Cognitive, Emotional, and Relational Characteristics of Master Therapists”.
- Orlinsky, D.E. (2001). Towards an Empirically Grounded Model of Psychotherapy Training: Four Thousand Therapists Rate Influences on Their Development. Australian Psychologist, vol. 36, no. 2.
- Pope, K.S., & Tabachnick, B.G. (1994). Therapists as patients: A national survey of psychologists’ experiences, problems, beliefs. Professional Psychology: Research and Practice, 25(3), 247-258.
- Pope, K.S., & Tabachnick, B.G, (1994). Therapists’ Anger, Hate, Fear, and Sexual Feelings: National Survey of Therapist Responses, Client Characteristics, Critical Events, Formal Complaints, and Training, Professional Psychology: Research and Practice, 24(2), 142-152.Scheier, M.F., & Carver, C.S. (1992). Effects of optimism on psychological and physical well-being: Theoretical overview and empirical update. Cognitive Therapy and Research, 16, 201-228.
- Stoeber, J. (2001). The Social Desirability Scale-17 (SDS-17): Convergent Validity, Discriminant Validity, and Relationship with Age, European Journal of Psychological Assessment, Vol. 17, Issue 3, 1015-5759.
- Vasco, A.B., & Dryden, W. (1994). The development of psychotherapists’ theoretical orientation and clinical practice. British Journal of Guidance & Counseling, Vol. 22, Issue 3.