CRFpq Design Article Critique
Y603
Amy Kleiner

1. Background information:
Andersen, B., & Anderson, W. (1985). Client perceptions of counselors using positive and negative self-involving statements. Journal of counseling psychology, 32(3),462-465.

2. Abstract:
Several studies have compared clients' perceptions of counselors who use self-disclosing versus self-involving statements. The present study examined the differences in clients' perceptions when counselors used either positive or negative self-involving statements. Subjects included 53 female and 29 male undergraduate students enrolled in a personal adjustment course at the University of Missouri. Participants' were randomly assigned to one of two conditions and then given a transcript of a counseling session to read. After reading the transcripts, participants recorded their perceptions of the counselor's attractiveness, expertness, and trustworthiness by completing the Counselor Rater Form (CRF). Participants were also measured on their perceptions of the appropriateness of counselor behavior and their willingness to see the counselor in the future. Results indicated that counselors using positive self-involving (PSI) statements are perceived as more attractive, expert, trustworthy, and appropriate than counselors using negative self-involving (NSI) statements. Further, participants expressed a greater willingness to see PSI counselors than NSI counselors in the future. Suggestions for future research in this area are provided.

3. Null hypotheses, alpha and p-level, and sample size per group:
The authors hypothesized that participants would rate counselors using both PSI and NSI statements similarly in terms of their expertness, attractiveness, trustworthiness, and appropriateness. They also hypothesized that participants would express a similar willingness to see both PSI and NSI counselors. In other words, the authors expected that there would be no differences on client ratings of expertness, attractiveness, trustworthiness, appropriateness, and willingness to see the counselor for PSI counselors and NSI counselors. The authors did not hypothesize about the effects of gender.

Five two-way ANOVAs were conducted to test the effects of transcript type and gender on each dependent measure. Each ANOVA tested three null hypotheses:

Ho1: mrow1 (PSI) = mrow2 (NSI)
Ho1: Type of self-involving statement has no effect on client ratings of the counselor's expertness/ attractiveness/ trustworthiness/ appropriateness/ or participant's willingness to see the counselor.

Ho2: mcolumn1 (male) = mcolumn2 (female)
Ho2: Gender has no effect on client ratings of a counselor's expertness/ attractiveness/ trustworthiness/ appropriateness/ or participant's willingness to see the counselor.

Ho3: mjk - mj'k = mjk' - mj'k' for all rows and columns
Ho3: Ratings of PSI and NSI counselor's expertness/ attractiveness/ trustworthiness/ appropriateness/ or participant's willingness to see the counselor will not differ because of gender.

The alpha level for each ANOVA was not pre-stated in the article. The F-ratio for main effect of transcript type on counselor appropriateness was significant at the p<.0001 level; counselor expertness was significant at the p<.005 level; counselor attractiveness was significant at the p<.0001 level; counselor trustworthiness was significant at the p<.005 level; and willingness to see counselor was significant at the p<.01 level. The F-ratio for interaction between sex of subject and transcript type was significant at the p<.05 level.

N = 81 PSI transcript (n=41; 16 males, 25 females)
NSI transcript (n=40; 12 males, 28 females)

4. Independent and dependent variables:
There were two independent variables: 1) Transcript type (two levels- PSI statements or NSI statements), and 2) Gender of participant (two levels- Male or Female).

There were five dependent measures: 1) Counselor expertness, 2) counselor attractiveness, 3) counselor trustworthiness, 4) appropriateness of counselor's behavior, and 5) client's willingness to see counselor in the future. These variables were measured by the Counselor Rater Form and two independent scales.

5. Instrumentation:
The Counselor Rater Form (CRF) was used to measure participants' perceptions of the counselor's expertness, attractiveness, and trustworthiness. The authors reported that previous investigations have shown the CRF "to be a reliable instrument that is capable of discriminating both within and between counselors (p. 463)." They did not mention the validity of the instrument, nor did they provide specific reliability or validity statistics.
Participants were also asked to rate the counselor in terms of their perceived appropriateness and willingness to see a similar counselor. Participants informally rated their perception of the counselor's appropriateness by indicating how congruent the counselor's behavior was with the behavior the participant expected the counselor to engage in. Secondly, participants rated their willingness to see a similar counselor in the future. The authors suggested that the above two scales served to clarify the nature of the clients' perceptions of counselor competence. No information is provided in terms of the validity and reliability of these scales.

6. Procedure:
Participants included 53 female and 29 male undergraduate students enrolled in a personal adjustment course at the University of Missouri. Students participating in this study received extra credit for their involvement. Participants were randomly assigned to one of two conditions (PSI transcript or NSI transcript), and then asked to read a written transcript of a counseling session. The transcripts for the two conditions were identical except for the inclusion of six PSI or NSI statements. A self-involving statement is a statement made by the counselor that consists of a personal, present-tense response such as "I'm glad to see that you made it" (PSI) or "I can feel your emptiness and that disturbs me" (NSI). Two 30-minute group administrations were conducted. After participants read the transcript, they were asked to complete the Counselor Rater Form and to rank the counselor's appropriateness and their willingness to see a similar counselor.

7. Statistical Analysis and Conclusion:
Two-way ANOVAs were performed to determine the effects of transcript type and gender of participant on each of five dependent measures. The present study is a CRF22 Fixed Effects Design (Model I). Though the authors did not specifically state that the two independent variables were fixed, the reader can deduce this by recognizing that gender is a fixed variable and the research question requires that transcript type also be a fixed variable. Hence we can assume that MSerror (within cell) was used as the denominator to figure the F-ratios.
Gender of participants had no significant main effects on counselor ratings. PSI counselors were rated more favorably than NSI counselors on all five dependent measures. Counselors using PSI statements were perceived as being significantly more appropriate, F(1, 80)=16.4, p<.0001; more expert, F(1, 80)=8.89, p<.005; more attractive, F(1, 80)=18.40, p<.0001; and more trustworthy, F(1, 80)=9.22, p<.005 than counselors using NSI statements. Further, participants expressed greater willingness to see a PSI counselor in the future, F(1, 80)=7.22, p<.01. The authors reported that a Bonferroni correction was applied to data analysis procedures in order to control for the multiple ANOVAs, and that the findings were robust enough to withstand this correction. The only interaction that was found to be significant between gender of participant and transcript type, F(1, 80)=5.79, p<.05, was related to the dependent measure of willingness to see a similar counselor. Males expressed a significantly greater willingness to see PSI counselors whereas females demonstrated less pronounced differences on this dimension.
The findings presented in the current study indicate that PSI counselors are rated more favorably than NSI counselors on all five dependent measures. The authors assert that the current findings offer partial support for previous research findings that have compared clients' ratings of counselor expertness, attractiveness, and trustworthiness when the counselors used either PSI or NSI statements. They also mention that their findings differ from other research that has found no differences on these dimensions. The researchers caution against concluding that PSI counselors are more effective than NSI counselors, and they suggest that a reexamination of previous studies comparing positive and negative statements is important before any conclusions can be made about the general effects of these counselor behaviors.

8. My comments about the present study:
The design chosen for this study was appropriate, and data analysis appears to have been performed correctly. No post-hoc procedures were performed, which is appropriate for a design with only two levels on each independent variable. However, I have a few recommendations about how the study and write-up could be improved:
1) The researchers made no hypotheses about the effects of gender on participant ratings of counselor behavior. Additionally, they provide no explanation for why gender was an essential variable to consider in this research situation. Thus one-way ANOVAs may have been sufficient to test the effects of transcript type on participant ratings. On the other hand, if the researchers were concerned that gender may be a nuisance variable, a block design may have been more appropriate.

2) Likewise, due to the absence of a hypothesis about gender, the research questions of interest to the authors were not clear. The reader is left confused about why gender was included in this study and what research questions the authors were interested in.

3) The authors mentioned that the CRF has been found to be a reliable instrument. However, they do not mention anything about the validity of the instrument, nor do they provide specific reliability or validity coefficients. Further, nothing is mentioned about the reliability or validity of the two rating scales used to measure participants' perceptions of counselor appropriateness and participants' willingness to see a similar counselor in the future.

4) Additionally, it is not clear how the two independent scales were formatted or administered. This information is necessary for enabling other interested researchers to replicate this study.

5) The authors mention that they utilized a Bonferroni correction to control for the multiple ANOVAs. However, in discussing the one significant interaction between transcript type and gender, the authors reported F(1, 80)=5.79, p<.05. If the Bonferroni correction was used properly in each of the five ANOVAs, it is not statistically possible for the interaction to be significant at p<.05 because an alpha set at the .05 level would be divided evenly between each separate ANOVA. Hence this is either a misprint, the authors misread the F table, or the authors did not properly perform the Bonferroni correction as stated.

6) Finally, the researchers do not discuss possible shortcomings of the present study. While the absence of such a discussion does not cause this article to be a failure, the write-up of the study would have been strengthened and the researchers would have appeared to be more legitimate had they provided their own views about how the study could have been improved.