|The Oral Interview
The Oral Interview is administered in order to determine whether international candidates for AI positions at Indiana University have a minimally acceptable level of general oral proficiency in English. More specifically, the test aims to assess the "readiness" of AI candidates to conduct classes in English, a matter which entails using English both to give lectures and to interact with students, many of whom may not have had previous contact with nonnative English speakers, and hence may not be accustomed to "accented" speech. The interview provides candidates with the opportunity to demonstrate that they can communicate freely in English. Although candidates are not expected to communicate in English exactly like native speakers, it is expected that their proficiency level of spoken English will not hinder undergraduate student comprehension of any given subject/lecture.
The pool of interviewers consists of approximately 20 instructors from the Intensive English Program in the Department of Second Language Studies. All have advanced degrees in linguistics and/or second language instruction as well as many years of experience in the field of Teaching English to Speakers of Other Languages (TESOL) and testing. Their training as interviewers, moreover, is on-going, with training and calibration sessions conducted quarterly, to coincide with each TEPAIC. Apprentice interviewers, besides undergoing the training, are paired with experienced mentors for their first two interviewing sessions. The training includes: a review of interviewing skills, including types of questions and methods of employing "probes" i.e. questions which progressively probe more complex topics and demand more sophisticated language skills); discussion of criteria employed in rater judgments; and calibration (i.e. focused adjustment and standardization of raters' employment of the measuring instrument) using previous interviews to confirm that all interviewers have internalized the rating criteria and that they apply the criteria in the same way.
Two raters (one male and one female), whenever possible) conduct a fifteen-minute interview with each international graduate student. During an initial introduction, candidates are asked to show a picture identification and give their current contact information (address, student identification number, telephone, department, etc.). The raters also mention that the session is not a "test" per se, as there are no correct or incorrect answers. Rather, it is a conversational setting designed to give the candidates the optimum opportunity to express themselves in a wide variety of topic areas. Since candidates are often extremely nervous, interviewers attempt to establish a casual atmosphere and a modicum of rapport.
After the initial procedures, the interviewers may proceed in a number of directions, often dictated by the candidates' responses. At some point, there is usually a discussion of the candidates' experience with various cross-cultural issues, ranging from comparing the climate and living conditions of their own country with those in the U.S. to comparing the educational system or student conduct in each country.
A second common thread of conversation is a discussion of the candidates' fields of study, giving them the opportunity to present some simple technical information to the raters, who, as a relatively naive audience, need simple, coherent, yet adequately detailed explanation in order to understand the specialized topics. (See Rationale below.)
Besides these two major conversational threads, candidates are given probes which require that they project themselves into both future and hypothetical situations, each of which require specific grammatical devices to indicate the difference between real and possible situations.
If interviewers notice specific problems with a particular area of competence, they may probe this topic further to discern whether this problem will be an impediment to understanding for a prospective undergraduate audience.
In an English proficiency exam for instructors of English-speaking undergraduate students, an oral proficiency measure is needed, because AI's, in addition to lecturing, must be able to comprehend student questions and respond to them with intelligible explanations. Secondly, a measure of candidates' spoken English, separate from a written examination, is needed, because of results from an examination that emphasizes written English (c.f. Dandonoli and Hennig, 1990; ETS, 1992). This is not surprising, given the research has demonstrated that speaking and listening are separate skills (Bachman, 1981; Dandonoli and Henning, 1990).
A measure of general English proficiency, as opposed to specific or special purpose English (e.g. English for chemists, or English for mathematicians) is included in the battery for several reasons. First, a field-specific approach would require separate examinations for each area of specialization. This would be very difficult because it would require the involvement of content-area specialists who were quite knowledgeable about language use and who had extensive experience in applied linguistics and in language testing. The participation of such uniquely qualified individuals would be needed both for input in the test development stages and for interviewing and rating purposes at every single administration of the test. Furthermore, the status of such specific-purpose tests is still controversial in the language testing profession. However, the TEPAIC Oral Interview does provide opportunities for candidates to discuss topics on general themes within their specialization, in the relevant context of explaining a general concept in their field to a novice learner. Such explanations typically do not go into technical depth required in true specific-purpose tests, and hence do not require expert judges in each content area to rate the performance. Furthermore, since the interviewers are not specialists in the candidates' fields, a candidate would not be marked down for providing inaccurate technical information in responding to such a question. The purpose of this type of question is simply to provide a thematically relevant context in which the candidate may speak to demonstrate her or his English language proficiency.
A second reason that general oral proficiency (as opposed to field-specific language ability) is targeted by the AI test is that in many aspects of their teaching, teachers need to use general English that does not relate directly to the content of lectures. For example, they need to be able to explain what homework is to be done, when it is due, how it can be submitted, what chapters to study for a test, how much each section of the test counts toward the students' grade, and so forth. Teachers also need to be able to comprehend questions raised by students, and students typically use general or colloquial language, as opposed to, or at least in addition to, formal or special purpose language specific to the subject being taught.
For the TEPAIC interview, a Holistic Rating System is used, as in well-known oral interviews such as the American Council on the Teaching of Foreign Languages Oral Proficiency Interview (ACTFLOPI) and the Foreign Service Institute (FSI) interview. Separate evaluation forms help examiners remember specific candidates after the interview. When the candidates finish the interview and leave the room, the interviewers complete the evaluation form. There is little if any discussion necessary and no "coming to terms or agreement". Both interviewers must individually and separately decide whether or not the candidate successfully completed the interview.
Holistic evaluation is generally used in language testing for open-ended production skills, such as composition grading and oral interviews; otherwise, given the constraints of time and material, the number of areas which discrete-item scoring would need to consider are so numerous as to make such scoring untenable. This is particularly true with oral interviews since the evaluation is being done as candidates proceed through the interview. As Buck (cited in Cohen, 1994) notes "language comprehension is, by its very nature, multidimensional, and testing it only increases the number of dimensions involved in the process. In such a case it is obviously not possible to say what each item measures" (p. 261).
The Oral Interview for the TEPAIC is adapted from several common models of Oral Proficiency Interview (OPI). It is a test of oral proficiency, as opposed to a placement, achievement, or diagnostic test. Thus, it is designed to give a measure of the overall communicative competence of the interviewee, not to test retention and comprehension of specific material (achievement tests), to assign speakers to a particular level of instruction (placement tests), or to give impressions of specific deficiencies in discrete aspects of oral production (diagnostic tests). One of the best known OPIs, the ACTFL, on which the TEPAIC Oral Interview is loosely based, is mainly a placement test, placing speakers into one of 10 levels of competence (3 intermediate levels each for novice, intermediate, and advanced, with an added category superior). In the IU TEPAIC, however, there is no need for such fine distinctions across all levels, since we are only concerned with ascertaining that candidates have the necessary communicative competence to give lectures, lead discussions, answer questions spontaneously, and negotiate communicative breakdowns. Therefore, all candidates for the TEPAIC interview should be at or above ACTFL Advanced Level. A detailed explanation of the competence required for each of the ACTFL proficiency levels is available at www.actfl.org.
Immediately after the interview, the two raters independently assign a holistic rating (high pass, pass, borderline pass, borderline fail, fail) and then compare their findings. In the majority of cases, they agree on the holistic rating. In the case of disagreement, there is no negotiation; each rater simply assigns a separate score, and the lower rating is given for the overall evaluation.
Having given the rating, interviewers then assess problem areas of individual language skills by means of the analytical assessment form. This assessment has no relation to the holistic score already assigned, and serves only to give the candidate additional information about possibly areas in need of training. It must be stressed that candidates rarely fail on the basis of one particular flaw in language use; usually there are problems in many areas, some of which, such as a lack of sophistication in vocabulary or grammar, are difficult to describe, precisely because they are indicated by the absence of more sophisticated vocabulary or grammar rather than the presence of any inappropriate elements.
The categories used on the analytical instrument are:
1. Aural Proficiency (Comprehension, Appropriate Response)
2. Pronunciation (Consonants, Vowels, Intonation, Word Stress, Enunciation)
3. Language (Grammar, Vocabulary)
4. Communicative Strategies (Self-correction, Appropriate questions, Communicative Initiative, Rephrasing, Awareness of Listener)
5. Fluency (Speed, Smoothness, Sentence stress)
The holistic evaluation groups the candidates into one of five levels: