Joint project with the University of Melbourne and Kwansei Gakuin University, with the collaboration of the Center for Applied Linguistics
Project goals This research project considers the reliability and validity of a set of measures of syntactic complexity typically used in L2 oral production research and seeks to determine how these measures relate to oral proficiency. It investigates to what extent various syntactic complexity measures are valid, useful, and reliable developmental indices at various levels of oral proficiency on a range of different tasks and across four foreign languages: German, Japanese, Spanish, and English. The following research questions are addressed:
- Which measures can be most reliably coded and how do aspects of specific languages affect intercoder reliability for the different measures?
- How well do these measures index syntactic complexity which represents a developing IL along a continuum of increasing oral proficiency? Which of these measures are effective predictors of oral proficiency at what levels? What is the relationship of the various measures to a range of proficiency levels?
- How do the different measures vary across task elicitation conditions?
Development of elicitation and test instruments
- An elicitation instrument was developed that consisted of three types of story-retelling procedures:
- A story retelling based on pictures, with an L1 story as initial auditory input (Ortega, 1999).
- A story retelling based on the 5-minute video clip “Alone and Hungry” (Klein & Perdue, 1992).
- A story telling based on pictures, divided in three subtasks prompted by high-level cognitive complexity questions (Zhang, 1987).
- In addition, the following information sources for determining language proficiency were developed or adapted from existing test instruments:
- Elicited imitation task: an elicited imitation task was developed and piloted with native speakers and L2 learners for each of the four target languages. The final items selected were 30 sentences of increasing length and complexity. The sentences were equivalent across the four languages in vocabulary difficulty, structural difficulty, and syllable length, which ranged from 7 to 19 syllables per sentence.
- Learner background information: A questionnaire was developed to collect and quantify information regarding learners’ formal instruction and naturalistic exposure to the foreign language.
- Self-report questionnaire of oral language proficiency, developed by the Center for Applied Linguistics.
- For Japanese and Spanish, all participants took the SOPI instruments developed by CAL.
All tasks and instruments were incorporated into a test booklet with instructions in English or Japanese, depending on the expected L1 background of participants, and two parallel forms of the booklet were created in order to counterbalance the order of the tasks.
Data collection
Data have been collected from four languages, all in foreign language higher education contexts. For each language, the data collected yielded a corpus of narrative productions by 40 students at two different proficiency levels, intermediate and advanced. For Japanese and English, baseline data on all tasks by native speaking college students in the target environment have been collected. The data were collected in the three research sites as follows:
University of Hawai‘i at Mānoa, US: (a) Forty L2 Japanese students enrolled in fourth semester courses or higher, to be divided into two proficiency bands according to their ACTFL ratings; (b) Forty L2 Spanish students enrolled in fourth semester courses or higher, to be divided into two proficiency bands according to their ACTFL ratings; (c) Twenty L1 English undergraduate students as ba