James Dean Brown Thom Hudson
The goal of this project is to develop computer-based tests (CBTs) for less commonly taught languages, with a focus on the languages of East and Southeast Asia. These CBTs will be designed primarily for operational testing purposes (proficiency, placement, and self-assessment) but will also be made available for research purposes. An innovative aspect of the project is the creation of interfaces for some or all of the tests according to World Wide Web standards. Web-based tests (WBTs), written in HTML format, that allow the administration and scoring of tests via the internet, greatly increase flexibility, applicability to multiple system platforms, and accessibility to users, while keeping costs low.
During the first year of the funding cycle (1999-2000), project staff will expand expertise in the use of the most advanced CBT development and administration software currently available, as well as statistical analysis software for item response theory analysis using small samples. (Small sample size is a perennial problem in the development of reliable tests for the less commonly taught languages, which typically have only a few hundred to a few thousand students nationally at any given time.) We will also develop expertise in adapting CBTs to the Web. English language prototype CBTs and WBTs will be developed for various skill areas (e.g., grammar knowledge, vocabulary, reading ability, and academic listening, but specifically excluding speaking ability) in the ACTFL ranges novice to superior. At the end of this stage, we will write up a complete report of what is known about CBTs and how WBTs can be used to enrich and enhance computer-based testing.
Official links will be formalized with the national associations for the teaching of each of the languages of East and Southeast Asia. Each of these organizations will be asked to assess the testing needs for that language (under NFLRC guidance), including an assessment of the need for various types of tests in computer format and problems and solutions associated with scripts and fonts, and to nominate a minimum of computer-literate, university-level language instructors for each language to participate in test development.
Based on these analyses of needs and feasibility for each language, five languages will be selected for CBT development. University faculty representing these languages will attend a two-week Summer Institute June 12-23, 2000, where they will learn test development procedures and techniques for computer-based andweb-based testing. In addition to adapting test prototypes by writing items for each language, these teams will carry out small scale but useful testing projects (such as the development of a self-assessment instrument or tests keyed to instructional units) during the course of the institute, for each of the five languages represented.
In the third stage of the project, tests suitable for national use will be developed for two of the targeted languages. The languages to be addressed, the skill areas to be addressed, and the test types to be developed (CBT, WBT, or both) will be selected at the conclusion of the Summer Institute, taking into consideration both national needs and the likelihood of satisfactory completion within the remaining two years of the funding cycle. For each language, nationally recruited teams of instructors will administer to their students the batteries of test items written by the language teaching experts during the Summer Institute. Results of the administration of these beta versions will be assembled and analyzed so the tests can be revised and improved. Working with the language teaching experts, the tests will be analyzed for reliability and validity. Manuals will also be created to make the testing and scoring procedures clear to future test users and to provide technical information abo