ASSESSMENT IN AN ONLINE FOREIGN LANGUAGE METHODS COURSE
June K. Phillips
BACKGROUND FOR THE PROJECT
As student standards in many disciplines took hold under the federal Goals 2000 Educate America legislation (http://www.ed.gov/legislation/GOALS2000/TheAct/index.html), they were followed in 2002 by the development of standards for teacher education programs (http://www.ncate.org/public/unitStandardsRubrics.asp?ch=4), under the auspices of the National Council for the Accreditation of Teacher Education (NCATE). Prior to this date, the NCATE approval process did not evaluate foreign language education programs according to the guidelines of their Specialty Professional Association (SPA), i.e., ACTFL standards. Now that the standards are in place, programs must meet them for accreditation. One of the key requirements for NCATE accreditation is a foreign language methods course. A generic methods course in education is not sufficient. Likewise, many state departments of education require this course to meet pedagogical competencies in their alternative certification plans. Alternative certification is becoming a more recognized way for states to meet teacher shortages and to provide a mechanism for people wishing to make a career change into teaching. Both NCATE and alternative certification have resulted in a growing audience for online instruction made up of students who are not able to take daytime courses on college campuses.
Colleges and universities of medium or large size have no problem offering a foreign language methods course to their undergraduate students. However, small colleges had often relied on a generic methods course for students seeking initial licensure. Their language departments often consisted of two or three faculty members whose expertise was in literature or cultural studies, and student enrollments for teaching did not warrant additional faculty with second-language acquisition or applied linguistics expertise. Candidates seeking alternative certification also had difficulty finding a methods course offered in the evenings or at a reasonable distance. To meet these growing needs, in 1998 ACTFL and Weber State University were awarded a federal grant under Title VI: International Research and Studies Program to develop an online foreign language methods course.
DEVELOPMENT PROCESS AND GUIDELINES
With the grant support, a course design team was assembled. A unique and collaborative aspect of the course lay in the combined expertise of a faculty rather than that of a single individual trying to cover fairly specialized concepts.
Online teaching was still in a beginning stage as the project began, although technology was rapidly advancing so that oral and written interactivity was increasingly possible. Principles established to guide the design work included intent to parallel the scope and coverage of grounded courses. Given that this was a methods course, the development team felt that the online instruction had to model good teaching and good performance assessment. Above all, the course would have to meet the ACTFL/NCATE standards for a foreign language methods course for students using it as part of an accredited program.
In the area of assessment, ACTFL/NCATE Standard 5 specifically outlines the knowledge, skills, and dispositions (NCATE mandated categories) associated with "Assessment of Languages and Cultures." Teacher Education candidates are charged with
As assessment components of the course were developed, this standard served as guidance. To review the full ACTFL/NCATE standards, see http://www.actfl.org/i4a/pages/index.cfm?pageid=3384.
The online methods course consists of twelve modules (for an outline of the syllabus, go to http://departments.weber.edu/forlang/forlang/online.htm). The first challenge was how to replicate the various kinds of formative assessments done in a grounded course. Since electronic portfolio submissions are a primary means of faculty/student communication and consequently, a chief tool for assessment, three types of portfolio assignments were designed; they required increasing degrees of preparation, complexity, and time. They will be described below.1. Check-in
The first assessment is the "Check-in." As the course was piloted, it became clear that the non-verbal clues provided by students in the classroom as the instructor explains, demonstrates, or writes on the board to expand and reinforce content were missing in the online course. By watching faces and expressions, instructors gauge whether students understand the concepts that are being taught. As a result, instructors regularly ask simple questions to check for comprehension. In the online course, however, there was a concern that students could work their way through a significant amount of material, possibly going off-track, without being assured of direction or comprehension. Thus the "Check-in" was invented. Students are told that their responses to "Check-ins" should be done on the spot; extra reading or investigation is not required; response time should be almost immediate. While the instructor feedback cannot be as instantaneous as in the classroom, it still occurs within 24-48 hours, and students report that just by pausing to collect thoughts and respond, the check-in seems to keep them alert to key ideas.
Below is an example of a "Check-in" from Module 10 on Assessment which is then submitted to the electronic portfolio:
In response to the "Check-in," the instructor confirms that students have grasped the idea or elaborates on the student's comment. In the classroom, a response from one student is shared by all; the intensity of the online course means that each student is answering each question (a good thing), but the instructor is responding each time, often repetitively (burdensome, boring).2. Reflect/Respond
A second type of formative assessment is intended to elicit a more thoughtful response. It may require thinking of an example to support the response, synthesizing several points from the reading assignment, or taking time to access supporting material. These assessments parallel homework assignments, or mini-lessons. As instructors respond to these submissions, they frequently elaborate on student responses, so a dialogue is begun. Sometimes, instructors might ask for an additional response if it appears that students need to reach a deeper understanding.
Below is an example of a "Reflect/Respond" from Module 11 on Culture which is then submitted to the electronic portfolio:
Both the "Check-in" and "Reflect/Respond" make sure that the students are on-track and that they are acquiring the content background needed for good instructional decision-making. These formative assessments also allow students to engage in smaller tasks before designing a larger project.
TOWARD SUMMATIVE ASSESSMENTS
Each module concludes with an assessment appropriate to its goal and content. Two modules have content knowledge, e.g., second-language acquisition theories as the primary focus. The module on the national standards ends with a self-assessment. The module on learner diversity requires either a paper investigating a subtopic with further research or a case study that appeals to students currently in teaching situations. The seven modules that focus on approaches to teaching specific concepts require a project as an end task. One might think of all these assessments as summative in terms of the concept such as teaching interpersonal communication or culture, but they are also formative in terms of the final course project, designing lesson plans for a week to ten days. The modular projects require students to create parts of lessons. They require that students demonstrate how to plan, how to organize a lesson, and how to explicate the instructional decisions taken. One element missing in the online course that is often incorporated in face-to-face classes is peer teaching. Consequently, the course requires more detail in the lesson plan, samples of materials, and a narrative about how the lesson is intended to evolve.
Students tend to develop their projects as word-processing documents or web pages that can be submitted to the electronic portfolio but they are more commonly sent as e-mail attachments. A sample lesson plan format is provided, but students involved in teaching (primarily those seeking alternative certification) may use a plan required by their schools. All plans must include specification of how the national and/or relevant state standards are addressed.
Below is an example of a Project from Module 12 on the Connections
standards which is based upon interdisciplinary learning:
Before submitting your project, please review the Rubric that will be used to assess it.
In order to further guide the project designs and to model performance assessment, rubrics are provided for the projects. The basic rubric remains constant but specific categories are highlighted or de-emphasized depending upon the topic. Here is the rubric used in the course.
The development team used Wiggins and McTighe's (1998) concept of backward design for the course. The culminating project for the course ties together the theoretical underpinnings, the standards, and instructional approaches in a product that is based upon the reality of classroom teaching. The final project was identified as an outcome before the course modules were developed; it was meant to provide evidence that the outcomes of planning, theoretical underpinnings, and good decision-making have been mastered. In grounded courses, a lesser degree of written material might be balanced by classroom teaching demonstrations. Lacking that in the online course, a significant narrative is required from students to provide reasons for the instructional decisions they make. The outline given for the project includes:
Rubrics are also provided for the culminating activity both to guide students and to provide a model for performance assessments.
REFLECTIONS ON ASSESSMENT IN THE ONLINE METHODS COURSE
The assessment components of the online methods course have been revised regularly. Initially, there were more portfolio submissions than is the case today. At the same time, there are still many, primarily because of the need to keep a good number of check-ins and reflect/respond items.Without these, students tend not to keep up or to go too long without feedback. For instructors, these smaller largely one-on-one formative assessments require regular log-ins and intensive feedback. The rubrics created for the end-of-module projects work quite well now as they, too, have been revised several times. In the near future, sample projects from other students will be posted to a website as examples; a bank of samples should assist students in seeing course expectations and how the rubric was applied. As students have become more technologically skilled and creative, the culminating projects are becoming better and better. The majority of submissions are now on a CD so that the plans incorporate visuals, websites, and well-formatted teaching and assessment materials. Another finding is that the several tests and self-assessments have been done without monitoring in a secure environment and that has not posed a problem.
The course has run for over five years now. While challenges exist, most have been resolved to our satisfaction. Questions remain, of course. With improved software that crosses platforms, we expect that the numbers of voice chats and increased use of instant messaging among students might reduce the need for so many check-ins and concomitant feedback by instructors. We would also like to link more clearly the projects done in modules with the culminating one so that less 'new' work needs to be created close to the end of the course. On the positive side, it is often the students themselves who want to show their improved competencies by tackling new material. Finally, a major reason that assessment comprises such a huge challenge in the methods course lies in its 'newness' to the students. We are asking them to develop assessments that they rarely experienced as language learners. The undergraduate students' own experience is that of having taken discrete-point and written tests almost exclusively. One would have expected that the decades-long emphasis on communicative language learning would have resulted in their having experienced first-hand some performance assessment. The graduate students who are mostly practicing teachers report that almost all their testing practice has also been discrete-point with heavy reliance on textbook testing packets. While these often include listening components, most items have single correct answers; consequently, performance assessment is a new concept for them as well. If the profession is ever to make significant progress in performance assessment, future teachers need to have had familiarity with the process as learners.
ABOUT THE AUTHOR
June K. Phillips is Dean of Arts & Humanities at Weber State University. She is a former President of the American Council on the Teaching of Foreign Languages. She was project director for the Foreign Language Standards and co-chaired the development of ACTFL/NCATE Program Standards for the Preparation of Foreign Language Teachers.
ACTFL Foreign Language Standards Writing Team. (2002). Program Standards for the Preparation of Foreign Language Teachers. Alexandria, VA: ACTFL. Retrieved August 1, 2005 from http://www.actfl.org/i4a/pages/index.cfm?pageid=3384.
Wiggins, G., & McTighe, J. (1998). Understanding by Design. Alexandria, VA: Association for Supervision and Curriculum Development.
Copyright © 2005 National Foreign Language Resource Center
Articles are copyrighted by their respective authors.