ASSESSMENT IN AN ONLINE FOREIGN LANGUAGE METHODS COURSE

ASSESSMENT IN AN ONLINE FOREIGN LANGUAGE METHODS COURSE

June K. Phillips
Weber State University

BACKGROUND FOR THE PROJECT

As student standards in many disciplines took hold under the federal Goals 2000 Educate America legislation (http://www.ed.gov/legislation/GOALS2000/TheAct/index.html), they were followed in 2002 by the development of standards for teacher education programs (http://www.ncate.org/public/unitStandardsRubrics.asp?ch=4), under the auspices of the National Council for the Accreditation of Teacher Education (NCATE). Prior to this date, the NCATE approval process did not evaluate foreign language education programs according to the guidelines of their Specialty Professional Association (SPA), i.e., ACTFL standards. Now that the standards are in place, programs must meet them for accreditation. One of the key requirements for NCATE accreditation is a foreign language methods course. A generic methods course in education is not sufficient. Likewise, many state departments of education require this course to meet pedagogical competencies in their alternative certification plans. Alternative certification is becoming a more recognized way for states to meet teacher shortages and to provide a mechanism for people wishing to make a career change into teaching. Both NCATE and alternative certification have resulted in a growing audience for online instruction made up of students who are not able to take daytime courses on college campuses.

Colleges and universities of medium or large size have no problem offering a foreign language methods course to their undergraduate students. However, small colleges had often relied on a generic methods course for students seeking initial licensure. Their language departments often consisted of two or three faculty members whose expertise was in literature or cultural studies, and student enrollments for teaching did not warrant additional faculty with second-language acquisition or applied linguistics expertise. Candidates seeking alternative certification also had difficulty finding a methods course offered in the evenings or at a reasonable distance. To meet these growing needs, in 1998 ACTFL and Weber State University were awarded a federal grant under Title VI: International Research and Studies Program to develop an online foreign language methods course.

DEVELOPMENT PROCESS AND GUIDELINES

With the grant support, a course design team was assembled. A unique and collaborative aspect of the course lay in the combined expertise of a faculty rather than that of a single individual trying to cover fairly specialized concepts.

Online teaching was still in a beginning stage as the project began, although technology was rapidly advancing so that oral and written interactivity was increasingly possible. Principles established to guide the design work included intent to parallel the scope and coverage of grounded courses. Given that this was a methods course, the development team felt that the online instruction had to model good teaching and good performance assessment. Above all, the course would have to meet the ACTFL/NCATE standards for a foreign language methods course for students using it as part of an accredited program.

In the area of assessment, ACTFL/NCATE Standard 5 specifically outlines the knowledge, skills, and dispositions (NCATE mandated categories) associated with "Assessment of Languages and Cultures." Teacher Education candidates are charged with

5.a knowing assessment models and using them appropriately
5.b reflecting on assessment
5.c reporting assessment results

As assessment components of the course were developed, this standard served as guidance. To review the full ACTFL/NCATE standards, see http://www.actfl.org/i4a/pages/index.cfm?pageid=3384.

FORMATIVE ASSESSMENTS

The online methods course consists of twelve modules (for an outline of the syllabus, go to http://departments.weber.edu/forlang/forlang/online.htm). The first challenge was how to replicate the various kinds of formative assessments done in a grounded course. Since electronic portfolio submissions are a primary means of faculty/student communication and consequently, a chief tool for assessment, three types of portfolio assignments were designed; they required increasing degrees of preparation, complexity, and time. They will be described below.

1. Check-in

The first assessment is the "Check-in." As the course was piloted, it became clear that the non-verbal clues provided by students in the classroom as the instructor explains, demonstrates, or writes on the board to expand and reinforce content were missing in the online course. By watching faces and expressions, instructors gauge whether students understand the concepts that are being taught. As a result, instructors regularly ask simple questions to check for comprehension. In the online course, however, there was a concern that students could work their way through a significant amount of material, possibly going off-track, without being assured of direction or comprehension. Thus the "Check-in" was invented. Students are told that their responses to "Check-ins" should be done on the spot; extra reading or investigation is not required; response time should be almost immediate. While the instructor feedback cannot be as instantaneous as in the classroom, it still occurs within 24-48 hours, and students report that just by pausing to collect thoughts and respond, the check-in seems to keep them alert to key ideas.

Below is an example of a "Check-in" from Module 10 on Assessment which is then submitted to the electronic portfolio:

Check-in : Post your answers to these questions in the web portfolio:

What are the common features that all rubrics share?
Why do proponents of rubrics believe that they can "empower learners?"

In response to the "Check-in," the instructor confirms that students have grasped the idea or elaborates on the student's comment. In the classroom, a response from one student is shared by all; the intensity of the online course means that each student is answering each question (a good thing), but the instructor is responding each time, often repetitively (burdensome, boring).

2. Reflect/Respond

A second type of formative assessment is intended to elicit a more thoughtful response. It may require thinking of an example to support the response, synthesizing several points from the reading assignment, or taking time to access supporting material. These assessments parallel homework assignments, or mini-lessons. As instructors respond to these submissions, they frequently elaborate on student responses, so a dialogue is begun. Sometimes, instructors might ask for an additional response if it appears that students need to reach a deeper understanding.

Below is an example of a "Reflect/Respond" from Module 11 on Culture which is then submitted to the electronic portfolio:

Reflect/Respond: In your electronic portfolio:

Analyze and chart the products, practices, and perspectives from the two learning scenarios in the standards book that you identified.
Add any additional information that you know and might expand upon.
Identify any relevant changes across time.

Both the "Check-in" and "Reflect/Respond" make sure that the students are on-track and that they are acquiring the content background needed for good instructional decision-making. These formative assessments also allow students to engage in smaller tasks before designing a larger project.

TOWARD SUMMATIVE ASSESSMENTS

Each module concludes with an assessment appropriate to its goal and content. Two modules have content knowledge, e.g., second-language acquisition theories as the primary focus. The module on the national standards ends with a self-assessment. The module on learner diversity requires either a paper investigating a subtopic with further research or a case study that appeals to students currently in teaching situations. The seven modules that focus on approaches to teaching specific concepts require a project as an end task. One might think of all these assessments as summative in terms of the concept such as teaching interpersonal communication or culture, but they are also formative in terms of the final course project, designing lesson plans for a week to ten days. The modular projects require students to create parts of lessons. They require that students demonstrate how to plan, how to organize a lesson, and how to explicate the instructional decisions taken. One element missing in the online course that is often incorporated in face-to-face classes is peer teaching. Consequently, the course requires more detail in the lesson plan, samples of materials, and a narrative about how the lesson is intended to evolve.

Students tend to develop their projects as word-processing documents or web pages that can be submitted to the electronic portfolio but they are more commonly sent as e-mail attachments. A sample lesson plan format is provided, but students involved in teaching (primarily those seeking alternative certification) may use a plan required by their schools. All plans must include specification of how the national and/or relevant state standards are addressed.

Below is an example of a Project from Module 12 on the Connections standards which is based upon interdisciplinary learning:

Project: Submit one lesson plan or a lesson video that you consider to be your best example of classroom work leading to student learning toward the Connections standard. Explain why you feel this is a good example of interdisciplinary or content-based learning.

Written lesson plans should follow the format you have used previously in this course.

Your lesson (written or videotaped) should be appropriate to a specific grade level or secondary school course. It is best to develop materials for a grade or grades that you anticipate teaching in the future. Be sure to be clear about both the content and language objectives of your lesson.

Guidelines for the portfolio submissions :

You may choose an elementary, middle, or high school grade level and content objective appropriate to that grade level. You may use the content as the point of departure of the lesson, indicating the language outcomes that will result from content learning. Or, you may choose a language objective and demonstrate how a specific content objective at that grade level can support learning a language objective. Plan out 2 or more activities that lead students toward growing content-based learning.

You may develop a supplementary unit appropriate to the school curriculum and cognitive maturity of students, e.g., a unit on the rain forest.

You may develop a series of lessons that is not directly tied to the school curriculum but deals with a topic of interest to students and that represents an opportunity for students to acquire new information through the target language.

In any case, be sure to explain why you believe these student activities are a good example of interdisciplinary or content-based learning. Where possible, make explicit reference to the content area school curriculum you have targeted.

Before submitting your project, please review the Rubric that will be used to assess it.

In order to further guide the project designs and to model performance assessment, rubrics are provided for the projects. The basic rubric remains constant but specific categories are highlighted or de-emphasized depending upon the topic. Here is the rubric used in the course.

Project Rubric

	Excellent	Good	Needs Work
Organization of Project	Clearly written with no errors in mechanics (L1 & L2) Lesson is cohesive and clearly sequenced Tasks are explained in exceptional detail Project is submitted on time	Clearly written with few errors (L1 & L2) Lesson shows adequate evidence of cohesiveness Tasks are explained in sufficient detail Project is submitted within 48 hours of the due date	Not clear with several errors (L1 & L2) Lesson shows frequent gaps in organization of sequencing. Tasks in insufficient detail Project is submitted later than 48 hours of due date
Selection of Materials	Correctly identify all materials as authentic or scripted Appropriate to proficiency level, difficulty level, and amount of work expected Appropriate to age -- typical interests (e.g., motivating for students to participate, engaging)	Correctly identify most materials as authentic or scripted Most material is appropriate to proficiency level Most material appropriate to age of students	No authentic materials and/or incorrectly labeled materials Materials inappropriate to proficiency level Materials not appropriate to age
Use of Technology	Choice of technology used is clearly appropriate to the task; can aid learning	Choice of technology is reasonably appropriate to the task	Choice of technology is questionable or no use of technology
Use of 3 Modes of Communication	Student assignment clearly reflects intended mode Assignment integrates modes where appropriate	Student assignment likely to produce intended mode Assignment usually integrates modes where appropriate	Student assignment does not reflect intended mode Assignment does not integrate more than one mode
Teaching Strategies and Techniques	Makes logical choices of teaching strategies to reach all learners	Some evidence of diversity of teaching strategies	No evidence of diversity of teaching strategies
FL Standards	Lesson demonstrates accurately connection to appropriate standard(s)	Lesson demonstrates potential connection(s) to appropriate standard (s)	No evidence of connections to standards

SUMMATIVE ASSESSMENT

The development team used Wiggins and McTighe's (1998) concept of backward design for the course. The culminating project for the course ties together the theoretical underpinnings, the standards, and instructional approaches in a product that is based upon the reality of classroom teaching. The final project was identified as an outcome before the course modules were developed; it was meant to provide evidence that the outcomes of planning, theoretical underpinnings, and good decision-making have been mastered. In grounded courses, a lesser degree of written material might be balanced by classroom teaching demonstrations. Lacking that in the online course, a significant narrative is required from students to provide reasons for the instructional decisions they make. The outline given for the project includes:

an overview that specifies age and proficiency level of students;
a unit plan and lesson plans for 4-5 days for undergraduates and 7 or more for graduate students;
designation of the content scope and coverage of the unit, especially as concerns cultural or interdisciplinary content;
evidence of how technology is used to advance language learning;
commentary on principles used in designing the unit and lessons;
formal assessment for the unit.

Rubrics are also provided for the culminating activity both to guide students and to provide a model for performance assessments.

REFLECTIONS ON ASSESSMENT IN THE ONLINE METHODS COURSE

The assessment components of the online methods course have been revised regularly. Initially, there were more portfolio submissions than is the case today. At the same time, there are still many, primarily because of the need to keep a good number of check-ins and reflect/respond items.Without these, students tend not to keep up or to go too long without feedback. For instructors, these smaller largely one-on-one formative assessments require regular log-ins and intensive feedback. The rubrics created for the end-of-module projects work quite well now as they, too, have been revised several times. In the near future, sample projects from other students will be posted to a website as examples; a bank of samples should assist students in seeing course expectations and how the rubric was applied. As students have become more technologically skilled and creative, the culminating projects are becoming better and better. The majority of submissions are now on a CD so that the plans incorporate visuals, websites, and well-formatted teaching and assessment materials. Another finding is that the several tests and self-assessments have been done without monitoring in a secure environment and that has not posed a problem.

The course has run for over five years now. While challenges exist, most have been resolved to our satisfaction. Questions remain, of course. With improved software that crosses platforms, we expect that the numbers of voice chats and increased use of instant messaging among students might reduce the need for so many check-ins and concomitant feedback by instructors. We would also like to link more clearly the projects done in modules with the culminating one so that less 'new' work needs to be created close to the end of the course. On the positive side, it is often the students themselves who want to show their improved competencies by tackling new material. Finally, a major reason that assessment comprises such a huge challenge in the methods course lies in its 'newness' to the students. We are asking them to develop assessments that they rarely experienced as language learners. The undergraduate students' own experience is that of having taken discrete-point and written tests almost exclusively. One would have expected that the decades-long emphasis on communicative language learning would have resulted in their having experienced first-hand some performance assessment. The graduate students who are mostly practicing teachers report that almost all their testing practice has also been discrete-point with heavy reliance on textbook testing packets. While these often include listening components, most items have single correct answers; consequently, performance assessment is a new concept for them as well. If the profession is ever to make significant progress in performance assessment, future teachers need to have had familiarity with the process as learners.

ABOUT THE AUTHOR

June K. Phillips is Dean of Arts & Humanities at Weber State University. She is a former President of the American Council on the Teaching of Foreign Languages. She was project director for the Foreign Language Standards and co-chaired the development of ACTFL/NCATE Program Standards for the Preparation of Foreign Language Teachers.

E-mail: jphillips@weber.edu

REFERENCES

ACTFL Foreign Language Standards Writing Team. (2002). Program Standards for the Preparation of Foreign Language Teachers. Alexandria, VA: ACTFL. Retrieved August 1, 2005 from http://www.actfl.org/i4a/pages/index.cfm?pageid=3384.

Wiggins, G., & McTighe, J. (1998). Understanding by Design. Alexandria, VA: Association for Supervision and Curriculum Development.