Changes in accreditation policies and institutional practices have led to the emergence of student learning outcomes assessment as an important, increasingly common expectation in U.S. college foreign language programs. This volume investigates contemporary outcomes assessment activity, with a primary focus on useful assessment, that is, assessment that is put to use proactively by foreign language educators. Authors approach the topic from distinct perspectives, ranging from a study of national trends in outcomes assessment practices, to reflections on assessment experiences by program leaders, to case studies highlighting language educators’ implementation and uses of outcomes assessment for diverse curricular and pedagogical purposes.

Contents

Foreword

Chapter 1

The Usefulness of Accreditation-Mandated Outcomes Assessment: Trends in University Foreign Language Program | John McE. Davis

Student learning outcomes assessment is now an integral part of university accreditation and program review, a reform framework designed to improve teaching and learning throughout U.S. higher education. This study explores the impact of accreditation-mandated assessment on college foreign language (FL) programs. Impact is defined in terms of assessment “usefulness,” or the degree to which educators in FL programs learn or act in productive ways as a result of assessment activity. FL faculty and staff from 100 post-secondary programs across the US were asked to report on their assessment practices in two areas: (a) the factors/conditions within their programs that help people to undertake assessment effectively (i.e., assessment capacity) and (b) the various ways in which assessment is used (i.e., the actions and instances of learning resulting from assessment activity). The study was generally focused on elucidating whether differences in assessment capacity (e.g., more or less effective leadership, assessment expertise) account for how faculty use or learn from assessment activity. Of special interest was whether the specific dimensions of assessment capacity leveraged on FL programs by accreditation review (e.g., institutional assessment requirements and support) are related to assessment usefulness. Findings suggest that while accreditation mandates have been successful in stimulating considerable assessment activity, programs with little assessment capacity beyond accreditation/institutional compliance lack the needed conditions for meaningful assessment use to occur. Rather, assessment usefulness—and important corollary consequences such as educational innovation and improvement—seems to issue from personnel-related factors originating within foreign language programs.

Chapter 2

The Uses of Use-Focused Assessment: Two Chairs’ Perspectives | Theodore J. Cachey, Jr. & Peter C. Pfeiffer

The chapter offers examples of useful assessment practices of different scope from departments at Georgetown University and at the University of Notre Dame. The perspective taken is that of long-time chairpersons who lead the assessment initiatives in both institutions. The focus is on how academic administrators can fashion and conceptualize useful assessment projects in their own environments.

Chapter 3

“Centering” Collegiate Foreign Language Departments Around Useful Outcomes Assessment: Challenges and Opportunities | Lance R. Askildson & Hiram H. Maxim

Due to the well-documented divisions within the curricula, personnel, and pedagogies of collegiate foreign language (FL) education, FL departments are often not structured in a way that facilitates collaborative approaches to programmatic issues. This structural deficit is particularly noticeable when departments embark on assessment-related work because of its inherently programmatic nature. To address this challenge to FL departments, this chapter discusses how language centers, because of their advisory and supra-departmental role on campus, can serve as vital partners to FL departments in approaching assessment from a useful and programmatic perspective. Specifically, and based on the experiences of the authors as language center directors at their respective institutions, the chapter presents four ways that language centers can provide leadership and support for useful assessment in collegiate FL education: (a) providing departments with expertise and guidance on useful assessment, an approach that is typically not well-understood among university faculty; (b) providing opportunities for assessment-related professional development through workshops, seminars, lectures; (c) providing a forum and location for collaboration and communication within and between departments; and (d) using the language center’s prominent position on campus to disseminate and advocate for best practices in assessment.

Chapter 4

The Uses of Accountability | Amanda Randall & Janet Swaffar

This chapter examines the initial assessment experiences of a research university’s German language program with 10 tenure-track faculty members and undergraduate and graduate PhD programs. The authors frame their case study by exploring why the administrative charge of engaging in assessment of those programs challenges traditions of academic autonomy and gives rise to faculty members’ concerns about bureaucratic make-work and the imposition of evaluative criteria that do not reflect the content and knowledge applications they teach. To involve all faculty members in articulating their specialist understanding of teaching, the department used a process contrary to that proposed in the university’s institutional focus on outcomes per se. Instead, the department looked first for concrete examples of what students have learned at different levels, and then asked faculty members to reflect about what teaching practices that learning suggests. The authors conclude by describing the essential component in the upper-division undergraduate program’s assessment: engaging faculty in a shared effort to identify practices that enable specific learning objectives within a curriculum.

Chapter 5

Formulating Effective Student Learning Outcomes Through Utilization-Focused Evaluation: A Case Study of a University Japanese Program | Shoko Sasayama

This chapter reports on a program evaluation conducted in a Japanese program at a U.S. university, with the ultimate goal of formulating effective student learning outcomes that are compatible with program goals, students’ needs, and instructors’ expectations. The evaluation was conducted over two years and consisted of three phases. In Phase I, student and instructor needs, program implementation, and possible ways to improve the program were explored, and suggestions for improvement were made to this end. Phase II, then, focused on monitoring whether and how these evaluation findings were put into use and what impact, if any, program changes had on student learning and program effectiveness. Upon confirmation of successful implementation of changes and their effectiveness, in Phase III, further innovations were implemented, including, in particular, the development of a clearly articulated set of student learning outcomes that consolidated what was learned in the evaluation findings thus far. The program evaluation not only achieved the ultimate goal of formulation of outcomes statements (known as findings use), but also improved communication among instructors (known as process use). It is my hope that the Japanese program will continue to thrive and contribute to students’ learning of the Japanese language, based on this program evaluation.

Chapter 6

Developing Learning Outcomes for First-Year Arabic at the University of Notre Dame | Ghada Bualuan & Amaya Martin

This chapter addresses the initiation of student learning outcomes assessment in the Program of Arabic Language and Culture (PALC) at the University of Notre Dame. It describes how assessment was used by Arabic faculty to modify the first-year Arabic language course sequence. Initially, the PALC assessment committee undertook the development of student learning outcomes for two first-year language courses in the Arabic major. The committee also implemented an assessment project to better understand how well the new student learning outcomes (SLOs) matched instruction and content in Arabic courses. Student feedback (via focus groups and an online questionnaire) was used to analyze which aspects of the curriculum encompassed the learning benchmarks articulated in the new SLOs (and which SLOs were unaddressed). In general, the committee found that current first-year courses did not provide sufficient instruction toward new listening and inter-cultural competence learning outcomes. The chapter also reports on lessons learned via the assessment process (e.g., assessment needs and challenges), as well as planned assessment work for the future. The chapter concludes by discussing how assessment greatly benefitted student learning and language teaching in the PALC—that is, we highlight the effectiveness of high-quality student learning outcomes and their positive impact on curriculum planning and the trajectory of student progress.

Chapter 7

Assessing the Intermediate Level: A Critical Juncture in German Outcomes Assessment | Hannelore Weber

In the spring of 2010, the German faculty at the University of Notre Dame embarked on an outcomes-focused assessment project. After preliminary discussions about the usefulness of assessing the four-year degree program in German, faculty members decided instead to concentrate on assessing the effectiveness and quality of the intermediate sequence of courses, considered a critical juncture prior to more advanced German study. In pursuit of this assessment project, the German section created student learning outcome statements for the major program, revised student learning outcome statements for the beginning and intermediate-level courses, conducted a focus group, undertook proficiency testing, and created and administered a student survey. After examining all the results gained by the use of these various instruments and procedures, the faculty identified critical aspects of the intermediate courses in need of review and planned to implement some changes to the intermediate courses. For the next step in the ongoing assessment process, and building on the experience gained from this initial project, the German section is now prepared to undertake the more comprehensive task of assessing the effectiveness of the German degree program as a whole.

Chapter 8

Journey Greater Than the Destination: A Department and Program Perspective on Utilization-Focused Assessment | Alessia Blad & Shauna Williams

In this chapter, we reflect on our multi-year journey with outcomes assessment in the Department of Romance Languages and Literatures at the University of Notre Dame. This assessment journey set out with a primary focus on utilization of assessments for the overarching purpose of enhancing departmental and program understanding and enabling gradual yet persistent improvement in our practices. We present our reflections by highlighting overarching qualities of the assessment process that stand out, namely,

• the creation and composition of an assessment working group,

• the reporting process, and

• faculty ownership of assessment.

Part one of our chapter describes the participants in the process, our assessment enterprise and approach, methods pursued for collecting and reporting data, and project benefits at the department level. Part two then explores how our assessment practices shifted to include finer-grained, program-level assessment work, based on a case study in the Italian program.

Chapter 9

A Tale of Two Cultures | Anna De Fina & Donatella Melucci

This chapter reports on an assessment project in which the Department of Italian at Georgetown University investigated student perspectives on the department’s student learning outcomes. The focus of the project was on writing and cultural learning, areas at the center of a previous curriculum renewal and development effort. The project involved the creation of a survey to shed light on student perceptions about their writing achievement and cultural learning, as well as on student opinions about aspects of the writing and culture curriculum and instruction more generally. The chapter describes the design and implementation of the survey and reports on main findings and implications for instructional practice and curricular design. The chapter also stresses the importance of outcomes assessment as a tool to evaluate the effectiveness of curricula and provide insights into potential adaptations necessary to enhance educational quality of language programs.

Chapter 10

Foreign Language Curriculum as a Means of Achieving Humanities Learning Goals: Assessment of Materials, Pedagogy, and Learner Texts | Marianna Ryshina-Pankova

Recently, there have been urgent calls (e.g., MLA, 2007) to explicitly conceptualize the goals of FL study beyond proficiency in a foreign language and in line with a more comprehensive humanistic learning framework characteristic of collegiate education in general. In view of this need to draw transparent connections between FL study and humanistic inquiry, we conducted a project that aimed to establish a framework for understanding and assessing the nature of “humanities learning” in the German Department at Georgetown University. This chapter describes the stages of the assessment initiative in its first cycle by (a) outlining the goals and intended uses of the assessment project, (b) reporting on the process of formulating the learning outcomes, and (c) describing an investigation of student writing performance that sought to determine whether students are achieving the specified outcomes. The chapter demonstrates the transformative impact of the program-internal and curriculum-based approach to assessment on the understanding of humanities learning as well as on tasks and pedagogical practices that can foster humanistic inquiry.