resources

Using Student Learning Outcomes Assessment in Language Program Evaluation

I. Articles and books on SLO (assessment) in general

II. Articles on SLO assessment in language programs

III. Comprehensive websites on SLO assessment

V. How-to sites/resources on SLO assessment

VII. Examples related to language programs

1. FL degree outcomes and assessment
2. Study Abroad outcomes and assessment
3. Outcomes assessment in general education programs

This resource list is intended for college foreign language educators who are undertaking student learning outcomes (SLO) assessment as part of their program evaluation activities.

I. Articles and books on SLO (assessment) in general

Astin, A., Banta, T., Cross, P., El-Khawas, E., Ewell, P., Hutchings, P., Marchese, T., McClenney, K., Mentkowski, M., Miller, M., Moran, E., and Wright, B. (1992). Principles of good practice for assessing student learning. AAHE Bulletin, 45(4).

Barker, J., & Folger, J. (1996). Assessing student achievement in the major: Assessment for program improvement. In T. Banta , et al., Assessment in practice: Putting principles to work on college campuses. San Francisco: Jossey-Bass.

Hernon, P., & Dugan, R. (Eds.). (2004). Outcomes assessment in higher education: Views and perspectives. Westport, CN: Libraries Unlimited.

Hernon, P., Dugan, R., & Schwartz, C. (2006). Revisiting outcomes assessment in higher education. Westport, CN: Libraries Unlimited.

Joint Committee on Standards for Educational Evaluation. (1994). The program evaluation standards: How to assess evaluations of edcational programs (2nd Ed). Thousand Oaks, CA: Sage.

Loacker, G., & Palola, E. (1981). Clarifying learning outcomes in the liberal arts. New Directions for Experiential Learning, no. 12. San Francisco: Jossey-Bass.

Middle States Commission on Higher Education (2003). Student learning assessment: Options and resources. Philadelphia, PA: Author.

Nichols, J. O. (1991). Institutional effectiveness and outcomes assessment implementation on campus: A practitioner's handbook. New York: Agathon Press.

Wiggins, G. (1998). Educative assessment: Designing assessments to inform and improve student performance. San Francisco: Jossey-Bass.

Find many more key readings at: The Teagle Foundation web-page on “Outcomes and assessment” http://www.teaglefoundation.org/learning/resources.aspx#assessment
This page provides an excellent set of articles on student learning outcomes and assessment in higher education (note that most of the articles are electronically available).

II. Articles on SLO assessment in language programs

Bachman, L. F. (1989). The development and use of criterion-referenced tests of language ability in language program evaluation. In R. K. Johnson (Ed.), The second language curriculum (pp. 242-258). Cambridge: Cambridge University Press.

Bernhardt, E. (2006). Student learning outcomes as professional development and public relations. Modern Language Journal, 90(4), 602-604.

Brindley, G. (1998). Outcomes-based assessment and reporting in language programs: A review of the issues. Language Testing, 15(1), 45-85.

Brindley, G. (2001). Outcomes-based assessment in practice: Some examples and emerging insights. Language Testing, 18(4), 393-407.

Hudson, T. D. (1989). Mastery decisions in program evaluation. In R. K. Johnson (Ed.), The second language curriculum (pp. 259-269). Cambridge: Cambridge University Press.

Liskin-Gasparro, J. E. (1995). Practical approaches to outcomes assessment: The undergraduate major in foreign languages and literatures. ADFL Bulletin, 26(2), 21-27.

Mathews, T., J., & Hansen, C., M. (2004). Ongoing assessment of a university foreign language program. Foreign Language Annals, 37(4), 630-640.

Morris, M. (2006). Addressing the challenges of program evaluation: One department’s experience after two years. Modern Language Journal, 90(4), 599-602.

Norris, J. M. (2000). Purposeful language assessment. English Teaching Forum, 38(1), 18-23.

Norris, J. M. (2006). Assessing foreign language learning and learners: From measurement constructs to educational uses. In H. Byrnes, H. Weger-Guntharp, & K. Sprang (Eds.), GURT 2005: Educating for Advanced Foreign Language Capacities: Constructs, Curriculum, Instruction, Assessment (pp. 167-187). Washington, DC: Georgetown University Press.

Norris, J. M. (2006). The why (and how) of student learning outcomes assessment in college FL education. Modern Language Journal, 90(4), 590-597.

Norris, J. M., & Pfeiffer, P. (2003). Exploring the use and usefulness of ACTFL Guidelines oral proficiency ratings in college foreign language departments. Foreign Language Annals, 36(4), 572-581.

Pennington, M. C., & Brown, J. D. (1991). Unifying curriculum process and curriculum outcomes: The key to excellence in language education. In M. C. Pennington (Ed.), Building better English language programs: Perspectives on evaluation in ESL (pp. 57-74). Washington, DC: NAFSA.

Phillips, J. (2006). Assessment now and into the future. In A. Heining-Boynton (Ed.), 2005-2015: Realizing our vision of languages for all (pp. 75-103). Upper Saddle River, NJ: Pearson.

Ross, S. (2003). A diachoronic coherence model for language programme evaluation. Language Learning, 53(1), 1-33.

III. Comprehensive websites on SLO assessment

North Carolina State University
Internet resources for higher education outcomes assessment
http://www2.acs.ncsu.edu/UPA/assmt/resource.htm
An exhaustive list of resources on assessment in higher education. The list includes web links to assessment handbooks, assessment of specific skills/content, institutional assessment plans, and assessment of educational effectiveness.

Central Michigan University
“CMU Assessment Toolkit”
http://academicaffairs.cmich.edu/caa/assessment/resources/toolkit.shtml
Excellent set of focused resources, examples, and recommendations on assessments for use in college programs.

Assessment in higher education
http://ahe.cqu.edu.au/online_articles.htm
A comprehensive resource list of online and offline articles, books, professional organizations, discussion lists, and conferences on student assessment in higher education.

IV. Definitions of SLO assessment

Frye, R. (1999). Assessment, accountability, and student learning
[Electronic version]. Dialogue, 2, 1-11. Retrieved March 8, 2007, from

http://www.ac.wwu.edu/~dialogue/issue2.pdf
A good starting point for understanding assessment, accountability, and student learning outcomes.

San Diego State University
Definitions of assessment of student learning outcomes.
http://dus.sdsu.edu/assessment/index.shtml?definitions

V. How-to sites/resources on SLO assessment

American Psychological Association. (2009). The assessment cyberguide for learning goals and outcomes (2nd ed.). Washington, DC: Author. http://www.apa.org/ed/governance/bea/assessment-cyberguide-v2.pdf

This recently updated guide contains many resources that guide assessment planning and process. Their guiding questions and principles are particularly useful.

Bresciani, M. J., & Schuh, J. H.
“Supporting student learning: Assessing the effectiveness of our work.”
http://www.ncsu.edu/assessment/presentations/assess_process/supporting_student_learning.pdf

A comprehensive module on SLO assessment explaining (a) the purposes of assessment, (b) how to write mission statement, goals, and learning outcomes, and (c) assessment strategies and tools.

Cabrillo College
“Student learning outcomes: Assessment & accreditation”
http://www.cabrillo.edu/services/pro/assess/assessweb/index.html
This site includes informative workbooks on writing student learning outcomes.

California Polytechnic State University, Academic Programs
“Learning outcomes assessment planning guide”
http://www.academicprograms.calpoly.edu/assessment/assessplanguide.htm#defining
This is a how-to guide on stating mission, goal, objectives, and learning outcomes.

Northern Arizona University, Office of Academic Assessment
- Faculty development workshops on various assessment topics (e.g., “Using assessment for the improvement of student learning”) can be found here. Some workshops have power point slides and handouts. http://www4.nau.edu/assessment/oaalibrary/workshops.htm
- Degree program assessment plan worksheet: https://www4.nau.edu/assessment/oaalibrary/documents/Degree%20Assessment%20Worksheet.pdf
- Criteria for reviewing degree program assessment plans: http://www4.nau.edu/assessment/documents/DegreeProgramRubric_July06.doc

San Diego State University
“Writing assessable goals and objectives”
http://dus.sdsu.edu/assessment/writingassessablegoals.pdf

Skidmore College
“Developing an Assessment Plan for Majors, Interdisciplinary Programs, and Other Academic Programs”
http://cms.skidmore.edu/assessment/Handbook/developing-an-assessment-plan.cfm

Southern Association of Colleges and Schools (Commission on Colleges)
Presentation handouts and slides of the 2006 SACS-COC Institute on Quality Enhancement and Accreditation, Orlando, FL.
http://www.sacscoc.org/2006SIpresentations.asp
Below are some of the many presentations related to SLO:
- Jones, B. H. “Defining, assessing & documenting quality outcomes in community colleges”
- Hoey, J. J. “Defining, assessing and documenting student learning outcomes”
- Nilson, L. B. “Linking teaching methods with student learning outcomes”
- Mince, R. “Using the results of assessment to improve student learning”
- Eck, J. C. “Using multiple measures of student learning outcomes”

St. Ambrose University
“Five fundamental questions on student learning”
http://web.sau.edu/assessment/The%20Five%20Fundamental%20Questions.htm
The website includes (a) five guiding questions for planning SLO assessment, (b) how-to guide for developing departmental assessment plan, and (c) a sample assessment rubric.

University of Colorado at Boulder
“Outcomes Assessment”
http://www.colorado.edu/pba/outcomes/
Particularly useful is the section on “Historical and ongoing assessment activities.” There are concrete examples of methods, the use of assessment results, etc.

University of Hawai'i at Manoa
“A utilization-focused approach to stating and evaluating student learning outcomes”
A how-to guide/manual for stating and assessing language learning outcomes. The manual emphasizes making outcomes assessment as useful a process as possible for program stakeholders (the guide can be accessed here).

VI. Assessment in college accreditation

Middle States Commission on Higher Education. (2005). Assessing student learning and institutional effectiveness: Understanding Middle States expectations. Philadelphia, PA: Middle States Commission on Higher Education. Retrieved March 9, 2007, from http://www.msche.org/publications/Assessment_Expectations051222081842.pdf
This document clarifies expectations on assessing institutional effectiveness and student learning outcomes. The guiding questions at the end of the document help faculty members create better institutional assessment plans.

New England Association of Schools and Colleges. (2005). Standards for accreditation. Retrieved December 18, 2009, from http://cihe.neasc.org/standards_policies/standards/
Standard 4.44 states approach to assessment of student learning.

North Central Association of Colleges and Schools Higher Learning Commission. (2003). Handbook of accreditation. Retrieved March 5, 2007, from http://www.ncahlc.org/download/Handbook03.pdf
Criterion three (Student learning and effective teaching, p. 3.14) in the handbook provides standards for demonstrating educational effectiveness through SLO assessment. The website also has resources on SLO assessment. Check out the power point handouts and pdfs on SLO assessment from Higher Learning Commission assessment workshop here.

Northwest Association of Schools and Colleges: Commission on Colleges and Universities. (n.d.). Accreditation standards. Retrieved March 7, 2007, from http://www.nwccu.org/Standards%20and%20Policies/Accreditation%20Standards/Accreditation%20Standards.htm Policy 2.2 under Standard Two (Educational program and its effectiveness) provides examples of self-guiding questions for SLO assessment.

Southern Association of Schools and Colleges: Commission on Colleges (2001). Principles of accreditation: Foundations for quality enhancement. Retrieved March 5, 2007, from http://www.sacscoc.org/pdf/PrinciplesOfAccreditation.PDF
Section 3.3. (Institutional Effectiveness) and 3.4 (Educational Programs) of the accreditation standards addresses learning outcomes.

Western Association of Schools & Colleges (2002). A guide to using evidence in the accreditation process: A resource to support institutions and evaluation teams. Retrieved December 18, 2009, from http://www.wascsenior.org/findit/files/forms/Evidence_Guide__Jan_02_.pdf
The guide introduces the notion and principles of what constitutes good evidence in educational program reviews. Many examples of evidentiary sources that respond to WASC standards are illustrated.

The Council of Regional Accrediting Commissions developed multiple documents on principles of student learning in 2004.
- Regional accreditation and student learning: A guide for institutions and evaluators
- Regional accreditation and student learning: Improving institutional practice
- Regional accreditation and student learning: Preparing teams for effective deliberation

Council for Higher Education Accreditation. (2002). Student learning outcomes workshop. The CHEA Chronicle, 5(2). Retrieved March 5, 2007, from http://www.chea.org/Chronicle/vol5/no2/Chron-vol5-no2.pdf
The document is a short list of key considerations for the accreditation body when judging institutional effectiveness and quality of education. A power point file of the workshop (“CHEA workshop on accreditation and student learning outcomes”) by Peter Ewell can be accessed here.

American Association for Higher Education. 9 principles of good practice for assessing student learning. http://ultibase.rmit.edu.au/Articles/june97/ameri1.htm

VII. Examples related to language programs

1. FL degree outcomes and assessment

Brigham Young University, College of Humanities
-Student learning outcomes assessment summaries for multiple language programs (and other units) at BYU; includes outcomes statements and examples of the variety of assessments used:
http://humassessment.byu.edu/

University of Arizona, College of Humanities
-UA Outcomes Assessment for language programs (and other units); includes outcomes statements and descriptions of assessments used in a variety of languages:
http://assessment.arizona.edu/hum/College%20of%20Humanities

Montana State University, Modern Languages and Literatures:
- SLOs for French, German, and Spanish majors and minors, and Japanese entry level program: http://www.montana.edu/wwwprov/assessment/modern%20languages--plan.pdf
- A summary assessment plan for undergraduate FL majors: http://www.montana.edu/wwwprov/assessment/documents/ModernLanguagesPlan-2007.pdf

- A summary of assessment results:
http://www.montana.edu/wwwprov/assessment/documents/ModernLanguagesUpdate-2007.pdf

San Diego State University
European Studies, and Linguistics & Oriental Languages:
http://dus.sdsu.edu/assessment/department_learning_goals.pdf

State University of New York, Buffalo:
http://www.buffalostate.edu/offices/assessment/gened.htm#
The website has a link to the SLO assessment for general education foreign
language requirement program (101/201, entry level program). Example assessment activities are also given.

University of Alaska Fairbanks, Department of Foreign Languages & Literatures:
http://www.uaf.edu/provost/outcomes/PlansPDFFormat/flba.pdf
This is a summary of a student learning outcomes assessment plan for their Bachelor of Arts in Foreign Languages program. They provide a map of intended learning outcomes, assessment criteria, and implementation procedures.

Stanford University, Language Center
First-year language objectives: http://language.stanford.edu/requirement/first_year.html
Examples of student learning outcomes for language requirement (in 11 languages).

2. Study Abroad outcomes and assessment

Forum on Education Abroad
Very useful position papers, guides, etc. on major issues in outcomes assessment of study abroad can be viewed here:
http://www.forumea.org/research-position_papers.cfm

3. Outcomes assessment in general education programs

Mesa Community College, Office of Research & Planning
http://www.mc.maricopa.edu/about/orp/assessment/
The web page contains information about (a) their assessment plan and results, (b) SLOs for general education, workplace skill, and developmental education programs, and (c) student orientations to the assessment activities.
- SLOs: http://www.mc.maricopa.edu/about/orp/assessment/outcomes.pdf
- Student orientation to SLO assessment (flash presentation):
http://www.mc.maricopa.edu/about/orp/assessment/main_web.swf

Montana State University
http://www.montana.edu/wwwprov/assessment/assessmentplans.htm
Assessment plans, results, and suggested improvements for all undergraduate degrees are listed, including department of Modern Languages and Literatures.

Northern Arizona University, Office of Academic Assessment
http://www4.nau.edu/assessment/assessment/liberal/definitions.htm
SLO assessment of the Liberal Studies program at NAU.

University of Illinois, Urbana-Champaign
http://cte.illinois.edu/outcomes/unit_assess.html
UIUC has SLO assessment plans for each department, stating department level SLO, methods, and timelines.

University of Maryland
http://www.umuc.edu/outcomes/pdfs/inst_plan_2007.pdf
This detailed report is an institutional assessment plan for the School for Undergraduate Studies and Graduate School of Management and Technology. Students’ expected learning outcomes are listed in Appendix A from p.30 to p.35.

University of Wisconsin Madison
http://www.provost.wisc.edu/assessment/manual/manual2.html#a4
The Office of Provost provides assessment plans and introduces various direct and indirect SLO assessment tools.