Using Interviews and Focus Groups in Language Program Evaluation
References marked with an asterisk (*) indicate those sources that we have found particularly helpful.
Bourque, L. B., & Fielder, E. P. (2002). The Survey Kit (2nd ed.): Vol. 4. How to Conduct Telephone Surveys. Thousand Oaks, CA: Sage.
Fowler, F. J., &
Mangione, T. W. (1990). Standardized
survey interviewing: Minimizing interviewer-related error.
Newbury Park, CA: Sage.
Frey, J. H., & Oishi, S. M. (1995). How to Conduct Interviews by Telephone and in Person (Survey Kit, Vol. 4). Thousand Oaks, CA: Sage.
Gubrium, J. F., & Holstein, J. A. (Eds.). (2002). Handbook of interview research: Context & method. Thousand Oaks, CA: Sage.
*Kvale, S. (1996). InterViews: An introduction to qualitative research interviewing. Thousand Oaks, CA: Sage.
NOTE: Kvale walks the reader through seven methodological stages of qualitative interview studies: thematizing, designing, interviewing, transcribing, analyzing, verifying, and reporting.
Maso, I., & Wester, F. (Eds.). (1996). The deliberate dialogue: Qualitative perspectives on the interview. Brussels: VUB University Press.
Mishler, E. G. (1986). Research
interviewing: Context and narrative.
Cambridge, MA: Harvard University Press.
Rubin, H. J., & Rubin, I. (2005). Qualitative interviewing: The art of hearing data (2nd ed.). Thousand Oaks, CA: Sage.
Seidman, I. E. (2006). Interviewing as qualitative research: A guide for researchers in education and the social sciences (3rd ed.). New York: Teachers College Press.
Weiss, R. S. (1995). Learning from Strangers: The Art and Method of Qualitative Interview Studies. New York: Free Press.
Wengraf, T. (2001). Qualitative research interviewing: Biographic narrative and semi-structured methods. London: Sage.
Fern, E. F. (2001). Advanced focus group research. Thousand Oaks, CA: Sage.
*Krueger, R. A., &
Casey, M. A. (2000). Focus
groups: A practical guide for applied research
(3rd ed.).Thousand Oaks, CA: Sage.
NOTE: A non-technical step-by-step guide on how to design focus group studies, select participants, moderate focus group discussion, analyze focus group data, and report results.
Morgan, D. L. (1997). Focus groups as qualitative research (2nd ed.). Newbury Park, CA: Sage.
*Morgan, D. L., &
Krueger, R. A. (Eds.). (1997). Focus
Group Kit (Focus Group Kit,
Vols. 1-6). Thousand Oaks, CA: Sage.
NOTE: This six volume set provides a straightforward blueprint for people conducting focus groups, illustrating a number of key steps involved in conducting focus group research. Below is the list of volumes included in the kit.
|(1) Morgan, D. L.
(1997). The focus group
guidebook (Focus Group Kit,
Vol. 1). Thousand Oaks, CA: Sage.
(2) Morgan, D. L., & Scannell, A. U. (1997). Planning focus groups (Focus Group Kit, Vol. 2). Thousand Oaks, CA: Sage.
(3) Krueger, R. A. (1997). Developing questions for focus groups (Focus Group Kit, Vol. 3). Thousand Oaks, CA: Sage.
(4) Krueger, R. A. (1997). Moderating focus groups (Focus Group Kit, Vol. 4). Thousand Oaks, CA: Sage.
(5) Krueger, R. A., & King, J. A. (1997). Involving community members in focus groups (Focus Group Kit, Vol. 5). Thousand Oaks, CA: Sage.
(6) Krueger, R. A. (1997). Analyzing and reporting focus group results (Focus Group Kit, Vol. 6). Thousand Oaks, CA: Sage.
*Stewart, D. W., &
Shamdasani, P. N. (1990). Focus
groups: Theory and practice.
Thousand Oaks, CA: Sage.
NOTE: The book helps the researcher attend to the intra- and inter- personal and environmental variables when designing focus group studies.
Vaughn, S., Schumm, J. S. & Sinagub, J. M. (1996). Focus group interviews in education and psychology. Thousand Oaks, CA: Sage.
*Boyatzis, R. E. (1998). Transforming
qualitative information: Thematic analysis and code development.
Thousand Oaks, CA: Sage.
NOTE: This book provides clear guidelines on the process of thematic analysis, a method to encode and find patterns in qualitative data.
Coffey, A., & Atkinson, P. (1996). Making sense of qualitative data analysis: Complementary strategies. Thousand Oaks, CA: Sage.
Denzin, N. K., & Lincoln, Y. S. (Eds.). (2005). The Sage handbook of qualitative research. Thousand Oaks, CA: Sage.
Erlandson, D. A., Harris, E. L., Skipper, B. L., & Allen, S. D. (1993). Doing naturalistic inquiry. Newbury Park, CA: Sage.
Glesne, C. (2005). Becoming qualitative researchers: An introduction (3rd ed.). New York: Allyn & Bacon.
Maso, I., & Wester, F.
(Eds.). (1996). The
deliberate dialogue. Brussel:
Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Thousand Oaks, CA: Sage.
Patton, M. Q. (2002). Qualitative research & evaluation methods (3rd ed.). Thousand Oaks CA: Sage.
Silverman, D. (2001) Interpreting qualitative data: Methods for analysing talk, text and interaction (2nd ed.). London: Sage.
Topics on Interviewing & Needs assessment decision aid
Heidler, Reeves, and Brackett describe the advantages and disadvantages of focus groups, interviews, and questionnaires, for the purpose of identifying needs.
of qualitative methods and analytic techniques: Common qualitative
Mahoney describes and compares the purposes, advantages, and disadvantages of common qualitative methods: observation, in-depth interviews, and focus groups. This site includes sample instruments in the Appendices.
O'Connor introduces an online module for conducting synchronous interviews (a real time online chat interview).
structured interviews for educational research
This article in Practical Assessment, Research & Evaluation provides a brief introduction to problems and solutions for using interviews in program evaluation.
interviews: Administering interviews
Colker provides a module on how to set-up, conduct, follow-up, and analyze interviews.
interviews: Preparing an interview protocol
In conjunction with “Administering Interviews,” Colker offers a module on how to prepare and structure interview questions and interview protocols.
use of qualitative interviews in evaluation
Sewell summarizes interview types, the designing of an interview, types of analysis, and ethical issues in interviewing. This site also has a short annotated bibliography which can be useful for learning more about interview methodology.
This very useful site from the University of Texas provides a step by step guide to planning and conducting interviews, and transcribing and coding data from interviews. In addition, it suggests ways of using interview findings for evaluative purposes, including how to report.
Topics on Focus Groups & Focus group fundamentals
Grudens-Schuck, Lundy-Allen, and Larson summarize the features of focus group methodology for evaluators to use in generating program-related information.
you call it a focus group?
Larson, Grudens-Schuck and Allen compare the use of focus groups with small and large discussion groups when developing and evaluating programs.
is a survey?
Scheuren illustrates planning of and data collection in a survey study, types of surveys, and concerns about error. Focus group techniques are described in chapter 5.
of Focus Groups in Program Evaluation
Using focus groups for evaluation
Marczak and Sewell describe the effective use of focus groups in program evaluation and outline how they can be used in Jacob’s (1988) five-tiered approach to evaluation.
Jacobs, F. H. (1988). The five-tiered approach to evaluation: Context and implementation. In H. B. Weiss & F. H. Jacobs (Eds.), Evaluating family programs, New York: Aldine DeGruyter.
New York State Teacher Centers Evaluation and Strategy Committee introduces focus group questioning strategies and methodological strategies when teacher development centers have limited time and resources available.
Alderson, J. C., & Scott, M. (1992). Insiders, outsiders and participatory evaluation. In J. C. Alderson & A. Beretta (Eds.), Evaluating second language education (pp. 25-57). Cambridge: Cambridge University Press.
Chaudron, C., Doughty, C., Kim, Y., Kong, D., Lee, J., Lee, Y., Long, M. H., Rivers, R., & Urano, K. (2005). A task-based needs analysis of a tertiary Korean as a foreign language program. In M. H. Long (Ed.), Second language needs analysis (pp. 105-124). Cambridge: Cambridge University Press.
Gattullo, F. (2000). Formative assessment in primary (elementary) ELT classes: An Italian case study. Language Testing, 17(2), 278-288.
Horwitz, E. K. (1985). Formative evaluation of an experimental foreign-language class. Canadian Modern Language Review, 42(1), 83-90.
Karava-Doukas, K. (1998). Evaluating the implementation of educational innovations: Lessons from the past. In P. Rea-Dickens & K. P. Germaine (Eds.), Managing evaluation and innovation in language teaching: Building bridges (pp. 25-50). London: Longman.
Kieley, R. (1998). Programme evaluation by teachers: Issues of policy and practice. In P. Rea-Dickens & K. P. Germaine (Eds.), Managing evaluation and innovation in language teaching: Building bridges (pp. 78-104). London: Longman.
Liskin-Gasparro, J. E. (1995). Practical approaches to outcomes assessment: The undergraduate major in foreign
languages and literatures. ADFL Bulletin, 26(2), 21-27.
Long, M. H. (2005). Methodological issues in learner needs analysis. In M. H. Long (Ed.), Second language needs analysis (pp. 19-76). Cambridge: Cambridge University Press.
Lynch, B. K. (1992). Evaluating a program inside and out. In J. C. Alderson & A. Beretta (Eds.), Evaluating second language education (pp. 61-99). Cambridge: Cambridge University Press.
Lynch, B. K. (2000). Evaluating a project-oriented CALL innovation. Computer Assisted Language Learning, 13(4-5), 417-440.
Mitchell, R. (1989). Second language learning: Investigating the classroom context. System, 17(2), 195-210.
Mitchell, R. (1990). Evaluation of second language teaching projects and programmes. Language, Culture, and Curriculum, 3(1), 3-15.
Mitchell, R. (1992). The "independent" evaluation of bilingual primary education: A narrative account. In J. C. Alderson & A. Beretta (Eds.), Evaluating Second Language Education (pp. 100-140). Cambridge: Cambridge University Press.
Pawan, F., & Thomalla, T. G. (2006). Making the invisible visible: A responsive evaluation study of ESL and Spanish language services for immigrants in a small rural county in Indiana. TESOL Quarterly, 39(4), 683-705.
Rea-Dickins, P. (2001). Mirror, mirror on the wall: Identifying processes of classroom assessment. Language Testing, 18(4), 429-462.
Roberts, C. (1998). Language and cultural issues in innovation: The European dimension. In P. Rea-Dickens & K. P. Germaine (Eds.), Managing evaluation and innovation in language teaching: Building bridges (pp. 51-77). London: Longman.
Slimani, A. (1992). Evaluation of classroom interaction. In J. C. Alderson & A. Beretta (Eds.), Evaluating second language education (pp. 197-221). Cambridge: Cambridge University Press.
Snow, M. A., & Brinton, D. M. (1988). Content-based language instruction: Investigating the effectiveness of the adjunct model. TESOL Quarterly, 22(4), 553-574.
Weir, C., & Roberts, J. (1994). Evaluation in ELT.Oxford: Blackwell.
Using focus groups
Dassier, J-L. P., & Powell, W. (2001). Formative foreign language program evaluation: Dare to find out how good you really are. In C. C. Maurice (Ed.), The Odyssey Continues: Dimension 2001. Selected Proceedings of the Joint Conference of the Southern Conference on Language Teaching and the South Carolina Foreign Language Teachers' Association (pp. 91-110). GA: Valdosta State University.
Lynch, B. K. (2000). Evaluating
a project-oriented CALL innovation. Computer
Language Learning, 13(4-5), 417-440.