O

No entries for "O"

P

Paltridge, B. (1992). EAP placement testing: An integrated approach. English for Specific Purposes, 11, 243-268.

Paltridge gives a thorough description of a performance-based, criterion-referenced placement test for international students entering into an English language academic environment. He assumes that language testing should generally operate as an intermediary between real-world language tasks and behavior, on the one hand, and theoretically motivated instructional practice on the other. It is essential that such tests be criterion-referenced, with criteria based on real-world language needs. Performance ability with respect to these criteria should be elicited using direct methods, that is, by causing examinees to use the language in authentic situations during assessment. Assessment should therefore also reflect the integrated nature of language use, that is, the integration of different modes of language use, of thematic content, and of rhetorical organization of elicited language samples.
Paltridge's resulting EAP placement test integrates the four skills (listening, reading, speaking, and writing), response modes, and university-level academic content into an exam that takes a little more than two hours. The test draws from existing sources for many of its task types and rating scales (including the IELTS and the OTESL exams). These tasks are graded according to difficulty and cognitive complexity based on needs analyses conducted in England in the early 1980s. Tasks also correspond with the instructional curriculum used for further language training at the university. The theme of all portions of the test centers around "the problems of studying abroad" (p. 250). Listening and reading portions of the test involve selected response formats, whereas the writing and speaking portions involve constructed response formats. The test begins with listening and reading, each section focusing on different aspects of the general theme. Then follows the writing section, where students respond to the theme presented in the previous sections by constructing a problem/solution type essay. The test concludes with an interview about the student's backgrounds and needs, and based on the FSI OPI format. The first two sections are evaluated objectively (right/wrong answers), and the last two sections are evaluated subjectively by raters using an analytic scale and divides language production into several categories and ability levels.
Decisions based on the exam separate examinees into two groups: those able to enter directly into specific university major programs and those required to receive further English language training. The exam feedback forms provide specific information about areas in which the examinee should receive more practice and instruction. The test itself is considered to have face validity and a desirable washback effect, in that it introduces examinees to the kinds of language tasks they will be faced with in university course work. Technical reliability and other kinds of validation measures are planned. Of special concern is the application of criterion-referenced statistical measures of consistency (versus traditional statistical estimates applied to norm-referenced tests) and investigation of the predictive validity of the tests with respect to admission of students into university academic programs. Finally, the assessment techniques used in the placement exam are also intended to inform on-going classroom evaluation of language development. This enables formative and individualized assessment that has a directly positive effect on student learning.

Paulson, F. L., Paulson, P. R., & Meyer, C. A. (1991). What makes a portfolio a portfolio? Educational Leadership, 48(5), 60-63.

Portfolios in the classroom can be related to the use of portfolios by artists. They reflect the student's own perceptions of the learning and production process, and they can enable students to take control of their learning. Portfolios bridge the gap between assessment and instruction. They are at the same time instruments for evaluation and instructional tools. Development of a portfolio should reference the following guidelines:
(1) Portfolios must show the process of student reflection on learning.
(2) Students should own the portfolio; they should therefore select what goes into it.
(3) Portfolios are not cumulative folders; inclusion should be meaningful.
(4) Student classroom activities should be conveyed by the portfolio (including metacognitive activities).
(5) The portfolio should, ultimately, contain only that information that the student is willing to make public.
(6) Portfolio purposes should not conflict (e.g., student versus school district goals); they should reflect instruction related progress.
(7) Student growth should be illustrated.
(8) Students should be given portfolio models.
Portfolios should, therefore, show student performance in context, involve the student as participant in assessment, and offer a place for students to become independent, self-directed learners.

Pollitt, A., & Hutchinson, C. (1987). Calibrating graded assessments: Rasch partial credit analysis of performance in writing. Language Testing, 4(1), 72-92.

Pollitt and Hutchinson try to develop a model of writing assessment that can simultaneously account for the difficulty of tasks and the performance ability of examinees. Traditionally, these approaches have been seen as separate ways of distributing students according to ability, the task difficulty model assuming that better examinees would do better on more difficult tasks, and the performance model assuming that raters could reference a graded set of criteria for any examinee performance. Using the item response theory Rasch model, the effect of both examinee performance ability and task difficulty can be examined in relation to examinee score effect. Using three theoretical models -- a set of five independent language writing functions developed by Finocchiaro and Brumfit (1983), a set of communicative competence criteria (appropriacy, ideas, and expression) based on Canale and Swain (1980), and a taxonomy of response ability (prestructural, unistructural, multistructural, and relational) developed by Biggs and Collis (1982) -- they implemented The English Language Skills Profile writing test. Fifteen resulting scores (task x criteria x rated level) were generated for all examinees.
Using Rasch analysis of the resulting scores, Pollitt and Hutchinson were able to identify reliability of the test, variance attributable to task type, task criteria, and examinee performance level. Analysis of the TELS revealed that for higher level examinees, that is for purposes of separating those examinees who are capable of showing competent performances (upper end of the rating scale), separation of tasks according to competence components was almost unanimous (judgments of tasks for expression were all more difficult to achieve competence in than judgments of tasks for ideas, which were all in turn more difficult than judgments of appropriacy). However, when examining the tasks according to the ease with which engagement (on the lower end of the rating scale, opposite competence, a kind of minimal ability level) was demonstrated, the characteristics of the task (but not the area of competence demonstrated) were determinant. This rich level of information, regarding the variability in examinee performance attributable to task type as well as the kind of competence exhibited, is not yielded by traditional correlational analysis. The authors also point out that such applicability could be extended to any assessment of performance judged against rating criteria.


National Foreign Language Resource Center Homepage

Language