------------------------------------------------------------------------------
---------------------------
[This is a slightly modified version of an article published in Assessment
Forum, Fall, 1995]
STUDENT INVOLVEMENT IN ASSESSMENT
Michael Scriven
Evaluation & Development Group
Inverness, California
August 4, 1995
Schools and the assessment profession have recently become increasingly aware
of the need to involve parents and community in the assessment process, at
least to the point where they can better understand the meaning of the
results. This is not just politically sensible but ethically appropriate.
However, it has only indirect relevance to pedagogy. For a number of reasons,
it seems plausible to argue that involving students in the assessment of
students has greater pedagogical advantages. Perhaps the leading reason is:
(i) the importance of internalizing the standards of merit implicit in
assessment; but there are also (ii) considerations of increasing the
credibility of the assessments, particularly important with adult students,
who are present only by choice. And with adult students, (iii) assessment of
the work of others is often part of the job, so involving them in the process
in school has another justification. However, while the involvement of
students in the assessment of student work is particularly important with
adult students, a strong case can be made that the processes discussed here
are worth using in high school, and exploring at levels below that. Hence it
may be worth setting out some of the steps on the path towards a high level
of involvement, such as various degrees of co-authorship of tests and
rubrics, and co-grading of the results.
For those sympathetic to this line of argument, it may sometimes be useful to
have a taxonomy or scale which reflects the range of options in this area.
For convenience, we might call it the Student Involvement in Assessment
Scale, SIAS for short. It has some value for descriptive purposes, where it
serves as a checklist, and it can also serve as a basis for outlining a plan
for a systematic process of moving towards some ideal level of student
involvement. The following is only proposed as a first draft towards a useful
SIAS, and criticism would be much appreciated. It is not a strictly
cumulative (monotonic) scale, especially since Levels 8 and 9 can be combined
with any earlier level; although the rest are commonly sequential. It is
strictly a taxonomy, but in practice as a hybrid between a scale and a
taxonomy. Apart from its direct applications, it is possible that the most
valuable function of this first effort may simply be to stimulate more
thought about the surprising variety of ways in which students can be
involved in assessment-and the reasons for so involving them.
Note that although the primary application here is the assessment of academic
skills, similar considerations apply, perhaps even more strongly, to
student-or adult-involvement in disciplinary or ethical assessment of student
behavior or attitudes. Analogous considerations apply to consumer involvement
in personnel and program evaluation.
Levels of student involvement are labeled numerically in what follows, and
then described in narrative terms; in some cases a short label for the level
is also indicated in single quotes, within a parenthetical comment. Some
comments on desirability or validity are interspersed, and would of course
not be part of a published scale. "Testing" in this section refers to the
process of administration of either a conventional kind of test or a
structured observation, etc., and the collection of raw data from this; the
conversion of that data into a grade, rank, or (merit) score-typically via
the application of standards-is treated as a separate step (here called
"grading" for simplicity) and is the other essential component in assessment
or evaluation.
Level 0 Student is tested and graded (or scored or ranked, but for
convenience we refer hereafter only to grading) and informed of grade. (The
'student as subject' level, and the baseline for the SIAS). In this and the
next levels, students or their parents should be informed that there is a
right of administrative and ultimately legal appeal, and the procedures for
it; that is also a (baseline) aspect of (potential) involvement.
Level 1 Students are assessed but grades are also explained, either (1.1) in
a general discussion in class, or (1.2) on an individual basis, in writing or
in conference. (This is the minimum version of the 'assessment as education'
sequence.)
Level 2 Students are asked for their comments on the assessment procedure
and on the grades, and these comments are (2.1) given serious consideration
by the teacher, not only for revising the test but for regrading the students
who underwent it on this occasion. The comments may also (2.2) be sent
directly to, or forwarded to, the teacher's supervisor, and taken into
account in the evaluation of the teacher, which may have some effect on the
seriousness with which the teacher considers the comments. (This is the
minimum version of the 'student as co-assessor' sequence.)
Level 3 Students are voting members but not a majority on the appeals
committee. Note: the students on the appeals committee will be different
from, and might or might not be older than those who can appeal. They might
(3.1) be appointed by staff, or (3.2) elected by the students whose appeals
will be considered. (This is a stronger version of the 'student as
co-assessor' sequence.)
Level 4 Students are asked, normally as an assignment for credit, to either
(4.1) suggest topics for, or (4.2) create some-or (4.3) all-of the test
items. 4.2 is an old familiar face in composition testing ("Pick your own
topic and write on it" sometimes with the addition "You will be marked on
your choice of topic as well as your essay"), and it has a deservedly bad
name because of the unavoidably variable standards in grading essays on
different types of topic. But that objection does not apply to non-essay
items, where simple standards of item quality can be applied, and the
instructor can pick good items from those suggested, and explain the reasons
for picking them. 4.1 is also familiar, for term paper topics in
post-secondary, and has much more merit when combined with discussions with
the instructor about the suggested topic. It is less common to have
discussions in class of suggested topics but well worth considering. (Here we
start the 'student as co-author' sequence.) 4.3 has the obvious problems
about abrogation of responsibility hence unfairness because there is no
control over low-coverage or biased tests.
Level 5 Students are asked to either (5.1) assist in creating, or (5.2)
create the scoring guide. 5.2 has been turning up lately, but seems very
shaky unless some of the later levels in this scale are already
included-which has not been the case in the examples I have seen. 5.1, on the
other hand, is sometimes a very valuable exercise in role-switching with the
teacher and hence a step towards understanding in depth-and eventually
internalizing-the standards of merit. (This is a further step in the 'student
as co-assessor' sequence, and of the 'assessment as education' sequence.)
Level 6 Students grade other students' work. In (6.1) the grades are
tentative, and are reviewed by the teacher, possibly (6.2) for a grade. In
(6.3) they are decisive. 6.3 seems both unjustified and unfair, and again
represents an abrogation of responsibility on the part of the teacher. Echoes
of 6.1 are seen in cross-age and other peer tutoring protocols but usually in
an innocuous form. 6.2 is a very strong move towards role-playing the teacher
as evaluator. (This is the 'maximum defensible co-assessor' step.)
Level 7 Students assess themselves, either (7.1) as an exercise, which is in
turn assessed by the teacher, or (7.2) for the record. 7.2 has turned up in
radical experimental education, at least since the 50s, but is unethical
(cheats and those with poor ability to see their own limitations are
rewarded), whereas 7.1 is a chance to assess a key indicator of success in
efforts to instill reflective or critical thinking. (The 'self-assessor'
level.)
Level 8 Students assess the teacher (8.1). While there may be a required
campus form, the thoughful teacher will usually wish to supplement this with
a short form of his or her own, and issue it around mid-term so that the
present class can benefit from changes they suggest. He or she will thereby
be able to role-model the constructive use of feedback, including (probably)
negative feedback. Of course, the feedback must be totally anonymous, and the
students, as well as the teacher, can benefit from assisting in designing
that feature of the administration of the evaluation form, as well as the
form itself (8.2).
Level 9 Here we explicitly move to the metacognitive level, which seems
important for three reasons: (i) to firmly fix the assessment process in an
appropriate intellectual map; (ii) to provide a language for talking about
more specific issues of the kind listed above; and (iii) to consolidate a
kind of knowledge that heads off much of the superficial, unjustified,
dissatisfaction about standard tests and grading that one often encounters in
students and, unfortunately faculty. We can distinguish two dimensions. (9.1)
refers to an understanding of the principles of assessment as applied to the
specific case of this course, with some generalization to other courses.
Concepts covered would include, for example, the commonsense concepts of
validity and consistency, types of test bias and how to avoid them, the costs
and benefits of testing (including opportunity costs), and how to evaluate
different types of assessment including some alternative assessment methods
that are not being used in this course. (9.2) is an orthogonal (breadth)
dimension to 9.1 and refers to understanding the connection of the assessment
of student work samples with: the assessment of: students, teachers,
athletes, curriculum, schools and colleges, and everyday assessment of
products and services, etc. (This is the upper end of 'assessment as
education'.)
Concluding thought A number of threads run through these processes, threads
that relate to deep issues about the nature and role of the student, the
teacher, the administrator, and the curriculum. The reader wil have noticed
references to some of these themes. In one the progress was from "the student
as subject", through "the student as co-author" to "the student as
co-assessor"-and "the student as self-assessor". Another concerned
"assessment as education", another "the student as teacher evaluator". As we
continue to rethink the role of assessment in education, these themes serve
to remind us of the deep connections that exist between assessment and many
other crucial issues in education-and perhaps also to remind us that
rethinking is tied to reinvention, and to new practices. It is surely true
that much of the creativity in assessment lies ahead-and much of it will
yield results under titles other than assessment.
****************************************************************************
* To post to the list: email mathedcc@archives.math.utk.edu *
* To unsubscribe, send mail to: majordomo@archives.math.utk.edu *
* In the mail message, enter ONLY the words: unsubscribe mathedcc *
* Words in the Subject: line are NOT processed! *
* Archives at http://archives.math.utk.edu/hypermail/mathedcc/ *
****************************************************************************