I have been enjoying this thread. Since assessment is the "bottom
line" of education, it deserves a fair amount more attention than it
gets. It is one of the more insidious aspects of part-time teaching,
for instance, that one is generally only paid for contact hours -
leaving part-timers with this conflict of interest: minimize your prep
and grading time to maximize your true hourly rate. Of course, the same
applies to full-time - but these folks are afforded a professional's
salary, obviating this conflict to a large degree. Still, many employed
on a full time basis learn to get by with a 25 hour work week by giving
the same set of tired old (or computer generated) MC tests year after
year. This offers yours truly a great mound of resentment when I'm
sitting at home evenings and weekends making careful, detailed comments
on heaps of FR (free response) tests.
Murphy Waggoner wrote:
<< In terms of preparation for an exam it is much quicker for me to
write a free response (FR) exam than a MC exam. Since I don't keep a
test bank and make each exam mostly from scratch, I find I would rather
spend more time grading exams and less time writing them. On the other
hand, I am usually rushed to grade the final so I often make it (at
least in part) a MC exam. >
Why not keep a test bank? If efficiency is the goal, that is. I
would say that the time saved by using a ScanTron more than compensates
for the longer prep time. Also, I suspect that many teachers are using
that test banks that come shrink wrapped with the text.
<< My main concern is evaluation of the students' needs. On a MC exam I
cannot watch for patterns in thinking while on a FR exam I can sometimes
find patterns of invalid reasoning and provide the student with
appropriate feedback. I also like to have the students respond to the
exam themselves looking for patterns of errors and that is difficult if
they didn't record their work in an orderly fashion. >
Right on. The idea is to making the test a significant learning
tool - for both student and teacher. Where are the misconceptions? By
carefully designing the range of choices in MC, some of this can be
captured - but it is at best a blunt instrument.
Deanna De'Leberto wrote:
<snip>
<< Teachers should provide students with information on the topics to
be tested and the item format (i.e. multiple choice, enhanced multiple
choice, short constucted response, extended contructed reposnse,
open-neded with multiple anwsers, essay, etc.) so that students know how
to prepare themselves for the test. Once students have experience
answering each item type and understand how each is scored, it then
makes sense to have students actually write some items (test developers
call this item-development) and even select a few items for a test (test
developers call this test assembly). My students would write items
based on the content and test specifications I would provide them
(interpreted as we discussed as a class the topics to be tested
(basically I knew what we would agree on beforehand) and decided how
many items to write on each topic and what item formats we would use).
For multiple choice items, it was required that they indicate the
correct key and show how to obtain it, and for each of the distractors
they needed to provide a rationale for how someone would obtain that
answer. >
<snip>
This is great, but it strikes me as requiring a lavish amount of
time. This might work in a HS setting, but at the community college
level we need to cover these topics double time. Maybe on the first
test, just to set a flavor? I'd sure like to try it, but I worry about
the time sacrifice. These students may not need to learn much about the
intricacies of test creation (though it couldn't hurt.) Your earlier
comment that teacher prep courses should have more of this is in the
pocket, though. The degree to which test creation and grading
techniques are trivialized in education truly amazes me. Keep on
posting these ideas, I really like 'em!
Martin Kalmar wrote:
<< I've been using multiple choice test items for the last two years,
with a little twist. My students have to write a brief ustification for
their choices, either explaining why their choice makes sense and/or
explaining why the other choices aren't as good. I've been pretty
pleased with the results. It has pretty much eliminated guessing.
Students who can't think of a good reason for a choice don't bother to
respond since I make clear that a choice without justification will not
earn any credit. >
Yeah, on the rare occassions when I've attempted MC, this is the way
I've done it. Problem is, you can't use the ScanTron then and so the
time savings are out the window. Do you asign partial credit for
multiple step problems? I think it's the issue of partial credit that
wrankles me most (can you tell I'm wrankled?) It's this statistical
"correlation" thing where most MC advocates find their last refuge: "The
results come out about the same, so why bother with all the extra
reading and writing?" - or some such lame dismissal. Well, reading and
writing is a BIG part of what education is all about - even in
'Rithmetic! Part of what a teacher needs to do is set a good example
regarding the importance of reading and writing - MC test don't cover
it.
The Old Pro wrote:
<< With over 150 students papers to grade, and giving quizzes/tests
several times per week, I think multiple choice, true-false, and
two-column matching is a real sensible way to administer tests.
Especially, if the questions and possible answer choices are given to
the students scrambled. >
<snip>
Well, yeah, I have that many too. Grading them is hard, highly
skilled work. But my grandparents had to work night and day in the Ford
factory, so I figure I've got it relatively easy.
Dr. Sallie Paschal wrote:
<snip>
<< I spend hours constructing tests which will accurately assess
students' knowledge. I take it very seriously. With the latest thread
on this list, I was beginning to feel badly about the fact that my
tests regularly include about 30-35% multiple choice items.
30-35% seems reasonable - it gets the students thinking apparatus
oiled and sets a context. I can go for that!
<<A couple of years ago, I attended a workshop taught by an ETS guru who
taught us HOW to consturct a well-written MS item. It must have a valid
stem and well-written foils. Since that time, I have used MS items on
every test I give. (They are often the most-missed items on my tests.)
Really?! This is actually kind of what I'm after in this thread. A
good understanding of how to use MC in a balanced, sensible, responsible
way. Does this guru have a book?
<<I felt the need to include items of this type because as the
instuctor, I can award NO partial credit for a wrong answer. Call me
soft...>
I assume you do offer partial credit on the FR portion of your
tests.
<snip>
Oh well, if you're still here, thanks for reading!
Cheerio,
Geoff Hagopian,
Palm Desert
****************************************************************************
* To post to the list: email mathedcc@archives.math.utk.edu *
* To unsubscribe, send mail to: majordomo@archives.math.utk.edu *
* In the mail message, enter ONLY the words: unsubscribe mathedcc *
* Words in the Subject: line are NOT processed! *
* Archives at http://archives.math.utk.edu/hypermail/mathedcc/ *
****************************************************************************