From Rate Your Students, via Ernie’s 3D Pancakes:
My classes are large, so I mostly use multiple-choice tests. One day, being one question short of a nice round number, I used this question: “The answer to this question is D. Be sure to mark D on your answer sheet.” The offered choices were: (A) This is the wrong answer. (B) This is the wrong answer. (C) This is the wrong answer. (D) This is the correct answer. Be sure to mark it on your answer sheet. (E) This is the wrong answer.
About 20% of the students got it wrong. One possibility is that they couldn’t read any English. Another is that they didn’t care. But one student had the courage to admit that he always marked B for every answer (true, that’s what he did) in hopes that all the answers were B.
One wonders how these students would have performed on Jim Harrick Jr.’s basketball exam?
Not being used to multiple choice tests it took me a while to realize that if there are five answers for which always one is correct and the score for a correct answer is 1 the score for no answer at all (or two answers or similar) should be 0 while the score for an incorrect answer should be -0.25 such that Student B goes home with the same score as somebody giving no answer at all.
Normally, my TAs did all of the grading for my lecture classes which is both a good and and bad thing. It is bad since I did not get to see what kind of answers I was getting to the questions, and it was good since if I knew how bad some of the answers are I would lose my sanity.
On one exam we in a real time crunch, so I decided to pitch in and do some of the grading. It was the loop the loop problem where a block slides down a frictionless ramp and then around a loop of radius R. The question is how must the ramp be for the block to make it around the loop. It uses the concepts of potential energy, kinetic energy, and centripetal force.
I was so disappointed by some of the answers that I went into class the next day and told my class that I would put that exact problem in the final. I did and many of them still got it wrong.
A few years back in an engineering physics class at UCLA, I did as an example problem in class one of the classic Gauss’s law problems. Thin charged wire, surrounded by a cylindrical shell of specified amount charge; calculate the E field everywhere yadda yadda etc. etc.
Then I gave it to them in identical form …
-on homework
-on the first midterm practice test
-on the first midterm
-it was done in the final review session
-on the final exam practice test
-on the final exam
Many many students out of 200 still didnt do it close to correct on the final.
Horses … water…
I have always planned (I guess because I’m evil) to give a multiple choice test where the correct answers are all “b”, but to deliberately make the questions a little easier to see who can overcome the weirdness of having to select all “b”.
I suppose this was because when I didn’t have a clue as to an answer, I’d look at the previous few and try and select a nice pattern. This happened more than I’d like to admit.
I had a true-false test in high school in some history class (section of the final, I believe) where all 50 answers were “true”. It really does mess with the mind.
I suspect it’s because some of the students ran out of time and randomly ticked off the last few answers. Heh, I’ve done that myself!
It’ll be interesting to see the results if this question had been Question 1 instead.
Multiple choice is easy to grade but can often be an inaccurate measure of a student’s knowledge. This is especially so when there’s no partial credit score (1/2 or 1/4 points for more plausible wrong answers e.g. when the difference between 2 answers is only in the sign).
A question to Peter Armitage – do you give a practice test and grade it? (Or do you let students work on it as an assignment?) In several of my classes I’ve given easier versions of exam questions as a quiz a week before the exam. I find that this lessens exam anxiety and also imporves scores.
Thanks for the shout-out. Your page must be very popular because it has really spiked our hit counter today. Many thanks.
RYS
citrine: Gave practice exams, but they were take home and not graded. Didnt give solutions either. But I did solve the harder ones in the class or review session before hand.
I used to give tests where the students would have to write Thomas Hobbes’s name. The name “Hobbes” appeared, correctly spelled, 4 or 5 times on the test sheet itself. About 20% of the students would, nonetheless, misspell it consistently; usually as “Hobbs” or “Hobs”.
I formed the opinion that you could write the correct answers on the board, and tell the students that you had done so, and 20% would still fail. This was one of those opinions that you’d describe as “half joking”; I have to admit I’m still surprised by finding that RYS has actually confirmed the number. (I guess it’s even more surprising that Harrick’s students did so well; he must’ve really drilled it into the kids how they were supposed to do their tests.)
(F)None of the above. No question, no answer.
At the start of all of my lower division (mostly freshman) classes, i would hand out a quiz titled: “review of what you already know” It really was a list of 25 instructions phrased as questions. Number one said: “Before proceeding, read all questions thoroughly first!” Number 25 read: “Do not respond to numbers two through twenty-four in anyway whatsoever!” Two through twenty four asked students to do silly things like use their writing implement to tap out a 5/4 beat, or rub their stomach as they pat their head, or twirl their paper on their writing implement. It was always hysterical to watch as much as 90% of the class performing these feats, and then get to the last question. I always made sure i was aware of those who had followed the first instruction. They would be the ones who asked the important questions that the others would need to know, but wouldn’t think to ask. It is sad that this worked for years. I suspect that example above has proven to sustain itself over a long period of time as well.
The other thing that always got me was the incessant note takers, who would carefully record every utterance. Faithfully they would transcribe the inevitable nonsense i would provide, just to check to see if they really weren’t paying any attention to what i said. Thank the stars that someone would actually challenge it; ooh to know someone was paying attention.
About 40 years ago I had a teacher that asked the following question:
“Do you know Heron’s formula?”
One student came in drunk and got it right. His answer? “No.”
Sean,
This is completely off-topic, but I just heard an interesting cut from an album by a Chicago-area pianist and composer named Jo Ann Daugherty. Are you familiar with her?
Peter–I had a similar experience, except it wasn’t anything as subtle and deep as Gauss’s law–it was, wind vector is blah, airplane’s airspeed vector is bleh, what is the speed of the plane relative to the ground?
Everyone was baffled.
Well since we are talking about teaching and such I reckon I will add in a couple of pennies:
One of the most striking things upon returning to college for me, after being in the professional world for seven years, was the level of instruction. Perhaps the audience here could answer a question or two? Are there any instructional standards that you are held to? For instance, is there a requirement that you follow any of the best practices in education? Without looking them up, could you list them? ( That is, do you look to apply them? )
We would hire and fire people based on their ability to apply best practices at work. Failure to do so would end in termination, although that wasn’t really an issue. We certainly would test potential hires to see if they knew best practices. ( This was in software.)
I have to say that the professors I have encountered don’t make the grade as educators: specifically they do not apply the best practices in education to their teaching and in so doing they destroy the learning environment.
They also destroy students. I have been next to some bright students when they have broken down and cried because the professor was no help: the professor didn’t want to help. This was a upper division physics class, and the student was an A level person. The net result is that person will not continue on in Physics. They decided on another path. I have seen this happen several times now with other hight GPA students.
What strikes me most is that by not learning and applying the best practices in education you not only undermine the learning experience, but you hurt the students, who are paying you to teach them.
For me that is the most disgusting aspect: destroying a young persons dreams, who, really, is paying the instructor to help them achieve their dreams.
I could rant on further, about no instruction in critical thinking, and application of critical thinking, but I won’t. 🙂
BTW when I speak of critical thinking and such thinks I mean as defined at http://www.criticalthinking.org. Among other things, I recommend “How to Improve Student Learning”.
I will say that I would fire anyone who constructed tests to play with their students. That is abuse: regardless of the students attitude. You should take teaching more seriously: in a very real sense, peoples lives are stake.
Take care
Chris W., I haven’t heard of her before. But the clips sound good, I’ll keep an eye out.
Hehehe. That’s too funny and I have no problem believing it. Also, a test with all B answers or something similar would freak me out.
blah blah blah “best practices” blah blah blah.
Yeah, there’s stuff like that coming out of School of Ed types; it gets rather strenously ignored in physics, because Ed types wouldn’t know a metric tensor if you smacked them across the head Misner, Thorne & Wheeler. I’m sure they’ve got copious data on how best to teach 1+1=2, but QFT? E&M? It is to laugh. If you don’t know the material inside and out, how could you possibly know how it should be taught?
(yeah, and their attitude is just endemic in business management too: “I may not know anything about supersonic aircraft, but I learned how to manage at a major retailer, so I can manage a defence aerospace company.”)
And while “critical thinking” is, in abstract, a good and useful thing, the term has been taken over by the ignorant as just another buzzword.
I’ll have to agree with both jim and Not. Some teachers are wrapped up in their selfimage of brightness, and some students are too.
It happened to me, I think; I thought probability theory was easy, so when I hit some snaggles and our whiz kid professor wouldn’t take time to help at a level I understood, I had to take 3 exams until I passed. Statistics for the same teacher was easier, since by now I understood I had to learn some of the basics too. Now those two are among my favourite subjects. Who could know? 😉
But students doesn’t all leave their brains behind when they enter the class room. The first few weeks at the university, new town and all, two of my peers entered late studying a map. The teacher, already known for his distaste of late enters, stopped his lecture and waited silently. The student who held the map sensed something was wrong, but thought quickly and laterally. He let his gaze sweep the room while starting to fold the map, and commented “And here we are!”. Even the teacher laughed.
Not a string theorist,
IP numbers are always helpful in recognizing the “same ole same ole” people, whose reiteration is might be curtailed toredundancy?
Do they lack, “degrees of freedom?” Oui! Non?:)
Professors and lecturers at a university do not get fired for bad teaching if their research is top-notch. The other way around is less certain. I was told this explicitly at a job interview recently.
Also, sometimes you get landed with a course that is not directly your research area. And then the bad teaching is just a manifestation of your own lack of understanding of the subject matter. For example, when you’re used to deriving electromagnetism from a Lagrangian in a covariant way, try going back to the old noddy way of messing about with E and B fields. And with the right amount of hybris and shoddy preparation you’ll find yourself confused and waffling in front of an even more confused audience. The only remedy is to make sure you really understand the material you teach.
And never ad lib unless you can clearly see the line of reasoning all the way to the end.
This is precisely the arrogant, boy’s club attitude that drives people from physics and will have us all out of a job soon. While I have little use for educational buzzwords, I have even less for professors too full of themselves to be concerned with their teaching effectiveness, or even conduct civil discourse. I hope, for all our sakes, that Not a string Theorist is Not a Professor. I am not a School of Ed type. I am a physicist who has read Graviation and How People Learn. It’s not that hard.
I will say that I would fire anyone who constructed tests to play with their students. That is abuse: regardless of the students attitude. You should take teaching more seriously: in a very real sense, peoples lives are stake.
Since this may have been directed at me, let me deconstruct this statement a bit. Jim seems to hold the view that university students are young fragile minds who are willingly and consciously engaged in a process upon which their very lives depend. I take teaching beyond seriously if by “seriously” Jim means some form of formalized pedagogical structuring of which he personally approves. I taught (now retired) in universities, first in ethnic studies, then in School of Ed. I am a School of Ed type, and the test i constructed, that seems so offensive to Jim, as designed by a brilliant education educator decades ago, for the very purpose of getting new young teachers to understand that their own future students will not, in all likelihood, follow their given directions. Since i am talking about teaching teachers and not about teaching k-12 curricula i see the process very differently. And i provided my examples above from that experience.
I do however agree to some extent with Jim in regard to the capacity of university professors to provide successful pedagogical performances. I tried for years to convince Education departments to provide teaching workshops as part of doctoral work for the other departments. Ph.D. candidates need to have been exposed to the concepts and philosophical underpinnings of good teaching. The centuries of assumption that simply by attaining a doctorate and crafting numerous papers and texts for voluminous publication bears little, if anything at all to teaching. Yet we go on and on with the process. Universities with large endowments can afford to hire better teachers through searching across other less financially enriched institutions, gleening the creme de la creme. This isn’t the best we can do.
The student skills necessary for university success, such as critical thinking and enhanced reading comprehension (and a dozen absolutely necessary others), are learned at much earlier ages than the last couple of years of high school. There is research that has demonstrated that some of these student and study skills (these are two very different things) are linked to infancy and early childhood reading. The problem in our culture is that many of the university students’ k-12 teachers had neither the extensive requisite training nor realistically the extra time to facilitate the growth of these skills. If i wanted future teachers to keep teaching like they were taught, i certainly would have never provided my students with a quiz that challenged their sense of process. Paying attention is fundamental, more so than critical thinking, yet it is for the most part a skill savored by a few and ignored by the many.
Hi Spyder,
Since this may have been directed at me, let me deconstruct this statement a bit.
I had in mind what JustAnotherInfidel wrote about.
“Jim seems to hold the view that university students are young fragile minds who are willingly and consciously engaged in a process upon which their very lives depend.”
I don’t see them as young, fragile minds. However, I definately see them engaged in a process that in many ways the rest of their lives depends on. Additionally, they are paying the instructors and the University to help them: so they are customers too, in a relationship that is fundamentally a helping relationship. (Teaching being a form of helping.)
The poor levels of instruction at the University that I attend leads many students to abandon their dreams. That is bad: certainly it is not what the students is paying for, and it is destructive to the student’s life.
As a suggestion, I would recommend that Universities simply state that they do not hold their instructors to any educational standards. That job performance is measured on research and not teaching. This would help the customer, the students, more effectively decide what schools to attend: they can know what they are purchasing.