Housekeeping
More fiddling around with the links over on the left.
I've kept a link to James Lileks well past the point where I deleted the rest of the links to people who annoy me, just because I really enjoy his writing about mundane things. The stuff about his kid, his dog, the weird bits of pop ephemera from bygone days, and all the rest are brilliant, and while I haven't read the site regularly in a while, I always enjoyed his weird ramblings about apolitical matters enough that I was willing to skip lightly over the occasional brain-dead blather about The War.
That link is gone now, because he finally crossed the line with his recent remarks about Salam Pax. It doesn't quite put him into Little Green Footballs territory, but I've had enough. I won't link to it, but you can get there via Matthew Yglesias or Daniel Drezner. Between them and their commenters, anything I might say about it has pretty well been covered.
On a more positive note, I added a long-overdue link to Michael Nielsen's blog. Michael literally (co-)wrote the book on quantum computing, and has some very nice thoughts on science education as well. Check it out.
I also added some more good links in the "Geek Stuff" section: two site's from Donald Simanek's collection of pages, and MC Hawking's Crib, just because.
Posted at 11:32 AM | link | follow-ups | no comments
Formal Logic Drinking Game
As most people picked up, the "quiz" in the previous post has to do with the formal meaning of "if... then..." in logic. The correct answer is that you need to turn over two cards. One commenter summarizes it nicely:
You don't need to turn over K, because we know that each card has a letter on one side, and a number on the other. So this card can't have a vowel, and it doesn't matter what the number is.
You have to turn over 2, because if there is a vowel on the other side, then you have a vowel/even pair, and the claim is false.
You have to turn over A, because you have to check that this vowel has an even number on the other side.
You don't have to turn over 7, because the claim isn't that only vowels go with odd numbers.
(It's always a good idea to highlight the best student answers in class... Plus, it saves me some typing.)
The book I got this from cites it in a section on cognitive science, and suggests that a formally equivalent but easier to solve problem is obtained by switching the cards to have suspected ages of restaurant patrons on one side, and drink orders on the other. You're then presented with four cards:
[16] [Coke] [52] [Gin and Tonic]
and asked to decide whether you need to go check ID's at the table.
The author presents this as an example of how context is really important, and that abstract problems are different than concrete ones. In support of this, he notes that two thirds of physicists given the original question at a conference picked the wrong answer, while nearly all American adults will get the second one right.
Problem is, I don't think this example really tells you anything about cognitive processes in human beings, so much as it tells you that Logic and English are different languages. The most common wrong answer is to say that you need to flip over the "7" as well, implicitly assuming that "if vowel then odd number" also means "if odd number then vowel." That's not the formal-logic meaning of "if... then...," though-- implication flows only one way. It is fairly close to the everyday English meaning of "if... then...," though, which is the source of the confusion. When people use "if" in normal conversation, they usually mean something closer to "if and only if," in the same way that the conversational "or" is generally closer to the "exclusive or" of formal logic.
Now, I don't mean to say that conflicts between specialized and everyday uses of English words are not important-- as the book points out elsewhere, the everyday meaning of "force" carries a connotation of intent that causes some confusion among introductory physics students, and similar misconceptions crop up to confound the teaching of the physics concepts of energy, momentum, and electric current, among other things. (Nor is this restricted to physics terms-- the lawyers in the house can no doubt provide numerous examples of the same problem in law.) But that's not how the example is presented-- it's put forth in a section about general results of cognitive processes, which I think is misleading.
The message of this specific example is not so much "context is important" as it is "giving words two different meanings is confusing." Which is a perfectly valid point, and one well worth keeping in mind, but not quite as broad a conclusion about human cognitive power as originally suggested.
Posted at 9:10 AM | link | follow-ups | 3 comments
Quick Quiz
I'm reading a book on physics pedagogy (it's a reasonably productive way to kill time while waiting for students who may be showing up to get their final grades), which opens with a long section about cognitive models of student learning. Buried in there are a number of examples meant to demonstrate some point or another, many of which are very good.
One of them bugs me, though, because I don't think it shows quite what it's being used to show. Out of curiosity, I'm going to reproduce it here. I'll post the correct answer, and my complaint about the question, a little later, but if you want to answer it in the comments, that would be somewhat interesting to me.
Here's the scenario: You're given four index cards, each with a letter on one side, and a number on the other. The cards are placed on a table so that you can only see one side of each:
[K] [2] [A] [7]
The claim made about the cards is this: This set of four cards satisfies the property that if there is a vowel on one side of the card, then there is an odd number on the other.
The question is: Which cards do you need to turn over to be absolutely certain that the cards satisfy this property?
Posted at 2:46 PM | link | follow-ups | 22 comments
Solving the Student Evaluation Problem
Blogging has been light to nonexistent lately, not because I'm deeply enamored of the SUV posts and wanted to leave them at the top of the page, but rather because it's exam week at work, and I've had to make up and give two final exams (both of them yesterday). That's done, and I finished tweaking the Excel file with the course grades around this afternoon, so the only thing left to do is to walk the grades over to the Registrar (we still do everything on paper around here), and then pick up my student course evaluation results.
I always dread this part of the term. Teaching is such intense work that it's hard not to become emotionally involved in it, and I end up taking the evaluation comments very personally. I got rotten evaluations last winter term, and it sent me into an unholy funk for weeks-- both because it was kind of a slap in the face to learn that the students didn't like what I was doing, and because the evaluations figure prominently in tenure and promotion reviews.
It's doubly annoying because there isn't even general agreement that the evaluation forms actually measure what they're supposed to be measuring. The Invisible Adjunct links a couple of pieces on the subject (one critical review, and one broadside). (The Adjunct, unsurprisingly, is your one-stop shopping source for student evaluation stories. There's also this recent post, whose comments point to some sharp comments from Letters of Marque, referring to a Michael Froomkin post on the subject. Stop me before I subreference again...) It's commonly known that there's a strong correlation between the grade a given student expects and the numerical ranks given on evaluation forms (references are available via the critical review linked above), and other superficial factors have been shown to strongly influence the way people rate lectures.
In the end, though, probably the best thing I've read on the subject comes from the recently-added-to-the-blogroll Signal + Noise, who weighs in with a personal reminiscence of his experiences reforming a course on probability theory for engineers. (Which also gets to my other reason for being interested in the subject right at the moment...) His comments fairly accurately reflect my own experience with teaching evaluations (including the taking-things-personally problem), and also mirror the conclusions of the critical review: evaluations are "highly reliable, in that students tend to agree with each other in their ratings of an instructor" and "at least moderately valid."
Upon reflection, the bad evaluations I got last winter made a few good points. The complaints were amplified somewhat by the fact that the class met at a bad time (4:05-5:10 MWF), but some of what they said was spot-on. Among other things, I had changed homework policies, and spent less time on problem-solving tips and repetitive drill. Those were generally mistakes, and I won't repeat them (indeed, after taking some of the comments to heart, I got the best evaluations I've received to date the next term, from pre-meds no less).
On the other hand, some of the complaints were a little off-base. We had just revised the syllabus for the course in question, adding a bunch of material, and a number of the comments were based on the increased difficulty level (something I tend to regard as a feature, not a bug). Those aren't especially helpful remarks, and are actually probably counter-productive when it comes to actually educating students, as opposed to just dealing out A's for memorizing a few formulae.
It's also true that "[T]here is no requirement, and little incentive, for students to give thoughtful comments. It is easy to give angry, vindictive, or unsupported comments; easier still to leave the entries blank." I've certainly gotten a lot of blank or one-word ("OK.") written responses, and one term I had two students give me 1's across the board on the numerical rankings (I suspect they were bitter for reasons unrelated to my teaching). The really depressing thing is that a couple of outliers can really badly skew the mean values (which are what the administrators and reviewers mainly look at), particularly in small classes. (My personal preference would be to go to the scoring system used in judged sports, and throw out the lowest and highest scores before doing the average, but that's got its own problems in small classes...)
In the end, the main effect of these things is to complicate the life of beginning faculty. In order to (as one colleague describes it) "solve the student evaluation problem," you need to strike a fine balance; teaching enough material to be providing a useful service while not irritating the students overmuch. Some people claim the current emphasis on student feedback is tipping things too far toward the "don't annoy the students" side, but student feelings aren't something you can ignore. After all, the purpose of education is not just to pound facts into growing skulls, but to kindle some interest in the subject for its own sake. Which isn't something you can accomplish by irritating people.
It's a tricky problem, especially since different sub-groups of students can have wildly different demands and expectations-- there's no one foolproof solution, or at least, I haven't found one.
Looking back over what I've written, I see that, as usual when I tackle anything that's actually complicated, I really haven't gotten anywhere with this. It may be that I'm just babbling to cover a case of nerves over the thought of getting my evals for this term tomorrow. I certainly don't have any really concrete policy suggestions to make on this subject-- in some ways, I hate the whole student evaluation concept, but I do understand why they're used, and it's hard to come up with a better suggestion.
I guess the only real conclusion I can offer would be to say that if, like a commenter on a previous post, you find yourself "shocked now at the idea of those student evaluations having any real impact," well, don't be. And if you find yourself staring at a bubble sheet on the last day of class, trying to condense your experience to a handful of responses on a five-point scale, try to keep the impact in mind.
Posted at 9:26 PM | link | follow-ups | 10 comments