So you’re listening to a talk, and the speaker introduces a crucial step which you know — or perhaps only suspect — to be completely incorrect. What do you do? Do you raise your hand and point out the mistake? Or file it away momentarily, planning to ask them about it in private afterwards?
And does your answer change if the speaker is a senior scientist who will some day be writing you a letter of recommendation? What if it’s a fellow graduate student giving their first-ever technical seminar, and you know them to be intimidated by all these smart people in the room?
A Lady Scientist and PhysioProf have been talking about these issues. The former wonders whether there shouldn’t be some solidarity among grad students not to make each other look bad during journal club presentations, while the latter says that good perceptive critical questions are always in order.
My own attitude is pretty straightforward, and close to PP’s: it’s never impolite or out of order to ask appropriately probing questions about the material being presented at a scientific talk, regardless of the status of the speaker or the audience. It’s science, and we’re all on the same side; it doesn’t do anyone any favors to hide the truth in order to save someone’s feelings. Science is bigger than any of us, no matter how young and inexperienced or old and respected (feared) we may be. Not only should listeners feel free to ask any reasonable question of the speaker, but speakers should be honest enough to admit when they have said something that might be incorrect, rather than twisting around to find justifications for a slip-up. We’ve all made them; or at least I have.
To the extent that there is any sort of competition going on, it should not be “speaker vs. audience,” but rather “all of us vs. the natural world.” However, having staked out that absolutist position, it’s extremely important to recognize that we live in the real world. For one thing, many audience members tend to blur the distinction between “asking a good question” and “being an asshole.” There are people out there, one must admit, who tend to view seminar questions as a venue for them to demonstrate how smart they are, rather than learning about the subject matter in an open and collegial environment. There’s no excuse for that, and the guilty parties deserve to be smacked around, if only symbolically. Still, it’s no reason for the rest of us to equate hard questions with egotistical puffery, nor to soft-pedal questions that really are sincere. The biggest benefit of a talk, from the viewpoint of the speaker, would be to actually learn something from the questions and comments offered by the audience.
The other complication is that there is a competition going on, whether we like it or not. I personally don’t like it, and would vastly prefer to live in a utopia of unlimited resources where such competitions were unnecessary. But in the real world, there is a limited collection of goods — jobs especially, but other rewards of the profession — and a large number of people competing for them. And that competition never turns off. Academics are always judging each other, inevitably, and will use those judgments when it comes time to recommend or hire or give prizes to each other. So a real seminar is not simply a value-neutral examination of the facts; it’s a social milieu, in which interactions have real consequences.
Which is not to say that we should ever shy away from asking hard questions. But there are different ways to ask hard questions, and there’s nothing wrong with choosing the tone in which such questions are asked to match the occasion. Graduate students giving their first seminars need to learn that they will get asked tough questions, and that it’s okay — it’s not a devastating critique of their worthiness as scientists, it’s simply part of the process to which we are all ultimately subject.
A common technique to help students ease into the responsibility of giving talks is to have students-only seminars where the faculty are not permitted. The motivation for such things is admirable, but ultimately I don’t think they are a good idea. (As a disgruntled senior colleague once said, “Sometimes I learn something from listening to the students.”) Breaking down the barriers between “faculty” and “students,” and beginning to think of everyone as “researchers” and “colleagues,” should be an important goal of graduate school. It can all be intimidating at first, but it’s ultimately beneficial to learn to treat these artificial hierarchies as administrative annoyances, not natural categories.
The most successful graduate students are the ones that start thinking of themselves as colleagues right away. Go to the seminars, sit in the front, ask good questions, participate in the informal discussions afterward. It’s a big universe out there, and we’re all struggling to understand it, and working together is our only hope.
It’s interesting to me how much of a cultural thing this is, not just within disciplines (although certainly there) but even department to department. Where I did grad school, people were very reserved in asking questions, and would certainly wait to the end of the talk regardless; in my current gig at a research institute, everyone is cheerfully merciless and if you get through a talk without being interrupted (and usually challenged) at least 5 times, it means no one was very interested in what you were saying.
I think there is a difference between asking for clarification, with the underlying assumption that there could be one, and discovering a fatal flaw (especially if it is a well-known fatal flaw). The former contributes to the conversation being productive, the latter does not. In my experience then most people would not bother pointing out the elephant in the room , one of the reasons why complete silence during a seminar is a bad sign…
The closest thing to a universal rule is one you implied above: Don’t be an asshole. You are not the local version of Gregory House, M.D. whose utter brilliance will allow you to behave like a jerk. Do Unto Others, etc.
Moshe, I know what you mean, but — if you were the speaker, would you prefer that everyone sat silently through your talk, knowing you were full of crap but reluctant to say so out loud? It’s not an easy question, that’s for sure.
Sean, I think you are saying that instead of an overt statement “You are wrong about _______”, you mean that a good option is to ask a smart Socratic question that will lead the speaker to realize the error, and then perhaps find a way to wiggle out of it gracefully? That seems to me the best way to go. It is right to care about embarrassing speakers, but remember that the audience is being abused somewhat by any mistake that isn’t corrected – that should matter.
Sean, I may have told you this, but when he was in the army during WWI, Lemaitre got cited for insubordination for pointing out to his senior officer/lecturer that there was an error in the artillery ballistics manual. His temerity cost him any chance of officer grade, and while he joked about it in old age, it was clearly one of those life-learning experiences that rankled.
Excellent post, Sean!!!!
I went to grad school at a place where there were many bigshots, but there was the concept of “faculty lunch” which the grad students were kept out of. I did one postdoc at a place where there were only semi-bigshots, and the lunch was open for all.
At which place do you think I produced more papers per year?
More importantly, which place do you think produced more papers per year?
I think the personality of the speaker has to be taken into account. When I was a graduate student, I sat in on a presentation by the late Nobel Prize winning physicist Julian Schwinger. My thesis adviser, who was also present, had the temerity to question something Prof. Schwinger said and got his head handed to him.
The context I had in mind is slightly different, where there are some well-known problems, certainly well-known to the speaker. Most recent example is a seminar I attended where the speaker derived a probability distribution which was non-normalizable and strongly cut-off dependent (aka completely arbitrary), both of the problems are so well-known that you probably could guess exactly the topic and perhaps even the speaker…This is an example where asking a question is not likely to generate new information.
But, I agree completely with the general sentiment, and everything you wrote above, just that your first sentence is a bit more specific than the rest of the post, which threw me off a little…
Not telling someone when you disagree is to not take them seriously, it is supposing that they aren’t capable of discussion and reasoned thought. Part of treating graduate students as scientists is presuming that we are worth engaging with. Of course, there may be people who delight in humiliating the speaker, but that’s a different story.
but Sean, how often does this happen? is it really common for a major point (rather than an aside, or a qualitative explanation, etc…) is totally flawed?
There’s a good Feynman story about this:
“I sometimes attended graduate physics seminars as a freshman to see if I might hopefully be able to absorb anything. Usually they were mostly over my head. I remember attending one seminar at which Feynman was also in attendance, sitting toward the front of a large lecture hall surrounded by a few colleagues. The lights were dimmed, and the seminar was presented by a graduate student from a prestigious East-coast university that shall go unnamed. As I recall, this was a presentation of his doctoral thesis work which he in the process of completing. I didn’t pay too much attention to all of the overheads, as the math was beyond me, and I noticed that Feynman’s head was bobbing and he seemed to be dozing. After about half an hour of rather tedious equations with no audience participation, Feynman’s head suddenly jerked up and he said loudly, “You can’t do that!” The confused grad student froze in confusion, then asked what it was that was wrong. Feynman made a brief reply, and then some of his colleagues, apparently in the belief that he hadn’t been paying attention and was confused, tried to defend the student. Feynman then jumped up in disgust and went to the board to scribble up a few equations. There was a somewhat stunned silence. From what I was able to gather, the student had somehow mixed equations related to wave and particle theories of light in a way that they couldn’t be used together. I don’t know if he was able to salvage any of this thesis work – but it pretty much put an end to the presentation. I felt a little sorry for the grad student!”
(taken from this website: http://www.apnoia.org/ivantic/fizika/feynman/index.html )
Moshe, yes, I certainly agree with you in that context — if the speaker is saying something “incorrect” that they are already aware of, and are willing to look past the purported incorrectness, there is little point in raising a stink about it.
M, it is certainly not very common — for a faculty-level speaker, I think I’ve only seen it happen once or twice. But for grad students, especially someone who is giving a talk at a journal club (and therefore about someone else’s work), it’s understandably more common; they haven’t been doing this stuff long enough to be familiar with all of the major kinds of mistakes. But making mistakes is part of the process; you can’t learn to juggle flaming torches if you’re unwilling to risk getting burned.
Don’t know about you, anonymous, but I’d prefer to find out then rather than when defending my thesis…. At least that way it might be salvageable.
“Gotchas” are something we do when we are being geeks.
The problem with “gotcha” is that in highly technical fields, responding accurately to one is hard to do live and on the spot, and neither side is likely to be able to demonstrate much rigour. This is why we circulate and review formal papers, *especially* on topics at the bleeding edge of existing theory, and geek out over coffee or AIM/MSN/Jabber.
Informal discussions in various media and settings are great for brainstorming, for hashing out partially-formed ideas, and for “gotchas” which are useful in writing up a paper in the first place (if only to add some wording that stems similar “gotchas”). Formal talks are less great for this, but often a better way of catching up with other people’s work with which you might not be intimately familiar.
In highly technical fields, it’s pretty hard to keep up with what Dr Jones is doing if you are busy with your own research, and when you make a few hours to listen to formal talks, the last thing you want is to have the talk derailed by a series of someone else’s “gotchas” which might be better answered in detail in writing, rather than in the margins of presentation foils (wow, sign of age there) or whiteboards.
Sure, it’s not much fun to find out you’ve given an hour long talk on something that is fundamentally wrong, but it’s alot worse to have a talk which is *reasonable* (even if partly wrong) be totally derailed by someone who is unprepared to fully justify the derailment (or make it up to the rest of the audience).
Faculty who make a habit of derailing students’ presentations — no matter how informal — are not helping a student learn how to deliver useful arguments or respond to criticism in a well-reasoned, rigourous fashion. Even in the Feynman anecdote above, there is an unpleasant smell of argument from authority.
I think it would have been more useful to have found the student afterward, gone through the apparent error, and helped walk the student through the options for clarifying the error after it escaped into the wild during the talk.
On the other hand, being exposed to bullies, unreasoned criticisms and out-of-the-blue demolitions of fundamental assumptions is also good preparation for “the real world”.
On the other other hand one can take this too far. By way of answering the unsourced Feynman anecdote, an “anecdote” at the link below is, sadly, an only mostly ridiculous exaggeration of the experience of a derailed formal talk.
Search for “1982 Symposium” in the page at the link. The name “Felton” is probably a joke name with no conflicts at the time of writing (1987) rather than a prescient reference to a very young Edward Felten, whose name seems to be misspelled frequently:
http://www.ccs.neu.edu/home/shivers/autoweapons.html
Then again, perhaps Olin Shivers remembered the future?!
Er… “unsourced” in my comment above was unfair and inaccurate. I simply did not see the “taken from this website” link at the bottom of Anonymous’s comment. My apologies.
More than 90% of the time I’ve been in such a situation (as an audience member), I’ve been wrong. Not necessarily wrong-wrong; often times the speaker has made an assumption that only later she’ll reveal. Perhaps I’ve only gone to very good talks; very rarely have I seen a situation in which a complete talk falls apart, and never has it been unambiguous.
Especially when going to a “near-neighbour” talk — i.e., one out of my subsubfield — I’ve learned to keep quiet when I see the FATAL FLAW OMG and bring it up only after a number of slides have gone by (or after the talk), and only in a very much “I think I am missing something…” kind of way.
In terms of doing it with beginning graduate students; often times they are the least aware of the background assumptions of the field, and I’ve seen them fall apart when questioned on it. I think there should be some solidarity going on.
…looking over the posts that you point us to, it seems there are two issues.
One is “at a journal club, do you hammer people?” Answer is yes — it’s an informal situation, you won’t derail anything, everyone’s on the same team, the work is embryonic, and you have the right to suddenly turn ten minutes of the club into an “educate me” thing, which is the “worst case” scenario.
The other is “at a formal talk, with slides and stuff, do you hammer people?” Answer is no — you are most likely wrong (75% of the talk will be peer-reviewed by those closer to the question that you) and you will derail a talk that other people want to hear and the presenter needs to give.
Sean, could you elaborate on your opposition to students-only talks? In my department, we have scads of talks that graduate students participate in — three different journal clubs that run weekly during the year and are really propelled forward by grad students and postdocs, plus special events/exams and the like, and plenty of outreach opportunities for public talks.
Our journal clubs are pretty ideal environments, I should say. The question from A Lady Scientist was actually pretty shocking to me, because I can’t imagine a journal club environment where it wasn’t TOTALLY clear that the weird, unexplainable, or just plain wrong stuff from the paper wasn’t the responsibility of the presenter! And there’s always a lot of conversation — it’s totally OK to say, as a presenter, that you don’t know the answer to the question, but if somebody else does know, they’re not going to just sit there silently since it’s “your week” — they’re going to share.
But we also have, during journal-club-offseason, a set of talks for grad students and postdocs. Partly this is to ease students, generally, into “giving talks in a friendly environment,” but I also think we’re pretty good about self-policing and being responsible rather than using the ‘easy’ audience as an excuse. I think most of us do a good job of using it as a *special* kind of opportunity. And, of course, it’s not always about giving “talks,” but a PARTICULAR talk — job talks, exam talks, important conference talks, potentially life-changing talks. Additionally, I don’t think this series decreases the number of talks students give elsewhere in the department, or decreases the amount of responsibility overall; it’s just an ‘extra’ commitment that we all make and all benefit from.
Our grads-and-postdocs only event also does a great job of helping to create and support a feeling of community. I have actually noticed that since we started having more events for grads & postdocs only, I have become much more forward about asking questions — not just at these events, but in general.
So, I think like most things in grad school, it’s a tool and an opportunity, and of course it can be misused, but I think it also serves (maybe “can serve”?) some specific purposes.
One thing that often helps, if we’re talking grad student, experienced (feared) scientist, academia, private sector, whatever…is the rapidly disappearing art of civility and manners. Altogether too many take the attitude of, “I’m a (***you enter here)*** so that gives me carte blanche to be an asshole.” To paraphrase Winston Churchill, “If you must kill a man, it costs you nothing to be polite.”
Annie, my objection is not very strong; more precisely, there are strong arguments, but on both sides, and I’m sympathetic. Having a place where grad students can make mistakes away from the totalizing gaze of the faculty definitely has its benefits. But undermining the student/faculty distinction, and beginning to treat everyone as equals, is also a laudable goal. I tend to feel that the latter wins out over the former, but certainly understand those who feel otherwise.
Hmm, maybe we could amend the rules: only grad students, postdocs, and faculty members who actually LIKE the idea of treating young’uns as equals ;).
A grad-student only talk series not only helps the students learn to be comfortable giving talks, but *also* helps the students learn to be good audience members in a group of their peers, including important skills like asking hard questions without being a jerk, asking questions when one is confused and has a good idea that it’s not their fault, and being polite because the tables will be turned fairly soon. Just as its easy to be intimidated giving a talk to senior faculty in your first couple years of grad school, it’s easy to be intimidated into not asking questions.
Both of these goals (and roles) are important, and I like grad student only sessions for exactly this purpose. It also helps remove the over-inflated sense of `grad student solidarity’ mentioned above in talks, as when it’s all grad students there’s no `them’ to embarrass one of `us’ in front of.
Pingback: HowTo: ask questions in a seminar « Entertaining Research
Pingback: The Truth, Respectfully : Anand’s Scrapbook
So do you interrupt the talk, or wait until the end and then ask?