18th
no lifeguard on duty. also, tarpit.
In the most recent issue of interactions, Jonathan Grudin provides this incredibly vivid critique of CHI’s tendency to exlude negative findings:
…it’s true that CHI is disposed to reject papers that report negative results. One program chair said that reporting negative results would “run the danger of discouraging people to go down a certain path… That could be amazingly damaging… Imagine not wanting to go down a negative result path even if we don’t fully understand why.”
I can’t imagine such timid technology developers. The image I see is the glistening lake that once covered the La Brea Tar Pits, luring thirsty mammals to their doom. After a creature thrashed about and was pulled under, the lake closed over it and calmed, as enticing as before, with no warning left behind for the next mammal to come along [3].
While the image of hapless HCI researchers being swallowed up by tarpits is incredibly powerful, this analogy also makes me think of a different problem with under-reporting negative results. Maybe as dangerous as the tarpits, I worry about the perfectly pleasant intellectual ponds that have been closed to swimmers due to unfortunate misshaps related to specific technologies, approaches or implementations. Working in collaborative technologies, I’m continually frustrated by how often the media space “failures” of the early- and mid-1990’s come up in reviews.
Because negative results are so rare, I believe that their chilling effect intensifies, allowing them to persist even when the technical landscape and social climate have changed dramatically. If we were only more open to admitting failure, we might do a better job of understanding and qualifying failure.
I always find Jonathan’s column on the history of HCI to be incredibly thought provoking. I was glad to see that this month was no different.
-link (requires acm digital library account)
