peterme.com   Thoughts, links, and essays from Peter Merholz
petermescellany   petermemes

Home

Archives
Archives before June 13, 2001

RSS Feed

Adaptive Path (my company!)

About peterme

Coordinates
Most of the Time
Oakland, CA

Interests
Current
American history around the time of the Revolution, figuring out how to marry top-down task-based information architecture processes with bottom-up document-based ones, finding a good dentist in San Francisco Oakland
Perennial
Designing the user experience (interaction design, information architecture, user research, etc.), cognitive science, ice cream, films and film theory, girls, commuter bicycling, coffee, travel, theoretical physics for laypeople, single malt scotch, fresh salmon nigiri, hanging out, comics formalism, applied complexity theory, Krispy Kreme donuts.

surf
Click to see where I wander.

Wish list
Show me you love me by
buying me things.

Spyonme
Track updates of this page with Spyonit. Clickee here.

Essays
[Editor's note: peterme.com began as a site of self-published essays, a la Stating The Obvious. This evolved (or devolved) towards link lists and shorter thoughtpieces. These essays are getting a tad old, but have some good ideas.]
Reader Favorites
Interface Design Recommended Reading List
Whose "My" Is It Anyway?
Frames: Information Vs. Application

Subjects
Interface Design
Web Development
Movie Reviews
Travel

 
Don't Get Out of Sorts! Posted on 09/30/2001.

For the project I am working on, we're about to conduct a card sorting exercise. While I know the basics of the practice, I poked around to see if I could turn up any detailed methodologies. I found a number of items of interest.

If you know nothing about card sorting, start here. A brief primer.

Combine card sorting with a survey on usage behavior.

Microsoft Research's take on Web usability, including card sorting.

I'm quite keen on Chiara Fox's affinity diagrams for card sorting, a visualization that helps understand the "slop" that will inevitably occur when asking more than one person to organize a set of terms--strong groups will emerge, but there will still be significant relationships across the groups.

5 comments so far. Add a comment.

Previous entry: "Heading back east."
Next entry: "For faceted classification wonks."

Comments:

COMMENT #1
A good tool for analyzing card-sorting results is IBM's EZ Sort.
introduction: www-3.ibm.com/ibm/easy/eou_ext.nsf/Publish/1876
download: www-3.ibm.com/ibm/easy/eou_ext.nsf/Publish/649

Their tool is actually 2 tools: one is an interface to administer the card sort electronically, the other does the cluster analysis. Ditch the first, but use the second - you get these great relationship trees that can show you some interesting results.

If you're going for extra credit, you can do what we did at Intuit at build a flash-based front-end (to replace tool #1) so that it can be administered over the internet. Add a page of instructions and an example and you've got a tool you can use with 100 participants, not just 5-10. However, I'd do both remote and one-on-one testing - there's a lot of value in hearing their reasoning while they're categorizing (or listen to them over the phone while they do it remotely).

Also, this is a no-brainer, but make sure you pilot the card sort. It's easy to taint your results on the cluster analysis if you've got a couple of poorly worded cards.
Posted by Chad Thornton @ 10/01/2001 07:59 AM PST [link to this comment]


COMMENT #2
This is about faceted classification rather than card-sorting... today the Whitney launched a visualization I created to show the history of internet art:

http://www.whitney.org/artport/idealine/

It uses a classification system where the facets are themes and technologies. I'd be interested in opinions on the piece... one of the implications of faceted classifications for information maps is that the same object can appear simultaneously at different locations. It's unclear to me whether this is a good or bad thing, but it's definitely different from traditional map behavior!
Posted by Martin @ 10/01/2001 08:51 AM PST [link to this comment]


COMMENT #3
EZ Sort caveat:

While IBM is very generous in providing EZSort for free, you sometimes get what you pay for. On a recent project we did card sorts with a large number of cards (130) - though we had 21 top level categories defined already, and just had people matching into those categories or combining them.

EZCalc produced out of memory errors with that large a data set. Jianming Dong (EZSort creator) could run the same dataset on his box with half a gig of RAM, so you may be able to just buy more hardware if you run into the same problem.

There are several Excel stat packages that include cluster analysis, these may be worth the time if you're planning on doing a lot of card sorts.
Posted by Jess @ 10/01/2001 09:23 AM PST [link to this comment]


COMMENT #4
MIT's Electronic Card Wall is a neat project that is currently under development.

"Using EWall, problem-solvers work through a web-based environment to collect, organize and view graphical and contextual information. EWall comprises an integrated set of tools and methods to deal with novel issues in Information Visualization, Information Communication and Information Management."

A glimpse of the project: http://dcg.mit.edu/
Abstract, and Links: http://ewall.mit.edu/
.
Posted by Alex Shapiro @ 10/01/2001 12:56 PM PST [link to this comment]


COMMENT #5
I think the very cool EWall, mentioned in comment #4, is an interesting example. On its face, it seems non-hierarchical since it displays an unstructured set of relationships.

But then my eye seems to introduce its own hierarchy. When I look at the graphs, I see a few big clusters (top level of hierarchy) with smaller subclusters within them (the next levels of hierarchy). Of course there are lines between these clusters, but the clustering is, for me, the dominant effect.

I think there may be something about the way the human brain analyzes a scene that is intrinsically hierarchical. The only way around this that I can think of is to have items appear in several places at once.

Of course all this is based on off-the-cuff introspection and could be completely wrong. I'd be interested to hear contrary opinions.
Posted by Martin @ 10/01/2001 01:57 PM PST [link to this comment]


Add A New Comment:

Name

E-Mail (optional)

Homepage (optional)

Comments Now with a bigger box for text entry! Whee!


All contents of peterme.com are © 1998 - 2002 Peter Merholz.