Does Cluster Analysis Cut the Mustard?

Malcolm Gladwell’s article, “The Ketchup Conundrum” offers an intriguing look at the intersection of marketing and cognitive science. The basic thesis revolves around how a seemingly infinite variety of products have emerged to satisfy discrete differences in consumers’ desires — so whereas 40 years ago, if you wanted mustard, you got French’s, now the mustard market is sliced and diced along a variety of vectors — color, spiciness, tanginess, etc. — with a product to suit most everyone.

Originally companies made One Type of Product, aiming for that which satisfied the most people — think of it as the lowest common denominator. In the 60s there were developed research techniques that demonstrated there is almost never a platonic ideal in food. Different people have different tastes. From the article: “There was no such thing as the perfect Diet Pepsi. They should have been looking for the perfect Diet Pepsis.”

To get a bit technical, the research analysis method is referred to as multidimensional scaling.

It’s relevant to the work I do, because, too often, information architects strive for that platonic ideal, which I think is borne of the tools we have at hand. On an information architecture mailing list, there has been a discussion about card sorting and cluster analysis. We use these tools to get a sense of the relationships that people draw between different types of content and topics — in card sorting you have people group concepts in piles, and in cluster analysis, you analyze those piles across many subjects to evoke patterns that can help you in designing a website’s structure.

(It was on the mailing list where I learned the term multidimensional scaling, in a response from Nathan Curtis.)

Concern has been raised on the list about whether cluster analysis is sufficient — it produces a single hierarchical presentation of the concepts, an analysis that, frankly, attempts to meet that lowest common denominator. I think we can learn from the packaged foods industry that such an approach falls short.

In fact, I’d argue there’s an irony that information architects, who work in a medium as malleable and multivalent as hypertext, which ought to mean it’s a lot easier to tailor content, presentation and organization to different audiences, confine themselves to One True Organizations, while the PACKAGED FOOD INDUSTRY, some seeming dinosaur of mass production, provides the variety of approaches that people seek.

The problem, as I see it, is having access to the tools that enable this richer analysis. We can’t all have expensive statistical software applications to perform this kind of analysis. Ideally, the tools to make this happen will become affordable and offered in such a way that they allow for play and exploration.

It’s also yet another data point that we have to get away from hierarchical structures to more faceted ones — allowing users to make their own “hierarchy” as they move through our information spaces.

[Side note to IAs -- the vast amount of noise notwithstanding, SIGIA-L still has the occasional nugget such as this. I so wish I could unsubscribe, but then I would have missed this!]

2 thoughts on “Does Cluster Analysis Cut the Mustard?

  1. or you could subscribe to/visit the new yorker, which has been delivering steadily and increasingly awesome stuff lately.

  2. We do have the tool that allows the richer analysis. It is called the brain and is a particularly good tool for pattern recognition ;)

    But you know, having all the fancy tools in the world will never make up for the fact that card sorting is a lightweight technique that people are trying to make scientific.