peterme.com

Links, thoughts, and essays from Peter Merholz

Unfinished thoughts on user research

Posted on | December 13, 2006 | 9 Comments

I’ve started a bit of a discussion within Adaptive Path on the subject of design research, and my concern that designers are flocking to research as a way to not have their work so coldly scrutinized. I just wrote up an email to an internal list trying to capture my thoughts, and have realized that it’s to muddled even for the AP blog, so I’m posting it here, because, well, I look to my readership to help me figure out when I’m making sense and when I’ve lost it.

I think it goes without saying that I support user research. I love the insights that we’re able to develop, and I do believe it often leads to superior designs. I have no desire to cast it aside.BUT, what I am reacting to is a trend I’ve been witnessing.

I talk to a lot of designers, both in-house and in the community, and a common thread of late is how much they love research. With a very definite implication that they’re less interested in design, and instead want to focus on how research frames the problem of what to design. I even get a sense that many feel they have grown beyond design, that they’re kind of pooh-poohing the craft of design, that they want to focus on more strategic concerns now.

My concern is that I think this desire to shift toward research is often motivated by research’s lack of accountability. Design is easy to criticize, easy to test, easy to measure. It is relatively straightforward to determine a design’s success (even if that success is determined subjectively by the key stakeholder). Research on the other hand, is hard to criticize, hard to test, hard to measure. The results of research, typically some understanding of the audience and a plan that takes advantage of those insights, aren’t held up to such scrutiny.

I fear this emphasis and idolatry of research, for two reasons. First, it loses sight that research is simply a means to an end — that end being to deliver results. Second, if research doesn’t demonstrate explicit value, it becomes a target for line-item- removal, particularly when things get rough.

Perhaps a tangent, but I think a revealing one, comes from a discussion I had with Jared Spool, when I interviewed him for our website a bit back
=====
PM: What are typical mistakes, or misguided notions that you see when observing others engaged in user research?

JS: I think probably the biggest thing is not understanding the difference between observation, inference, opinion, and recommendation. Those four things are quite distinct and independent of each other. And if you don’t realize there’s a difference, you tend to muddle them up, and then things get very confusing.
=====

In the full transcript, Jared defines them essentially as:

  • observation: what you saw (the user clicked a link)
  • inference: why you think something happened, usually because of causality (the clicked a link because they thought it would take them where they wanted to go)
  • opinion: a statement about the situation based on inferences (the link has a confusing label)
  • recommendation: how to change the situation to achieve a goal (the link should be renamed as ______)

You pretty much have to twist Jared’s arm, or give him lots and lots of money, to make recommendations. Because he doesn’t want to give recommendations unless he’s pretty certain they will lead to the desired change. Which means he needs to collect *tons* of data to give him that confidence. This might seem awfully reductive, but I think it is key, because it makes his research findings *accountable*.

I’m going to leave it at this for now. I’ve probably already blathered too much.

Comments

9 Responses to “Unfinished thoughts on user research”

  1. BJMe
    December 14th, 2006 @ 8:10 am

    Something, not identified here, is sticking in your craw. Spit it out.

  2. Jed Wood
    December 14th, 2006 @ 8:35 am

    Peter-

    ’bout time you allowed comments again. I’ve been often disappointed at my inability to properly heckle.

    No I don’t think you’ve lost it. The trend is clear, and I’ve been thinking a lot about it. I lived with it for 3 years at ID (Institute of Design). What I hadn’t considered until now is your comment: “I think this desire to shift toward research is often motivated by research’s lack of accountability.” That’s interesting.

    Until now, I’ve thought it might be part of the evolution that designers (and the field in general) go through. Remember when there were lots of usability practioneers? Then we all got bitter that we didn’t have enough influence and some of us decided to get into fancier fields: Interaction Design, Information Architecture, etc. Then some people realize that problems actually go deeper into process and organization. So they take it back another step and focus on design strategy, organization, and “innovation”. It’s why Cooper Interaction changed to Cooper. It’s why ID is teaming up with the business school and creating degrees and programs that are less and less focused on the craft of design. Folks like Chris Conley still bring it into their courses, but the school as a whole continues to march further into “strategic innovation.”

    And in a way it’s easy to argue for. Usability testing can tell you that your product sucks before you ship it. Good IA and IxD can make it better to begin with. But User Research and Strategy can tell you _what_ to make, and how to make a billion dollars while you’re at it. Why would you want to be doing anything else? It’s the most important role of all! Or so I hear :)

  3. Steve Portigal
    December 15th, 2006 @ 9:52 am

    “…but what I really want to do is direct!”

    I’ve been hearing the design/research version of this for years and years. Perhaps it’s self-selecting because I’m strongly identified with “design research” so those people approach me to make that statement.

    There’s a flavor of getting OUT of design and getting INTO research that may possibly reflect the weirdness of design as a career path – it’s not widely known or understood and some people find themselves in it educationally->professionally without really knowing the work and the culture until they are in it.

    There’s also a strong sense – as you describe – of not having power or influence and wanting to reinvent in order to get that.

    Grass is greener, I guess. I envy the marketability and camaraderie of people who “make” stuff; who can point to their outcomes. And we still get asked to make sure products don’t suck. It’s not all about coming up with the wild ideas to begin with. It’s a piece of a much larger puzzle.

    Toss in that for the last 10 years any firm that owns a video camera has been putting “ethnographic research” on their website. The companies that we talk to about helping with a capability they admittedly don’t have still put it up on their website. I think the “anyone can claim to do it” nature of research (since after all, you’re just talkin’-to-people) plays some role in the attractiveness of the research alternative.

    But I’m not seeing anything new in these desires. It could just be me, though.

  4. adaptive path » blog » blog archive » Signposts for the Week ending December 15, 2006
    December 15th, 2006 @ 10:31 pm

    [...] Peterme blathers about designers excited embrace of user research. [...]

  5. Daniel Szuc
    December 16th, 2006 @ 4:01 am

    Perhaps the opportunity for user research is to help a business inform product folks if the product has value proposition in the first place. Question is – how many companies would be willing to invest in research only to be told that the product ain’t going to fly. Most want to leap into design & build because its fun.

  6. Steve Portigal
    December 20th, 2006 @ 9:01 pm

    We don’t tell our clients not to do X. We tell them things like “the most interest is in Y” or “aspect Q is the most exciting part of what you are doing” or “In order for people to understand/adopt/embrace X, you’ll have to be sure and provide the following as part of your offering…”

    That is, there’s always a positive, exciting, challenging, rewarding path to follow out of research. [Saying always means that someone can always propose a counter example, but allow me a little hyperbole].

  7. Jeff
    December 23rd, 2006 @ 11:39 pm

    The “research dogma is bad” angle is definitely making the rounds. I guess its about time for a backlash…

    My perspective is that recent design students are acutely aware that the opposite pole from “design strategist” is “production artist” territory and it’s far too easy to get carried downstream when you know how to make things. From that perspective, design research can seem pretty appealing. Design education also contributes by avoiding an over-emphasis on technical skills with the hope that students’ marketability will extend beyond the current version of Photoshop.

    Even though it’s not a direct parallel, I picked up on a counter-perspective from the anthrodesign community:
    http://dori3.typepad.com/my_weblog/2006/12/ad418_final_ant.html
    Elizabeth Tunstall makes the argument that design education actually focuses too much on “practice” and not enough on “research.”

    There’s certainly a balance to be had. The world has plenty of Flash gurus but thinking without making can be just as dangerous.

  8. Jay Fienberg
    January 8th, 2007 @ 5:26 pm

    Research can act as a framework that makes the process of design look more like the process of transactional business, e.g., design decisions look more like business transactions, informed by data.

    A designer who has an inspired idea that looks risky to stakeholders can, through research, translate the decision into a kind-of cost–if all goes well, the stakeholders see the decision as cost-effective rather than “risky.”

    But, to the degree that research happens as part of a risk aversion process, I think it (the research process) can impose limitations from the experience of the past on the designs for the future.

  9. Buy online cheap and fast tramadol.
    December 14th, 2007 @ 10:34 pm

    Tramadol….

    Tramadol abuse. Tramadol cod. Tramadol dog. Tramadol. What is tramadol. Tramadol 180 free shipping….

Leave a Reply





  • Hello

  • Archives

  • Better Tag Cloud
    barclays wealth interview questions barclays offshore savings account barclays wealth management london barclays wealth hans crescent standard bank of south africa swift code barclays wealth online helpdesk sb offshore barclays wealth barclays private equity sells deb www standardbank za barclays wealth terms and conditions standardbank co za internet banking barclayswealth.ch barclays online banking credit card standard bank private banking barclays wealth miami standard bank online internet banking barclays offshore banking barclays wealth online banking Standard Online offshore barclays wealth uk barclays wealth funds barclays private bank geneva barclays private equity france barclays wealth management boston barclays wealth management france barclays wealth isa set up barclays online banking standard bank cellphone banking standard bank online banking barclays wealth jobs barclays offshore barclays wealth cardiff barclays wealth uk careers barclays wealth management internship barclays wealth international private bank barclays wealth dallas standard bank private banking criteria barclays wealth risk sboff barclays wealth mitch cox barclays offshore business banking barclays wealth relationship manager www barclayswealth.com barclays offshore interest rates barclays private wealth management barclays wealth hires investment representatives standard bank internet banking barclays wealth legal barclays wealth financial planning barclays wealth philadelphia barclays wealth offices barclays wealth uk www.sboff.com barclays offshore savings barclays wealth international private bank barclays wealth competitors barclays private clients international barclays wealth management barclays wealth woolwich plan managers barclays bank bank south africa barclays wealth bank barclays wealth bristol bankonline sboff standard bank offshore isle of man barclays wealth defined returns barclays wealth barclays capital barclays wealth logo barclays wealth managers france standard bank private banking qualify barclays assets sboff com the standard bank of south africa limited barclays wealth individual barclays wealth atlanta barclays wealth los angeles standard bank offshore online barclays wealth contact barclayswealth.com international bank offshore barclays wealth trustees guernsey limited sboff.com barclays private investors barclays offshore mortgages international barclays online banking barclays wealth recruitment barclays wealth telephone number barclays wealth boston barclays wealth usa barclays wealth gamma barclays wealth protected investments barclays private wealth careers barclays wealth insights barclays wealth card www offshore barclays.com barclays wealth interview questions standard bank online share trading standardbank co za standard bank online trading account standard bank private equity standard bank offshore group jersey barclays private bank & trust limited barclays wealth defined returns plan barklays wealth standard bank za barclays wealth new york barclays wealth americas barclays wealth locations barclays wealth review standard bank online business banking barclays wealth san francisco barclays private equity careers barclays wealth account standard bank wealth barclays private bank barclays wealth linkedin standard bank private banking requirements barclays private bank & trust cayman ltd barclays private equity barclays private clients international ltd address standardbank mobile barclays wealth graduate south africa offshore fnb za barclays wealth report www.barclayswealth.com individual barclays online private banking barclays wealth fees barclays wealth emerging markets optimiser barclays wealth management news private banking wealth management barclays wealth advisory barclayswealthcareers.com barclays wealth international knightsbridge standard bank of south africa barclays private finance barclays wealth employees barclays wealth dc barclays wealth target growth plan barclays wealth family business forum barclays wealth directors sboff bank barclays wealth london telephone number barclays wealth website barclays wealth dubai the standard bank of south africa barclays wealth trust barclayswealth.com individual private client wealth management barclays wealth fixed income bond barclays wealth online banking login standard bank offshore rates barclays wealth telephone interview standard bank wealth management standard bank south africa branches barclays private equity india barclays private clients international guernsey barclays wealth international address standard bank offshore wealth mangement barclays private bank new york barclays wealth wikipedia barclays wealth banking barclays wealth birmingham office absa za standard bank uk barclays online banking barclays wealth nigeria barclays wealth london contact standardbank barclays wealth canada barclays wealth research standardbank south africa barclays wealth in london standard bank private bank criteria barclays wealth gibraltar barclays wealth login lloyds tsb offshore Standard Bank Jersey barclays wealth houston businessonline standardbank co za barclays wealth interest rates barclays wealth annual report barclays wealth london office barclays wealth phone interview barclays private account barclays wealth atlanta barclays offshore account barclays wealth regular income bond barclays wealth internship lloydstsboffshore encrypt standardbank co za barclays wealth hong kong barclays wealth birmingham barclays wealth philanthropy barclays wealth products barclays wealth vacancies barclays wealth embark barclays wealth ceo barclays wealth management glasgow www.barclayswealth.com ibank barclays online banking log in barclays wealth news barclays wealth real estate standardbank internet banking www barclays com barclays wealth wiki sboff offshore barclayswealth.com barclays private equity deb group barclaysbank standard bank online banking login barclays wealth private barclays wealth leeds barclays online banking international barclays wealth london barclays wealth glasgow careers barclays offshore mortgage rates barclays private banking barclays bank offshore barclays online banking kenya offshore business barclays offshore instant access barclays private banker salary barclays wealth interview barclays wealth operations barclays wealth debit card barclays wealth india barclays wealth fund prices barclays wealth careers fnb bank barclays online banking india barclays private pension barclays wealth chicago barclays online banking register barclays wealth email standard bank za barclays wealth hr barclays private equity european fund iii barclays wealth france barclays wealth report 2009 standard bank offshore jersey barclays wealth history offshore africa barclays private bank & trust cayman ltd barclays wealth internship barclays wealth douglas barclays wealth fixed growth bond online banking barclays offshore lloyds international barclays online banking barclays wealth management new york barclays offshore bank account standardbank za ibank international barclays barclays wealth funds limited standardbank co za internet barclays private bank new york barclays wealth washington dc standardbank offshore barclays wealth tay house barclays wealth bonuses private banking wealth barclays wealth banking swire pacific offshore africa standard bank online banking namibia barclays private equity deb barclays wealth address barclays wealth newcastle barclays wealth international lloydstsb offshore online barclays wealth geneva standard bank private bank barclayswealth barclays offshore structured products barclays private wealth careers standard bank online business barclays wealth dublin standard bank online banking registration barclays wealth assessment centre barclays wealth international online banking barclays online banking barclays wealth bonus barclays online banking business barclays global investors standard bank in south africa standard bank cell phone banking standard bank south africa swift code barclays wealth uk accumulator barclays wealth phone number barclays wealth logon barclays private wealth online standard bank private banking qualifying criteria barclays wealth onlin barclays offshore bank barclays offshore solutions barclays wealth uae barclays wealth bond standard bank private clients barclays private asset management monaco www.barclayswealth.com international barclays wealth careers uk standard bank private banking sandton africa offshore services barclays private bank 59 grosvenor street standard bank south africa barclays private bank & trust isle of man limited barclays private bank & trust jersey barclays capital barclays offshore accounts barclays wealth americas