Why Focus Groups Are Bad For Making Design Decisions
Tuesday, June 2nd, 2009Found on Johnny Holland
Found on Johnny Holland
The local Washington DC Interaction Design Association (IxDA) chapter recently conducted an IxDA ‘09/IA Summit ‘09 “redux” mini-conference, allowing members of the DC user experience community to hear abridged versions of select presentations from each conference. Jared Spool’s presentation focused on Amazon.com’s smart design decisions and how these decisions have made Amazon a successful business. In his presentation, Jared stressed that just copying Amazon’s design isn’t necessarily a good idea and doing so may have unintended consequences.
When working with clients we are often asked to “make it work like Google”, or “we want to do it like Amazon does”. These requests are understandable, because both Google and Amazon are really good at what they do. Google allows us to find information we are searching for quickly and easily, and Amazon makes buying stuff so easy that instead of going out to the store and buying something, we will wait the couple of days it takes to receive an item in the mail. It makes sense that a client would like to copy the success of either Google or Amazon, but just copying the features and interfaces of other sites is not necessarily a recipe for success.
Amazon.com sells a lot of stuff. They sell a lot of stuff because they provide an amazing user experience; they have designed a system that facilitates users buying stuff quickly and easily. I buy a good number of items from Amazon.com every year because they have great prices, I can find what I want easily, and I am given a lot of information to make a decision on whether or not to buy a certain item. If fact, there are people who go to Amazon.com first just for the reviews even if they think they might buy the item somewhere else. There is no doubt that Amazon has created an incredibly useful recommendation system.
Yet, just because this system is effective for Amazon, doesn’t mean you should copy it for your site. In fact this might be an incredibly bad idea. For example, Target.com uses leases Amazon’s platform for their own e-commerce site. Target’s recommendation system works in nearly the exact same fashion as Amazon.com. Yet, if you search for an alarm clock on Target.com, you will get a ton of alarm clocks that are rated very low (or not rated at all). If I were in the market for an alarm clock, I would be wondering why Target sells so many crappy ones.
In his presentation, Spool examines why this system works so effectively for Amazon but tends to fail for Target. Amazon and Target.com both sold a few million copies of the popular book “Harry Potter and the Deathly Hallows”. Yet, if you look at this book’s Amazon page, there are over 3,000 reviews for the book, where the Target.com book page only has 9 reviews. It is obvious that Amazon users are much more likely to post review than the typical Target user. In fact, if you look at the quality of Amazon reviews vs. Target reviews, the Amazon reviews are also much more useful. Amazon reviews are often well written, detailed, and provide a lot of useful pro and con information about the product. Target reviews are typically shorter and contain less useful information than their Amazon counterparts. The fact that Target gets less review for each item also makes the reviews less useful. People are more likely to write a poor review of a product if they have a bad experience than write good reviews when they have a good experience. Thus, Target.com has a lot of alarm clock that are rated very low as not a lot of people are revved up to write a review of that new alarm clock they just bought. Amazon, on the other hand, typically gets a very large amount of reviews for its products. Since products on Amazon tend to get a large number of reviews, there is more likely to be high quality reviews of the a product on Amazon. A product with only 9 reviews on Target is not going to have the same reliability of information for a user to decide to buy or not buy a product.
So, the next time a client asks “Lets do what Amazon is doing”, remember that successful designs are always successful in a specific context. Copying another company’s successful design will not always bring the same success to your company. As user experience designers, it is our job to design the right system for the correct context. It is likely that your context is very different from Google’s or Amazon’s, so copying what they do is not likely to be successful for your situation. As always, informed design that takes into consideration the unique situation of the company’s business, users and technology will always win out over copying successful designs of others.
Last week, Jesse James Garrett gave the closing plenary speech at the Information Architecture Summit. At one point in the speech, he refers to the ongoing struggle in the UX community about definitions, boundaries and in-fighting between the many groups that form the UX community (mainly IAI and IxDA). This struggle has been evident on the message boards of these communities. One consistent point of contention in these discussions seems to be the disagreement on what our job titles should be.
Some say “you cannot design an experience”, therefore the title “User Experience Designer” is invalid and should not be used. Job titles such as “Information Architect” and “Interaction Designer” define two different yet related sets of activities, all under a single umbrella typically known as “User Experience”. Garrett contends that “There are no information architects. There are no interaction designers. There are only, and only ever have been, user experience designers,” and hints that the community should quit fighting about what we call ourselves and move on with designing and creating great products.
I, for one, am less concerned with the accurate semantics of what we call ourselves. “Industrial Design” isn’t concerned with designing “industries”, yet you don’t hear the industrial design community complaining about the semantic meaning of their title. Yet, the craft of industrial design has been around for around 100 years and is a well established discipline in the design of products. The title “User Experience Design” has been around less than 15 years, and her young age is showing in the contentious dialog we maintain among ourselves. Continuing the discussion, despite its often heated tone, is a good thing as it helps the community to define what we do.
Still, the marketing of our profession to businesses is suffering because of our lack of unified terms to label what we do. Because of the nature of our work, we DO get caught up in semantics and meaning of words. Yet, this can be detrimental in the long runs as it clouds the message we send to other about what exactly we do in the process creating products and services.
I am more concerned with how our profession is perceived by others. I believe we should decide upon a unified title for our profession and stick with it. “User Experience Designer” seems to be the term that most people outside of our profession recognize. Even if we cannot “design an experience”, we certainly are in the “user experience” business, and our designs do have an influence on what that experience will be.
I think it is more important that we have a recognizable title that represents the broad set of skills and specialties that encompass our profession. While we may do interaction design, information architecture, usability assessment, etc., “User Experience Designer” should be our branding. It is the name in which others recognize what we do. We should probably embrace it. The activities we perform under that title will obviously grow and change over time, and we should focus our efforts on continually creating and refining our craft. Yet, we should probably cease continually creating and refining our title.
Please read the transcript of Jesse’s plenary: http://jjg.net/ia/memphis/
I have read a lot of books about user experience. I think the first UE book I read was “Don’t Make Me Think” by Steve Krug. Since then I have read many books about information architecture, interaction design, user research, design deliverables, etc. These readings have helped to shape my knowledge and opinions about what UE is, and how it should be practiced.
I have a list of books that I have been recommending to people who are interested in becoming a user experience practitioner. The list below is a good starting place for a UE newbie who needs a place to begin exploring the vast amount of information that exists about user experience design.
This book was first published in 1988, and its contents are still very relevant today. After reading this book, I knew that user experience design was what I wanted to do. Don’s books are easy reads, insightful, forward thinking, and entertaining. Read all of his books, but read this one first.
Great overview of what interaction design is, and how it is practiced.
You’ll like Mike Kuniavsky’s broad selection of practical user research methods–presented clearly and usably. It demonstrates how to discover what is in users’ heads, and suggests how we might balance those considerations with business objectives.
Why software design is broken. Alan Cooper (former software engineer) explains why and what to do about it.
While “The Inmates are Running the Asylum” book tells you what is wrong with software development, this book tells you what to do about it in detail.
This is the only book I know of that focuses completely on UE deliverables.
A complete overview of what information architecture is and how it should be practiced.
Currently I am reading “Sketching User Experiences” by Bill Buxton. I am half way through it, and I am thinking I should add this one to my UE newbie list.
Good Design Faster was a work shop presented by Adaptive Path members Leah Buley and Brandon Schauer at User Experience Week 2008 in San Francisco. This workshop focused on how to create great design while keeping a strict, deadline driven schedule. This workshop practiced what it preached by packing a lot of good content into a small time frame and focused on understanding the process rather than finishing with a result.
Here is a common scenario: you are finishing up your user research and have plowed through all of the qualitative and quantitative data. You have a pretty good idea of the direction your design should go, so you sit down and begin wireframing. Yet, wireframing at this point of the design process is probably a mistake, as wireframes are not a good tool for exploring a large number of design concepts. Wireframes are good at outlining the details of a single design approach, and creating them tends to make you focus on the details instead of exploring multiple concepts. Instead of wireframes, we need a tool that avoids this unnecessary detail, allows for quick review of many different design concepts, reveals the best of multiple solutions, and gets everyone’s input and buy-in.
Quick sketches are really the best tool to use at the conceptual stage. They allow you to go through many ideas quickly and evaluate the good and the bad. Sketching also forces you to avoid unnecessary details, which only get in the way of creating a lot of ideas really quickly.
There are two types of sketching: exploratory sketching and refining sketching. In exploratory sketching, we are trying to produce as many ideas as possible. A tool the presenters provided for doing this was a 6-up template. The 6-up template is basically a blank piece of paper that contains six sections. For each concept you are trying to illustrate, you force yourself to come up with 6 completely different ideas. Often it is difficult to get more than a few ideas out, so tools such as word lists, inspiration libraries, and conceptual models can be used to spur ideas. Word lists are a list of interaction paradigms such as ‘according menu’, ‘rating system’, or ‘drag and drop’ that helps a designer to spur ideas that might not have normally been associated with the problem at hand. Inspiration libraries are a collection of images depicting different interface designs (see Peter Morville’s search screen Flickr page). You use these images, much like the word list, to spur ideas that you might not have normally associated with the design problem you are exploring. Finally, concept models allow you to define a problem as a range of characteristics. These ranges can be a linear spectrum, a 2 x 2 Cartesian chart, or a multi-celled grid. Defining design problems as range of characteristics allows you to frame the problem in a way that can help you visualize a design approach.
Once you have come up with six different solutions to a problem, you then begin to refine your approach by a process called refining sketching. With refining sketching, you take the best ideas contained inside of your six sketches and condense them into a single sketch. Here, instead of exploring as many ideas as possible, you want to focus on fewer, better ideas. A 1-up template (which is basically a single section version of the 6-up template) is used to refine these sketches into a small number of high quality ideas. On this template it is important to use line weight and shaded areas to create a visual hierarchy that emphasizes the ideas that you are trying to convey. Also, create interesting labels names to give your page and features context.
A major theme that emerged throughout the conference was that ideas are not scarce or fragile, but they are actually really cheap. Generating of ideas is really easy. The challenge is going through these ideas and finding the best ones. Sketching allows you to do this, and sketchboards allows you to present these ideas in a way that is both visible and easily iterated. Sketchboards are basically a large piece of paper where you attach all of the 1-up sketches you create for your product/service. Sticky notes can be added to the sketch board to categorize ideas. Also, inputs to design such as your design tenants, personas, etc. can be added to create context around your concepts.
The purpose of these sketchboards is to facilitate conversation. Sketchboards should hang on the wall of your workspace, which will encourage people to discuss their contents. Wireframes tend to reside in files confined to our hard drives making them invisible to the rest of the team. Sketchboards are out in the open where they are easily seen and scrutinized. The visibility of your design concepts allows you, your team, and other stakeholders to visualize and discuss all of the ideas for the product/service. In addition to casual encounters with the concepts, formal design review sessions should be scheduled to evaluate the design direction with both the team and the client/stakeholders. During this session, the sketchboard is presented by the design team, outlining all of the concepts for the design, how they fit together and how the design inputs influenced the concepts. At this time, everyone is allowed to discuss, vet, approve, shoot down, question or modify any of the concepts presented on the sketchboard. The key here is to document this discussion on the sketchboard itself. When you are finished, your sketchboard should be filled with annotations outlining the resulting conversation around the design concepts. Writing directly on the sketchboard itself helps participants remember where and how concepts where presented and how the conversation affected the various designs presented. When finished, the design team has a clear picture of the discussion around each of the concepts, how they were successful or unsuccessful, and how to approach the next step in the design process.
The next step in the design process could be wireframing, or it could be more conceptual sketching. After the sketchboard review, you should have a clear idea if you are ready to commence with the detailed design or not. If there are still questions that need to be answered, iteration on the sketching and sketchboard exercise may be necessary. If you now have a pretty clear idea of the direction the design needs to go, then you are ready to begin the detailed design of wireframing.
The great thing about the above techniques is that you can crank out ideas very quickly. In fact, it suggested that combining these techniques with an agile development process is a way to get to the ‘Good Design Faster’ goal. This quick design can be achieved by using a design schedule organized into five-day ‘sprints’. A five-day sprint might look like this:
Start sketching- each member begins with their 6-up templates to explore many different design approaches to the problem, and concludes with the 1-up template containing a solid design concept.
Tuesday:
Begin to assemble the sketchboard. Share and review the sketchboard with team members and other stakeholders. This should be a combination of casual hallway conversations and a formal design review.
Wednesday:
Begin to create high fidelity wireframe designs.
Thursday:
Share the high fidelity wireframe designs in a formal design review session. Refine designs.
Friday:
Continue refining process and complete the designs.
These design sprints can start over the following week with the next round of design problems to solve. However, sprints also should only be held consecutively for three weeks. The intensity of these sprints tend to lead to burn out, so designers will need a week off to help them refresh for the next round of sprints. The fourth week could be spent tweaking and presenting the wireframes to the client, and also planning for the next series of sprints.
The goal of this process is focusing on having the right ideas at the right time, generating lots of ideas, and creating design activities that everybody can get involved in. Wireframing too early can pull a designer into details that are not necessary at the design concept stage. Sketching allows you to explore many ideas quickly, and allows you to involve stakeholders at the conceptual stage. By the time you get to presenting wireframes, your stakeholders/client is already familiar and has signed off on the design concepts contained in the wireframes. This helps to ensure client buy-in of the designs, and eliminates unnecessary work on wireframing the details of concepts that will not work or will not be accepted by the client.
So, get out your pens, your paper (both large and small), and begin sketching ideas for your next design project.
Jared Spool talks about why experience design is so important
Apple released its new iLife ‘08 suite of applications yesterday. I am intrigued by the new iPhoto and how it helps to organize your photos. Changing the name of a collection of images from ‘Albums’ to ‘Events’ is particularly interesting. This change of terminology not only simplifies the photo organizing process, it also makes users think differently about how they categorize their images.
Apple chose to organize images around events, which reflects the circumstances around which most pictures are taken. If you go on a vacation in the Outer Banks, NC, you typically want to keep these photos together. You take pictures at your child’s fifth birthday party, you want all of those pictures in one place. Apple makes it really easy to do this. From what I can see from the demos (I have not been able to use it as of yet), images are automatically organized by the day they were taken. Since most pictures are taken sequentially over the course of an event, organizing images into in this way becomes simple. iPhoto’s interface makes it incredibly easy to string together multiple days into a single event, or split a day into two or more events.
Arranging photos into albums is the old, physical-world way to organize photos. Typically, we fill our photo albums up until they are full of pictures. So, our organization is based upon how many pictures the album holds. In the digital world, the size of the album is no longer relevant. So, the old term of ‘album’ and the organization scheme associated with albums has become obsolete. Apple has recognized this, and has thus changed the name of organizing images in iPhoto to ‘Events’. This not only makes organizing photos easier, it helps the user to think differently about the process.
Could this small change in terminology be a tipping point in how users organize photos?
I want a Wii, but tracking one down seems to be a difficult task. I’m not sure if Nintendo is purposely trickling out supplies to retailers to maintain the mystique, or if they just seriously underestimated the demand. Either way, they have blown out their competition by creating a gaming system that meets a single purpose: it is fun to play!
The Wii doesn’t have the latest 3D graphics chip, or have 5.1 surrounds sound. It doesn’t have a 1080i high definition picture or play Blue-Ray DVDs. There seems to be no shortage of gaming systems with these features at your local department store. Yet the Wii, with its unimpressive graphics, has been almost impossible to find since its release in November, 2006. Why is this?
I think that that Nintendo has found a niche that overlaps both the hard-core and casual gamer. Its unique game controller is truly revolutionary, and it is a great platform for playing games that are just plain fun. The Wii sports game that comes with the system features bowling, tennis, golf, baseball and boxing (if you haven’t had the pleasure of hitting your Wii friend in the face using the game controller/nun chuck combination, then you haven’t lived). Instead of sitting on your butt pushing tiny buttons to incur actions in the game play, the player actually engages in movements similar to the playing the real life version of the game. Some people have claimed to actually lose weight by playing the Wii on a daily basis.
I believe that this is a classic case of technology driven vs. people driven product design. Donald Norman talks about the transition from technology driven products to people driven products in his book The Invisible Computer. He talks about how when technology makes something possible that was not possible before, it drives the design of products. Early adopters accept these products because it allows them to do something they were unable to do in the past. As the technology advances, the demand for more features begins to drive the process. At some point products become so feature laden that transitioning it beyond the early adopter audience becomes more difficult. Designing products from the viewpoint of the people that actually use the products helps to transition the product from the realm of the early adopter to the hands of the general consumer.
A great example of this phenomena is the cell phone. Up until recently cheaper, faster, and smaller components have driven our cellphones into a feature laden mess. Sure, there is a web browser on my phone, which is really cool because I can access the internet anywhere I can get a cell signal. Yet, i have to push 13 buttons on my Verzon VX9800 to access the browser and type in a URL on my phone. Many tasks on this phone take weird combinations of button pushes to accomplish things. Yet the iPhone only takes two ‘clicks’ before you can type a URL into Safari. iPhone = people-driven design. Verizon VX-9800 = technology/feature driven design.
Paradigm changing products such as the Wii and the iPhone are great examples of people-driven technology, and will help the user experience community solidify themselves into the product design process. The Wii has found an audience with the regular non-hard-core-gamer crowd. This is why the PS3 is looking at price reductions, while stores cannot keep Wiis on the shelf. Will my local department store PLEASE get some in stock soon……