Make Your Surveys Not Suck for UX Projects

If the word “survey” makes you want to shudder, you’ve come to the right place. Building a survey that doesn’t suck for your UX research project is pretty tough. First of all, surveys are very tempting to use because they get you a lot of numbers and they verify your assumptions about your product. However, those are also the downfalls of surveys – they verify possibly false assumptions and can get you “good” numbers that back up those made-up assumptions. And that sucks. This post breaks it down and helps you put together a survey that doesn’t suck.

Why am I writing this? A few months ago, I was tasked with running a survey to collect data about message editor usage for AWeber. I had no idea how to properly conduct a survey and so a search began on the best way to tackle this beast. This post is a short collection of tips on how to put together a survey that doesn’t suck.

Before we get started

Surveys are a great supplementary method to other qualitative tests. Surveys can collect both qualitative and quantitative data.

Writing your questionnaire

A survey != questionnaire. A survey is the method and a questionnaire is the actual thing used to ask the questions. Got it? Good! This post will outline some best practices on writing that effective questionnaire.

Set a Goal

This should come as a no-brainer, but figure out what you want to find out first! Determine, based on this goal, whether a survey is the most appropriate method of research. A few months back, our UX team at AWeber was given the task of finding out the biggest reasons why people were using a previous version of our message editor. A survey was appropriate because: 1. we were looking for quantitative data and 2. we already had qualitative research from which we drew our theories and assumptions.

Collect your research

Gather your relevant user interviews, usability studies and whatever other qualitative research you may have. Comb it for possible answers to your initial questions. As Caroline Jarrett states a relevant, and therefore a well researched question, creates trust between the researcher and the research participant.

I like to write all possible answers onto sticky notes, and then, card sorting them into relevant groups. I prefer writing my answers first and going all Jeopardy on them by writing my questions after I have gathered relevant answers. At this point after you have your groups of answers, you have, what seems like a  mess of information. You have answers to questions not yet written in seemingly relevant groups. So, what kinds of questions should be asked?

Asking the Right Types of Questions

DO ask: closed, open, general, measurable, hypothetical, comparative, and neutral questions
DO use branching logic to ask more specific questions after asking general questions
DON’T ask: leading questions, biased questions, unclear questions, subjective questions, and irrelevant questions
Examples of Questions not to ask:ashville airport survey questionThis is a great example of a question that’s trying to do too much at the same time. It’s asking me about my perception of an airport, however, the answers refer to about 15 different things: personal experiences, prices, flight routes, other airports, flight schedules, etc. You can do a few things to fix this question depending on what you’re trying to find out. Simplifying it and breaking it down into a few different questions is one way.
cornellSo, this is probably one of my favorite questions. I received this in an email from our alumni relations office. As soon as I started filling this questionnaire out, I knew I was in for a treat. For this particular question, I know what’s being asked, but my answers are terribly subjective. I’m being asked to put a number on my feelings based on memory. How in the world do you get value out of such an arbitrary number? These types of questions are particularly dangerous because you get a LOT of great numbers, but they don’t mean anything because everyone’s answer is so subjective.

Analyze Your Research

Don’t skimp on analysis for your research. Make sure you put in the work before AND after running your survey. Following these few simple steps will help you get good data for your UX research!

Putting on Your User’s Metaphorical Shoes – Accessibility

Alexis* knew things were a little different as soon as she touched her computer. She just needed to do a simple task, using simple software that she’s been using for ages, and be able to use her screen reader to do so.

She couldn’t. The software had just released a “newly improved” function, which left Alexis in the dark.

Alexis noted that this new release was “totally inaccessible using screen readers.” So, she did what any sensible person would do and she tried to access an older version of the product, which her screen reader was able to work with. She said that there is “supposed to be a big green button”, which would enable her to get to the former version of the software. In reality, though, she was “not seeing any buttons at all!” The “big green button” was there. However, the css was set to “display: none” on the corresponding element of the pop up menu which would have given Alexis access to the previous version.

The “newly improved” product became a nightmare for Alexis to use because she couldn’t even access the old product. Not only did the new product not support screen reader use, but the website was structured in such a way that the old product became inaccessible as well! So what did Alexis do? She called the help line of the product, where support guided her through and sent her a work-around hack to get to the old product. Were it not for superior customer service, Alexis would have without doubt looked for an alternative product to use.

A similar issue came up with a popular forms website where screen readers were not recognizing radio buttons. screenshot of question with radio buttons The screen reader would simply read the options to the question “What is your favorite fruit?” in sequence “Apples Bananas Oranges Mangoes Kiwi Pears Grapes” without indication of a radio button anywhere.

How did this happen? How can it be prevented? Why wasn’t this noticed?
This happens all the time, but why? We know about Section 508, but why stop there?

This happened because of a lack of empathy for the user who may have special needs. It’s difficult to put yourself in your users shoes when they have different needs, different abilities, and different goals. However, that doesn’t make this task any less trivial. As a matter or fact, by keeping those with different needs in mind, we can help those who may not realize they have a need. As UX designers we must always remember to design for the lowest common denominator. For example, just like The Bradley allows those who have visual impairments to feel the time, it helps many others in similar situations where you can’t look at your wrist or cell phone to be able to feel the time. You can feel the time in a dimly lit room, at a happy hour when you don’t want to pull out your phone, or during dinner at the in-laws. Kel Smith mentions another great case study in Digital Outcasts. The design of wheelchair ramps on sidewalks not only helps wheelchair users, however, also allows for use by families with strollers, those rolling their suitcases, and just makes for easier walking on and off the curb. Therefore the tool gains universal use and purpose.

How can you, then, put yourself into the mindset of users who are different than you? Luckily, there are plenty of tools, which you can use to get yourself in the spirit.

Tools You Can Use To Put Yourself In Your User’s Shoes
It goes without saying that using some of these tools are not replacement for proper qualitative analysis and testing of your product. However, they can work as good rule-of-thumb tools during your design/creation process.

I recommend selecting two or three tools, and sticking with them for the most part. Otherwise, if you’re looking for a much longer list, I recommend taking a look through this list on the W3 website.

What kind of tool do you use to ensure your product/website is accessible? What do you do to prevent accessibility issues? What impact have your efforts had on those who may not necessarily realize they need the extra accessibility?

*names are changed

Design in the Future

Smashing Magazine is a web design destination for information on best practices, current trends, and even little tutorials on how to master emerging techniques and new creative suite  functions. Related to the topic of patterns and textures in design as in Annette Tietenberg essay “The Pattern Which Connects”, this post (from about a year and a half ago on Smashing Mag) describes a now-common trend of incorporating textures into websites:

It is interesting to see how textures around you, such as the weave of the fabric from your oxford shirt or grouting between tiles or leaves in a tree, have become instrumental in making a website, a thing completely digital, more organic and natural.

The paradox between making something digital look more natural and something tactile and concrete look modern, clean, and sleek is a fascinating concept which may signal the future of design.

An example of this is the following this wedding website. It is a beautifully put together piece of work displaying a marriage of information and peaceful outdoor scenes*.  As the viewers, we are absorbed by the scenery on the page. It takes us from the computer and into a park, in front of a monument, or onto a staircase. We are there with this couple celebrating their joy, even though we have no idea who these two people are.

We are moving from nature to a representation of nature. A representation that can change at the click of a mouse button or a remote control.

This trend is also reminiscent of a Star Trek concept known as a holodeck where an empty room on the spaceship Enterprise serves the function of creating an artificial environment so real that the occupants of the Enterprise use it for recreation or simulated battles.

Is this where the future of design is heading? Is decoration meant to be contained in a neat sleek digital box to satisfy our ephemeral tastes?

*pun intended

This post originally appears here.

To judge or not to judge

I’ve recently run into an instance where I was researching a solution for an issue with a project I was working on. The project involved taking a paper form, scanning it into a computer using Acrobat, using OCR text recognition to extract information, and sorting the form into appropriate folders in Sharepoint based on the information extracted from the PDF.

I ran into a product that would extract certain fields into Sharepoint through stackoverflow, however, when I saw the vendor’s website I became incredulous. Why should I trust a product made by this vendor when their website doesn’t even function up to my standards?

The website looked modern enough upon my first inspection. However, upon closer examination, I realized it wasn’t responsive, their forms’ javascript wasn’t validating correctly, and they ignored basic web development rules such as inserting images in places where some styling would have been more appropriate.

I took a screenshot of the two different phone numbers and their validation result.

Which one is a real number?

Which one is a real phone number?

I decided to give this product a shot. I’ll let you know how it goes. Until then, would YOU trust a digital product if the vendor’s website had such huge errors? Is this a case of judging a book by its cover or being a cautious customer?

Don’t Click On Phishing Attempts

Lesson One: Don’t Click On IT!
Phishing attempts are prevalent everywhere: I’ve seen the most creative excuses for getting private information stolen. Those gems never cease to amaze me. Sometimes… ok, most of the time, I feel the thought process of my our customers is something like this:

clicking decision map

But seriously, people generally fall into the following neat categories when dealing with clicking remorse:

  • The gullible: “But Thunderbird didn’t tell me that it was that fish email.”
  • The apologetic: “I’m so sorry, I had no idea that this was a bad email. I don’t think I’ll be able to sleep at night knowing I’ve done your IT department wrong!”
  • The defensive: “I didn’t do anything. I never give my information to anyone. Someone else must have given out my password. I’ve never had this happen, ever. Maybe my wife gave it out.”
  • The sheepish: “I cannot believe I just did that. *face to palm*

There are a few cases where I feel such empathy for users, I just want to read their email out loud for them to prevent phishing from happening. We’re very familiar with those cases: the elderly, those who face language barriers, and those who don’t use technology as obsessively as I do. I’ll even do creative email reading interpretations. That’s not very possible, so we have to take precautions.

So, how do you protect yourself then from phishing? Don’t click on it. If it looks real and asks for your password it’s a phishing attempt. If it says it came from the “Master IT Desk” and it needs you to “confirm” your password, it’s a phishing attempt. Just. Don’t. Click.

Front End Dev & UX Designer. Dancewear designer. Mouflon fan.