This is part of a series of posts that will discuss bad experiences. The goal of these is not to complain, but rather to relate ideas about UX to poor implementations. To be honest, it’s really easy to have the best intentions in mind only to mess up the execution. Hopefully UX designers can take note and beware the pitfalls that exist.

I'm very fortunate to have a loving (and growing!) family. My wife and I recently welcomed our third child into the world.

I'm very fortunate to work at a great company that is willing to invest in User Experience and listen to UX professionals on many different topics.

Somehow survey design has been a thorn in my side professionally and personally.

As I have settled into my role at Auto-Owners Insurance, I have been fortunate enough to fulfill the role of research lead for UX on our internal UX consulting team. It's given me more insight into how the company can improve our quantitative and qualitative research. This includes surveys.

Especially in this remote-work-first world, our ability to perform "normal" UX research (contextual inquiry, moderated in-person usability studies), we have come to rely on remote research. One area that the company has further pursued is surveys. And man, there are a lot of good uses for surveys, but even more bad uses. I've begun advocating for a more thoughtful approach to sending out surveys, which has been well received. Good survey design doesn't happen on accident, and we know it.

It seems as though not all companies know this.

My wife just gave birth to our third child. We are ecstatic. Momma and baby are both doing well.

We were surprised then that the nurse told us that my wife needed to speak to a social worker about her home life before discharge.

The electronic medical record system (called Epic) used by the hospital included a patient care app that had documentation and videos to which we had to respond in the form of a survey. The survey included a rather lengthy questionnaire about depression. And this is where the problems started.

Epic reports that this questionnaire helps to catch depression symptoms sooner than without. I believe this, and I'm thankful for a hospital care team that wants to prevent and treat depression as quickly as possible. But the survey itself commits cardinal sins; poor question ordering and inconsistent question polarity.

THE HORROR!

I know it seems nitpicky, but surveys should use the same polarity across each question. Imagine these as the first two questions in a survey asking you about the foods you eat:

Do you like pizza more than hamburgers?
Do you dislike hamburgers more than tacos?

If I didn't know better, I would be likely to report that I like pizza more than hamburgers (I do) and that I like burgers more than tacos(I don't). My intent as a participant is lost because I read "like" into both questions.

This is bad enough when my love for tacos is called into question. Imagine now the questions are:

In the past two weeks, have you been motivated to do everyday things in your life?
In the past two weeks, have you thought about harming yourself?
In the past two weeks, have you laughed or made any jokes?

It would be very easy to quickly answer "yes" to all of these; My brain could take the shortcut of reading the first and last questions and assume that answering "yes" is the "good" health response.

That's what my wife did. Since she indicated that she had thought about self-harm in the past 2 weeks (even though she hadn't), a social worker was called.

More terrifying is if I'm depressed and I have thought about self-harm. I answer "no" to all of them, trying to sandwich the content... and end up misreporting my self harm.

Depression is no joke- I offer this example because I think it's important to recognize the effect of design decisions on user intent.

In this case, a survey falsely flagged that my wife was at-risk, causing a delay in our discharge and distress in my wife.

In a worst-case scenario, a survey could falsely let a struggling mom fall through the cracks.

Surveys are great ways to collect information. But if we aren't careful with how we ask them, there can be consequences to our data, our products, and others lives.