3 Tips for better mobile surveys

With the average number of mobile survey completes ranging from 30% to 60%, designing for mobile is not optional.

In this most recent research on research, jointly conducted by Dynata, Forsta and MaritzCX, we explore the survey presentation of three commonly used question types: grids, Net Promoter Score, and lists.

Using a multi-cell design, we tested each presentation condition in a survey among 2280 participants. We investigated data comparability, data quality and participant experience.

Smartphone Ownership

98% of Millennials and 95% of Gen X own smartphones, with Boomers not far behind at 85% (Dynata Global Trends Report)

98%

Millennials

23%

Gen X

85%

Boomers

Global Web Traffic

In Q1 of 2019, mobile devices (excluding tablets) generated 49% percent of global website traffic (Statista)

49%

Smartphone only internet

One-in-five Americans are now ‘smartphone only’ internet users at home, up 7% since 2015 (Pew Internet)

Global Web Traffic

Mobile ad spend will top $93 billion in 2019, which is $20B more than TV (eMarketer)

1. Grids

Grids are one of the most commonly used question types, yet we know they are problematic, often receiving high levels of straight-lining (where participants select the same response option for all items in the grid) and survey dropouts.

We tested three ways of displaying grids (standard matrix grid, grid bars and progressive, also known as card sort within Forsta Surveys.

The Recommended Approach

• No significant score differences across devices
• Lowest % mobile straight liners
• Highest completion rates (comparable with grid bars)

2. NPS

Net Promoter Score

(NPS) is a widely used key performance metric to monitor brand health so it’s critical that the data is accurate. This is challenging on mobile as the NP is calculated from a wide 11-point scale.

We tested two ways to display the NPS question; a standard approach using radio buttons and the FocusVision Decipher approach using a grid bar that displays consistently across desktop and mobile devices.

3. List responses

Long answer option lists can have a detrimental impact on mobile survey data. The longer the list, the more likely the response provided is from the top of the list (primacy effect). This is especially challenging when randomization is not an option give the nature of the list.

We tested two ways to display the list question, a standard list displaying all items and open-ended text box lists where participants need to type in their answer(s).

Each has their strengths and draw backs, so usage depends on what is most important for the research question at hand.