Monitoring platform usability

What I did

  • Educate and get buy-in on the approach
  • Launch, recruit, analyze, and socialize results every quarter

View results

While working as the sole researcher at a company, I need to look for opportunities to do more with less. I constantly need to balance the volume of requests that I receive, the priorities of the business, the potential impact that any study may have, and the readiness of stakeholders to take action on the results.

One of the ways that I do this is with the System Usability Scale (SUS) survey. This survey is a tried and true way to measure users' perception of holistic usability (i.e., as opposed to measuring actual usability by directly observing users complete tasks in a test). I use it because we have limited time and resources for usability testing -- the SUS helps us keep an eye on the platform's overall usability over time, identify potential issues, and form hypotheses about what needs to be fixed or deserves greater investment. This is also helpful in organizations with relatively low research maturity, where stakeholders do not have hypotheses that researchers can design studies around.

The SUS contains ten likert questions that address various aspects of usability and result in a score out of 100. It has been shown to correlate highly with Net Promoter Score, but provides much more actionable data to product decision makers. I also added an open text question, “What else should we know about your experience?” to add more richness to the score.

I track the score over time and tag PMs to view the relevant feedback. This often affects prioritization, inspires design exploration, or leads to a study to better understand the feature that appeared as a strong theme in the open text question.

For example, we routinely scored lowest on the statement, “I needed to learn a lot of things before I could get going with [the product].” This led to the growth pod prioritizing and ultimately redesigning the onboarding experience, and the docs team streamlining onboarding docs.

Please note, for confidentiality I have used dummy data in the results.