May 29, 2020 50:20 E43

#43 - UX Benchmarking: Demonstrate Design ROI with Kate Moran of NN/g

UX benchmarking may seem like a lot of work, but Kate Moran is here to show you how to do it effectively. She's VP of Research & Content at Neilsen Norman Group and leads UX teams to better benchmarking, teaches newbies how to get started, and explains this complicated subject with clarity. She joined Erin and JH on our very first live episode to explain how UX benchmarking can help teams show the ROI of their work.

She walked through how benchmarking can help get stakeholders on board, how to choose the right metrics early on, and most importantly, how to translate that to real ROI.

Our very first live podcast was a great learning experience and a ton of fun! We really enjoyed the interactive aspect, and our audience asked a lot of thoughtful questions.

Highlights

[2:01] Kate explains what UX benchmarking is
[3:37] How to choose benchmarking metrics
[12:01] The difference between summative and formative studies, and why you need to distinguish between them.
[17:21] Why context matters when evaluating benchmarking metrics
[21:28] How to translate benchmarking results to ROI
[29:11] Kate talks about case studies from NNg's ROI for Usability report
[35:34] Q&A - How do you limit bias in unmoderated studies with non-users and users?
[38:16] Q&A - How do you measure time spent on a task? Stopwatches aren't great.
[39:59] Q&A - How do session replay tools fit into this?
[41:10] Q&A - What happens when your stakeholders have different metrics for success?
[44:57] Q&A - If a participant thinks they completed a task successfully, is that a success?
[46:42] Q&A - How do you benchmark for emotional aspects, like how fun a product is?
[49:07] Parting words of wisdom

Kate's recommended resources

NN/g full-day seminar: Measuring UX & ROI
NN/g articles about benchmarking, ROI, and quantitative research
Jeff Sauro’s articles about quantitative UX research (he also has very useful confidence interval and statistical significance calculators): measuringu.com
Jeff Sauro’s book about benchmarking: Benchmarking the User Experience
Two great books for getting started with statistics: Straightforward Statistics and The Cartoon Guide to Statistics

Episode Video

Creators and Guests

Host

Erin May

Senior VP of Marketing & Growth at User Interviews

Host

John-Henry Forster

Former SVP of Product at User Interviews and long-time co-host (now at Skedda)

Guest

Kate Moran

Kate is VP of Research & Content at Neilsen Norman Group and leads UX teams to better benchmarking, teaches newbies how to get started, and explains this complicated subject with clarity.