Lyndsi [0:00:00]: Measurement isn't always about having, like, a perfect scorecard?

Lyndsi [0:00:02]: It's about asking better questions.

Lyndsi [0:00:04]: Right?

Lyndsi [0:00:05]: And so what does success look like for our users.

Lyndsi [0:00:08]: What's the smallest clear sign that we're on the right track to start there.

Lyndsi [0:00:14]: And I think this outcome, you know, driven measurement helps you do that because it gives you something to anchor to.

Lyndsi [0:00:20]: It helps your team focus like, less on the noise and more on the signal.

Erin May [0:00:28]: Hey.

Erin May [0:00:28]: This is Erin May,

Carol Guest [0:00:29]: and this is Carol Guest.

Erin May [0:00:31]: And this is Awkward Silences.

Erin May [0:00:33]: Awkward Silences this is brought to you by User Interviews.

Erin May [0:00:37]: The fastest way to recruit targeted high quality participants for any kind of research.

Ben [0:00:48]: Hey, it's Ben.

Ben [0:00:48]: Welcome to Awkward Silences.

Ben [0:00:49]: On this episode, we're talking to Lyndsi Lee, the senior manager of Ux church at Turnitin.

Ben [0:00:55]: And our topic is measurement.

Ben [0:00:57]: Measuring user experiences, whether or not we should use something like N p or net promoter score, Csa sat, customer satisfaction, Lyndsi has a great person to take us through this because her career started in educational assessment.

Ben [0:01:12]: And through a circuit secured path and involved a stop along the way at a remote interior design company more on that a second.

Ben [0:01:19]: She landed at Turnitin where she's combining her interest in design, assessment and as of late Ux research.

Ben [0:01:25]: And so it's a great pod if you're someone who is struggling maybe with trying to get your stakeholders to think more broadly about how to measure experience, maybe they're really focused on and maybe obsessed with something like N mps or Csa sat.

Ben [0:01:38]: She has this framework that she takes us through, which is called outcome based assessment.

Ben [0:01:43]: It lets you tailor your research to a product suite, regardless of your industry vertical or whatever it is you're working so It's really useful.

Ben [0:01:52]: I think you'll leave with something that hopefully you can put into practice next week.

Ben [0:01:55]: So here's my conversation with Lyndsi Lee.

Ben [0:01:57]: Enjoy.

Ben [0:01:57]: Lyndsi, welcome to Awkward Silences.

Lyndsi [0:02:05]: Thank you.

Lyndsi [0:02:06]: Thank you.

Lyndsi [0:02:06]: Good be here.

Ben [0:02:07]: Yes.

Ben [0:02:07]: Very excited to talk with you about measurement, impact, Roi, all things that are very important to not just researchers, but the companies that they work for.

Ben [0:02:16]: Before we jump into that N mps csa sat and all the rest.

Ben [0:02:19]: Tell us a little bit about your path.

Ben [0:02:21]: To turn it in because you didn't start in a quote unquote research job.

Ben [0:02:26]: What was your role before you transitioned to what you might call your research career now?

Lyndsi [0:02:31]: Yeah.

Lyndsi [0:02:31]: Yeah.

Lyndsi [0:02:31]: So buckle up for this one because it's a winding one.

Lyndsi [0:02:35]: So I come from, like, a big family of educators, You know.

Lyndsi [0:02:41]: But for the longest time, I was convinced that I would not follow that path.

Lyndsi [0:02:45]: So when I was in college, my mom encouraged me, like, several times and, you know, not subtly, like, to go into education, and I was like, no way.

Lyndsi [0:02:56]: It's not for me.

Lyndsi [0:02:58]: Okay.

Lyndsi [0:02:58]: Great for you, not for me, You know?

Lyndsi [0:03:01]: I got my bachelor's in psychology, and I thought that I might go, like, the clinical route.

Lyndsi [0:03:06]: But eventually, I don't know.

Lyndsi [0:03:08]: I I just started veer in a direction, the...

Lyndsi [0:03:12]: And I got my mass in school counseling Okay.

Ben [0:03:15]: I

Lyndsi [0:03:16]: got a specialist degree, and then I became certified as a school psychological examiner.

Lyndsi [0:03:20]: Mh So, you know, after declaring that I would not go into education, I spent ten years in public education.

Ben [0:03:30]: Has one does?

Lyndsi [0:03:31]: As one exactly.

Lyndsi [0:03:32]: So I supported students on the counseling side.

Lyndsi [0:03:36]: And then I was able to support educators through these educational evaluation.

Lyndsi [0:03:41]: So when a teacher had concerns about, like, a student's learning, we'd work together to create a plan, and sometimes that plan, like, included individual testing.

Lyndsi [0:03:51]: And so I was the one who carried out those evaluations.

Lyndsi [0:03:54]: And that experience, it really gave me, like, a front row receipt to how instructors and educators make decisions and the information that they need to, like, do their job.

Lyndsi [0:04:05]: Mh.

Lyndsi [0:04:05]: As well as, you know, how deeply they care about just doing right by their students.

Lyndsi [0:04:10]: They truly truly care about these students.

Lyndsi [0:04:12]: So...

Lyndsi [0:04:13]: So ten years later, eventually, started to just kinda crave something more creative.

Lyndsi [0:04:20]: I'm just a really...

Lyndsi [0:04:21]: I'm a creative person Always have been And I just...

Lyndsi [0:04:24]: I needed something more creative.

Lyndsi [0:04:26]: And so I've always loved interior design.

Lyndsi [0:04:30]: And so I took courses got certified and started designing rooms like remotely for this online interior design company.

Lyndsi [0:04:38]: So much fun.

Lyndsi [0:04:39]: It was.

Lyndsi [0:04:40]: Yeah.

Lyndsi [0:04:41]: It was great, and it was a lot of fun.

Lyndsi [0:04:43]: I I got to do a lot of things with them.

Lyndsi [0:04:45]: And so...

Lyndsi [0:04:45]: But this company was considered tech, you know, because they're online.

Lyndsi [0:04:50]: Right?

Lyndsi [0:04:50]: And so it really opened my eyes to some new possibilities.

Lyndsi [0:04:54]: And again, it was it was a lot of fun, but I started thinking, like, well, what's this.

Lyndsi [0:05:01]: And what's this?

Lyndsi [0:05:02]: And so that's where I, you know, first discovered Ux design, and I had this kind of like epiphany that Ux is, like, where psychology and technology meet.

Lyndsi [0:05:12]: And those are two things that I'm actually, you know, really good at and have experience in.

Lyndsi [0:05:17]: And so dove deep into that got another certification.

Lyndsi [0:05:21]: I'm all about some certifications.

Lyndsi [0:05:23]: Yeah.

Lyndsi [0:05:23]: Worked on, you know, some real products and consulted on some independent projects to build experience.

Lyndsi [0:05:29]: And then when the opportunity came it turn it in, it just felt kind of like the perfect fit because it combined everything that I had done up to that point, which was, you know, psychology, technology, education.

Lyndsi [0:05:45]: I actually even design because it was for a designer role.

Lyndsi [0:05:48]: You know?

Lyndsi [0:05:48]: And so I realized I could...

Lyndsi [0:05:50]: At that point, like, circle back to, you know, my roots and continue kind of supporting educators, but doing it at a much broader scale now.

Lyndsi [0:06:00]: And helping them by building tools that just help them do their work better.

Lyndsi [0:06:05]: And so, anyway, if there's one thing I've learned kind of along the way it's that paths into Ux research just are not linear.

Lyndsi [0:06:14]: Mine certainly wasn't linear.

Ben [0:06:16]: Sure.

Lyndsi [0:06:16]: But I've heard, you know, many, many, like, interviewing people.

Lyndsi [0:06:20]: Now as a manager, they talk about their their path into Ux research and it's just...

Lyndsi [0:06:26]: They're all winding, And that's...

Lyndsi [0:06:27]: I think what really makes this work so kind of rich and meaningful that we all have different...

Lyndsi [0:06:33]: We bring different experiences to the table.

Lyndsi [0:06:36]: And I think that's really cool.

Ben [0:06:38]: Yeah.

Ben [0:06:38]: I can already hear and recognize why you're doing so well there turn it in, you know, your background and evaluation and testing probably got you to sharpen your your ability to take a, you know, create instruments and assess whether or not something is real or needs to be, you know, gone after and then you're pivot to interior design and design more generally?

Ben [0:06:55]: How you think about, you know, the orientation and relationship to things?

Ben [0:06:58]: What skills do you find that you carried with you across these roles and that you rely on even today in your work?

Ben [0:07:04]: Whether or not it's for f turnitin in and otherwise, But, like, are you...

Ben [0:07:07]: I mean, we'll talk about how you're know, your approach and thoughts around assessment with N mps and C.

Ben [0:07:11]: But what are some of the other things that you've taken with you and that you think Google will carry on no matter what role you go to?

Lyndsi [0:07:17]: Yeah.

Lyndsi [0:07:17]: That's a good question.

Lyndsi [0:07:18]: I think, honestly, I've taken things with me from each experience, and I like the fact that I kind of have this, like, diverse experience.

Lyndsi [0:07:29]: So, you know, the education piece, obviously, like, the experience in education and knowing how the education system works.

Ben [0:07:37]: Mh.

Ben [0:07:37]: You

Lyndsi [0:07:38]: know, inside and out.

Lyndsi [0:07:38]: And but then going to interior design and had to work more creatively there.

Lyndsi [0:07:46]: And I don't even just mean, like, design wise.

Lyndsi [0:07:48]: Like, just...

Lyndsi [0:07:49]: There were a lot of solutions that I had to come up with that had to be very creative, whether it...

Lyndsi [0:07:56]: But, you know, like, with budget or with small spaces or, you know, I had to think outside the box to really, like, to get to a solution.

Lyndsi [0:08:06]: And I think that helped me obviously, to work in research and design.

Lyndsi [0:08:11]: And so I think the balance of, like, creative and analytical skills have helped me kind of come together to this role.

Ben [0:08:20]: Mh.

Ben [0:08:20]: And what value did you find in certifications?

Ben [0:08:22]: Because I know a lot of our audience are folks who are either earlier in their career and they're wondering if that might be a good fit for them or there...

Ben [0:08:29]: Maybe folks like you who are transitioning from a different kind of industry what value did you find in the certifications?

Ben [0:08:35]: How did that help you, along your path?

Lyndsi [0:08:38]: Yeah.

Lyndsi [0:08:38]: That's another good question.

Lyndsi [0:08:39]: So there's a lot of talk conversation around, like, certifications and boot camps And Yeah.

Lyndsi [0:08:45]: Should we...

Lyndsi [0:08:45]: Or should they?

Lyndsi [0:08:46]: Should they not?

Lyndsi [0:08:47]: And what I personally got out of the certification was not...

Lyndsi [0:08:51]: I don't...

Lyndsi [0:08:52]: I didn't think about it, like a checkbox box.

Ben [0:08:55]: K.

Lyndsi [0:08:55]: Like, I'm gonna put this on my resume and say that I have this certification.

Lyndsi [0:09:00]: You know?

Lyndsi [0:09:00]: I didn't think about it like that.

Lyndsi [0:09:02]: I wanted to immerse myself and learn from these courses, you know?

Lyndsi [0:09:08]: And I did.

Lyndsi [0:09:09]: I learned a lot from the courses that I didn't really know before because I had not worked in Ux before.

Lyndsi [0:09:15]: And so anything that it could teach me about Ux and about, like, how it works was beneficial valuable to me, But I'll also say that I did one other thing that was beyond the certification.

Lyndsi [0:09:29]: And I found this kind of immersive.

Lyndsi [0:09:32]: It wasn't a course.

Lyndsi [0:09:34]: It was, like, a project.

Lyndsi [0:09:36]: Okay.

Lyndsi [0:09:37]: And that's where I worked with a Pm.

Lyndsi [0:09:40]: I worked with an engineer and I really kind of worked on real products and real projects because one thing that the certifications in the boot camps can't give you is those real world experiences in the constraints.

Lyndsi [0:09:56]: So Yeah.

Lyndsi [0:09:58]: And I see that.

Lyndsi [0:09:58]: Like, whenever...

Lyndsi [0:09:59]: Again, if I'm interviewing somebody or something like that.

Lyndsi [0:10:03]: If they bring projects, if they're fresh out of, you know, boot camp or something, it's it's very formulaic, the design, the whole process.

Lyndsi [0:10:11]: And Sure.

Lyndsi [0:10:12]: But if you have the opportunity to put yourself into a project, a real world project, then I would say absolutely do that because that was the most valuable thing that I could have done for my.

Ben [0:10:26]: Yeah.

Ben [0:10:26]: I imagine, you know, that being able to speak to how you reacted to constraints, how you reacted it to things like timeline and budget.

Ben [0:10:33]: Those are things that we often as practitioners.

Ben [0:10:37]: We kinda feel...

Ben [0:10:38]: We feel constrained.

Ben [0:10:39]: We don't wanna have our research timelines shortened or our design resources cut, but that's the nature of many businesses.

Ben [0:10:45]: You know, the unlimited budgets.

Ben [0:10:46]: Those are not going anywhere.

Ben [0:10:48]: So I expect that's why you're so efficient as a leader and impactful as a practitioner because you are able to deploy rigor as we'll talk about in a second where it's able, But you know they're, like, well, you, I can't have it everywhere.

Ben [0:10:58]: And is that Something...

Ben [0:10:59]: Is that advice that you would give to someone who's maybe job searching now?

Ben [0:11:02]: They might have a passion for, you know, user experience design or quantitative research, and they might have this more niche passion of theirs, but, you know, they might be going after roles that are more broad.

Ben [0:11:12]: Did that help you or or what advice would you give someone who's interviewing now and has this passion, but might be, you know, applying for a role that's a little different than it.

Lyndsi [0:11:20]: Yeah.

Lyndsi [0:11:20]: I would just say, like, you know, don't box yourself and thinking that there's only one way to use your passion.

Lyndsi [0:11:27]: So, you know, like, sometimes the connection between, like, what you love to do and the the work that you do isn't obvious, like, right away, and that's okay.

Lyndsi [0:11:36]: I would say, like, stay open to exploring from different angles, even if they feel a little unexpected and obviously, I'm just I'm pulling from my experience here.

Lyndsi [0:11:47]: And again, my experience is all over the place, but you know, for me, it wasn't a straight line, and I approached my career from a few different angles before, pulled all those pieces together.

Lyndsi [0:11:58]: And again, every experience taught me something valuable.

Lyndsi [0:12:01]: Even if it didn't seem like, vote on track at the time because I could tell you, like, oh, I ahead at all mapped out.

Lyndsi [0:12:08]: Like, that's, you know, I would be a liar if I told you that.

Lyndsi [0:12:12]: Sure.

Lyndsi [0:12:13]: I didn't have it mapped out.

Lyndsi [0:12:14]: I thought I did at the time.

Lyndsi [0:12:16]: You know, I thought that I had these things mapped out.

Lyndsi [0:12:19]: But then I stayed open.

Lyndsi [0:12:22]: And I would...

Lyndsi [0:12:24]: Again, the advice I would give is like, trust that the skills and the instincts that you're building now will show up later in ways that you might not expect, and you've been taking small steps to explore those new interests or invest in learning when something sparks your curiosity, I'm all about the curiosity, and I'm sure a lot of us are, you know, in in this profession, obviously.

Lyndsi [0:12:46]: But give yourself like, permission to evolve, you know, your career doesn't have to look like anybody else's blueprint.

Lyndsi [0:12:54]: You don't have to go one, two, three, four.

Lyndsi [0:12:56]: Right.

Lyndsi [0:12:57]: It's okay to build as you go.

Lyndsi [0:12:59]: And so I guess that would be my experience.

Lyndsi [0:13:02]: Great my my advice.

Ben [0:13:05]: I appreciate that.

Ben [0:13:05]: I think that it's accessible for someone to reflect on what it is they like about the work they're currently doing and maybe the gaps and what they're getting.

Ben [0:13:13]: I love your story about how you...

Ben [0:13:14]: Through a, remote interior design company found out about tech broadly defined of going from in a school and a school environment counseling and and advising educators and students to now turn it in.

Ben [0:13:27]: And so let's turn unintended there.

Ben [0:13:29]: To turn it in where you got exposed to a different kind of part of research, which is...

Ben [0:13:35]: I don't know what would you we even call these, the N mps and the csa sad.

Ben [0:13:39]: These are like, the ways we measure experience measure impact.

Ben [0:13:42]: Let's start there at the high level.

Ben [0:13:43]: When you say n mps and Csa set.

Ben [0:13:45]: What are these things?

Ben [0:13:46]: How would you define those?

Ben [0:13:47]: As a category?

Lyndsi [0:13:49]: I have such a visceral reaction, and I shouldn't.

Lyndsi [0:13:51]: I shouldn't.

Lyndsi [0:13:52]: But I do.

Lyndsi [0:13:53]: So I don't know if you should ask me what to call these things like, on air because it's problem, you know, I'll probably have, like, bad or something.

Lyndsi [0:14:00]: No.

Lyndsi [0:14:01]: Just kidding.

Lyndsi [0:14:02]: So I do have kind of a complicated opinion on Cis and Mps.

Lyndsi [0:14:07]: And, you know, it might not be that complicated, actually.

Lyndsi [0:14:11]: We might all have this opinion.

Lyndsi [0:14:13]: And maybe all of our opinions are complicated.

Ben [0:14:16]: But need to voice them.

Ben [0:14:17]: Yeah.

Lyndsi [0:14:18]: Yeah.

Lyndsi [0:14:18]: Exactly.

Lyndsi [0:14:18]: But, you know, I've had to report on these in the past as many of us probably have had to report on these type of metrics.

Ben [0:14:26]: Mh.

Lyndsi [0:14:26]: And I do think that they can be useful in certain context.

Lyndsi [0:14:30]: I think that they can tell you, you know, that something is happening, but they don't tell you, like, why users feel the way they do or what you should actually do about it.

Lyndsi [0:14:40]: Yeah.

Lyndsi [0:14:40]: And I think honestly, in my experience, I've seen teams end up just reaching for these metrics because they know they should measure something after a product ships, but they're not always sure, like, what to measure you know, they're not experts in psycho or Mh.

Lyndsi [0:14:56]: Quantum metro.

Lyndsi [0:14:57]: I don't...

Lyndsi [0:14:58]: Is that a word.

Lyndsi [0:14:58]: I don't know if that's it.

Lyndsi [0:14:59]: It is isn't.

Lyndsi [0:15:00]: Yeah.

Lyndsi [0:15:01]: Yeah.

Lyndsi [0:15:01]: So these broad scores kind of become the default and you know, that's fine to a point, but especially, like, in the more complex products.

Lyndsi [0:15:10]: I think those scores can hide a lot of, like, real pain points.

Lyndsi [0:15:15]: And also, you might have, like, a decent N ps score because you've got brand trust or, you know, you've had users for a really long time or you don't have a...

Lyndsi [0:15:26]: They don't have a better option, You know, but that doesn't mean like, the day to day experience is actually good.

Lyndsi [0:15:31]: And let me get off on this tangent for one second.

Lyndsi [0:15:35]: So I'm sure we've all had this experience, but I have seen N Ps questions in the most absurd of places, you know, like...

Lyndsi [0:15:45]: Yes.

Lyndsi [0:15:45]: Okay.

Lyndsi [0:15:46]: So, like, I'm you know, looking at my medical records online and it's, like, really, would I would I...

Ben [0:15:54]: Yes.

Ben [0:15:54]: I just got a blood test.

Ben [0:15:55]: I don't know if I would.

Ben [0:15:56]: And N mps for those who are newer is net promoter score often Lyndsi zero to ten.

Ben [0:16:01]: Right?

Ben [0:16:02]: How likely would you be to recommend this product service to a friend Yeah.

Ben [0:16:06]: And it does show up.

Ben [0:16:07]: Yeah.

Ben [0:16:07]: It does show up in a bunch of different weird places.

Lyndsi [0:16:10]: It it does.

Lyndsi [0:16:10]: So how how likely would I be to, you know, recommend my medical evaluation app or records app to a friend.

Lyndsi [0:16:19]: Like, am I gonna go to wine night with my girlfriends to be, like, you guys...

Lyndsi [0:16:23]: Guess look at this app that my dog dashboard

Ben [0:16:27]: board.

Ben [0:16:27]: Yes.

Ben [0:16:28]: My, Yeah.

Ben [0:16:29]: Whatever.

Ben [0:16:29]: Yeah.

Lyndsi [0:16:30]: Amazing.

Lyndsi [0:16:30]: No.

Lyndsi [0:16:31]: I'm not gonna do that.

Lyndsi [0:16:32]: And Some people might, And I'm not judging that.

Lyndsi [0:16:35]: No.

Lyndsi [0:16:36]: Like I'm not judging what goes on at one night.

Lyndsi [0:16:38]: That's fine.

Lyndsi [0:16:38]: Like.

Lyndsi [0:16:39]: But, you know, my friends and I have other things to talk about.

Lyndsi [0:16:42]: We're not gonna talk about.

Lyndsi [0:16:43]: Sure.

Lyndsi [0:16:44]: But Yes.

Lyndsi [0:16:44]: So that's my small tangent with that piece of it.

Lyndsi [0:16:47]: But then you've also got, like, analytics, and I'm not knocking analytics.

Lyndsi [0:16:52]: They they've got a place in research for sure.

Lyndsi [0:16:56]: And they can also tell you like, what users are doing or not doing or why they're But they they can't tell you why they're not doing it or doing it.

Lyndsi [0:17:05]: Right?

Lyndsi [0:17:05]: So analytics, when I say analytics, you know, I mean, obviously, like, engagement scores.

Lyndsi [0:17:10]: That's a popular one that a lot of people like to fall back to is engagement.

Lyndsi [0:17:14]: If you're just looking at these N mps or Csa sat, which is, you know, how satisfied are you with the product?

Lyndsi [0:17:22]: Right.

Lyndsi [0:17:22]: You end up just with this kind of big fuzzy picture and, like, you know maybe something's off without really a clear idea of how to fix it.

Lyndsi [0:17:32]: And you just have this kind of arbitrary maybe target to say, like, we want our Mps sport to be, you know, fifty six.

Lyndsi [0:17:40]: And then why?

Lyndsi [0:17:42]: Why do you want it to be?

Lyndsi [0:17:44]: You know?

Lyndsi [0:17:45]: Like, and what are you gonna do about it?

Lyndsi [0:17:47]: If it's not, Yeah.

Lyndsi [0:17:48]: Basics six.

Lyndsi [0:17:49]: You know?

Lyndsi [0:17:49]: So that's really kind of oof.

Ben [0:17:54]: Yeah.

Ben [0:17:54]: And And I think it's popular because it's maybe simple to a false

Lyndsi [0:17:58]: exactly times.

Ben [0:17:59]: And in our day and age, you know, I'm sure you've seen this in our user experience community more broadly.

Ben [0:18:04]: We are regularly being asked to justify our impact and designate our Roi.

Ben [0:18:09]: And so I get it why teams, Who might be reporting to product stakeholders, engineering stakeholders or, like, marketing your sales folks, you know, the sort of front of the house teams, I understand why they gravitate toward that.

Ben [0:18:21]: But you alluded to it that a lot of our work doesn't neatly roll up?

Ben [0:18:24]: I mean, if you're work doing are like, a classic usability study and your...

Ben [0:18:27]: You are improving the onboarding flow, then, yes, maybe a a C score will rise.

Ben [0:18:31]: But what are the approaches that we ought to add to that?

Ben [0:18:35]: I don't think you're suggesting that we do away with these things they have a value, but what might we as researchers or experienced professionals start thinking about adding to these other scores that can help us get a more holistic picture that maybe our stakeholders and the wider business might sort of get along with.

Lyndsi [0:18:51]: Yeah.

Lyndsi [0:18:51]: I'm definitely not advocating for just, like, doing away completely.

Lyndsi [0:18:55]: And, you know, I've seen these provocative statements before, like, do away with Mps or do away with Csa.

Lyndsi [0:19:01]: And it's like, no.

Lyndsi [0:19:02]: Okay.

Lyndsi [0:19:03]: Let's take a step back.

Ben [0:19:05]: Yeah.

Lyndsi [0:19:05]: And look at all of this together and, you know, we can all live in harmony here.

Lyndsi [0:19:10]: So I think honestly, instead of just, like, measuring general satisfaction.

Lyndsi [0:19:14]: Right?

Lyndsi [0:19:16]: Or looking at behavior just in this vacuum, we can start asking ourselves, like, What if we got really specific about the outcomes that users need.

Lyndsi [0:19:26]: So there's this framework that I've used in my work that's been really effective for this, and it helps break the experience down into these atomic pieces.

Lyndsi [0:19:36]: Okay.

Lyndsi [0:19:37]: And so what they are are like, small, foundational outcomes that users might not even articulate out loud, but they maybe quietly, like, make or break how they experience feels, You know?

Lyndsi [0:19:50]: And so I do wanna be clear on something though, I did not invent this framework.

Lyndsi [0:19:55]: I wanna give credit where it's due.

Lyndsi [0:19:57]: You know, it came out of Amazon Prime, videos work on customer experience outcomes.

Lyndsi [0:20:03]: Okay.

Lyndsi [0:20:03]: But what's really great about it is that it can be adapted to any product any org, It doesn't have to follow, again, x y z formula.

Lyndsi [0:20:13]: You can take it and do what you need to do with it.

Lyndsi [0:20:16]: And I think that's why I really like it.

Ben [0:20:19]: Yeah.

Lyndsi [0:20:19]: But the big idea here is that, like, you're not just tracking feelings or behaviors, but you're getting down to whether you're helping your users succeed in ways that matter.

Lyndsi [0:20:31]: To them.

Ben [0:20:32]: Okay.

Ben [0:20:32]: It sounds a little bit like a job to be done, but exactly.

Ben [0:20:35]: I guess, so let...

Ben [0:20:36]: Let's dig into the atomic outcome.

Ben [0:20:38]: Can you give us some examples or Yeah.

Ben [0:20:40]: Questions that someone might be asking or could ask themselves kinda get to these outcomes.

Ben [0:20:44]: Let's take a, like, a digital or a software experience as our example here.

Lyndsi [0:20:48]: Right.

Ben [0:20:49]: What would be an example of one and what are some questions that we can use to get to them.

Lyndsi [0:20:52]: Yeah.

Lyndsi [0:20:52]: And I'm glad that you said that jobs be done It's very jobs to be done.

Lyndsi [0:20:56]: Not like, reinventing every single wheel here, You know, it's taking frameworks that we already know and that we've practiced before and just bringing them in.

Lyndsi [0:21:05]: And so so you can start by just, like, pulling everything in that you already know which would be, like, past research, user interviews, feedback, support conversations, anything you've got.

Lyndsi [0:21:17]: Just pull it in, and then you look at the deeper needs behind them.

Lyndsi [0:21:22]: So, like, the jobs to be done.

Lyndsi [0:21:23]: And so from there you get your team together, obviously, you can do this remotely due it in nero or whatever your whiteboard is and draft outcomes based on what users are trying to accomplish.

Lyndsi [0:21:36]: And so you have to make sure that these are not the features that they use.

Lyndsi [0:21:41]: But Yes.

Lyndsi [0:21:42]: What they actually want to, like, achieve or feel.

Lyndsi [0:21:45]: Right?

Lyndsi [0:21:45]: And it helps to, like, organize them, you know, by product or journey stage, and definitely keep them feature agnostic because...

Lyndsi [0:21:54]: Okay.

Lyndsi [0:21:55]: So example.

Lyndsi [0:21:57]: So instead of saying, users can bulk upload files, which is feature, you know, Right specific.

Lyndsi [0:22:04]: You could say something like users can complete repetitive tasks without friction.

Ben [0:22:12]: So...

Ben [0:22:12]: That could apply to multiple parts of your experience on.

Ben [0:22:15]: I see.

Ben [0:22:15]: Okay.

Lyndsi [0:22:16]: You could Yeah.

Ben [0:22:16]: Mobile We could go all over the place.

Ben [0:22:18]: Okay.

Ben [0:22:19]: I see.

Lyndsi [0:22:20]: These are, like, modular almost.

Lyndsi [0:22:22]: You can move these things around.

Lyndsi [0:22:23]: But the phrasing, it just keeps the focus on the experience and Mh.

Lyndsi [0:22:27]: The product can evolve and the features can evolve and the need is still the same.

Lyndsi [0:22:32]: Right?

Lyndsi [0:22:33]: Right?

Ben [0:22:33]: Yeah.

Lyndsi [0:22:34]: And so your team just will continue to do that.

Lyndsi [0:22:37]: And until you feel like you've documented all the important outcomes.

Lyndsi [0:22:40]: I found that, like, after you do this with your team like, let it simmer for a few days.

Lyndsi [0:22:46]: Okay.

Lyndsi [0:22:46]: You know, and come back to it.

Ben [0:22:48]: Yeah.

Lyndsi [0:22:48]: Because then you might realize, like, Oh, these two can be combined into this one or this phrasing feels off you know, just like with any documentation.

Lyndsi [0:22:57]: Yeah.

Lyndsi [0:22:57]: So that's kind of the first phase of this.

Erin May [0:23:01]: Awkward interruption.

Erin May [0:23:02]: This episode of Awkward Silences like every episode of Awkward Silences brought to you by User Interviews.

Carol Guest [0:23:09]: We know that finding participants for research is hard.

Carol Guest [0:23:11]: User Interviews is the fastest way to recruit targeted, high quality participants for any kind of research.

Carol Guest [0:23:17]: We're not a testing platform.

Carol Guest [0:23:18]: Instead, we're fully focused on making sure you can get just in time insights for your product development, business strategy, marketing, and more.

Erin May [0:23:27]: Go to user interviews dot com slash awkward to get your first three participants free.

Ben [0:23:33]: And shout out to the Cs and support folks out there.

Ben [0:23:35]: They have always been such a great resource for, you know, if you're joining an organization, you're trying to figure out the lay the land, what has already been done or certainly, if you're a user researcher you're trying to figure out what's the most painful thing that maybe your product or engineer, design stakeholders don't know about talk to your successor or support folks do a little ride along.

Ben [0:23:52]: Digitally or in person they are, the tip of the spear.

Ben [0:23:55]: Okay.

Ben [0:23:55]: So Lyndsi, I a follow up question.

Ben [0:23:57]: So it sounds like at the end of a couple of these exercises?

Ben [0:24:00]: You might have a very busy whiteboard with a bunch of different potential?

Ben [0:24:04]: I mean, I guess depending on how broad your experiences.

Ben [0:24:07]: How do you prioritize what?

Ben [0:24:08]: How do you kinda get to the thing you really want to to start?

Ben [0:24:11]: Because I think that's one of the attractions and downsides of something like N mps As it tries to be a catch all, but it can't.

Ben [0:24:17]: So how do you and your team or how have you and your were thought about prioritizing of those?

Ben [0:24:21]: Yeah.

Ben [0:24:22]: Atomic outcomes.

Lyndsi [0:24:23]: A great question.

Lyndsi [0:24:24]: And, yes, circling back to the support folks.

Lyndsi [0:24:27]: They are kind of the un sun heroes sometimes.

Lyndsi [0:24:29]: Definitely.

Lyndsi [0:24:30]: And the untapped resources, I think, for research teams a lot of times as well, you know, that we don't think about going to them.

Lyndsi [0:24:36]: So...

Lyndsi [0:24:37]: But, yes.

Lyndsi [0:24:37]: So you're gonna wanna, like, stress test these outcomes.

Lyndsi [0:24:41]: Okay.

Lyndsi [0:24:41]: And, ideally, this would be done with users, but.

Lyndsi [0:24:44]: You know, we work with what we've got, for example, support or Cs or customer facing folks.

Lyndsi [0:24:51]: But if you are working with real users, it could be through interviews or surveys or even, like just simple cognitive testing, but you would ask things like, does this goal make sense to you?

Lyndsi [0:25:05]: You know, would you describe it in this way?

Lyndsi [0:25:07]: How important is it compared to other things you're trying to do.

Lyndsi [0:25:10]: You would come up with a question framework on how to get these outcomes, this is leading to the prioritization, You know.

Lyndsi [0:25:18]: But it's making sure you're capturing like, what matters and not just, like, what sounds nice in a

Ben [0:25:25]: workshop.

Ben [0:25:25]: Yeah.

Lyndsi [0:25:26]: You know?

Lyndsi [0:25:26]: Because we can tend to do that Sometimes.

Lyndsi [0:25:28]: Oh, of course.

Lyndsi [0:25:30]: Caught up in ourselves.

Ben [0:25:31]: I think that's why so much of was, like personas or have liked personas as we get to use a cool name and we might be able to partner with a designer and come up with an Avatar, and then well, we check back in a few quarters in that, you know, Sally the shop or Jimmy the...

Ben [0:25:45]: I don't know.

Ben [0:25:46]: The bargain hunter doesn't quite stick in the same way, but we really liked it.

Ben [0:25:49]: It really sounded.

Ben [0:25:50]: At it...

Ben [0:25:50]: I know.

Ben [0:25:51]: I'm a sucker for branding too, and sometimes they can get in the way of the real research outcome or the real ability to improve for our customers.

Ben [0:25:57]: So I appreciate that caveat.

Lyndsi [0:25:59]: Yeah.

Lyndsi [0:25:59]: Exactly.

Lyndsi [0:25:59]: You've gotta keep yourself grounded.

Lyndsi [0:26:01]: Yeah.

Lyndsi [0:26:01]: All the time.

Lyndsi [0:26:02]: I do love a a good looking personas as well.

Lyndsi [0:26:04]: But...

Lyndsi [0:26:04]: Oh, yeah.

Lyndsi [0:26:05]: So, obviously, not every outcome is critical if everything's important, then nothing's important.

Lyndsi [0:26:11]: Right?

Lyndsi [0:26:12]: And so narrowing the list to, like, ten to twelve top outcomes, keeps it focus and manageable and sets you up, like, for measuring them in a meaningful way.

Lyndsi [0:26:22]: And so that's how you kind of narrow them down is take them through with users and asking them honestly, I've seen card sorting done here.

Ben [0:26:35]: Oh cool.

Lyndsi [0:26:35]: Where it's, like, here's, like, most important here is, like, yeah.

Lyndsi [0:26:40]: And here is, like, least important.

Lyndsi [0:26:42]: And so put the outcomes and so that the users could then, like, sort the outcomes in the columns, and then that's how you can come up with those ten to twelve.

Ben [0:26:53]: I love that.

Lyndsi [0:26:53]: Yeah.

Lyndsi [0:26:53]: That's a really good activity for that one.

Lyndsi [0:26:55]: So...

Ben [0:26:56]: And I would almost...

Ben [0:26:57]: I would like to do that with my product stakeholders, you know, sometimes product managers get very narrow and what they're working on.

Ben [0:27:03]: But how do you get buy in or communicate these with stakeholders?

Ben [0:27:07]: Are those stakeholders part of this process at any point?

Ben [0:27:11]: Because I know that's another thing that some researchers and experience pros faces that they internally among the research folks or Ux folks get alignment, and then they take it out sort of in the wilder organization and they face some pushback either because it's they haven't done education around it or they're not aligned on road roadmap.

Ben [0:27:28]: How would you advise folks in their practices to make sure that their partners and stakeholders are along for the ride with the atomic outcomes.

Lyndsi [0:27:36]: Yeah.

Lyndsi [0:27:36]: So I would actually encourage everybody to get strategic about this.

Lyndsi [0:27:40]: So...

Ben [0:27:41]: Okay.

Lyndsi [0:27:41]: I'm the product person that's, like, the most...

Lyndsi [0:27:43]: Not agreeable necessarily, but, like, that you know will be open minded to work with.

Lyndsi [0:27:50]: You know?

Lyndsi [0:27:51]: And then take them through the whole process, don't hand over anything.

Lyndsi [0:27:56]: Like, they...

Ben [0:27:57]: Great call.

Ben [0:27:57]: Yeah.

Lyndsi [0:27:58]: Yeah.

Lyndsi [0:27:58]: They need to be there from the beginning.

Lyndsi [0:28:00]: If you have a a big product suite, you know, then start with one product, start with one experience, start with one Pm, you know, and then take them through it, build it together, and then when it starts to work, and everybody starts to see, like, Oh, that's how we can measure our experiences, it catches on.

Lyndsi [0:28:23]: It really does.

Lyndsi [0:28:24]: And so because it's hard to measure experiences for product people for Ux people for everybody.

Lyndsi [0:28:31]: So if they see, like, okay.

Lyndsi [0:28:34]: There's a way to measure, then it catches on.

Lyndsi [0:28:37]: So Yeah.

Lyndsi [0:28:37]: That would be my advice.

Lyndsi [0:28:38]: Just to find the the most open minded product manager first.

Ben [0:28:43]: Yeah.

Ben [0:28:43]: I don't think there's anything wrong with trying to identify a champion and leveraging that person because what I'm hoping what would happen in this example is that that person's product suite or they're...

Ben [0:28:52]: What they're working on.

Ben [0:28:53]: Would get better faster than those or more me than those.

Ben [0:28:56]: Who aren't working with the research team?

Lyndsi [0:28:59]: So okay.

Ben [0:29:00]: That leads me to a follow question.

Ben [0:29:01]: When you've done this in the past or when you've seen this work for other teams, does that help in the kinds of things you're reporting, You know, sometimes researchers are like, okay.

Ben [0:29:09]: I found this thing, but how do I make it stick with the stakeholders, It sounds like if you've got a an agreed upon outcome you can say, and here's how it will impact and affect this outcome.

Ben [0:29:18]: Is that how you are thinking about using these identifiable outcomes as they kinda show up in the deliverables and the ways that you put your findings to work?

Lyndsi [0:29:26]: Yeah.

Lyndsi [0:29:26]: I mean, it even shows up in, like, the way that designers design, and, like, because if they are working on something for this product, and it's like, okay.

Lyndsi [0:29:37]: Well, which outcome does it map to.

Lyndsi [0:29:39]: And if it doesn't map to an outcome, like, this this important.

Lyndsi [0:29:43]: Like, you know, maybe we need to reevaluate this.

Lyndsi [0:29:46]: And sometimes, yes, it is and we need to add an outcome to the dashboard.

Lyndsi [0:29:51]: But sometimes maybe it's not.

Lyndsi [0:29:53]: And so I think that that's one way that it helps.

Lyndsi [0:29:57]: But when it really helps is when we start measuring these outcomes.

Lyndsi [0:30:01]: Because at this point, we've just identified the outcomes, but which is is fine.

Lyndsi [0:30:07]: If we wanna stop there, we can stop there.

Lyndsi [0:30:09]: But measuring those outcomes is, I think really kind of what gets everybody on board.

Ben [0:30:16]: Okay.

Lyndsi [0:30:17]: And I think there's really practical ways to do it.

Lyndsi [0:30:20]: You could do it.

Lyndsi [0:30:21]: I'm a whole lot of different ways.

Ben [0:30:23]: Yeah.

Ben [0:30:23]: What are some of the ways that you've found it successful?

Ben [0:30:25]: Is it a, like, a monthly survey with users?

Ben [0:30:27]: Is it a in app pop up, how how might you have to identify some outcomes, kind of vet them with your stakeholders?

Ben [0:30:32]: How do you start putting them to work and putting them in front of your customers or users wherever they may be.

Lyndsi [0:30:38]: Yeah.

Lyndsi [0:30:38]: I mean, once you get sophisticated with it, you could use a mix of methods and add it, you know, all of it to a score, but I'll not go there yet.

Lyndsi [0:30:47]: I'll just say that I think I have found that the survey method works well, like, a pop up.

Lyndsi [0:30:54]: Yes.

Lyndsi [0:30:55]: But it could be, like, a pop up to an external survey, if it's a little longer, you know?

Lyndsi [0:31:01]: Because if you've got this one product and you've got, like, ten or twelve outcomes that you want to measure, it might need to be, you know, an external survey somewhere else.

Lyndsi [0:31:11]: But you could just even something as simple as, like, how easy or difficult is it to x y z?

Lyndsi [0:31:19]: And then just have a consistent scale for every question so that the comparison will be easy and then make sure you, you know, you've got your sample size big enough that you can trust the signal obviously.

Lyndsi [0:31:30]: But what's important though is that you measure each outcome individually rather than kind of collapsing them into one support...

Ben [0:31:38]: Important.

Ben [0:31:38]: Okay.

Lyndsi [0:31:38]: Yeah.

Ben [0:31:39]: And why do you do that?

Lyndsi [0:31:40]: Because then you know exactly where you're at, like, in the experience, what is going on and where things are maybe going sideways.

Lyndsi [0:31:51]: Yeah.

Ben [0:31:52]: A little bit.

Ben [0:31:52]: Sure.

Lyndsi [0:31:53]: But again, you can adapt this any way that you want for your org, but one way that Eye like is, like, the traffic light system.

Lyndsi [0:32:00]: Okay.

Lyndsi [0:32:01]: So just, like the red yellow green, you know, you don't have to have these complex scores you know, associated to each outcome.

Lyndsi [0:32:09]: Mh.

Lyndsi [0:32:10]: They could just be red yellow green.

Lyndsi [0:32:11]: When when they get to the product.

Lyndsi [0:32:13]: But when they get to the stakeholders, they just get red yellow or green.

Lyndsi [0:32:16]: Behind the scenes, we know, you know, what the scores are and all of that and what went into it.

Lyndsi [0:32:22]: But green, you're meeting the outcome, which is a strength area, yellow is, you know, mixed or inconsistent, which is an opportunity area, and then red means falling short and the users are feeling it.

Lyndsi [0:32:34]: Yeah.

Lyndsi [0:32:34]: Which is a growth area.

Lyndsi [0:32:35]: And so I think the power of this approach is that you're not just saying like, you know, we need to improve the N mps, you know, which it's...

Lyndsi [0:32:44]: That's vague and disconnected, but you're saying like, this specific part of the experience is breaking down, let's fix that, you know?

Lyndsi [0:32:53]: And that's really powerful I think because you can all rally around that and actually take action and it's specific enough to, like, guide real decisions and not just be like, well, let's raise the Mps.

Ben [0:33:08]: Like, yeah.

Ben [0:33:08]: Yeah.

Ben [0:33:09]: It it ends up you'd you said this when we were preparing for the conversation that one of the benefits of the outcome approaches that it provides a lot of clarity.

Ben [0:33:16]: And I think what it also does in addition to that is it sort of turns down the temperature on the politics that can sometimes happen in product development You know, you might have that one and not trying to throw shade on any Pm, but, like, there's usually a not butt period.

Ben [0:33:27]: There's usually that one Pm that's been there for a minute and their product often will get a little more love or get priority on the road map or get sooner.

Ben [0:33:35]: And so this way, it democrat sizes and levels the playing field?

Ben [0:33:39]: What are some other the benefits?

Ben [0:33:40]: The team gets clarity on what's working and what's not?

Ben [0:33:42]: What else have you seen both from the research, the design side as well as the stakeholder side when a team or a company or a product group shifts to this more outcome approach?

Ben [0:33:51]: Way of of looking at the experience.

Lyndsi [0:33:54]: Yeah.

Lyndsi [0:33:54]: I think that what I really like is that you kind of stop chasing these, like, hand goals, like, make it easier to use, You know, like, my.

Lyndsi [0:34:03]: What does that mean?

Lyndsi [0:34:04]: Right.

Lyndsi [0:34:04]: Like, make what easier to use, Which part, like, where, and you start kind of focusing on, like, the y behind the design, and then I also really like that it helps with prioritization because if you've got two ideas on the table, which we often do.

Lyndsi [0:34:19]: Right?

Lyndsi [0:34:20]: You can ask, like, which one helps us improve more outcomes or which one will help us move, like, a red to a green?

Lyndsi [0:34:28]: Mh.

Lyndsi [0:34:28]: I think that's just a much more productive conversation than to say, like, which one feels more important.

Ben [0:34:35]: Oh my gosh.

Ben [0:34:35]: Yes.

Ben [0:34:36]: That feel.

Ben [0:34:37]: And it's important when we're doing user statements because that empathy is important or not, but the feel when you're trying to do product prioritization.

Ben [0:34:44]: Who's gonna win?

Lyndsi [0:34:46]: Subjective?

Lyndsi [0:34:46]: How do exactly.

Ben [0:34:47]: How do you determine between two?

Ben [0:34:49]: I feel like this is Well I feel like this is simple important.

Ben [0:34:51]: Well, what do you do And as the research for your caught between multiple stakeholders for you both wanna please.

Ben [0:34:55]: Yeah.

Ben [0:34:55]: That's such a great call out.

Lyndsi [0:34:57]: Yeah.

Lyndsi [0:34:57]: So you can just point to it and say, like, well, this is what, and this is why.

Lyndsi [0:35:01]: Yeah.

Lyndsi [0:35:02]: And this is where we should go.

Lyndsi [0:35:03]: You know?

Lyndsi [0:35:03]: It's directional and it's actionable.

Ben [0:35:07]: Yeah.

Ben [0:35:07]: I love that.

Ben [0:35:07]: This is what and this is why.

Ben [0:35:09]: And so is that...

Ben [0:35:09]: Would that be one of the pieces of advice that you have for teams who might be or Ics we're listening to this and maybe they're at a a startup up or a smaller company or maybe we'll say less mature user experience or design focused company and they wanna start measuring Maybe they've heard from their executives or their managers like, how do we know this is working?

Ben [0:35:26]: How do we know your work what you're spending your time doing is making an impact?

Ben [0:35:30]: You know, is that the kind of advice Is that what you wish more teams would start doing, especially on measurement?

Ben [0:35:35]: I think we get kind of obsessed with quantifying?

Ben [0:35:37]: Like you said, when use these fuzzy inputs like, do the users like it?

Ben [0:35:41]: Does it feel better?

Lyndsi [0:35:42]: Yeah.

Lyndsi [0:35:42]: Yeah.

Lyndsi [0:35:43]: I I think that...

Lyndsi [0:35:44]: Especially, like, with the smaller teams and the lower resource teams like, just start small.

Lyndsi [0:35:48]: Like, start with a few outcomes that your team feels like are important to users and then go vet them.

Lyndsi [0:35:55]: With the users obviously.

Lyndsi [0:35:57]: So then it's again, it's not just, like, a feel pain.

Ben [0:36:01]: Yeah.

Lyndsi [0:36:01]: And then once those are embedded then that start measuring just a few of them.

Lyndsi [0:36:06]: And I think the other advice I have is, like, it doesn't have to be perfect.

Lyndsi [0:36:10]: It just has to be meaningful.

Lyndsi [0:36:12]: Right?

Lyndsi [0:36:13]: So I think teams a lot of times just get stuck and trying to find either, like, the perfect metric or the mix of perfect metrics and when they can't, they just default back to the easiest one which is like, Mps c sat, and it's not, you know, it's just because it's difficult to try to find those perfect metrics, and you're never gonna find the perfect thing.

Lyndsi [0:36:36]: But measurement isn't always about having, like, a perfect scorecard.

Lyndsi [0:36:40]: It's about asking better questions.

Lyndsi [0:36:42]: Right?

Lyndsi [0:36:42]: And so what does success look like for our users.

Lyndsi [0:36:46]: What's the smallest clear sign that we're on the right track to start there.

Lyndsi [0:36:52]: And I think this outcome, you know, driven measurement helps you do that because it gives you something to anchor to.

Ben [0:36:58]: And helps that.

Lyndsi [0:36:59]: Your team focus like, less on the noise and more on the signal.

Lyndsi [0:37:02]: Yeah.

Lyndsi [0:37:02]: And that's what I kinda love about.

Lyndsi [0:37:05]: This framework more.

Ben [0:37:06]: Absolutely.

Ben [0:37:06]: And it makes me...

Ben [0:37:07]: It reminds me of something that I'm hearing more teams do and that is create sort of customer advisory boards or internal panels, not certainly, you know, you can create a large panel if maybe you have a difficult to reach population, but if you're thinking as you're listening to Lyndsi, like, how do I talk to users quickly Like, well, that's what a customer advisory board could do.

Ben [0:37:26]: If you have twenty to twenty five folks who represent sort of your core user group or segment and maybe they're that they're in that champion or evangelist?

Ben [0:37:34]: They'll tell you if something isn't working.

Ben [0:37:36]: They'll tell you if an outcome statement doesn't resonate.

Ben [0:37:38]: And so, you know, you send them a quick pulse, share with them a survey, grab ten or fifteen minutes.

Ben [0:37:42]: Is that Lyndsi, how you often think about this?

Ben [0:37:44]: It sounds like it's scrap it doesn't have to be...

Ben [0:37:47]: Like you said, perfect and perfectly laid out and and and take a long time.

Lyndsi [0:37:51]: I've done, like, both before.

Lyndsi [0:37:53]: I've had customer advisory boards where, you know, we've, like, met once a month or you could just send them a quick email.

Lyndsi [0:38:01]: Yeah.

Lyndsi [0:38:01]: Like you said, get the pulse on something, while simultaneously talking to users in user interviews, you know, like, if you can have a couple different methods going on, That's obviously great.

Lyndsi [0:38:13]: But again, if you need to just get scrappy, gets scrappy.

Lyndsi [0:38:16]: Like, work with what you've got.

Lyndsi [0:38:18]: And if that is, you know, just talking to, like, ten users over here or if it's having this customer advisory board, just start.

Lyndsi [0:38:26]: Yeah.

Lyndsi [0:38:27]: To start.

Lyndsi [0:38:27]: That's really advice that I would have.

Ben [0:38:30]: I appreciate that.

Ben [0:38:31]: I wanted to ask two follow ups.

Ben [0:38:33]: One, how often do you typically or would you recommend folks revive or revisit the outcome statements?

Ben [0:38:39]: Like, is there a good moment when you would recommend team checking back in, like, a a road map revisit or at the end of the year, when have you found that revisiting the outcome statements makes sense.

Ben [0:38:50]: Because you don't wanna be doing it too often and such we lose momentum and you don't build enough information.

Ben [0:38:54]: So when have you found that it works to revisit those?

Lyndsi [0:38:57]: Yeah.

Lyndsi [0:38:57]: I mean, you could do it every six months.

Lyndsi [0:38:59]: You could even do it honestly every twelve months, but Okay.

Lyndsi [0:39:02]: That comes with a caveat of, also, if you are working on outcomes, You know, like, if you've got three red, let's say, and you're working on those, and you push out a quote fix for those, like, a new release.

Ben [0:39:19]: Sure.

Lyndsi [0:39:20]: You might want to rerun it.

Lyndsi [0:39:22]: After, you know, after the release is has been out for a few weeks or something.

Lyndsi [0:39:27]: So I would say, maybe once every six months or if you've got changes that you wanna measure.

Lyndsi [0:39:33]: Those changes.

Ben [0:39:35]: So I appreciate that.

Ben [0:39:36]: That's that's really useful.

Ben [0:39:37]: And then the second question, when we started our conversation, you talked about how you began as maybe considering clinical psych, and you've got involved in counseling in school, you have this through line from my Poe of, like, a a good sharp understanding of quantitative value and assessment.

Ben [0:39:53]: And I think for some, not all user researchers.

Ben [0:39:55]: That's sort of the, like, oh gosh.

Ben [0:39:57]: I'm a numbers afraid person and Ai can and large language models and workflows and all the things that have been battering us left and right.

Ben [0:40:06]: A lot of it is quantitatively oriented.

Ben [0:40:08]: You know?

Ben [0:40:08]: Maybe we're partnering with data science or we have a quan person we're working with.

Ben [0:40:13]: Is there, you know, a skill that you think today's user experience professional really should learn from the more quantitative assessment side or something that they they often misunderstand about doing this sort of more quantitative assessment work, because it's clearly helped you in your work in your career, but you know, for someone who's listening to this and is a little, has a little number fear to them, what advice might you give for them to kinda get over that and put it practice?

Lyndsi [0:40:37]: That's funny.

Lyndsi [0:40:37]: If you would asked me this, like, a few years ago, I would have said, like...

Lyndsi [0:40:42]: Take a course in r, you know, just to get familiar with it.

Lyndsi [0:40:46]: But I don't know that I would even say that now.

Lyndsi [0:40:48]: And I'm sitting over here looking for my...

Lyndsi [0:40:50]: So I have this book that, honestly, yes, I have Actually, I've got a couple of books, but I have this background in psycho and and all of this.

Lyndsi [0:41:00]: And still, I refer to books, like, I look at them all the time and one of our product people is, like, she's got her book out, you know.

Lyndsi [0:41:09]: So it's Yeah.

Lyndsi [0:41:11]: Anyway, but point being, I would have said, take a course in r.

Lyndsi [0:41:15]: And still, if you wanna take a course in just to become familiar with it.

Lyndsi [0:41:19]: I think that would be great.

Lyndsi [0:41:20]: But like you said, Ai is, you know, been battering us, like, I think we all...

Lyndsi [0:41:29]: Not just people who are kind of scared of numbers.

Lyndsi [0:41:32]: I think we all need to learn how Ai can help us with quantitative metrics going forward.

Lyndsi [0:41:40]: I think that is kind of the way forward.

Lyndsi [0:41:43]: And...

Lyndsi [0:41:43]: We hear about, you know, Ai all over the place, all of us just it's it's everywhere.

Lyndsi [0:41:48]: Right?

Lyndsi [0:41:49]: But I think it's really kind of the future.

Ben [0:41:53]: For sure.

Lyndsi [0:41:54]: And it's supposed to help us do things more efficiently and why can't it help us do quantitative research more efficiently or at least help us understand the numbers more efficiently.

Lyndsi [0:42:05]: Right?

Lyndsi [0:42:05]: Yeah.

Lyndsi [0:42:06]: So that's where I would probably start along with carrying your textbooks around my either.

Ben [0:42:11]: Yes.

Ben [0:42:11]: I was gonna say you're...

Ben [0:42:12]: I'm wanting you to bring out this orthopedic thing Like, here's what I use everything.

Ben [0:42:16]: It's this eight inch thick thing that, I hear you.

Ben [0:42:18]: I had a a small amount of stats training years ago, and I found that Well, I might not pull the specific stats, you know, the, an equation, being able to, like, know roughly what a standard deviation is and be able to ask for, you know, know the difference between a mean and a median and when to use those things.

Ben [0:42:34]: Like, you can plug those into an Ll and Long as you can have that baseline, you'll be able to interpret.

Ben [0:42:39]: You know, that's what so many of the Ai adjacent researchers are saying, like, you know, vet the tech, you know, trust the tech, but also verify.

Ben [0:42:47]: And so if you have that foundation, you can know what you're looking for.

Ben [0:42:49]: Because if you don't know what you're looking for then.

Ben [0:42:51]: Right.

Ben [0:42:51]: You might deliver something that might end up hurting or harming and experience down the line.

Lyndsi [0:42:56]: Yeah.

Lyndsi [0:42:56]: Sample size two is a big one.

Lyndsi [0:42:58]: You know, right now, Unlucky to have a very good font person on my team, but, like, anytime, I wanna know anything about...

Lyndsi [0:43:06]: I'm, like, sample size.

Lyndsi [0:43:07]: Yeah.

Lyndsi [0:43:08]: Is that a good sample size.

Lyndsi [0:43:09]: Is that a, you know, like.

Lyndsi [0:43:10]: And so even if you don't have a quan person on your team, you can still, like you said, go verify.

Lyndsi [0:43:18]: Yeah.

Lyndsi [0:43:19]: But go to Ai and figure out, like, what your sample size should be.

Ben [0:43:24]: Yeah.

Lyndsi [0:43:25]: Understand what a standard deviation is.

Lyndsi [0:43:26]: Like, a even just a basic Linkedin learning or something or any of those courses, like, a basic stat intro.

Lyndsi [0:43:36]: I think would be good and then go to Ai and, like you said, as long as you know what you're looking for.

Ben [0:43:42]: Yeah.

Ben [0:43:42]: Yeah.

Ben [0:43:43]: It'll help you reinforce or figure out.

Ben [0:43:44]: Yeah.

Ben [0:43:45]: It's kind of a your teaching assistant or your your intern.

Ben [0:43:48]: Yeah Alright.

Ben [0:43:48]: A couple of other resources that you would recommend for folks that you found useful just in general, Maybe not necessarily about measuring experiences, but just user experience, research or design, resources that you like.

Lyndsi [0:43:59]: Yeah.

Lyndsi [0:43:59]: Well, there is a book called measuring the user experience.

Lyndsi [0:44:02]: Looks pretty good.

Lyndsi [0:44:03]: So just fyi I on that one.

Lyndsi [0:44:05]: But I know that there is no way that you haven't heard this one, but I do recommend Erica alcohols book.

Lyndsi [0:44:13]: Just enough research.

Lyndsi [0:44:15]: I'm sure you've heard that plenty of times.

Lyndsi [0:44:16]: It's just approachable, and it's a little snarky, and I like that.

Lyndsi [0:44:21]: You know, it's just very to the point.

Lyndsi [0:44:24]: And I think, honestly, it's good for, like, designers or Pms or anybody, like, that wants to know more about research.

Lyndsi [0:44:31]: So...

Lyndsi [0:44:31]: I love that one.

Lyndsi [0:44:33]: And then I'll recommend a podcast as well.

Lyndsi [0:44:35]: It's called hidden brain.

Lyndsi [0:44:37]: Yep.

Lyndsi [0:44:37]: And So.

Lyndsi [0:44:39]: Yeah.

Lyndsi [0:44:39]: It's not about Ux per s.

Lyndsi [0:44:41]: Right?

Lyndsi [0:44:42]: But it's just a good reminder of, like, how people think and how they feel and how they make decisions.

Lyndsi [0:44:47]: So I mean, I guess it's kind of like a psych major drone of sometimes with.

Lyndsi [0:44:51]: Right.

Ben [0:44:52]: Totally.

Lyndsi [0:44:52]: It just helps me to, like, zoom out sometimes and think more deeply about, like, user psychology and why people do the things that they do.

Lyndsi [0:45:01]: And it's so central to good research.

Lyndsi [0:45:03]: You

Ben [0:45:03]: know.

Ben [0:45:03]: And it makes complete sense.

Ben [0:45:05]: I mean, it hindsight here, but your attraction to outcome oriented and research.

Ben [0:45:10]: I mean, so much of that I imagine is alluded to in a in a podcast.

Ben [0:45:13]: Like hidden brain that talks about, like, our underlying motivations and the goals we have.

Ben [0:45:18]: And I think...

Ben [0:45:18]: Right.

Ben [0:45:18]: That's such a great way to end because the real beauty of the framework that you've shared with us today is it...

Ben [0:45:24]: It helps us as ux professionals.

Ben [0:45:26]: I say that, you know, if you're in product design, research.

Ben [0:45:28]: It helps you pull away from your particular experience because I I think sometimes we can get sort of bogged down, like, I work in this kinda of tech and it has to have this kinda outcome like, well, are you dealing with people trying to do a thing in this world?

Ben [0:45:41]: Well, like, start there And I think Lyndsi, that's what I love about this approach.

Ben [0:45:44]: Is it it helps us to sort of pull ourselves up, meet with our stakeholders, think about what we're actually trying to help people do.

Ben [0:45:51]: So get out there everybody and start thinking about your experience in outcome, atomic outcomes, and I think you'll see your experience get better.

Ben [0:45:58]: Lyndsi.

Ben [0:45:59]: Thank you so very much.

Lyndsi [0:46:00]: Yeah.

Lyndsi [0:46:00]: Thank you.

Erin May [0:46:08]: Thanks for listening to Awkward Silences, brought to you by User Interviews.

Erin May [0:46:11]: Theme music by Fragile Gang.

Erin May [0:46:13]: Hey, there Awkward Silences listener.

Erin May [0:46:25]: Thanks for listening.

Erin May [0:46:26]: If you like what you heard, we always appreciate a rating or review on your podcast app of choice.

Erin May [0:46:32]: We'd also love to

Carol Guest [0:46:33]: hear from you with feedback, guest topics or ideas so that we can improve your podcast listening experience.

Carol Guest [0:46:38]: We're running a quick survey so you can share your thoughts on what you like about the show, which episodes you like best, Which subjects you'd like to hear more about, which stuff you're sick of and more just about you.

Carol Guest [0:46:48]: The fans have kept us on the air for the past five years.

Erin May [0:46:51]: We know surveys usually suck.

Erin May [0:46:53]: See episode twenty one with Erica alcohol for more on that.

Erin May [0:46:56]: But this one's quick and useful we promised.

Erin May [0:46:58]: Thanks for helping us make this the best podcast it can be.

Erin May [0:47:02]: You can find the survey link in the episode description of any episode or head on over to user interviews dot com slash awkward survey.