Erin: [00:00:00] So Luke Fraser, awesome to talk with him about how to actually make democratize research happen in a big old and tightly regulated and enterprise company.

JH: [00:00:14] Yeah, it's hard to imagine a more regulated industry than insurance. And so the fact that he was able to find ways to do user research within Liberty mutual is just really cool and a lot of great advice there, I think for other people, um, to, to leverage.

Erin: [00:00:28] Yeah, absolutely. I'm always inspired by people who can work within constraints to make things happen anyway. You sort of don't let the obvious excuses given the way of doing what you really want to do. So we've had tons of examples of being able to do that successfully that I think will be inspiring and useful for lots of folks

JH: [00:00:48] Yeah. And if nothing else, check it out so you can hear how to get someone from legal to start attending daily stand ups. That's a fun little insight as well.

Erin: [00:00:55] Yeah, absolutely.

Luke: [00:00:57] I think some of my research practices are great, but there's also some, some really other great ones out there that I don't know. I think what I. Would I am not lenient on is, , respecting the individual, um, respecting their time and respecting their story. Hello, hello everybody and welcome back to Awkward Silences.

Erin: [00:01:34] We are here with Luke Fraser. He's the founder and managing director at paper ventures. He has a ton of experience working in large fortune 100 companies and tightly regulated environments, and we're going to draw on that expertise today to talk about how to really get folks comfortable with. Doing user research in those kinds of environments.

So thanks so much, Luke, for joining

Luke: [00:02:00] Thanks so much for having me here. Yeah.

Erin: [00:02:03] We got JH here too.

JH: [00:02:05] I am also here a little under the weather it's just me like my flu game. So it should be a, should be a classic episode by those standards.

Erin: [00:02:11] You might start to hallucinate by the end, but that'll be very entertaining for all of us. So Luke, thanks. Thanks for joining us. You have a lot of experience with, with working, in these kinds of environments that can be a little bit tricky to get people interested in what we all love, which is user research.

Could you tell us a little bit about. What makes it difficult for people to be comfortable with user research from your experience?

Luke: [00:02:38] Thanks again for having me on. So, yeah. Um, had the opportunity to work within big firms, small firms, sometimes in the agency setting. Have found in, in a lot of the, the bigger fortune 100 companies, there's a lot of hesitancy early on in trying to execute some of the, field user research that designers and researchers and PMs might want to do.

And when I say that it's in my head, things like, you know, we want to design for, parents were bringing home their first child, or people who are doing home renovations, things like that. And so a researcher, designers. The first, first thought might be, great, let's go into the home and, and let's talk to them about home renovations, in their kitchen that they're renovating or, or, as they're getting the, the baby's room ready, things like that.

But a lot of large organizations just might not have that in their DNA to quickly go out and recruit some folks and go talk to them. So three kind of barriers that, that I've seen is one, just lack of familiarity of, of how to execute their own research game. A lot of firms or, a lot of larger organizations have outsourced that to, to research firms.

And then, two other concerns. One is, harm to the employee or risk to the employee doing the research offsite. And then three, any liability to the company of, of putting their employees out there. Those are kind of the three main barriers that I've seen that have stopped large organizations from, from actually executing and building their own internal research capability.

Erin: [00:04:05] Great. So let's, let's dig into those. Uh, wherever you want to start, which is the most, common or the most difficult barrier to get over, would you say.

Luke: [00:04:16] I think. My experiences working in, in large financial services organizations, a hundred plus year old organizations. They just have a different type of legacy and different type of risk profile. So , for me as an employee in the past, the harm to the employee and harm to the company are kind of the biggest barriers.

But I actually think a lot of folks think that to execute research is, is a lot bigger deal than it, than it actually is. Um, like when you distill it down to the, the most basic thing you're doing is you're, you're having a conversation with another person. Things we do all the time.

Early on, if folks are like, well, I didn't even know how to execute, research or start talking to a user, I would just ask them to think of a potential user in their own friend or family network and go have an informal conversation with them and come back with the lens of how do we use that conversation to inform feature development or a new product and kind of show them that logistically it's not that difficult and it's, it's as complicated or complex as you want to make it.

JH: [00:05:16] It seems like, the lack of familiarity and like, not in the DNA. I can imagine kind of getting over. Right. Like, do some research, grab an analyst, you know, find some users who are willing to talk to you. Like. It. Those kinds of feel like solvable with like a little elbow grease and Googling and whatever it may be and some internal goodwill building.

Getting over the hurdles around like the concerns about harm to employees or liability for the company for me as like a non-legal person. Feel like much harder barriers to get over cause like I don't, I wouldn't know how to make those arguments to the people that care about that stuff to, to convince them that like there's not a liability risk or, or things like that is how do, how have you seen that navigated.

Luke: [00:05:55] Yeah. I think that the best way that I've seen that done is, is thinking about, three things. I'd say, uh, compromise, um, involvement and then figuring out. Like, what's your, what's your SLA or what's your service level agreement? In organizations where I've seen this happen, we really tried to , understand the concerns of, of HR and legal, and then almost just kind of like prototype together ways that we could achieve our research goals and they could achieve a level of, comfort. So, for example, early on in building a research capability, one of the compromises that we made with these stakeholders was great. We can recruit whomever we'd like, but we'll always meet with them in a public, space.

So while the purist in me would have been like, well, we need to be in context, if the next best thing is meeting in the local library with them to have an early first conversation, like that's a win win. I think. The second way that I think is super, super helpful here is, is getting these folks in.

In our, in my former case, it was, you know, different types of counsel and, human resources representatives. Getting them involved in the research. So bringing them along to conversations, maybe at the local library or the Dunkin donuts, for them to really understand like, Oh, this really is just a conversation with somebody else.

And that has helped with momentum because they can see, that it is pretty low key. They can see when there's a product being developed as a result, or are new feature set as a result. They get to see that it was informed by some of the research that they permitted.

Erin: [00:07:28] Get involved. Feels like just good advice always for user research, right? When you're working in organizations for people who are a part of it and see the benefit of it and that it's not scary, whoever they might be are going to become champions of user research.

Luke: [00:07:45] Yeah one of the best outcomes of some of this was, one squad I was on, we were building a new adjacent insurance product and digital sales and claims platform to test it in the market. And we didn't fully tell our legal team that we were doing it initially.

And when they found out, um, they didn't find out too late, but when they found out we got to the awesome agreement that one of their team members would be on our scrum team. And so, every single morning, we all did our stand ups and, we had cards on the board for legal as well as for, you know, design and development and things like that.

And it was a wonderful, way to have, I guess, a stakeholder that you've traditionally wouldn't, you know have on your, on your product team to be on there. you know, certainly there were, trade offs there, but we were just able to understand each other so much better and, and we like to talk a lot about that.

Each member on the team had a, had a super power that we didn't want to block. as another teammate. And so really were allowed like counsel to do their thing and make sure we were operating legally within the States we were launching and then let the PMs do their thing as well. And some of the best moments are when, you know, this particular individual on the, on the squad would be a huge advocate in executive team meetings because, cause they were on early research and, and they could stand up for, you know, Susie from Medford or something like that.

JH: [00:09:06] That's really cool. Now I'm just like fascinated by what sort of standup updates the, uh, the council person was giving but,uh probably don't need to go down that tangent.

Erin: [00:09:17] Lots of acronyms. GDPR. Compliance. Um, I also love when you talked about how you kind of figured out how you're going to make this happen. This happens a lot where you're basically user researching your own team, right? You're doing interviews, talking to people, trying to understand what they really care about where their objections might be.

I think you said you prototyped, right? Possible ways of making this happen, and then I'm sure you iterated as you have this council on your, so it's, I think that's so interesting, bringing those same kind of skills into your internal teams and figuring out how to function effectively.

Luke: [00:09:53] Right. Yeah. That, that's a good call out. I think what I'd add to that too is, is reminding your internal teams that The decision we make today is it can be valid for, for a week or two, and then we can revisit it. and just like this, this itself is an experiment and we'll keep you, keep you involved in all of the, the data and output and we can, we can all iterate accordingly.

Yeah.

Erin: [00:10:18] So, so harm harming people is not so good. Um, no one wants to do that. And you've got legal and HR kind of stakeholders in that. You went, you talked a little bit about how, how legal has gotten involved. What about on the HR side?

Luke: [00:10:32] Yeah. Similarly, we would, bring HR, legal, and then, in the kind of financial services and insurance world, there's also product development folks that are building the financial instruments that support the experiences. also called product management, which can make it sometimes confusing in a large company.

And we would identify those folks early on, get them on the project team early on and, and really bring them out with us. And what was really cool is that, you know, in, in trying to understand the broader organization in everyone's goals. We would identify ways that, we could help them in a future project.

You know, that team in particular was focused on, new product and service innovation. And sometimes that meant, uh, internal innovation. And so that was a really cool kind of unplanned research project, as well. But, you know, nothing crazy, but just the reminder that , getting everyone involved, as early as possible to prevent surprises and unexpected news to people is just always a good idea, especially if it's , a new practice

JH: [00:11:30] the advice and the tips you've shared about like how to navigate this all make a ton of sense to me. I guess the question I have is when I hear, you know, like legal on a here HR, I guess probably hopefully incorrectly, but like I assume that means like stuff's going to take awhile. Was this a process that once you got it rolling like took a long time to get some of these experiments out orby pulling people in early and getting them on the team, were you all able to actually make some pretty like rapid progress and figure this stuff out.

Luke: [00:11:56] Yeah. I'd say on the initially, after we tried to understand the barriers and some of the concerns from the stakeholders, we came up with our own approach and service level agreements and, shared them with those folks and, and made it as, Hey, here's how, here's how we see, we're going to approach this.

And if you see huge red flags, please stop us. So, almost a, in insurance, you'd call it a desk filing. But you know, here's what we're doing and we're going to move forward with this, unless, unless there's a big issue. on the second piece, I think having counsel as part of our, like on our scrum board was incredibly helpful in terms of timing, , because you didn't just shoot an email into the abyss and then hope that it hope that it turned around. you saw that there was, uh, a story for someone, and that was priority and that was taking, you know, X amount of time, and if it was going to be delayed, that's totally fine, but we're really super lucky to get, that visibility and those updates. I, I'd also say that, we, through our, through our partnership with legal, I learned the right questions to ask, of when a decision had to do with legality, whether it had to do with internal comfort, And then third, to ask, what is the kind of expected value of taking this risk?

Um, especially when we're working with such small sample sizes. oftentimes we would work together to just really understand like, how big is this risk What's the chance that it happens? And then as the business, are we able to accept that risk and move forward?

Erin: [00:13:24] Yeah. That we hear that in, in organizations of different sizes a lot, right? The what you're going to learn what from talking to five people and we're going to spend what on on that and. Is when you talk about lack of familiarity, right? Part of that is we don't know how to do this. We've outsourced that, and part of it is we don't know why we should do that, or if we should do that.

Have you had success with selling the value of doing a user research, qualitative research in the first place within these organizations?

Luke: [00:14:35] Yeah. And you know, I think, I think a lot can be said up front in any team. You should really try to make their case of, you know, here's the approach we're taking. Here's. here are other organizations that leverage this. It's best practice to do these types of things.

but nothing is a replacement for your either research chair or your opportunity share, to really drive home that, you know, Erin, you mentioned five people and I, I think all of us are probably agreement in this call that. There's, there's almost an infinite amount of, of salient stories and anecdotes and quotes, , and data to gather from talking to five people, and it's on the research team and the, the, the PM or whoever's the team synthesizing all of this to, to really prove that it's valuable throughout the project.

So one thing that we kind of built into our practice. We were doing a lot of, um, early stage product, ideation and testing. And so we would maybe do a few weeks of research and then we would make sure that, we had a research share that was in person and we would block off like two or three hours for it.

And we tried really hard not to make it . Like a Skype meeting. because what we really wanted to do is immerse the group, the stakeholder group as much as possible, and increase the information saturation of everyone in that room and have the real time to say, you know, one of the themes that we came across was, was X, Y, Z, and, and meet Erin. Erin is a small business owner in downtown Boston, and share an anecdote about Aaron, and oftentimes.

It took until that output share for folks to say like, Oh, I get this and I get Y gives you a different output than sampling 500 people online.

Erin: [00:16:28] Yeah. Yeah, totally. What about the dollars and cents piece of it, right. we've seen studies that say, right, for every dollar invested in UX, you get back two to a hundred dollars or things like that. Is that a convincing argument for folks you've worked with?

Luke: [00:16:44] I can't say I've ever used that. what I have done, which is probably, I guess it's less out outcome oriented, but say, so this research is probably going to happen. Um. In some capacity. The traditional way of doing it is going to take you four times longer and probably be four times more expensive.

And, uh, what I can deliver is I can speak to the same amount of people. I can do it in. You know, a quarter of the time and quarter of the budget, that's what I think. Um, just in my experience of working with different recruitment firms and, their minimums and their timelines and turnarounds and they can do great work as well, depending on the project needs.

But I think that recruiting through your own networks, peer networks, your employee networks, Facebook, Craigslist, user interviews, and doing your own logistics I think it's lean. It's mean, it gets your whole team just so much closer to it.

So I would just make a cost, uh, argument to that.

Erin: [00:17:44] no, it's interesting. I was thinking of it coming from the other way, should we invest in this? But you're coming at it, right. Of course. From, we already have these big budgets for research and we've been doing it this slower, faster, fatter way, and we can actually, uh, get the same result faster, better, cheaper.

Luke: [00:18:01] Yes. I think where, where I've had, I've received questions of value Is great. We believe in the research that has been done. How do we know this is valid beyond the 12 people you spoke with? and then I think that's where it's fair to try to pull in a, a larger quant survey, but even that can be done, fairly quickly and fairly cheaply.

JH: [00:18:25] To jump back to something you shared a little earlier about, um, you know, the expected risk versus the expected value. I was wondering, did any of the risks like come to fruition? So it seems like you as a team were pretty good about, you know, what could go wrong in a situation and how to mitigate, like meet in a public place and things like that.

were there any instances where some of those risks actually came to the surface and you guys had to navigate around that?

Luke: [00:18:47] Uh, not in my experience so far. what we always did was, we always screened individuals on a phone call at first. And, you know, that's helpful for a number of reasons. One, to see if they fit the qualifications that they, that they indicate. Two, to understand if they'll be a, great participant, if they have a lot to say on a, on a certain subject.

and then third, finally to, to just do our best with intuition on if their intentions are sound. And so that tends to be, a good screen. And if, if any, if at any point our gut said that it wasn't a good idea, we didn't, we didn't go do it. I think the only thing that, okay, I'm lucky, but I think the only negative outcomes, which I think every researcher has come across as just like.

Oh, this wasn't the best interview we've ever had or, oh, we made a recruitment mistake here. but luckily haven't had anything else. I think, if we're honest with our intentions for the research, most folks are just super willing to help and kind of, I've found that user research just reminds me of the good in, in humanity almost.

Um, I'm sure there are researchers who say the exact opposite.

Erin: [00:19:57] Not a lot we talk to, but I guess they don't feel like joining the podcast so much.

Luke: [00:20:02] Yeah. I was talking to someone this morning about a previous project and they asked, you know, why do people agree to talk to you? And you know, there's compensation, but I actually. I think the best participants, you know, the compensation is, is nice and I think people should be compensated for their time.

but there are people who, who either have a pain or have an excitement that means so much to them that they want to share. And oftentimes researchers are in. Are organizations that are trying to figure out the best ways to help them. And so if that's the true purpose, um, I think most people want to be helped and most people want to share their story.

So, usually the intentions are good, but that might be some rose colored glasses on my part

Erin: [00:20:51] Yeah we, we did a whole episode on that. We talked to a bunch of participants and asked them, well, you know, why do you participate in, yeah, money's. Or compensation, right? Some folks to other kinds of thank yous, but it's gotta be part of it to compensate for the time.

But really the emotional drivers are, um, like what you said, but I want to be heard. You know, my, I want someone to care about my opinion. wanting to learn about companies and their innovation. Right. , yeah. So some, some cool motivations that drive interesting

conversations.

Luke: [00:21:23] Oh, I agree there. an objection that, that I've heard a lot executives at companies who are, who are listening to research shares there, uh, there sometimes is skepticism about. You know, who would respond on Craigslist to do this, or who are these serial or these serial interviewees type things.

Um, so there's definitely some skepticism about, about that out there.

JH: [00:21:50] You mentioned getting this started in one of your roles and kind of like within your team for the project you were working on, is that something that once he got started and once you had some success there that like, that broke down the barrier and like propagate it to other teams in the organization?

Or did it kind of stay local to the way that your squad and group works.

Luke: [00:22:09] I'm not sure if it's spread throughout the entire organization. Um, but it, it definitely was something that our teams within our division. Continued to do and could do without a significant pushback, which was, which was really great. , what was great at, you know, is it really big companies? So there are a lot of different teams.

Doing these things was great, as is once all of these teams gotten in touch with each other to say, Oh, you're using user interviews to recruit people. Oh, that's awesome. Uh, and then one team would say, well, we have this space that we don't use. If you want to use that for your. You know, dinner conversations.

and so I think once teams got to see that certain tactics were, were successful, uh, then they could adopt them or it was just easier for them to make the case. and I think, , one of the, one of the core innovation teams in the, innovation labs there did a really good job eventually of, um, of trying to at least have tabs on.

Best practices that were being successful across the org and just making them known so that folks could share them. another thing that I just brought this up, oftentime we got pushed back on, bringing cash to interviews as well. So like one thing we could share with folks that was really easy was, you know, go buy visa gift cards and you won't run into that, barrier.

but yeah, I don't know if it officially spread like that, but there are a lot of people doing really good things and at least we could contribute some things to, to the larger organization.

JH: [00:23:33] That's awesome. What was the pushback around cash.

Luke: [00:23:35] I don't know if it was fear that we would become theft targets. I, I dunno. Um, and I'm pretty lassiaz faire, like last night, fair person. Like I. I have, I have no problem. Like going to interview someone in their home. Um, but my risk, I guess my risk tolerance is very different. But yeah, we would, uh, by the number of visa gift cards we needed and activate them and, and uh, and hand those out.

Erin: [00:24:01] You know, it's a lot easier with User Interviews. Amazon gift cards. Yeah.

I just had to have a slide that plug in there. So compromise. You talked about compromise. I think, you know, there's a lot to unpack there in terms of, in any kind of negotiation or compromise situation and not, you're negotiating with lawyers, right? So, God bless. how do you. Good at it. Um, you talked about, well, like, Oh, we want to go into people's homes.

now of course, like we're imagining like worst case scenario, like silence of the lambs. I dunno, like something terrible is going to happen when you go on this person's home. = , but most people are nice and, and not crazy. So in that, in that case, you.

A compromise to going to a public space. how have you worked through various compromises that you've made and where to draw the line or where to push more or where no, this is really important to the research and if we don't do it this way, why do we bother you? No. Have you learned anything through the various compromises that you've made over the years.

Luke: [00:25:02] Yeah. I've learned to be as clear as possible on, my vision of the tradeoffs of making certain choices. you know, one for example is if, especially in like a, uh, a client, uh, setting where you want to have a one on one conversation and. The client team wants to be there. and that might be five of them.

We're like, I don't think a 6 v. 1 conversation is a, is a productive environment for anyone. Um, but it keeps the client informed. You know, I will offer my belief of a best practice. I but ultimately. You know, if there's a given amount of pushback, I'm happy to, to experiment with a couple of conversations in certain ways.

Um, knowing that, I think some of my research practices are great, but there's also some, some really other great ones out there that I don't know. I think what I. Would I am not lenient on is, , respecting the individual, um, respecting their time and respecting their story. so making sure that, you know, we are, incredibly transparent and honest about our intent and about who we are.

and then once we have their conversation or have their, have their story or, uh, have all of their learnings that, We stay true to that story. And then, uh, as whatever agreement we came to, , we respect, you know, confidentiality, you know, legally we have to, if there's certain, you know, HIPAA things involved there too. So I, I guess I, if I feel like anything is, is not respecting our participant, that's where I put my foot down. But otherwise, I'll, you know, give my strong opinion and, and see where it goes.

That's another, that's another thing from a regulated industry. , we've done a lot in the health and wellness space, and so making sure that we, if we're talking about a health experience or health history.

Um, that we are, making sure that we're abiding by all HIPAA requirements with that data and that data storage and confidentiality. so if anyone's working in a, an industry where they might touch that, that's definitely a point where I would, uh, check in with counsel on that just to make sure. I'd also, double check what your participant agreements look like, to make sure that, like if you are using someone's photo that you've taken that.

The language is in there and that they get to see that contract at least 24 hours before, uh, you have the interview with them, just as a matter of, of respect.

Erin: [00:27:37] What did we not ask you, Luke? What else should we know?

Luke: [00:27:40] Hmm. I would plug user interviews. I do, I do. To everyone. What's, what's your, uh, referral bonus here in G H? . Uh, the one is like, do your own risk assessment of the permission versus forgiveness balance. And in many organizations, folks might, folks might find that just going out and doing it is the best path for them. can't, can't assess that from afar.

Uh, the second I'd say is, um, sometimes having an outside catalyst is helpful here. So, um, I've been in organizations where. Because we had done a small engagement with an outside design firm their output was fantastic and we were able to point to that and say. This is the process that they went through.

And we want to, we want to emulate that. Um, so you know, if there's many cases where an outside catalyst is helpful, but, sometimes that's useful. And then the last one is, uh, I would advocate that everyone on the product or project team, play a role in recruitment at some point. Um, that's just like a principal that I have, , maybe you don't do it forever, but the quality of your research is larger depending on the quality of your recruiting. And so just understanding what matters, what doesn't matter, logistics of all that I think is, uh, is super useful. And I actually, uh , 90% of the time. Like won't not be the recruiter for my projects that I do.

Um, just because I love it. , but I also think like that's the, a lot of work is done in the recruitment process.

Erin: [00:29:12] sure. Hmm. You love recruiting. I think you might be the first person who has said that

Luke: [00:29:18] Um, I don't know if I said love, uh, but I definitely appreciate it. And, um, no, that, uh, time is only your friend in recruitment.

Erin: [00:29:28] why don't you do it so because it, because of the time, because your time is better spent doing other things?

Luke: [00:29:34] Oh, no, no. I only, I love, Hmm. I said love. Uh, I will always be, I will always be involved in recruitment because I think it's, I think it's so important, and I think, Mmm. Especially if you're recruiting for a difficult population or a specific population, having time on your side is, is only a good thing. Mmm.

Because you never know in that last, I always would tell, my advice to my teams is always, if you have to decide between getting our recruiting post up on like Friday or Monday, do whatever you can to get it up on Friday before you leave. Because those two days of just letting it sit out there is just,

JH: [00:30:15] That makes sense. On the, uh, on the first one you mentioned about like permission versus forgiveness. I have this like fake fictional thing in my head where a product manager and a designer at a large organization, you know, goes off and does research that wasn't fully sanctioned and then they get fired for it.

And then I just see like on medium, a headline, right, of like, I got fired for doing user research. And it's just like super viral all over design, Twitter, and this person has a million new offers to choose from. And I think, I think in a lot of cases, that person would end up okay. But, , this is all totally fictionalized in my head, but I think that's how it would go.

Luke: [00:30:56] I think once I've gotten advice of, of just make sure whatever you're doing isn't going to land on the front page of the Boston globe tomorrow. That was, that's like career advice I got once.

‍
Erin: [00:31:09] Well, I think that's a good place to leave it