Erin: Hi everyone. Welcome back to Awkward Silences. We are thrilled to be here today with Sonya Badigian. She is a UX and Content specialist at Marketade. And today we're going to talk about remote research, which is something near and dear to all of our hearts. So thank you for being with us, Sonya.

‍
Sonya: Yeah, thank you both so much for having me.

‍
JH: Nice to meet you. Thanks for joining.

‍
Erin: Cool. So let's get right into it. So you are a fan of remote research. Why?

‍
Sonya: Yeah, so just a short bit of background. I went to school for human computer interaction, and a big part of that was teaching us how to conduct to UX research in person. So like contextual inquiries, really meeting people in their environment, going to where they are. And that was always explained to me as an in person process.

‍
Sonya: Now, out of grad school I started working for Marketade, which is a completely remote company. So my exposure to remote research was not necessarily my choice. It was more of the function of the company that I was working with, who I felt really passionate about working with and has been great ever since I started. So my first exposure to it was on the job and I, at first because of my grad school experience, I kind of thought, wow, this isn't quite the full UX research experience. This isn't how I was taught to do it. And I felt like maybe it was missing something.

‍
Sonya: And after probably a few months on the job, a few studies under my belt, I started to feel differently. Like, wait, now I feel like we're getting better results here, what's happening? Why is this working as well if not better than how I was taught to do this research in grad school?

‍
Sonya: I've come around to a couple of different sort of thoughts about it. My first is that actually the whole goal of UX research is to meet people in their environment and context, and frequently we're just not able to do that being a remote company having clients all over the country. So in a way, the remote meets that goal better than having them come into an office somewhere, having them in an uncomfortable environment.

‍
Sonya: So eventually I realized that, okay, so the point of research is not to necessarily have the people there in person. That's not the goal, it's to be in their environment. So I feel like remote really accomplishes that.

‍
Erin: I love what you said about, I love those moments where you learn these theories in college and then you get like a real job, and apply these theories and it really brings those theories to maybe a more real or creative place where you can actually do things in a more effective way and relax. That's really interesting background.

‍
Erin: So remote makes it possible to meet people in context where they are. What's that look like from your experience?

‍
Sonya: Yeah. I think one of my more recent favorite stories was I was interviewing someone who was a fire protection engineer, and they were ... we were doing a usability test for a fire and life safety standards association. So a lot of these folks, it seems like work from home and so they have home offices and we were getting them to be looking at like a prototype at home, in their office, wherever they logged in from, which was frequently their home office.

‍
Sonya: This guy was doing his work and there were kids yelling in the background, there was a dog barking and my initial thought is, Oh God, it's going to be a bummer showing the client this because there's a lot of background noise. But also on the other hand, people have distractions at home and that's where that guy really works. So it was really kind of an empathy building exercise too, to understand what it looks like to be in this person's home or to watch them trying to do their work or get an insurance quote or pay their bills or whatever that the cases that we're testing. You get to see a real slice of that.

‍
Sonya: Maybe their baby's crying and I have to step away. That's happened to me during usability tests and they're like, "I'm so sorry. One second." And I'm like, "No, this is a window into your life. This is exactly what we want."

‍
JH: Do you think I, I guess, I was just going to put like the disclaimer on there of, I think there's probably some stuff that still needs to benefit from being tested in person. Like physical products come to mind, but would you put that disclaimer on it or do you think you can find creative ways around that limitation for any type of test?

‍
Sonya: Yeah, no. I think there is a disclaimer on that. Absolutely. I think that first off, you're totally right. Physical products, I think, are best tested in person. Also, there are some products that kind of get used in a more contextual manner like around the house. Like of example, it's not your t.v, it's this sort of space for software to be. And it's great because when we're testing internet based or computer based or mobile based software, the user can join me in a remote meeting on that device while using that device and sharing that device's screen.

‍
Sonya: But there aren't yet like webcams ... maybe I'm behind the times in webcams on t.v and I don't know if you can join a zoom meeting from your t.v or share your t.v screen so, you know, we had a proposal for a client who was kind of offering this sort of t.v based software and our research proposal for that ended up being in-person testing because it is important to see the user using it in their home context. But it didn't make sense to have them you know, setting up the camera behind their t.v couch while they were using the remote and all that sorts of thing.

‍
JH: Yeah. That makes sense. I think there's some allegations that Samsung's t.v are spying on us. Maybe they're doing some market research. The other thing that I think people, I don't know if you agree with this, but on remote stuff, is ... you mentioned like meeting people where they are at and actually seeing them in their environment which makes a ton of sense. But I think the other piece is the diversity that remote allows you to get. Obviously in person you can still get a wide variety of people and backgrounds but geographic diversity is a real thing too right?

‍
JH: People have different vernaculars and different norms and all that stuff. And so I think that's another big advantage in my mind. Do you guys try to be mindful of that when you recruiting people mobily?

‍
Sonya: Oh yeah. Absolutely. Actually, not only are we mindful of it in the normal day-to-day recruiting that we do but we even had at one point a usability test that involved a voice recognition software. So we had to recruit people with different dialects to participate. And that would have been impossible had we just been limiting ourselves to in-person testing we would have been trying to find someone with a Minnesota accent or southern accent, southern California accent, or someone speaking English as their second language. You know, that would have been impossible. If we've been trying to bring those people in in one small city.

‍
Erin: What are some of the tools or tips that you like to use to really bring out some of those advantages of remote testing to make people comfortable and keep them in their natural context?

‍
Sonya: I find it ... you know, a lot of the work has been done for me. People in their homes generally, they are comfortable. But other times they might be ... you know, I had a nurse off of her shift in her break room on her mobile phone one time and it's like that's when you probably would be trying to get an insurance quote, I guess. So that makes sense. You know, I think the schpeel comes into play here. Like the five minute thing you tell people before you get going and hit record on the meeting. I think that's where you have space to make people comfortable and to feel like they're free to share what they would normally do in real life.

‍
Sonya: And there is a couple of ways that I do that. First off, I tell them that I don't have anything to do with building the software. So they don't have to worry about offending me. I want it to be a conversation. I want it to be honest. And if they're on their own device, which is another huge part of not having people come into your office to test your software because they have things like password managers and other types of software that interact with what you might be testing.

‍
JH: Do you do anything else to try to get the rapport or talkativeness? Like where you do some kind of idle chatter to start or ... ?

‍
Sonya: You know, I don't do a ton of idle chat. I don't necessarily think that these people are expecting that when they log in for one of these. And I also ... you know, they're frequently on a schedule too. Because they are thinking 'Oh I can do this from my home. Let me squeeze this in, here and here.' You know, if we let them go five minutes early they're psyched. But I do think that rapport building is important. We can do that in other ways. Sort of in how you phrase the schpiel, how you present the schpiel. You can add a little bit of personality to it without necessarily having them talk about their day.

‍
Sonya: But in addition I do think that one rapport building thing that you can do is that you can ask questions before the usability test as kind of what we call a pre-test interview just to get a little bit more context around what we might be testing. So, you know, I don't just put the software right in front of them. But I'm not necessarily asking what they had for breakfast either.

‍
Sonya: I'll usually ask about things like, for example, we do have this financial services client as a company and one of the things we were testing was an app for roadside assistance with a vehicle. So instead of asking have you been in an accident recently? How often you call for roadside service. And of course part of that gets covered in the phone screeners too. But didn't need the details. Seem interesting in what the experience has been and then using their experience as the scenario for the usability test.

‍
Sonya: And that's where I think that some of the realism really comes in. And of course I guess that applies to having an in-person test as well. But I don't think the importance of that can be understated is that you shouldn't be given every user you test with the same scenario. You should be building a scenario based on their responses and with the rapport you build through those responses.

‍
Erin: Are you typically doing usability tests or are you also doing more discovery? If not, ethnographic style research or ... what sorts of research ... gonna guess you are doing a variety of creative research. But what do you do when you do remote research?

‍
Sonya: Yeah. So frequently we're doing usability tests and occasionally we will have a client come to us with ... you know, like "What would people think of this service if we were to offer it?" Kind of question. And in those cases we end up talking a lot about kind of what people currently use and then in some ways it ends up being a usability test of other services that are similar. Because I'll say things like, 'oh, well can you pull that up?' And that's what I love the remote testing because they got it book marked on their computer, they can show me the software they're using.

‍
Sonya: You know, you can really see their work flow.

‍
JH: Kind of like the flip side of them being in their own environment is that you have a little bit less control over the environment. Have you found ways to kind of minimize technical glitches or challenges getting the video to work, or ... I guess how do you work around some of that stuff where a little bit may be more as outside of your control than an in-person session?

‍
Sonya: Yeah. That's a good question. One thing I did run into recently, was someone really having trouble screen sharing. That's something I see occasionally, is people just can't seem to get their screen share going, whether they can't find the button on zoom or they, you know, I don't know what the case is. But-

‍
JH: I would like to say that it's not a technical thing, is our CTO can also not use screens very capable, sort of it's just a difficult thing. So it's not a-

‍
Erin: Or the camera.

‍
JH: Yeah. It's not a slight towards the participants.

‍
Sonya: Right. No. Of course it is not. So, what I did most recently is I was able to share my own screen with the prototype and then give them control of my mouse unlike all my meeting software and that was a great work around. I mean it was mostly great because then I would go to look at the script then maybe pulling my mouse the other way. I was like, "gosh.'

‍
Sonya: But you know, trying to make do for sure with technical difficulties. You know, taking advantage of the chat, if there is audio issues. You know, can you see me? Yes. Can you hear me? No. Okay call in. And giving instructions. I've found I've become a very good patient instruction giver through moderating remote usability tests as part, one of the things I love about the job.

‍
Erin: What are some of the other tools you use? You mentioned Zoom. Is that your go to or do you use what participants might already have or ... what-

‍
Sonya: Yeah. We primarily use zoom for the usability tests. I think there was a time when we were occasionally using go to meeting and we kind of transitioned to zoom a little bit more as a team as well for internal staff meetings and got more comfortable with that. And that seems to be a little bit more user friendly than go to meeting. Nobody- this isn't a plug. But I've found that users seem to have a little bit of an easier time getting their audio and their screen share set up.

‍
Erin: I was just gonna ask what do you use for prototypes or anything else that you might have for, in your go to toolkit.

‍
Sonya: Yes. So a lot of the times with our clients we're testing live environments which is really exciting. But when we're not, we sometimes build act share prototypes or our clients will. And I even have gotten to the point of using act share for ... like low fidelity prototyping as well. Just wire frames, I'll do an act share. At first I think coming out of grad school I was like, 'well balsamiq is for wire frames, and act share is for high fidelity and then eventually it just kind of got to the point where I was like, no I think act share is for everything.

‍
JH: I was gonna say, are you mostly doing stuff on, like people on their computers or do you do it on their phone or is that very test to test?

‍
Sonya: Probably 50/50. Some clients are really interested in ... you know, or it just makes sense to do it as a desktop app where they don't have necessarily their mobile web or an app but then the other half of the time we are using mobile phones and testing apps in mobile web experiences. So we even use zoom for that and we just tell people, you know, if you can download the zoom app to your phone before hand, it's free. Then they ... it's super easy to get them going on that.

‍
JH: Awesome. What's the set up look like on your side? Do you have note takers also calling into the meeting or are they looking over your shoulder or I guess, do you have a note taker?

‍
Sonya: No. I'm my own note taker but I do record the sessions. So I'll usually just watch them back. Sometimes I like double speed or something like that. Just so I'm making sure that I ... I'm catching everything. But frequently I find that the best way to start compiling results or to... if that's what the usability test requires or the client requires is just do it immediately after the session. And you know, everything is fresh in my mind. I'll make note of things they struggled with, things they said. I'll make notes to myself to go back and look up an exact quote. But I mostly just do it myself.

‍
Erin: Back to the tools, I'm hot on tools this month.

‍
Sonya: Yeah.

‍
Erin: Are you using anything special to collate those themes or google docs, or ... what do you kind of, what your flow after a session?

‍
Sonya: Yes. We're like fully a g suite company. We use google docs internally for basically everything whether that's slides or sheets or documents. And then we do use base camp in terms of just internal work flow which has been great. So my first company by using base camp and I like it so far. Although I think we are in two, not three ... I think they have an update so we're still in two. But it's been working great for us. And yes, so what I'll do is, we like to present findings to clients in one of two ways: sometimes the client will just say, we just want your impression of how this went. We'd be interested in seeing some user quotes or having access to the videos, but we just kind of wanna know what the next step should be for us or what your recommendations are.

‍
Sonya: And in that case I'll go straight to slides. Then I'll present those to the client but in other cases, and this when I think the work gets really fun is when we get to ...this is kind of the only in-person work I do truly is when we get to go to the client. And I'll bring those videos, those recorded videos of the usability tests and we'll run a full day workshop where we bring in the team. Like seven to ten people that are gonna be kind of involved in the next steps for that client's usability test or project or whatever the case may be. And we'll have them watch the videos and then they're my note takers. Sort of they're taking the notes on the videos, putting them up on the wall, and then we spend the whole afternoon basically just shuffling notes around, finding themes, finding problems and taking next steps.

‍
Sonya: So I guess to answer your question, my favorite kind of note taking is when the client does it. And that's something I'm getting to do more and more and I love that.

‍
Erin: Awesome. And in those cases you're clipping kind of ... are you pre-editing and finding the best moments in bringing this to the client? You bring the whole thing?

‍
Sonya: I bring the whole damn thing. It's amazing. And our clients love it. So I'll trim off ... you know, if I sneeze in the middle of it I'll cut that out. Or, I'll trim the schpiel at the start and I'll trim the part where I'm telling them how to, get their compensation. But the whole thing sticks and it's great.

‍
Erin: And do you kind of stick to the ... five people is enough or do you wait till you... the themes emerge? What's your process for how do you figure out how many people to talk to for a remote usability test?

‍
Sonya: We generally do five to six. Sometimes we'll have a no-show and it's four. If I feel like people aren't starting to repeat themselves I'll get another person or two. But I basically have not yet done a project where people are not repeating themselves after four or five usability tests. You can really see those themes start to emerge. So we stick to that about-

‍
JH: Does it matter little. I'd be curious to hear your thoughts on ... we're talking about all these advantages of remote sessions for products that can enable it in terms of, you get a better diversity of participant, you get to meet them where they're at, it's often cheaper in terms of incentive or finding the people. You can probably schedule a little more back to back than you can when you are walking people in and out your buildings, you can go through them faster, the recordings are great. There's all these advantages, but I feel like when you still talk to people it still seems like you're settling when you do online.

‍
JH: In person is till like the gold standard and people are like, you know, "If I have to, I'll do it online." Give a sense for why that is. Is it just our thinking hasn't caught up to the tools or because previously the only way to do it was kind of in person. So that just sort of have some nurture behind it? Do you have any thoughts on why people haven't kind of clicked on the value of it?

‍
Sonya: Yeah. That's a good question. I think that, you know, it's not something I've encountered quite sort of much in my work because my whole company is remote. We really push it, our clients love it. So I haven't necessarily come across a client that gets involved with me and then says, "I don't know about this remote thing." That said, I did kind of experience a little bit of that in grad school and I think that there is this sense that like, convenience sometimes affect quality. Like if it's a convenient way to conduct these interviews they probably aren't as high quality. And I just think that's a misconception.

‍
Sonya: I think that regardless of how you do these, they're hard work and they're empathy building and they're really valuable. I don't think we have to be able to reach out and touch someone to understand how they are using a piece of software.

‍
Erin: Probably not a good idea these days anyway.

‍
Sonya: Right.

‍
Erin: Too much stranger touching.

‍
Sonya: Right.

‍
Erin: Yeah. I think that's a great point. Like, no pain, no gain. The amazing thing of technology obviously is enabled us to get more with less.

‍
Sonya: Yeah. We've talked about some of the times where remote just isn't a possibility. But for a lot of digital projects certainly, there are a lot of cool advantages.

‍
JH: Yeah.

‍
Erin: Yeah.

‍
JH: I think what you're getting at ... I think feels right in the sense of like, I'm gonna mess up the quote, but the idea of in theory there's no difference between practice and theory but like in practice there is. And I feel like in theory if you could do any type of session you'd pick in-person, but when you are actually getting into the practice and you do use the online sessions. I think people are then are quick to recognize the benefits, but it might just be that in isolation you're thinking about it a little bit more abstractly people prefer in-person and when you are actually doing it day-to-day like you are the benefits of online are little bit more clear.

‍
Sonya: Right. And I also think there is this sense of our fields somehow being based on physical ... like the research that's set on physical products. Because user experience research is not exclusively a digital field and it really evolved out of industrial design and product research and just for today's technology which is so frequently digital or on a screen I should say. So I think that there is a sense in order to legitimize the work that the more our work looks like the work of ... or the more our day-to-day looks like the day of an industrial product researcher the more legitimate the work is. So I think that here we have we kind of have a little bit of bias to overcome in terms of making the point that remote research is valid.

‍
JH: Yeah. And that makes sense. I mean sort of much stuff in the digital space evolved out of, physical or analog concepts. It's all the old skeuomorphic stuff or like the desktop and the folder. Or like the old notes app that had like the lined paper and stuff on it.

‍
Sonya: Yeah. And I think especially in academic spaces. I went to grad school at Carnegie Mellon which has a fantastic industrial design and product research. Grad program as well. So in some ways I think the human computer interaction and the UX research side of that sort to mimic that in the digital space and prioritized in person research as a result.

‍
JH: Cool.

‍
Erin: Yeah. I love that. It's something we're really interested in is this kind of research needs to be very whatever. Like how the book says it needs to be and expensive and hard and only the privileged class of PhDs who've done ethnography in Africa would ever really allowed to do this kind of work. Versus let's democratize getting insights and talking to your customers and -

‍
Sonya: Yeah.

‍
Erin: It doesn't have to be that hard and obviously everyone can benefit from learning some methods and particularly I think being able to extract the signal from the noise and things like that. But I think that's a fun conversation and what you're saying about transitioning kind of academic theory into real life and finding where you're making trade offs and where you're actually getting something better. That's a super interesting part of that.

‍
Sonya: Oh absolutely. Yeah. And I think even in the recruiting side of things there's a lot of times that we have this sense of who is the ideal recruit for something. I mean you'll get these people on the phone or on an online meeting with you and you're like, "well you still have the recruiting requirements but there's something not realistic about the way that we're now talking about this experience." I've had experiences where you ... like someone slips through recruiting somehow. And you're like, "wait, you mean you're not in the market for insurance? Why am I on this call with you?" And then they are like, "well, you know, you had me get a quote before the meeting. That actually looked really good. So I went back in and took another look at it and pulled it up." And I'm like, "I was just about to talk to you about imagining we calling a quote. And you just went and did it." And just those kind of magic moments of like, well that wasn't supposed to happen but awesome.

‍
Erin: We can work with this.

‍
Sonya: Yes.

‍
JH: Yeah. I think it's more number two, like different approaches or techniques have different benefits or upside to them. But they all also have their own little hiccups that are going to happen. So like we talked about the technical glitches on online sessions. But in-person ones, maybe the person can't find the office or they can't find a place to park or-

‍
Sonya: Oh yeah.

‍
JH: Or the meeting room got double booked somehow cause Outlook does that, right? There is a whole host of things that can happen on that side too. I think sometimes we look passed those and, you know, things can go wrong in any stage.

‍
Sonya: Oh absolutely. And my suspicion, although I'm not sure if this is true, and ideally your user compensation should take care of some of this, but I think that there are less no shows. And people are way more willing to reschedule with you if something doesn't work out if they know that they're not rescheduling a drive or hopping on the metro or something like that.

‍
Erin: Definitely. We see that in our numbers as well. Yeah. Sure.

‍
Sonya: Yeah.

‍
Erin: Cool. What else, if anything, should people know about remote research and how to just crush it?

‍
Sonya: Yeah. I think that ... I mean of course this isn't specific to remote. I think just being flexible. Because you're gonna get people logging on and being like, you know, "I can't hear you"". Or like, "one second, my dog's barking and I got to take my dog out". Or like, "let me switch computers, this one doesn't seem to be working." And I think that ... you know, in some ways I treat every remote research session like a little movie I get to watch and interact with, from wherever I am and wherever that person is. And it's such a gift to get that little insight into that person's life, whether it's something "uninteresting' as paying their water bill. You know, just getting to have that screen pop up and they're... Maybe they're in their work uniform or maybe they have their lunch and their mouth is full and they're trying to tell you what they think of entering their credit card number and they're ticked off and ... I don't know.

‍
Sonya: I think that it is still all about empathy, it's just in a little bit of a different way. And I think that just meeting people where they are and also making sure that you updated zoom before you try to log in and all those sort of little technical things can also help you crush it. So you have to be extra organized.

‍
Erin: Organized and flexible.

‍
Sonya: Yeah.

‍
JH: It's one of those things to right, where it's a trend, it's just gonna keep growing so getting the skills and getting comfortable with it is gonna become more and more valuable for a bunch of different reasons and a bunch of different contexts. Even if it's something you're not doing a ton of today, easing into it and starting to experiment with it could be valuable in the years to come.

‍
Sonya: Yeah. One thing that we do is when we were doing , I think a little more in-person, and when we do so occasionally when we do some in person testing, but using a remote test the week before as a back up in case someone's a no show, that has A, come in super handy and is B, a great way to sort of get into remote research if you're not already doing it. Because it's low pressure ... if one of your in-person people doesn't show up maybe the client will see it, but also maybe they won't and it's an opportunity for you to get familiar with the new tool.

‍
JH: Totally. I love that sentiment of why can't it be both?

‍
Sonya: Yeah.

‍
JH: There's no reason that the study only has to use one method to talk to people, right?

‍
Sonya: Oh yeah. And clients love that when you are like, "oh there is a no-show?" And they're like "oh no." And you're like "don't worry we have this video to watch."

‍
JH: That's great.

‍
Sonya: Yeah.

‍
Erin: Well, thank you so much for joining us.

‍
Sonya: Yeah. Thanks sort of much for having me. This is lovely.

‍
JH: This was fun.

‍
Erin: Thanks for listening to Awkward Silences, brought to you by User Interviews.

‍
JH: Theme music by Fragile Gang.

‍
Erin: Editing and sound production by Carrie Boyd