Roberta - 00:00:00: If you're a leader, building a team. I think having the lay of the land and seeing what tools are out there very helpful. You're building a team from scratch. You can get kind of like acclimated as to what tools are in which buckets. Because when you're on a team, you're usually using a particular stack, so there's no way that you'll be able to touch all of these tools.

Erin - 00:00:22: This is Erin May.

JH - 00:00:24: I'm John Henry Forster and this is Awkward Silences.

Erin - 00:00:29: Silences. Hello, everybody, and welcome back to a very special episode of Awkward Silences. Today. We're here with Roberta. Hello, Roberta.

Roberta - 00:00:46: Hello. Hello. Good to be here again.

Erin - 00:00:48: We got JH here too.

JH - 00:00:49: Yeah. Use one of those AI things to write on my line so you can see.

Erin - 00:00:54: We don't know. It's impossible to say. And today we're going to talk about our fourth annual UX Research tools map, which we're very excited to have out into the wild. And let's jump right into it. That's your cue Roberta.

Roberta - 00:01:09: Oh, sorry. Cool. Yeah. Super excited about this year's Tools map. It's kind of really one of those moments when it's super meta because I always would wait for the tools map to come out every single year. It's funny, I actually was looking at my Twitter the other day and I saw, like, I think it would have been like, 2021 before I came to the team. I was like, oh, it's the best. Time of the year. The tools. Maps coming out. And then it's my first time working on it with the team this year. Really excited to be involved this time around.

Erin - 00:01:44: Awesome. Yeah. So, this year, this is our fourth one, maybe just for a very quick history lesson. So, the first one, I think I give Dennis some credit here. He was like, you know those things with all the tools on the matte volume escape. We should do one of those. It's like, cool, cool. I'm with you. Let's do it. How can we make it, like, different and better and ours and that's kind of the kernel of the idea this Tools map came from. For me, the thing I was always really convicted to was that a tool could belong to multiple categories. I think the thing that has always bugged me with those things is everybody gets shoved into a box. Like, literally, you're just put into a box and, like, forget about whether the boxes are the right boxes or not. You know, you've got IA issues from the beginning. Inevitably, you end up in a box. You're like, I guess you have to put me somewhere. But really, I belong in multiple. And from a user's perspective, when you're shopping for a tool, often it wanted to do multiple things for you, multiple use cases, and to not be able to see them all at once, just not very usable situations. This is how we end up with the subway map to begin with. Did kind of three years of the subway map, you know, obviously adding more tools, fine tuning the different lines and categories. Last year, we had a real designer, Holly, to make it look super dope and then come to this year, we're like, okay, this landscape is getting bigger and bigger. Every year, there are more and more tools. How do we add more tools while making the thing remain usable? And maybe this is also an opportunity to play with the trope of the subway map and make a new theme. And so, this is how this is born. We have this medieval theme, which is very much the creation of Katrina, who literally has, I think, a masters in medieval history. So not an accident there. And, yeah, we're able to keep the multiple category requirements, but do it in a totally different way. And with this new theme. And their response has been really great so far. So, it's been fun.

JH - 00:03:58: Yeah, it's super cool. So, if anyone listening has not seen it, go check it out. I've actually been pretty removed from it. So, Roberta, Erin, you all more in the mix here. So, I'm curious, as we look to evolve in the fourth year beyond the subway map, once we came up with this idea, did it click right away? Like, that's it, or there are a lot of ideation or ideas left on the floor to get here.

Roberta - 00:04:17: There's definitely a lot of ideations. Like, Erin mentioned Katrina. Really? This is her brainchild that she worked on. I remember when we were in the first kickoff call, I was, like, telling her about how I've been watching the Amazon reboot of, like, Lord of the rings, and then we kind of went down from there. It's really interesting, though, because we hear when we do research internally as well from customers all the time about how many tools they're juggling. And we've tried to map it out on the research team, how many tools we hear within a particular study, and it could be anywhere from, like, 20 to 30 tools. And then we zoom out now and we look at the map, and it makes sense. Yeah, we're hearing around 30 tools every single time we do interviews because we're hearing over 230 tools when we put together a map. It's a lot that we're juggling, and even internally, we're juggling a lot of tools as well when we're working on projects, too. So, it's been a kind of evolution over the last few months.

Erin - 00:05:15: Well, tell us a little bit about you said 20 or 30 tools. I think if you really sit down and think about how many tools you're using, 30 is less than it sounds like, maybe, right? Like, how many are we using right now just to have this call? Forget about the hardware, right? It depends on how much. Do you want to really include everything, but I've got chrome, I've got yeah.

JH - 00:05:39: Zencastr, Google Docs, Slack.

Erin - 00:05:41: Zencastr, we've got Slope over, so you get to 30 pretty quickly. But maybe you could tell us just a little bit for folks who might be interested about some of the tools we really rely on in our UX research stack.

Roberta - 00:05:53: Yeah, it's funny, I was like, counting earlier. It's about 30, so it makes sense. We have everything from tools that we use for planning. We use Asana internally when we're planning studies or scoping. We have passive insights tools. So, things like Google Analytics, Zendesk, Mixpanel, Mode. We use User Interviews for recruitment, we use Recruit and Research Hub, and that's an initiative we've been focusing on a ton. The latter half of this year, we have over like, four or five different survey tools that we use Survey Monkey, Autopilot, and then to pull everything together, because we have all these different tools, we have tons of insights tools. So, we use Google Sheets, Drive, EnjoyHQ. We even use Notion write ups for summaries from the product teams. And then my favorite, I'm a Nero fan girl. Everyone knows that. So, when we're actually analyzing synthesizing insights, we use everything from Nero to Google Sheets to do that. And there's a lot, there's even, like, Zoom if we're doing Async, or Maze if we're doing concept testing or usability testing. So, there's a lot involved when you really sit down and think about it and look at everything.

JH - 00:07:05: Yeah. So, if I'm somebody who's like, this map is really cool. I like it into the nerdery of medieval stuff, what's the best way to actually make use of it? So, like, once you've had the initial shock and like, this is really cool. Is it that you need a tool in each kind of quadrant or realm or whatever calling these things, or is it more to a discovery mechanism? I didn't know that tool existed. I didn't know there's a tool for that. How do you think research teams can best leverage it?

Roberta - 00:07:30: I think it's kind of all of the above. Initially. If you're a leader, building a team, I think having the lay of the land and seeing what tools are out there very helpful. When you're building a team from scratch, you can get kind of like acclimated as to what tools are in which buckets. Because when you're on a team, you're usually using a particular stack, so there's no way that you'll be able to touch all of these tools. If you want to narrow down in fill in the gaps of specific tools, you can kind of search and discover really love the search feature in the tool. And you can also just go to each little land and say, all right, do I have my tool needs met here? Here. As I was building the team at User Interviews, that's what I did. I went like, category by category to kind of fill in the need. There were certain tools I didn't need to purchase because we were already using them, but in the kind of lands we didn't have filled, I would go through. And I actually looked at the tools map at that point to see what's out there, what is everyone else using to kind of stay up to date and then identify the needs?

Erin - 00:08:32: Yeah, you know, our hunches heading into the economic climate we're in, a lot of folks listening. We're not listening. Just folks are thinking about their stacks and their budgets and how do I reduce the number of tools that I'm using to save budget? Just anything I can do to maximize certainly the ROI, but also to reduce the cost. And I think the direction a lot of folks go is to focus on more of an all in one so you can reduce the number of tools. The idea being fewer tools, fewer dollars, saving money. But a lot of times those all-in-one tools can be very, very expensive and kind of come with maybe some features you aren't really using and might seem like a bargain and might not be. So, you know, we hope that the math can also be useful for folks to really figure out what's actually going to give you the coverage that you need, where your focus, where your team is, where you want your team to be next year? And are you actually better off with maybe three or four tools that are interoperable, that talk to each other, that work well together, but that might save you a lot of money instead of just one tool? So I think, really think about it's not the number of tools you have. It's about avoiding unnecessary overlap so you're paying twice for the same functionality. No one wants to do that. And just, you know, making sure you have the best tool for the things that are most important to your business. Of course, I wouldn't be doing these interviews justice if I didn't mention that quality participants should always be important to you.

JH - 00:10:05: Yeah. All right, quick awkward interruption here. It's fun to talk about user research, but you know what's really fun is doing user research, and we want to help you with that.

Erin - 00:10:15: We want to help you so much that we have created a special place, it's called userinterviews.com/awkward for you to get your first three participants free.

JH - 00:10:26: We all know we should be talking to users more, so we went ahead and removed as many barriers as possible. It's going to be easy. It's going to be quick. You're going to love it. So, get over there and check it out.

Erin - 00:10:35: And then when you're done with that, go on over to your favorite podcasting app and leave us a review, please.

JH - 00:10:43: What's nice when you do go, the more kind of curated charcuterie board of tools is if you realize you're not using a tool, it's easier to remove it right within all-in-one tool. You are a little bit more locked in and it's hard to go maybe adjust your contract or terms to change that. Whereas we have five or six different testing tools for all these different methodologies. The team is really rarely using this one, we're not getting value from it like you can just kind of turn that off and without disrupting the other stuff that you're doing, which is a nice benefit. We obviously really think about ourselves as being a foundational piece of that participant management, participant recruiting side of things. And we talked about this at length that participants are really important to research. As you were referencing Erin we're on the User Zoom podcast ranting and raving about this. But we do think that stuff, that's one where it's hard when it gets siloed if you have a contact with Roberta in one tool and then a contact with Roberta and another tool, you can't really keep track of that. We do actually think that the centralization and all in one on the participant side can be really valuable. But when it comes to methodologies and some of the other tools, choosing the solution that's best in breed can be a nice kind of complement to that.

Roberta - 00:11:50: Yeah, I can't really echo that enough because it's really hard to be a master of every single tool when it comes to methods. You're giving a tradeoff somehow to the either participant experience or researcher experience when you're aggregating the tools for all-in-one solutions. And so that's why we've kind of built our own stack internally to be really what's the best tool that meets our needs for this particular method. And then starting of course with User Interviews for the recruitment and then the integrations make it really seamless on that side as well.

Erin - 00:12:24: Roberta, what tools have you been excited about this year? Either new ones you've tried for the first time or new use cases for existing tools, new cocktails combinations of tools for different methods.

Roberta - 00:12:37: Yeah, I have been really loving when I joined the team, we already had a contract with Grain, which is like a video recording software and they do transcriptions super easy and seamless and it's just internally, our team uses it all the time so that's been really cool to see. And then we started we rolled out EnjoyHQ earlier this year for insight sharing, insights management and so that's been really great to aggregate kind of passive insights from Zendesk. We have active notes from interviews in there too. I think that's kind of like what I get really excited about for the future of our space. Of course, these are interviews and like how are we iterating on our product, but insight sharing and how do you share the knowledge that you're gathering from these sessions? Whether it's surveys or interviews, that's where the knowledge lives. And the idea that maybe there's machine learning to help you with coding and stuff like that to make our lives there's. Researchers’ kind of easier as well. That's where I think kind of the next experimentation and the next rise of tools are going to be happening in the future as well.

Erin - 00:13:44: Yeah.

JH - 00:13:46: Grains a fun one.

Erin - 00:13:48: I know the product team has been really active in building out integrations this year. How do you see that fitting into sort of the different areas of the tools map and how we're thinking about the landscape and connecting these different areas together.

JH - 00:14:04: Yeah, I mean we put a lot of thought and effort into prioritizing what tools we're going to integrate with because we want to add the most value for our users. So, it's a mix of which tools are people using a lot and getting a lot of value from and making sure those play nicely with UI. So, there's a lot of intention there. And then also, you know, what tools are kind of emergent or coming onto the scene and looking really exciting for folks. So I think we've been really excited to partner with Sprig. Their concept testing stuff that's coming seems really cool. I'm excited for folks to get their hands on that. The survey tools have been really powerful. It just takes like a lot of the effort and time out. I know Paolo, one of our PMS has been leading the integration strategy has like a very real example of talking to researcher who manually was collating like who they tracked in User Interviews with a survey tool and probably spent around honestly 10 hours on it and something now they can do in like 10 minutes with the integrations. We're really excited about that. And then I think as we get into 2023, we're going to continue to make it really easy for tools to build into us. So we've been doing a lot of the early integrations through some tight partnerships with these tools and that's been really fun. But one thing we do want to open up is if you're a new tool and you're trying to drive some awareness to your tool and get it in front of people, build it into User Interviews yourself and let researchers kind of discover it that way. So that's something that will be easier and easier for teams to do. So that's something that's interesting to folks should reach out to us. We love to get you on the early wave of that.

Erin - 00:15:20: Thinking of all the map analogies here. So we're building bridges to other territories. Well, of course you have to build a nice competitive moat. You're going to think of all these medieval drawbridge dragons.

JH - 00:15:35: Yeah, I know. And there's some tools I haven't played with yet that I hear a lot about. Like I know we've been hearing about Alchemy more and more as one of these personally as a product person to explain the new tools, some things I want to check out myself as well.

Erin - 00:15:46: Yeah, for sure.

JH - 00:15:47: Since it is a map with a lot of fun stuff on there to kind of what you're doing with the metaphors and everything. Erin, did you all have any fun little details on here that were your favorites in terms of like names or other little artifacts that stood out to you? When you look at it, there's so.

Erin - 00:16:00: Many fun Easter Eggs that you never know what people are going to grab onto when you want something like this. Will you hope that some of the obsession over details is noticed by a few? And I think the names of the little territories has been a source of a lot of fun for people. What is it? The imperial walled city of end platforms as a personal favorite or GDragon for Google, there's a lot of fun.

JH - 00:16:25: Oh, nice. I don't think it's that one.

Erin - 00:16:28: Microsoft and Google have their own little islands.

JH - 00:16:31: I like the Transcription Shire always sticks with something about that. Feels like it could be a real place, sort of.

Roberta - 00:16:37: It does. Yeah. I really like the eye. That's my favorite one. Analytics.

JH - 00:16:46: Yeah. The names are cool.

Erin - 00:16:48: Cool. Well, it's been fun. Other thoughts, tools?

JH - 00:16:53: Any last Easter Eggs behind the scenes? Things you can share about the map. I know it's a big effort and should there be fun little outtakes or stuff to share with people, if you have any?

Erin - 00:17:00: Well, no, I want to definitely applaud Katrina and Holly, who worked incredibly hard to put this together. I think it's always really cool to do something if you stick with it year after year, whether on a continuous basis, like with a digital product or, you know, on a once-a-year basis, as with a PDF. Love to hate them. He's a lovely it's fun to stick with it and see it evolve and get better over time and also to kind of pass off the range for others to do better than maybe you did on your first go around and to watch them kind of takeoff. So, they did incredible work. So very proud of them. And give them a shout out for that.

JH - 00:17:41: Yes, very well deserved. Not to get ahead of myself, but I'm curious if we're going to double down on them. I know next year.

Erin - 00:17:49: It's been funny to watch people. It's like Zelda like. It's medieval. It's kind of this pan fantasy sort of situation. So, there's lots of directions you could go with it.

JH - 00:17:58: Yes. Some people saying you use it for like a D&D campaign. That request for the physical version. Do you think we're going to get some exclusive merch someday?

Erin - 00:18:06: I don't know. I cannot confirm or deny these rumors.

JH - 00:18:10: I think we need a printed version. I'm going to keep advocating for that. Listening?

Roberta - 00:18:13: I think so.

Erin - 00:18:15: Fantastic. Well, it's been fun. And if you're listening to this, stick around. We've got end of year kind of trends, predictions, trends, all that sort of fun stuff coming up in a few weeks with Roberta as well. So thanks for listening.

JH - 00:18:30: The look back and I did last year.

Erin - 00:18:34: We should do that. It's good for you. All right.

JH - 00:18:37: Cool.

Erin - 00:18:38: Thanks, everybody.

JH - 00:18:38: Take care.

Erin - 00:18:42: Thanks for listening to Awkward Silences brought to you by User Interviews.

JH - 00:18:47: Theme music by Fragile Gang.