Nancy [0:00:00]: Human beings are unpredictable and complex, and we are with human beings the whole time we're doing research.

Nancy [0:00:07]: You have to be able to think on feet or be prepared for something unexpected to happen.

Nancy [0:00:14]: When we are interviewing participants, but it's really a guided conversation.

Nancy [0:00:22]: It could take so many different pathways, and you need to be ready to identify that Aha moments that think that you weren't expecting and dig into that and go down that rabbit hole.

Erin May [0:00:38]: Hey.

Erin May [0:00:38]: This is Erin May,

Carol Guest [0:00:39]: and this is Carol Guest.

Erin May [0:00:41]: And this is Awkward Silences.

Erin May [0:00:44]: Awkward Islands this is brought to you by User Interviews, the fastest way to recruit targeted high quality participants for any kind of research.

Ben [0:00:58]: Hello, and welcome to Awkward Silences live.

Ben [0:01:00]: The only podcast that brings together insights, design, research, Ux, all things human centered design thinking and research.

Ben [0:01:09]: I'm Ben.

Ben [0:01:10]: And my guest today is the Vp of qualitative research at C+R Research, Nancy Baum, Nancy.

Ben [0:01:17]: Welcome.

Nancy [0:01:18]: Thanks for having me.

Ben [0:01:19]: And you and I, we talked earlier in the summer at the crux event that User Interviews put on.

Erin May [0:01:26]: Okay.

Ben [0:01:26]: And we were talking about among many other things, just the role and importance and standing of qualitative research today.

Ben [0:01:36]: That is.

Ben [0:01:36]: Yes.

Ben [0:01:36]: Twenty twenty five.

Ben [0:01:37]: It's been sort of buffet by a couple of different innovations, which we'll get to, although, oh, it's still so relevant important.

Ben [0:01:47]: And if anything, all of the new data and automations and ways of working nec the core foundation of what makes qualitative research great now and great then hence the name of our conversation the enduring value.

Ben [0:02:02]: So we're going to try to talk about where qualitative research today reflect a bit about where it's come from, and then hopefully make some projections about where it's going and how you and your team or your organization can continue to make decisions with the why that Q can often generate.

Ben [0:02:19]: So lots of things.

Ben [0:02:19]: But Nancy, why don't you start us off with your background because it informs a lot of what you do at C.

Ben [0:02:25]: So let's start with your background in anthropology and some of your weaving pads to get to Cn.

Nancy [0:02:32]: Yeah.

Nancy [0:02:32]: Yeah.

Nancy [0:02:33]: So I did start.

Nancy [0:02:34]: My kind of research career studying anthropology.

Nancy [0:02:38]: So I have a master's degree applied anthropology, studying cultural anthropology and also have a master's in public health.

Nancy [0:02:47]: So a lot of where I started was very, not surprisingly academically based and kind of learning about the traditional forms of qualitative research from the lens of ant anthropology and social sciences.

Nancy [0:03:03]: And from that, Ivan a conducted research for a couple of nonprofit doing qualitative research for them and specifically understanding barriers in utilizing, services for individuals summary from Food security.

Nancy [0:03:22]: So doing research for local food banks in the area that I was living in.

Nancy [0:03:27]: And then I kind of jumped around from a to a couple different kind of qualitative slash research places, being either the loan researcher, the loan qualitative researcher, learning some different things being exposed to some different environments and researching different topics.

Nancy [0:03:45]: And then landed where I am was at Cn, and I've been with cn for about eight years and work with a whole team of policy to researchers and moderators, and we conduct to research across a range of industries of populations and topics.

Nancy [0:04:01]: So we do research for nonprofit.

Nancy [0:04:04]: We also do research for durable goods manufacturers for Cp companies for tag finance, insurance, healthcare care.

Nancy [0:04:13]: So just a broad range of different types like topics that been research.

Nancy [0:04:18]: And so what is all of that and I feel like, I I wear a lot of different hats.

Nancy [0:04:22]: And the type of work that we do and then bringing in the academic training that I've got, and then here we are today with this wide range of of topics and industries.

Ben [0:04:35]: Applied anthropology makes me think that you well I'll ask instead of declaring.

Ben [0:04:39]: Was eth demographic, immersive sorts of methods part of your training?

Ben [0:04:44]: And if so, Yeah.

Ben [0:04:45]: How how has industry?

Ben [0:04:48]: Because I know shout out to Epic, the the good folks at Epic who look at eth pornography in industry.

Ben [0:04:53]: What was that your method methodical approach that sort of embedded ness and immersion?

Ben [0:04:58]: And how did that translate to your work in industry?

Ben [0:05:01]: Where you asked to do those sorts of things to sort of embed and observe what walk me through how that method translated from academic to industry.

Nancy [0:05:10]: Yeah.

Nancy [0:05:10]: So eth demographic research that or eth pornography actually started in anthropology.

Nancy [0:05:17]: They were the creators of it so the And so, yeah, you can imagine, you know, people spending years at a time living with a group of people in deeply and hurting themselves with their way of life, the activities that they do on a date basis, the rituals that they experience, talking with them about their values, their relief systems, their perspectives about how they navigate the world and their realities.

Nancy [0:05:44]: And so we are taught to be very observe in where we are I immersed ourselves, but also to purchase aches.

Nancy [0:05:56]: And so one of the methodologies is participant observation, which today, a lot of people mis apply it to observing participants, but it's being a participant and an observer.

Nancy [0:06:08]: And so some of that involves conducting the same kind of activities that others arguing doing in order to understand the processes.

Nancy [0:06:17]: And then have the context from their beliefs as well.

Nancy [0:06:21]: So some of those things I continue to take on in the industries that we work with now.

Nancy [0:06:27]: I think the most difficult thing is that, you know, we don't have two years to go spend with people.

Nancy [0:06:32]: Sure.

Nancy [0:06:32]: And for the most part, we're studying kind of our own culture so to speak in our own population.

Nancy [0:06:39]: So there is a lot of foundational understanding that we already have.

Nancy [0:06:42]: And there's a lot of research projects that are constantly ongoing.

Nancy [0:06:45]: So in a sense, we are researching these populations for years years, but it's not that we are living in their home with them

Ben [0:06:53]: for

Nancy [0:06:53]: this time.

Nancy [0:06:53]: So it's more on the smaller scale of getting close with people and really needing them where they are.

Nancy [0:07:00]: So if it's in terms of seeing how someone uses your product, and it's it's a machine.

Nancy [0:07:08]: Then going into their moment watching them use it and taking time to really observe the different ways that they're using it, the worker that they have, the...

Nancy [0:07:17]: And the distractions that they have all around them in their home, whether it's a pet or child or phone calls and things like that and really getting that context.

Nancy [0:07:25]: And and just getting really close to those types of things.

Nancy [0:07:29]: Her could be shopping with them.

Nancy [0:07:31]: And really seeing how they are exploring the products that you sell and the competitors that are shelves around you and being able to observe those creators behaviors.

Ben [0:07:42]: I have heard of some especially market research or consumer insights type research functions using those in homes to really get a lot of that context, and I know some of the the better researchers and research teams are those who are trained in ant logical methods namely ant enter eth pornography.

Ben [0:07:59]: I ask that because I'm I'm curious your time and C eight years there, you've been helping a wide range of teams and companies today.

Ben [0:08:09]: When you go into an engagement again with a sort of composite customer.

Ben [0:08:13]: I'm asking it a paint with a very broad brush.

Ben [0:08:15]: How do they understand qualitative research.

Ben [0:08:18]: Is that just anything that's not quantifiable data?

Ben [0:08:21]: How do you and your team typically hear executives or leaders talk about, Q?

Ben [0:08:27]: Is it a conversation that you have to have to get to an agreed upon understanding walk us through what the pulse is or what the view of qualitative research today from industry is.

Ben [0:08:38]: What is it??

Nancy [0:08:39]: It's it's vast.

Nancy [0:08:40]: So there are...

Nancy [0:08:41]: There are a lot of varied interpretations of what qualitative research.

Nancy [0:08:45]: And what qualitative research can or can't do.

Nancy [0:08:49]: And so our job is to be a consultant to our stakeholders are to our clients and really understand first and foremost, what are they trying to achieve in terms of do they wanna learn?

Nancy [0:09:03]: And what are they hoping to do with what they learn?

Nancy [0:09:06]: And from there, we can then provide recommendations on if it makes sense to conduct a certain methodology.

Nancy [0:09:13]: And it can start as broadly as...

Nancy [0:09:15]: Helping them understand whether it's quantitative research or qualitative research based on the type of research questions that they have.

Nancy [0:09:22]: Sometimes to sell come to us with a research question that they think this is qualitative, but it's actually better done quantitatively because they want to understand the magnitude of something or they wanna validate...

Nancy [0:09:34]: Validate something.

Nancy [0:09:35]: But if they come to us and they really wanna deeply explore something, obviously, that would be then qualitative research.

Nancy [0:09:42]: And from within that, we ask many questions about, again, going toward to their goals?

Nancy [0:09:48]: So what are they hoping to achieve?

Nancy [0:09:50]: What are they help me to understand?

Nancy [0:09:52]: And then that we can then provide recommendations on if it's better to do a focus group if it's better to do it in home eth pornography.

Nancy [0:10:00]: If it's better to do an online eth photography and talk to a national sample of of purchase pens across the country and get into their homes kind of in a remote way, So all of that comes with the conversations that we have with our clients very early on in the process so that we can be the best consultants and the best stewards of of the research that we do.

Ben [0:10:26]: And I wanna get to some of those methods that you alluded to the in homes, the online maybe diary sorts of approaches but staying at this theoretical level, we I, I won't speak for everyone.

Ben [0:10:36]: I often hear qualitative and teams talk about it it's value primarily as understanding the why behind the what or the how behind the the when.

Ben [0:10:45]: Is that still how executives and decision makers and companies who are trying to pivot to this much faster moving somehow automated age where data seems to be infinitely accessible?

Ben [0:10:56]: Are they still struggling with that why beneath what they're seeing in trends?

Ben [0:11:01]: Is that what what's leading them to your doorstep in qualitative work.

Nancy [0:11:06]: Yes.

Nancy [0:11:06]: Yes.

Nancy [0:11:06]: I would say that the why is the main reason, and and, of course, than the how.

Nancy [0:11:11]: I think even that can sometimes be forgotten because sometimes we hear that data really, you know, drives decisions that stakeholders really want to have the numbers behind things.

Nancy [0:11:25]: And sometimes they forget that they have to conduct some qualitative research and under to understand the why.

Nancy [0:11:33]: And so Yep.

Nancy [0:11:34]: We we have our stakeholders come to us saying that, you know, you know, x number of people are abandoning it our website on the home homepage.

Nancy [0:11:42]: But we don't know why.

Nancy [0:11:44]: And so qualitative research then helps to understand what is happening, identify those opportunities and to prioritize those opportunities.

Nancy [0:11:53]: I think one of the other really important things to take it even if a layer further is that another reason to conduct qualitative research and and a benefit of it for our stakeholders is is about context and meaning.

Nancy [0:12:10]: And two of those things is is really ultimately what we're all trying to achieve.

Nancy [0:12:16]: When we're conducting qualitative research.

Nancy [0:12:18]: We want to understand that context behind that choice or that decision.

Nancy [0:12:24]: That someone is making quantitatively in a closed ended question of what what is revolved around that decision.

Nancy [0:12:32]: That they may...

Nancy [0:12:33]: And also to understand the meaning behind the words that they use.

Nancy [0:12:38]: So we all use the same words, but we have different meanings behind them, we have different implications to kind of what we're saying and how we're saying it and so much of what we do.

Nancy [0:12:50]: It's really deeply important to understand when words are being used whether it's by a brand or by a consumer that we understand what the meaning is behind it, what the intention is behind it so that we can help brands kind of meet a consumer in the middle and communicate to them in a way that fits the meaning that they have for that that term or that concept or that belief?

Ben [0:13:17]: I so appreciate that you're mentioning context and decisions in your response there because I think for a lot of mixed methods or qualitative leaning practitioners, they struggle with the Roi for the listener I'm using air quotes, although it is a real thing.

Ben [0:13:31]: It's not fake.

Ben [0:13:32]: But I think, to your point about how might we influence stakeholders to see the value of and invest in non quantitative research or non quantitative approaches of asking questions.

Ben [0:13:43]: You've laid it out there to my ear perfectly.

Ben [0:13:45]: It's giving you more context around the data point, you as a Pm might pull open a dashboard and see usage is up by this number, but why?

Ben [0:13:55]: What's happening around?

Ben [0:13:57]: And I think it also matches qualitative research really matches the context, our environment in which we live?

Ben [0:14:02]: Our customers are not some of them might be opening up their laptop and going to a bank's website or a an e commerce site in a quiet controlled environment, but it's more likely that they're on a train commuting into work, switching between windows, juggling five different things for good or not.

Ben [0:14:20]: And so there's a lot of things that qualitative research can be to tease out and then make recommendations around for, you know, builders of of products and systems, marketers of those systems.

Ben [0:14:30]: I think, for me, qualitative continues to be important not just because of that environment, but also, finally, users have so much choice.

Ben [0:14:38]: And so making a decision building a system marketing a thing with them in mind when you can tailor an experience or share that you've actually listen, that really makes a difference.

Ben [0:14:49]: I talk to my students all the time and and they lead with authenticity, trust, You know, they they know they can probably find a product just about anywhere.

Ben [0:14:58]: They wanna like the brand that they are buying it from.

Ben [0:15:00]: They wanna trust the company that's marketing.

Ben [0:15:03]: And so Are those some of the questions?

Ben [0:15:05]: Let let's move again.

Ben [0:15:06]: I understand high level here, But are the...

Ben [0:15:08]: Certainly the questions might be around market fit?

Ben [0:15:10]: But do you have brands struggling with how we can...

Ben [0:15:12]: I guess, what are some of the questions that qualitative research really helps drive for the customers that you work with or do you think are best suited for?

Ben [0:15:20]: We've got the idea of the why, but are there, like, how do we position ourselves to be data savvy?

Ben [0:15:25]: Is it...

Ben [0:15:26]: What questions are is q best suited to answer or better

Nancy [0:15:30]: Yeah.

Nancy [0:15:30]: So I think and just to kind of tap on to what you're were saying before too, I...

Nancy [0:15:34]: That point of authenticity is is really key as well.

Nancy [0:15:39]: I'm glad you brought that up because we can't be good to storyteller retailers without that contact.

Nancy [0:15:45]: And we can't be authentic without having that context as well.

Nancy [0:15:51]: And so, yes, we we definitely have brands today that are more concerned and increasingly concerned with being authentic.

Nancy [0:16:00]: And and not really seeming like they're they're communicating to their consumers in a superficial way and that they're truly resonating with them and their life, whether it's through communications and showing just a single moment that really resonates with someone we're using words that really resonate with them or a moment that encapsulates a struggle or problem that someone has.

Nancy [0:16:26]: So a lot of what we do ties into identifying those those pain points, those struggles, the desires, the hopes and the dreams that that people have.

Nancy [0:16:37]: So we really understands what truly matters to them and going back to the the point of context skin.

Nancy [0:16:44]: Sometimes the harsh truth is that your your brand is such a small part of personal.

Nancy [0:16:50]: Life.

Nancy [0:16:51]: And you can't really know that from a data point.

Nancy [0:16:55]: You know that from spending time with person and really deeply understanding what is important to them and what matters to them and and talking through other elements of their life, and not not just focusing on this key questions that you wanna have answered, but taking it a step back and and Like, hey, you know, tell me about what your of typical day looks like.

Nancy [0:17:19]: Tell me what happens across your week, what what are the most important things you in your life.

Nancy [0:17:24]: What are you trying to achieve what are your goals.

Nancy [0:17:27]: And then having that context about that individual and then the sample of individuals that you're talking to, you can better situate how how much your brand or your product does or doesn't matter to them.

Nancy [0:17:41]: So that you're being even more effective and not trying to come off as as trying too hard.

Nancy [0:17:46]: If it's something that's not a huge part of their life.

Ben [0:17:49]: That's so helpful.

Ben [0:17:50]: I'd like to move to the the theory of Q to more of this practice.

Ben [0:17:55]: Specifically, given that you're a leader of of qualitative researchers, qualitative thinkers, what does today's call researcher use to do their work?

Ben [0:18:04]: You alluded to online open ends I expect or diary studies?

Ben [0:18:09]: I'm guessing that moderated sorts of sessions are also core to the work you and your team to sketch out those techniques?

Ben [0:18:15]: What is today's qualitative practitioner doing to get at that why in that context?

Nancy [0:18:22]: Yeah.

Nancy [0:18:22]: So at its core, we're primarily it's humans talking to humans first and foremost, but we are conducting live, qualitative research, we are conducting asynchronous qualitative research And we have various methodologies.

Nancy [0:18:39]: Kind of with in that.

Nancy [0:18:40]: So we will do one one interviews, focus groups.

Nancy [0:18:44]: I mentioned before shop longs or d longs or dried longs, in home eth.

Nancy [0:18:49]: In the online portion of it, we are doing online discussion words, diaries, journals will do kind of shopping missions where shoppers will asynchronously kind of go into stores and do a scavenge for hunt of different activities and and provide pictures and videos and immerse less in that experience that they have when they go to a specific location, whether it's a store, or restaurant or or something else.

Nancy [0:19:19]: And so we are primarily still conducting at its core the same methods, but putting ourselves in different scenarios, putting ourselves in different environments adapting to the the fact that, again, we can't spend two years looking at home someone.

Nancy [0:19:39]: There are consumers that live across the country, and we can't fly to ten different cities in one a week, but we can talk to individuals online.

Nancy [0:19:51]: We can have them provide video responses to activities that can complete activities online where they are getting us as close as possible to a process or or things that they're going through on their they're on a daily basis and providing pictures and all of those details to get us even more context.

Erin May [0:20:14]: Awkward interruption.

Erin May [0:20:15]: This episode of Awkward Silences like every episode of Awkward Silences brought to you by User Interviews.

Carol Guest [0:20:21]: We know that finding participants for research is hard.

Carol Guest [0:20:24]: User Interviews is the fastest way to recruit targeted, high quality participants for any kind of research.

Carol Guest [0:20:29]: We're not a testing platform.

Carol Guest [0:20:31]: Instead, we're fully focused on making sure you can get just in time insights for your product development, business strategy, marketing, and more.

Erin May [0:20:39]: Go to User Interviews dot com slash awkward to get your first three participants free.

Ben [0:20:45]: I wanna revisit how you and your team compile, synthesized, analyze those things into really compelling reports, but I wanna revisit you said humans interviewing humans.

Ben [0:20:56]: So let's take a quick pause and and jump into what I'm confident listeners and viewers of this will be familiar with and that is or if not, Ai moderated sorts of research tooling.

Ben [0:21:08]: One of the many, many, many different outgrow of Ll in research specifically is that there are lots of great, which is to say beautifully designed, very well marketed, sometimes very useful tools wherein in artificial intelligence, agent or an Ll will interact with a human flesh and bone participant user customer, and that's how the moderated session works.

Ben [0:21:33]: I wanna get your take on this is your team using it, where are you if so and how might this change the work that you and your team are doing?

Nancy [0:21:44]: Yeah.

Nancy [0:21:44]: So we are using an Ai moderator for some of our projects.

Nancy [0:21:49]: It's a capability that we offer And we usually tag it on to quantitative projects that need a lot of speed and efficiency, but it's...

Nancy [0:22:01]: We provide it kind of with the caveat yet that it's not replacing, you know, deep exploratory qualitative research.

Nancy [0:22:08]: A lot of our clients understand that.

Nancy [0:22:11]: But we do explain and consult with our clients and help them know kind of what it can provide them what it can't provide.

Nancy [0:22:18]: We wouldn't be good stewards of qualitative research if we suggested that this was a replacement of Sure.

Nancy [0:22:24]: Conducting sixteen one on one interviews.

Nancy [0:22:27]: But in the age of always perpetually trying to get things done quicker or faster, cheaper, this is a way that can help our clients be more efficient, whether it's cost efficient, time efficient, and get a little bit closer to understanding a little bit of that fun time or that meaning that's needed to then create a well thought through quantitative research tool.

Nancy [0:22:52]: And so it kind of works in conjunction with that to help then create a better quantitative survey.

Ben [0:23:01]: It's really clarifying because I have heard some teams taking those Ai moderation tools.

Ben [0:23:06]: They and their team or a few moderators will conduct some of the early, sort of not piloting, but they'll take the early interviews, get a sense of sort of lay of the land, sharpen and find the moderator guide.

Ben [0:23:18]: And then if they feel comfortable maybe deploying that Ai moderator so that they can do many more interviews than they might...

Ben [0:23:26]: Or as You said, maybe get some different geographic or, you know, go and get a really nice sense matched population.

Ben [0:23:33]: Are there other use cases that you are either currently experimenting with or are or are having success with re Ai moderation.

Ben [0:23:40]: I wanna talk about how if you're using Ai for of your analysis this...

Ben [0:23:43]: And synthesis in a second, but let's stick with moderation.

Ben [0:23:46]: Is Ai helping in any other ways or might it?

Nancy [0:23:49]: Yeah.

Nancy [0:23:49]: And again, it's it's another tool that kind of complements quantitative research.

Nancy [0:23:54]: We can provide additional probes and ob in order to encourage a a deeper or thoughtful open end response, then follow up on some key points that were mentioned in that purchase participants response.

Nancy [0:24:11]: So instead of just getting kind of a one sentence answer, we can push for for a little bit more get a little bit more detail because we know that there's a lot of times when you're going into open ends and looking through your a command that wish I could've have just sure.

Ben [0:24:25]: Oh, yeah.

Nancy [0:24:26]: Thing.

Nancy [0:24:26]: One more thing.

Nancy [0:24:26]: And so that that provides a little bit more of that context in the situation of doing quantitative surveys and getting a little bit more out of that.

Ben [0:24:37]: Is your team using Ai in any other ways.

Ben [0:24:38]: Generating moderation guides, drafting plans.

Ben [0:24:42]: I'm I'm saying this because if you might drop this into chat we, when we when we ran our Ai research report last year.

Ben [0:24:49]: These were some of the examples that researchers mentioned.

Ben [0:24:51]: They were very hesitant or they were hesitant to let it have too much exposure directly to participants, but they were more comfortable having it as a thought partner or a c designer.

Ben [0:25:00]: How are you and your team Are thinking about her already using Ai beyond moderation.

Nancy [0:25:05]: Yeah.

Nancy [0:25:05]: So for the qualitative researchers, we're using it a lot more on the analysis side, and kind of administrative.

Nancy [0:25:14]: So helping us be a little bit more efficient with certain tasks that could just be done quicker if if Ai can do it either organizing thoughts or just seeing something in in a pid way are just some smaller examples of how we're reusing it.

Nancy [0:25:31]: But we...

Nancy [0:25:31]: As a team, we kind of approach using Ai right now as an exploration and as kind of a test.

Nancy [0:25:42]: It's something that we are continuously kind of pressure testing it and We we use several different programs for Ai analysis and have tested out several different ones, and anybody who is a qualitative researcher on this call knows that there are platforms that have Ai analysis a kind of attached to them.

Nancy [0:26:04]: So we've we've tested a lot of those.

Nancy [0:26:07]: And we kind of play around a little bit and see if we give it this, what is it gonna spit back out at us.

Nancy [0:26:15]: And so we're finding some different things where we'll have kind of moments of magic where it's like, oh, my gosh.

Nancy [0:26:20]: Came up with something, really amazing that has just changed our world today.

Nancy [0:26:25]: And then there's other times we're, you know, we're pushing it.

Nancy [0:26:27]: We're pushing it.

Nancy [0:26:28]: We're pushing it, and it's just not working as well as as human brain would.

Nancy [0:26:33]: So I think one of the the ways that I have approached it and specifically of the programs that I use, I think of it as a research assistant.

Nancy [0:26:43]: And I provide it with some tasks that I already kind of know the answer to or at least, I'm almost there, but I wanted to maybe provide me with some suggestions or maybe it's going to spark an idea for me that I wasn't going to think.

Nancy [0:27:01]: And so I'll I'll have it complete tasks in that way, but a lot of times, I'm actually coaching it and and providing it feedback and saying, that's close.

Nancy [0:27:11]: However, what I really want is this this or this?

Nancy [0:27:14]: Because frankly, if you were to just put the transcripts in, and say, summarize these interviews or give me themes whatever the prompts might be.

Nancy [0:27:25]: It comes out kind of vanilla.

Nancy [0:27:27]: And our reports are...

Nancy [0:27:30]: We have to tell a story.

Nancy [0:27:31]: And we...

Nancy [0:27:33]: Ai only get to that first layer.

Nancy [0:27:36]: It can help kind of categorize the chunks of data that you have and qualitative of research, it is data.

Nancy [0:27:44]: A lot of people don't think it that way, but we are getting data points.

Nancy [0:27:47]: The words that people are using, the quotes, the verbatim, those are data points.

Nancy [0:27:52]: And so it is summarizing for us, some of those themes.

Nancy [0:27:56]: But we then have to take it those next steps.

Nancy [0:28:00]: We are taking those themes and we are turning them into insights based on the observations that we've done based on what we know about our stakeholders, what their goals are, what their needs are, what the team already knows and understands.

Nancy [0:28:16]: And then we are also prioritizing those insights, and identifying those key opportunities.

Nancy [0:28:22]: And I have not seen Ai be able to do that for me.

Nancy [0:28:27]: And if it can get any workflows, it requires a lot of my input.

Nancy [0:28:31]: And so really, at this point, it is something that again, is a research assistant, but we are still the ones who are taking it those other steps for steps.

Nancy [0:28:44]: Two, three four five that I listed.

Nancy [0:28:46]: And and taking it to that level that we are creating something that is actually useful for our stakeholders, and they know what to act on once we provide them with the story the deliverable the insights.

Ben [0:29:00]: Yeah.

Ben [0:29:00]: I have heard much the same from other teams.

Ben [0:29:02]: The Ai as an augment to work you're already doing, not a replacement.

Ben [0:29:07]: Its use is dramatically increased when you the researcher have done your due diligence, and are bringing the context.

Ben [0:29:14]: It doesn't know.

Ben [0:29:15]: The Doesn't know the context of your stakeholders of your...

Ben [0:29:18]: I mean you can...

Ben [0:29:19]: As you said, like, you can try to prompt engineer that and spend a lot of your time hoping that it gets it.

Ben [0:29:24]: But, yeah, I have found success where I've already done a lot of the the coding and I'm telling it about the codes, and then it's looking for some of those codes.

Ben [0:29:31]: But I already have a good sense of those codes.

Ben [0:29:33]: So I can gut check and cross check and and really, like you said, of course correct when needed.

Ben [0:29:38]: It it is really...

Ben [0:29:39]: It lacks a lot of that Nuance.

Ben [0:29:41]: And is someone who's a big fit on verbal, payroll linguistics, it doesn't right now.

Ben [0:29:46]: Have a sense of any of that.

Ben [0:29:47]: And if you've moderated even in a few sessions with customers, you can add that to, as you said there to the story of sharing, this verbatim says this, but here's the video where you can really hear this customer is confused or delighted or whatever the thing is.

Ben [0:30:02]: And so let's move there the outputs, the deliverables, the reports.

Ben [0:30:07]: This is, I think where qualitative research really shines because we are still thankfully, humans building things for other humans to use to do things.

Ben [0:30:16]: Get stuff done.

Ben [0:30:17]: And so when you're putting together reports, deliverables, workshops, the various things that see in our delivers for its clients what are those things that really make, a statement for stakeholders, deliver the value, make their eyes widen?

Ben [0:30:32]: What are those things?

Ben [0:30:33]: You will you alluded to, to, verbatim?

Ben [0:30:36]: You talked about not wanting to have things as vanilla, what really, moves the needle with qualitative research when you're sharing.

Nancy [0:30:44]: So some of that, I would take it a step back to that relationship that you have with your stakeholders and the conversations that you have at the beginning of the project to really understand their needs, where they're going to take this research, how they're trying to act on it.

Nancy [0:31:02]: And then you will create deliverables that resonate with them and are useful for them, because there could be...

Nancy [0:31:10]: I can't just say, like, o videos.

Nancy [0:31:13]: Because with some stakeholders, videos, are not going to be helpful for them or just lots of verbatim.

Nancy [0:31:20]: So it's really being thoughtful and customizing your deliverables to that research project to your stakeholder their needs.

Nancy [0:31:28]: And making sure that you are prioritizing the things that they're needing in order to to act upon the learnings and achieving their goals.

Nancy [0:31:38]: So yes, videos are helpful and I would say, especially when it comes to, behavioral aspect and the nonverbal key like you mentioned before.

Nancy [0:31:48]: A video is, you know, fifth prefers a thousand worth of a thousand words then a video is, you know, ten million words.

Nancy [0:31:55]: And I find it a lot with Ux research that when you see someone having difficulties or not doing something.

Nancy [0:32:05]: Even if it's just one individual, you can paint such a better picture than describing it in a people bullet at points.

Nancy [0:32:13]: But and so making sure that that you're really illustrating those nonverbal cues and and helping individuals get as close as possible to those users and really making sure that your stories are human centered are user centered are shopper centered and that that is actually driving the insights and that you're providing evidence that that is driving the insights.

Nancy [0:32:39]: We're also finding that more and more clients and stakeholders don't want sixty slide reports.

Nancy [0:32:45]: They want to know in as heavy way as possible what those key points are and So making sure that you are really tight with how you we're describing the insights while still straighten that balance of providing detail in context is really important.

Nancy [0:33:03]: And so I'm not gonna be able to think of it the exact same way that another person at Cn put it.

Nancy [0:33:10]: But it's it's kind of that if you are able to write a report or provide a deliverable in a pity way, it shows just even more so your skill as a researcher, and that you are able to take it one step further and not just spit all the information out and hope the reader prioritizes and identifies sculpt the points, but that you are able to su describe what has happened why it happened and what the implication is for your clients for your stakeholder.

Ben [0:33:46]: I think that's such a yes, underlying bold, I think, for a lot of especially more early career folks.

Ben [0:33:53]: They look at that report as a time to throw everything that they've done into a into a a single repository.

Ben [0:34:00]: And that's your index or your appendix.

Ben [0:34:03]: You can do that.

Ben [0:34:04]: What you just described, Nancy, the what...

Ben [0:34:07]: So what now what or whatever the framework that you like, that is the primary goal of that deliverable.

Ben [0:34:14]: I mean, you're you...

Ben [0:34:15]: I mean, you're you're working with clients who have you have deadlines, the...

Ben [0:34:20]: Your clients have deadlines, and so there are that framing it in that way shows that you've respected and thought about their own context that you've applied many of the same principles, of, a researchers to your actual partners there.

Ben [0:34:32]: And, yeah, that's that's the goal.

Ben [0:34:33]: I think, too often researchers are lower on creativity in that output time, and they...

Ben [0:34:41]: I mean, it's...

Ben [0:34:42]: These are our babies.

Ben [0:34:42]: We've interviewed the the participants.

Ben [0:34:44]: We've been maybe sift through the hundreds of open ends.

Ben [0:34:47]: Thankfully, we've largely haven't had to transcribe it ourselves.

Ben [0:34:50]: Though there are some of us probably on a call who remember doing that.

Ben [0:34:53]: Our stakeholders just don't have that...

Ben [0:34:55]: It's not that they don't care.

Ben [0:34:57]: It's just that they need a different sort...

Ben [0:34:59]: A different level of information.

Ben [0:35:00]: So I really appreciate that you are and your team are thinking about delivering what we found why it matters and how you can execute action on it.

Ben [0:35:08]: Are there skills that you think?

Ben [0:35:11]: Let's linger now a bit on skills for today's qualitative researcher and maybe tomorrow's qualitative researcher.

Ben [0:35:17]: What are the things that are still evergreen that you think no matter how robust large language models get or how lean teams get the effective qualitative researcher working industry today needs to be able to fill in the blank.

Ben [0:35:32]: Give us a couple of things that you think will be just go to skills, mindset approaches, habits that a good qualitative researcher and effective one should have.

Nancy [0:35:42]: So some of this going back to my ant training, I can't help, but link ant anthropology and qualitative research together in my mind.

Nancy [0:35:51]: So we're gonna nerd out for a second.

Nancy [0:35:54]: I'll do and get back to it.

Nancy [0:35:55]: But so one of ant oncology you'd taught me a ton of kind of soft skills and ways to how to navigate the world and the workplace.

Nancy [0:36:05]: And one of the biggest things and maybe the most foundational thing that I learned in anthropology is the concept of Eth.

Nancy [0:36:14]: And that means that it's basically, the idea that one's own culture, their views, their beliefs is the right or the normal one.

Nancy [0:36:24]: And therefore, all other cultures, norms beliefs are judged on that standard.

Nancy [0:36:30]: And once I kind of wrapped my head around that concept and that way of thinking, It opens me to this whole world of, there are so many different perspectives.

Nancy [0:36:41]: There are so many different behaviors.

Nancy [0:36:43]: There are so many different ways of exploring, experiencing the world.

Nancy [0:36:48]: That are kind of unchecked and I wanna find out all about them.

Nancy [0:36:52]: And so through that, it kind of unleashed this curiosity, and this just constant perpetual need to understand as much specials as I can about why if people do the things that they do.

Nancy [0:37:06]: And then also this lens of taking what I observe about people, their beliefs their behaviors, their perspectives.

Nancy [0:37:16]: Is not necessarily right or wrong, but as one of the many ways.

Nancy [0:37:22]: And in order to be a good qualitative researcher, you have to be not judgmental.

Nancy [0:37:26]: You have to be able to accept that there are many of ways of doing something.

Nancy [0:37:34]: There are many different for ways of seeing something that there are many for truths for people out there.

Nancy [0:37:41]: And if you can't accept that tenant, you are not going to do really d thorough rigorous, hopeful qualitative research because your job is to go out there and identify all the different ways that people are thinking about something.

Nancy [0:37:58]: People are talking about something that people are doing something.

Nancy [0:38:01]: And then taking that and finding out where those patterns exist, where those themes are, where it's a priority versus not a priority.

Nancy [0:38:11]: So with that that constant curiosity that idea of being non judgmental and also being patient, I feel our very, very core skills that are needed to be a good qualitative researcher and to be a good moderator as well when you're are one up one with a purchase.

Nancy [0:38:32]: And then a lot of those skills kind of kind of go from there, and you can kind of bills on that.

Nancy [0:38:38]: And so Another one is being able to be flexible and the adaptable.

Nancy [0:38:44]: Human beings are unpredictable and complex, and we are with human beings the whole time we're doing research.

Nancy [0:38:52]: You have to be able to think hundred feet or be prepared for someone unexpected to happen.

Nancy [0:38:59]: When we are interviewing participants, but it's really a guided conversation.

Nancy [0:39:07]: It could take so many different pathways, and you need to be ready to identify that Aha moment that think that you weren't expecting.

Nancy [0:39:17]: And dig into that and go down that grab hole.

Nancy [0:39:20]: But then also know when to stop going on that rabbit a hole.

Nancy [0:39:23]: And have all of the questions that hold unfold this unexpected discovery but then be able to move on to the next thing and cover the topics that you wanna cover.

Nancy [0:39:33]: So making sure that you can be flexible in that way goes so far in this in this type of work.

Ben [0:39:41]: And the best product managers, the best team or department leads that I've had the pleasure of collaborating with they're working around.

Ben [0:39:49]: They all have the skills, They might not be qualitatively trained researchers like yourself.

Ben [0:39:53]: But they don't think of personas or, segments as static.

Ben [0:39:58]: They understand that people as they go through the various live courses might change the way they relate to tax preparation software or do holiday shopping if they're not apparent one year and they become a parent or care worker the next year, that might change Met it will, how they're going to approach the journey map that they take or the experience map that they take.

Ben [0:40:19]: I think about personas I think about jobs to be done.

Ben [0:40:22]: A lot of these frameworks that product centric companies use, qualitative researchers think those are useful to a point.

Ben [0:40:30]: Right?

Ben [0:40:30]: Like, their use full until they they stop adapting.

Ben [0:40:34]: And so that curiosity around the outlier that one person who did that thing that one way or didn't quote unquote get it that for you and I, Nancy, that would be where we would go.

Ben [0:40:43]: Like, what's going on there.

Ben [0:40:45]: What is it?

Ben [0:40:45]: And that's how we got I don't mean to say that accessibility research or thinking inclusive about experiences has happened recently period, new new paragraph, thinking about accessible experiences, thinking inclusive about your experience that benefits everybody not just folks who might have different access needs or assistive technology needs.

Ben [0:41:03]: But thinking about those outliers who might not have been served by your product experienced whatever, thinking in that way just like you said they being curious, not like ex ex away, I'm teaching a a data driven decision making course, and we were reviewing what counts as an insight.

Ben [0:41:17]: And we reviewed this really long from an information scientist, like, three part into definition of an insight.

Ben [0:41:24]: And one of the...

Ben [0:41:25]: Key parts of an insight are not having a closed mind to unexpected results.

Ben [0:41:31]: Insights are most likely when you pursue the unexpected.

Ben [0:41:36]: When you don't sort of say, like, a, that's probably not something we need to care about.

Ben [0:41:39]: And it might always be the case that you're gonna unlock some new business opportunity or new way or new, but you might.

Ben [0:41:45]: And a qualitative researchers self free reflex that not being too centered on one's own view and self, not letting that be a blind, I think is useful certainly for qualitative researchers, but I would expect stakeholders more broadly.

Nancy [0:42:00]: At being opened to the idea of uncertainty.

Nancy [0:42:02]: I think a lot of people like to live in a black and white world, but

Ben [0:42:08]: who are

Nancy [0:42:08]: qualitative researchers do well in that kind of fuzzy gray area in and around the moroccan white and being comfortable with not having a perfect answer to something.

Nancy [0:42:21]: Is also really important skills To I have But but then also navigating needing how to speak about that to your clients because your clients may want, I guess or no answer What should we do with this?

Nancy [0:42:32]: And so taking that and, helping them understand that Know, maybe there isn't a clear answer.

Nancy [0:42:39]: Maybe there's a couple different pathways you could take with us, or it could just be that this area that's feeling a little fuzzy right now can be validated or explored in quantitative research then that's where we get the more definitive.

Nancy [0:42:53]: But qualitative research can can provide some of the the fuzzy answers at first, but give you something to then dig into even more so in

Ben [0:43:03]: the quantitative research.

Ben [0:43:03]: What skills for not just any researcher, but certainly, someone working into today's business environment who might be using qualitative open ended analysis or bumping into these sorts of things.

Ben [0:43:15]: What what skills are maybe given too much weight or over hyped?

Ben [0:43:18]: And what do you think is not given enough to attention.

Ben [0:43:21]: So if maybe if someone's listening to this in their job seeking and they're wanting to develop skills or maybe someone's hiring for a researcher, what are skills that are maybe given too much weight or over height and what do you think those are that don't get in get enough love?

Ben [0:43:35]: Yeah.

Nancy [0:43:36]: So I'll I'll start with the the ones that I feel like are getting enough love.

Nancy [0:43:40]: And I think some of those things I've kind of alluded to and in some of the the previous points to that just the the skills around being kind of innate curious being patient being non judgmental and accepting of different perspectives.

Nancy [0:43:56]: I think those are very, very important skills that don't show up necessarily at a doc job description of a qualitative researcher, but really make you a good researcher good analyst, a good interpreter of data, and and also good storyteller.

Nancy [0:44:12]: I would say that being a really well adept at certain software or platforms or things like that, sometimes can be over emphasized which for a lot of what we do, it really comes down to what you were able to do with human beings and and how you understand human beings and the way you talk and the way you listen.

Nancy [0:44:38]: We also talk a lot about being an active listener and being a good moderator.

Nancy [0:44:44]: And in a sense is a tape skill.

Nancy [0:44:48]: But I would say that one that isn't appreciated enough worse talked about enough is being a good deep listener, which I see as I step beyond an active listener.

Nancy [0:44:59]: So that, you know, there's skills that you're taught in order to listen and engage with someone, but being a deep listener is much more than just those verbal amount verbal cues that would we do as we're having a conversation with someone.

Nancy [0:45:16]: But really fully nursing yourself in that sixty minutes or two hours that you have with that person, and remembering what they are telling you and taking in what they're telling you and acknowledging their beliefs, their presence, their details that they're sharing with you and going back to it and hoping to like, we've got story together as you're talking and and really talking with them about their experiences and saying, you know, you mentioned before that this was really important to you, and now you're saying that something different help me understand why is there this this contradiction or building out why they have these contradiction at building out the complexity of them as a person?

Nancy [0:46:03]: And a lot of that comes from deeply listening to them and truly understanding all of those different facets around them.

Ben [0:46:11]: We'll, close here, you've been at C r for a years leading qualitative research.

Ben [0:46:16]: You've been doing qualitative research longer than that.

Ben [0:46:19]: We had a lot of questions leading up to our conversation.

Ben [0:46:22]: We solicit some from our listeners, and a lot of them were around making sure that Q has a seat at the table or advocating for its value in this ever quantifying world.

Ben [0:46:34]: We've talked directly in directly about how we can get some of that value throughout our conversation.

Ben [0:46:38]: But I wanna close with how can qualitative teams, researchers, folks who are working with open ended unstructured data.

Ben [0:46:46]: How can we make sure that this stays valuable for the next say, eight years.

Ben [0:46:49]: How are you going to for your team at Cn, make sure that these sorts of approaches and ways of asking questions and ways of seeing the world are valuable, not just for industry but for non.

Ben [0:47:00]: So essentially, make a prediction as to why, you believe in our bullish about qualitative research going into twenty twenty six and beyond.

Nancy [0:47:09]: Well, I will say some of it comes maybe with a little bit of packing that...

Nancy [0:47:14]: We will always leave it.

Nancy [0:47:16]: There will always be those questions, and you can't answer those questions without it.

Nancy [0:47:20]: But in reality, I think it's important to talk with your stakeholders and really make sure that you are understanding what they want to know.

Nancy [0:47:32]: And having those early on conversations, I can't say enough about really just having those early on conversations, understanding what they need, understanding the questions that they have, and being able to effectively explain to them, coach them, consult with them about why they need to understand the wise.

Nancy [0:47:53]: And how they won't be able to achieve what they need to achieve without it.

Nancy [0:47:59]: I think the company, a a team could could go months and months and months, years doing quantitative research.

Nancy [0:48:06]: And get some questions answered, maybe for the sake of savings, time and money.

Nancy [0:48:11]: But in the long run, it's really not going to provide any savings because you're going to have to then backtrack and you're going to to catch up and and go back to square one and you'll realize that you need to spend more fine, but your customers with your users with your shoppers and go back to understanding those lies, understanding their context, their meaning, their values in their beliefs.

Ben [0:48:37]: I love it.

Ben [0:48:37]: I hope that anyone listening to this who's qualitatively minded brings a little bit more of that swag or confidence and maybe even cock ness to their stakeholder conversations because it is valuable.

Ben [0:48:47]: Any company that wants to exist more than one quarter, should know.

Ben [0:48:51]: Should meet the demands of and expectations of their customer shoppers and consumers.

Ben [0:48:55]: Nancy Baum, Vp of qualitative research of C+R Research.

Ben [0:48:58]: Thank you so so much for the time.

Ben [0:49:00]: I really appreciate it.

Nancy [0:49:02]: Thank you so much for having me.

Erin May [0:49:10]: Thanks for listening to Awkward Silences brought to you by User Interviews.

Erin May [0:49:13]: Theme music by Fragile Gang.

Erin May [0:49:15]: Hi there Awkward Silences listener.

Erin May [0:49:27]: Thanks for listening.

Erin May [0:49:28]: If you like what you heard, we always appreciate a rating or review on your podcast app of choice.

Carol Guest [0:49:35]: We'd also love to hear from you with feedback, guest topics or ideas so that we can improve your podcast listening experience.

Carol Guest [0:49:41]: We're running a quick survey you can share your thoughts on what you like about the show, which episodes you like best, Which subjects you like to hear more about, which stuff you're sick of and more just about you.

Carol Guest [0:49:51]: The fans have kept us on the air for the past five years.

Erin May [0:49:54]: We know surveys usually so See episode twenty one with Erica alcohol for more on that.

Erin May [0:49:58]: But this one's quick and useful fully we promised.

Erin May [0:50:01]: Thanks for helping us make this the best podcast it can be.

Erin May [0:50:04]: You can find the survey link in the episode description of any episode or head on over to User Interviews dot com slash awkward

Carol Guest [0:50:11]: search day.

Carol Guest [0:50:12]: Bye