John [0:00:00]: I'm gonna go look for quotes in this interview as an Ai?

John [0:00:02]: And then I'm gonna be a person that verifies are those really quotes.

John [0:00:06]: And then I'm gonna have another one that's like, is that really strategically relevant.

John [0:00:09]: And so you see where I'm going is, we kind of argue, oh that thing was lousy.

John [0:00:14]: It doesn't work But, you know, really good intros intersection of how are we really, really, really doing this source?

John [0:00:20]: What is an insight?

John [0:00:21]: How are we generating?

John [0:00:21]: And is there an opportunity for these tools to actually support us in some of that work in different ways and granted.

John [0:00:28]: Things like making interview guides or things are more straightforward.

John [0:00:32]: Yeah.

John [0:00:32]: That's better suited.

John [0:00:33]: But on the cutting edge, is there an opportunity for these things to sort of up their game.

John [0:00:39]: So they're not just sort of fun toys, and they might be really interesting strategic partners.

Erin [0:00:47]: Hey.

Erin [0:00:47]: This is Erin May,

Carol [0:00:49]: and this is Carol Guest.

Erin [0:00:50]: And this is Awkward Silences.

Erin [0:00:53]: Awkward zones.

Erin [0:00:55]: This is Brought to you by user interview is the fastest way to recruit targeted high quality participants for any kind of research.

Ben [0:01:07]: Hello everybody.

Ben [0:01:08]: Welcome to Awkward Silences live.

Ben [0:01:09]: I'm Ben, and I am so excited to be bothering to...

Ben [0:01:14]: I'm gonna go ahead and call them friends.

Ben [0:01:16]: Certainly, there folks who I read and listen to regularly.

Ben [0:01:20]: I follow their work closely as you should too.

Ben [0:01:22]: And we're going to be doing two things.

Ben [0:01:25]: We're going to be reviewing a bit of what's happens and what continues to happen in our space broadly to find user experience researchers, designers, product folks, folks who wanna learn about humans and how to make stuff better for them.

Ben [0:01:37]: And then we're gonna reflect on how that might present itself in the New Year.

Ben [0:01:41]: So we're gonna make some predictions, don't hold us to them, but we're gonna do our very best to ground doing what has happened and then set some guard rails so that you can jump into the New Year with your team or company, maybe more confidently.

Ben [0:01:53]: And joining me to do this thankfully because you don't want me doing that strategy setting for you.

Ben [0:01:58]: Are two folks who again have been thinking and writing and working in this space for some time.

Ben [0:02:02]: First is Maria Rosala, Maria is the director of research at Nielsen Norman Group.

Ben [0:02:07]: She leads an overseas research.

Ben [0:02:09]: At NN/g.

Ben [0:02:10]: You're likely aware of them, where she trains teams around the world on a variety of Ux topics as well as working with clients to help them improve the user experience of their products and services.

Ben [0:02:19]: You've probably heard her, read her work, Maria welcome.

Ben [0:02:21]: It's nice to see you.

Maria [0:02:23]: Thank you so much.

Maria [0:02:23]: Thanks for having me, and Ben absolutely.

Maria [0:02:26]: I can consider you a friend.

Ben [0:02:28]: Oh that's I love that.

Ben [0:02:29]: That makes my makes my holidays.

Ben [0:02:31]: And speaking your friends, John Whalen, and I go way back if you search John Whalen in Youtube with and put up my name, You'll find an old des out people nerds webinar with him educating me on the young people.

Ben [0:02:41]: We did a Gen z webinar on how to tailor experiences to this up and coming buyer segment, which is now, I guess, maybe not the youngest buyer segment.

Ben [0:02:49]: John is the founder and CEO, really take experience.

Ben [0:02:52]: He specializes in memory problem solving attention language, calculation and numerical processing, but as of late, he's been helping businesses understand how to make Ai products and services resonate with humans really sound like human, And he's been teaching a lot of master classes on the.

Ben [0:03:06]: Welcome, John.

Ben [0:03:07]: It's nice to see you again.

John [0:03:09]: Thank you so much, Ben.

John [0:03:09]: It is definitely a pleasure.

Ben [0:03:11]: Well, let's get right into it.

Ben [0:03:13]: I thought we could start at ten thousand feet, And that is as I was telling you before we jumped on.

Ben [0:03:18]: I just was hoping each of you could muse a bit on what you see research meaning to businesses, companies and teams.

Ben [0:03:26]: And by that, I mean, you know, two or three years ago, a lots of folks unlikely are still today, not obsessed but focused on democrat.

Ben [0:03:34]: We did research Ng has as well on head counts.

Ben [0:03:38]: And those are sort of vac between, yes, we still want to expand whereas others are kinda contracting.

Ben [0:03:44]: And then often means that non researchers product, design folks are taking on some research activities.

Ben [0:03:50]: We've also got as we'll talk about it in a bit Ai moderation tools.

Ben [0:03:53]: These things are all, you know, research adjacent.

Ben [0:03:56]: But I'm just curious from your P, Myriad I'd love to start with you.

Ben [0:03:59]: In the customers or pardon the the leaders you speak with and the engagements that you work with.

Ben [0:04:04]: Does research still kinda mean the same thing that did four or five years ago or how has it changed?

Maria [0:04:10]: Yeah.

Maria [0:04:10]: I think that's a really great great point around democrat amortization Ben in terms of, like, what's been happening over the last few years in terms of, like, who is doing research.

Maria [0:04:19]: Right??

Maria [0:04:19]: What we kind of consider research to be.

Maria [0:04:22]: Obviously, you know, ten, fifteen years ago, research was done by people who...

Maria [0:04:27]: Have research titles or, you know, were usability experts and, you know, there were a small handful of methods that were commonly being utilized within teams.

Maria [0:04:36]: And obviously, over the last couple of years, we've seen the shift where we're now having more and more people interested in research, which is fantastic.

Maria [0:04:44]: So we have, like, product managers, like, asking, you know, can we have more research?

Maria [0:04:49]: And we have you know, even designers like, starting to do their own research rather than waiting and relying on a search to be available to do that.

Maria [0:04:58]: So the demand has grown as a result, you know, research departments and research teams have had to look at, you know, the demand and go, how do we cater towards that.

Maria [0:05:07]: And, you know, gate keeping isn't always, you know, dutch saying, oh, no.

Maria [0:05:11]: Only researchers can conduct, you know, every type of research.

Ben [0:05:15]: Mh.

Maria [0:05:15]: It's not the answer to kind of help product teams get answers more quickly.

Maria [0:05:19]: So smart teams have started to look at, well, what can we kind of allow the people to do?

Maria [0:05:24]: How can we kind of share research insights faster with those individuals?

Maria [0:05:29]: Maybe help themselves serve a little bit.

Maria [0:05:31]: Maybe these kind of really simple tactical studies that people are doing.

Maria [0:05:34]: Do we really need to have, like an experienced researcher running, you know, usability tests on some really simple Ui elements, maybe not, maybe we can support some non researchers to do, you know, simple un moderated tests or Mh.

Maria [0:05:49]: A simple prototype test or help them set up a survey.

Maria [0:05:52]: Right?

Maria [0:05:53]: So I think that's happening in a lot more organizations these days and research is starting to be seen by people as, you know, something that is accessible to more than just someone with this these sort of specialized skills.

Maria [0:06:05]: Of course, there is a lot of debate around, like, what is considered research and what's considered something like customer feedback.

Ben [0:06:14]: Yeah.

Maria [0:06:14]: So that's where...

Maria [0:06:15]: You know, or it's up to kind of teams to kind of draw that distinction and kind of communicate, like, hey, this is what researchers do.

Maria [0:06:24]: Right?

Maria [0:06:24]: They think about the study design.

Maria [0:06:25]: They think about, you know, the quality of the inputs of information that's coming in.

Maria [0:06:30]: They think about, right, you know, not just summarizing what's in a transcript.

Maria [0:06:34]: They think about taking it once step further and sort of drawing meaning and linking to, you know, contextual important contextual business information and helping people make good decisions.

Maria [0:06:45]: So Mh.

Maria [0:06:45]: You know, I think there still is a lot of work happening within organizations to try and define that as, of course, as you mentioned, like, are lots more now tools that use Ai that have been popping up on the scene that are supporting powder, right, people who do research to do more of these sort of, you know, research activities.

Maria [0:07:04]: And so the challenges is really for us as research just to kind of, like, define.

Maria [0:07:08]: Like, what do we really mean is.

John [0:07:11]: Yes.

John [0:07:11]: It.

John [0:07:12]: And Yeah.

John [0:07:13]: Yeah.

Maria [0:07:13]: Where this is.

Ben [0:07:15]: I really appreciate that reminder and that callback to democrat, because it's gonna I from my Po continue to to grow in importance and relevance as budgets are tightening.

Ben [0:07:23]: John, when you work with customers who or clients who come to you.

Ben [0:07:26]: Do they have a sense of what they mean by research?

Ben [0:07:28]: Are they asking different questions, you know, is brilliant tackling?

Ben [0:07:31]: Not just focused on Ai and having them say, like, help us figure this out, but, like, the sort of spirit of the question.

Ben [0:07:38]: Do you think that has evolved and changed just customer point out.

Ben [0:07:42]: You know, there's, like, there's the Cx world.

Ben [0:07:44]: There's the voice customer world.

Ben [0:07:45]: There's a longstanding standing tradition in market research.

Ben [0:07:47]: So it's it's not a totally new thing, but what do you see or how have you seen research the term and practice of all this year?

John [0:07:54]: Yeah.

John [0:07:54]: I think there's a couple pieces that I've seen.

John [0:07:56]: So, I think as you know, I've been doing these interviews with founders of these Ai tools.

John [0:08:01]: I can't help but say here's what was sort of logically possible We can say It's a good idea.

John [0:08:06]: But what's logically possible.

John [0:08:07]: And so I'm getting asked, you know, hey, we used to take seven weeks to do a body of research do, and I'm a Phd researcher.

John [0:08:15]: I love doing great in up thoughtful research.

John [0:08:18]: But I think we're getting pressure to say well, with all the ai stuff.

John [0:08:22]: This group on the web says we can do research in minutes and hours.

John [0:08:25]: You know?

John [0:08:26]: Right.

John [0:08:27]: So so you said seven weeks, they say eight

Ben [0:08:30]: hours.

Ben [0:08:30]: Seven minutes.

Ben [0:08:31]: Yeah.

Ben [0:08:31]: Yeah.

John [0:08:32]: Yeah.

John [0:08:32]: So what is it?

John [0:08:33]: So I...

John [0:08:33]: I think that's one piece is just pragmatic.

John [0:08:35]: I think there's pressure there that sets new for us to deliver, deliver faster, maybe deliver at scale with Ai tools delivering internationally.

John [0:08:46]: So I think it's both a good...

John [0:08:47]: You know, it could be a very positive thing where, you know, we suddenly, I have skills now in thai and you know, Brazilian Portuguese that I would argue I didn't have before Ai.

John [0:08:57]: So you know, there's ways to potentially be more inclusive with a very small team, for example.

John [0:09:02]: And so I I guess that's one piece that there's this whole new world, and also with things like prototyping, So we're sort of un undo now in the scale we can do in our...

John [0:09:13]: In maybe qualitative research.

John [0:09:15]: We're also amazingly agile with new prototyping tools to sort of don't have limits there.

John [0:09:21]: And so just that ability to iterate and to try new things and try it in a more broad fashion sort of opens up so many new worlds, and I'm not even gonna touch synthetic users yet, but I might have just we'll get there.

John [0:09:34]: And we'll get through.

John [0:09:34]: So, you know, what is research?

John [0:09:36]: You know, some folks are like go, Am I doing I'm using those kind of things as well.

John [0:09:40]: Yes.

John [0:09:40]: And so I probably had pretty strong feelings there, but we'll be gentle.

John [0:09:44]: And so I think those are some of the big things was just...

John [0:09:47]: And lastly, if you are using some of these Ai tools, just the financial constraints are coming down.

John [0:09:53]: Right?

John [0:09:53]: So I can do or anyone on my team can do more studies per year, so it can be more cost effective to do research than it was in twenty twenty two.

John [0:10:02]: Even with recruiting.

John [0:10:04]: So that just opens sort of a whole new approach you could have to research.

John [0:10:10]: So Think it's really, really interesting days.

Ben [0:10:13]: Maria, are there skills that you and your team are hearing from community or from leaders or...

Ben [0:10:18]: From founders or executives who are like, I really need my team certainly, again, we'll get to, like, competencies around maybe Ai prompting and sort of using these new tools.

Ben [0:10:28]: But for researchers who are listening to this or watching this.

Ben [0:10:30]: In twenty twenty six, Do you think there's something they ought to prioritize or be thinking about or doubling down on because it's only going to be more important?

Maria [0:10:39]: I certainly think having a little bit if there is, like, you know, some of these new Ai features or, you know, like, for example, Ai interviews.

Maria [0:10:46]: Like, a lot of platforms are integrating.

Maria [0:10:49]: This is a new feature.

Maria [0:10:50]: So if you have access to something like great question.

Maria [0:10:52]: Like, there's now an Ai interviewer, I believe it's very soon to be launched.

Maria [0:10:56]: Then you can experiment with, And I think, you know, if you can get access to those tools and try out a study or two with them, just to see what their capabilities are, you're gonna be in a better position to have a conversation with maybe it's powder.

Maria [0:11:11]: Right?

Maria [0:11:12]: Or who wants to do a study, and they're thinking of, like, let's just have Ai moderate it.

Maria [0:11:17]: Right?

Maria [0:11:17]: You can be in a position to say, yes.

Maria [0:11:19]: I think this will be fine for an ai moderate study, or actually no.

Maria [0:11:23]: I think we really need a human moderate study.

Maria [0:11:26]: For this one because it is more complex.

Maria [0:11:28]: Right?

Maria [0:11:29]: So I think you gaining that familiarity with what's out there.

Maria [0:11:33]: If it's available to you, try it.

Maria [0:11:35]: Right, experiment with it, play with it.

Maria [0:11:38]: That's gonna be super important not only in terms of you being able to see where it can be using your workflows, but also in being able to provide guidance and advice to those people who are reaching out who need support because we know that a lot of researchers right now are sort of providing support, Yeah.

Maria [0:11:56]: You seems to do research.

Maria [0:11:58]: So, you know, being knowledgeable about what's out there and how it works and what limitations are right now is going

John [0:12:04]: to Yeah...

Maria [0:12:05]: And this is something that I'm working on.

Maria [0:12:06]: I'm like, we ran some studies looking at Ai interviewers.

Maria [0:12:09]: We've also played around with, you know, Ai features in various different tools.

Maria [0:12:13]: And these tools are changing all the times.

Maria [0:12:15]: So I'm

John [0:12:16]: gonna be doing this.

Maria [0:12:17]: Not into twenty twenty six.

Maria [0:12:18]: It's never the case but well, right now, it's not the case that you run a study and you'll learn.

Maria [0:12:22]: Okay.

Maria [0:12:23]: Here the limitations and those limitations all going to exist next year.

Maria [0:12:26]: Yeah.

Maria [0:12:27]: These tools are gonna get a lot better.

Maria [0:12:29]: So I'd say, you know, get experimenting with it.

Maria [0:12:31]: Try and, you know, have conversations with your other colleagues and in how they're using it.

Maria [0:12:37]: I also really much to encourage people.

Maria [0:12:39]: If you haven't really been using any of these, you know, new tools and new features or even just Jennie chat bots that much in your work.

Maria [0:12:46]: You are not alone.

Maria [0:12:47]: I've been speaking to.

Maria [0:12:49]: Yeah.

Maria [0:12:49]: Ux, practitioners research designers, and many of them either only just got access to something in their organization.

Carol [0:12:57]: Right.

Carol [0:12:57]: Because

Maria [0:12:58]: organization had inc, like, you know, incredibly strict protocols.

Maria [0:13:00]: But also, you know, some of them have just haven't had the time been so strapped to really spend time exploring and experimenting, and there really hasn't been very much support coming down from leadership or from ops teams.

Maria [0:13:13]: You know, with guidance on how these tools can be used.

Maria [0:13:17]: So you are absolutely not alone.

Maria [0:13:18]: There's many, many people out there who are still not sure how they can incorporate this into their workflow.

Maria [0:13:24]: So hopefully, that kind gives people a bit a bit of risk Yes.

Maria [0:13:27]: Instead Yes.

Maria [0:13:28]: Have a go that they're not going to be left behind.

Ben [0:13:31]: And John, thank you for dropping your, your Ai skills for research made and Lincoln, and I will share that out for folks who are watching your live watching this after.

Ben [0:13:38]: But John, what would you add to that?

John [0:13:39]: Yeah.

John [0:13:39]: I know.

John [0:13:40]: We will...

John [0:13:40]: Literally in that class, But what what we do, but I encourage everyone to do.

John [0:13:44]: You don't be do it in my glass at all is, so I guess, I try to say, I really want to learn better prompting because it just can make things more powerful.

John [0:13:52]: I want them to try Ai moderation just to understand even what it feels like as a participant.

John [0:13:58]: And also kind of how well does it do?

John [0:14:01]: So, you know, you could have ask you a question about how you use Ai in everyday life and say, I really like ice cream sandwiches.

John [0:14:09]: They're yummy and see how it responds or say, I feel like And be really uncomfortable.

John [0:14:12]: And how does it respond?

John [0:14:14]: And so kinda get a feeling for where are we right now with these tools and also see ultimately if you're bring these properly.

John [0:14:21]: What's the level of thoughtful it has and follow on questions or how it's interpreting things.

John [0:14:28]: So there's Ai moderation.

John [0:14:29]: Also, I think there's so many tools now they're trying to give you summaries or insights that just, again, I love to encourage people to have a field, they know so well.

John [0:14:40]: So if they have studied scientists, then great.

John [0:14:43]: Let's see how a scientist perform with this, let's see how it does on interviews, You already know the answers to, and see how it summarizes them as well.

John [0:14:52]: And so, basically, I guess I'm trying to say, really...

John [0:14:55]: You have to do a little bit of the hard work of comparing what you would do with what it did.

John [0:15:00]: And then to sort of understand.

John [0:15:02]: That's where it is right now.

John [0:15:03]: And lastly, I just...

John [0:15:05]: I encourage people to also try...

John [0:15:07]: I know I brought up once, but just...

John [0:15:09]: I I do think that synthetic users are gonna be more of a being that man a designer might ask about or someone else.

John [0:15:15]: And just having, informed a opinion about all these things.

John [0:15:19]: It doesn't mean you have to use the model in your work.

John [0:15:22]: It means just that not having a visceral reaction to it, but having a, you know, evidence driven, you know, decision making that you're doing.

Ben [0:15:29]: Yeah.

John [0:15:30]: And they're different, you know, for folks that might be deciding on a potato chip cover that might be a different thing than I've got something for people with anxiety that's an Ai tool and it's critical to get right.

John [0:15:41]: You know, there can be different reasons why these could be appropriate in different situations or not.

John [0:15:46]: So what's right for you in sort of forming that because also the people that I'm meeting, like I'm sure you are Maria are asking what our teams do and where should we go from here.

John [0:15:56]: So will all be the group that are helping to make this the first wave of using these tools in our work because it they didn't really exist in twenty twenty two.

Ben [0:16:06]: Yeah.

Ben [0:16:06]: I love each of those responses, they lead into what I've was going to share, which is when I talked to leaders and Maria, it's exactly what you said, a senior or half leader half practitioner of a international media company was chatted with me about how he feels that he's more of a an ops person even though it's not in his title.

Ben [0:16:25]: He's John doing what you're you're saying wherein and he's testing the tools to evaluate their fit.

Ben [0:16:30]: Mh.

Ben [0:16:31]: He's thinking about governance Maria, which I wanna talk about here in a second.

Ben [0:16:34]: He's Yeah.

Ben [0:16:35]: You know, figuring out with the data science team in this part of the company, how that can work with the voice of the user data the researcher, he's best positioned to not technically fit the things together, but he knows what piece of data and what information can create the insight that the business needs, which to me sounds like, you know, one of the core functions of a researcher.

Ben [0:16:56]: Yes doing the research, but then s synthesized, making sense of and put it into practice.

Ben [0:17:01]: And so I'll be interested to see how many more research leaders in particular are performing that sort of odds like function of pulling the different levers.

Ben [0:17:09]: But, Ray, you wrote and our pre conversation doc that this idea of oversight experimentation and governance is going to be more and more important to teams.

Ben [0:17:18]: Can you talk us through what, first of all you mean by those things as it relates to research specifically?

Maria [0:17:24]: Yeah.

Maria [0:17:24]: Definitely.

Maria [0:17:24]: I think those are really great points then in terms of, like, the...

Maria [0:17:28]: We're sort of aptly positioned to kind of start doing more of the that ops work.

Maria [0:17:32]: And really, that should fall to operation.

Maria [0:17:34]: Sure.

Maria [0:17:34]: Those the, like, individual contributors.

Maria [0:17:36]: But certainly, you know, in the last year and a half, you know, a lot of teams have sort of been put in this position where they have to figure it out themselves.

Maria [0:17:46]: So it's sort of, like, experiment, but we don't know.

Maria [0:17:49]: How to use Ai experiment with it and you tell us, like, you know, what else some useful use cases.

Maria [0:17:56]: And I've spoken to practitioners who can talk about like, these kinds of schemes within their organization lenders.

Maria [0:18:02]: You know, and one of my colleagues Laura klein were a great great article about that, which is just sort of her message is like, stop telling people to try and figure it out themselves.

Maria [0:18:11]: Like, you need to provide oversight, and you need to provide, you know, a strategy for.

Maria [0:18:16]: How these tools can be utilized effectively.

Maria [0:18:19]: Yeah.

Maria [0:18:19]: And provide, you know, proper training and proper guidance around it rather than expecting.

Maria [0:18:23]: People individual contributors who are very very busy to try and figure out doing all this experimentation, And then this communication.

Maria [0:18:31]: Right?

Maria [0:18:31]: See does it even how I'm using it Does it even apply...

Maria [0:18:36]: Could it be useful in some other use context like that's that's a whole endeavor, it's a whole project.

Maria [0:18:41]: So I'm hoping that in twenty twenty six, we see people being a bit more strategic about this.

Maria [0:18:46]: And move away from, you know, these assignments like, hey, you know, put in a spreadsheet, an example of how you're using Ai in synthesis.

Maria [0:18:55]: And no one reviews it.

Maria [0:18:57]: No.

Maria [0:18:57]: No action items after it.

Maria [0:18:59]: Right?

Maria [0:18:59]: It's pointless activity.

Maria [0:19:01]: And it leads to what Laura was talking about in the article around performance theater.

Maria [0:19:05]: Right?

Maria [0:19:06]: People going.

Maria [0:19:06]: Look at my prompts, look at, you know, how I've used it.

Maria [0:19:09]: So I think if we really want it kind of ensure that people are utilizing these tools safely, efficiently, effectively, and we have a guard rails around it.

Maria [0:19:19]: It really needs to be driven by by leadership like, Mh.

Maria [0:19:23]: Recognizing that this is a considerable piece of work.

Maria [0:19:25]: What our work workforce to be, you know, efficient.

Maria [0:19:28]: We need to provide times kind of, like, really identify, like, what are some challenges right now And then take user centered, human centered of approach.

Maria [0:19:37]: So we're like, what are problems right now in our teams.

Maria [0:19:40]: Right?

Ben [0:19:42]: Exactly that our stakeholders have that we can do.

Ben [0:19:44]: What are their job jobs that they're trying to get done.

Ben [0:19:45]: Yeah.

Maria [0:19:47]: Rather than you how can we use Ai and when plug it in.

Maria [0:19:50]: So I hope we we move more to thinking more from a human sense perspective next year and take a time just to pause and reflect and really sort of do that necessary research internally to figure out how we can actually utilize.

Maria [0:20:04]: It might look different from one organization to the of course.

Ben [0:20:07]: Yeah.

Ben [0:20:07]: John, go ahead.

Ben [0:20:08]: Oh,

John [0:20:08]: no.

John [0:20:08]: I was just gonna say, thank you, R.

John [0:20:10]: And I was gonna say that the technical chair I think for Maria is talking about is radical common sense.

John [0:20:14]: So you know, I so appreciate it.

John [0:20:17]: And you know, folks...

John [0:20:18]: And, you know, they're...

John [0:20:19]: On one hand it's like, yes.

John [0:20:20]: We'd love you to do your research with the one tool we're providing, and you can use any tool in the whole world as long as it's c copilot.

John [0:20:26]: And so, you know, which has its moments, But it's so hard for individual contributors says say, yes.

John [0:20:34]: I'd like to use this tool and, like, well, is it talk to compliant?

John [0:20:37]: What...

John [0:20:38]: How exactly is it where are servers located in the world.

John [0:20:41]: How is it dealing with Gdpr compliance?

John [0:20:44]: Has it deal with the way?

John [0:20:46]: We work work with things?

John [0:20:47]: Will we be connecting with our specific clients, world the Vp?

John [0:20:50]: Like, either all these things are way are things that we all have to be so sensitive of, and you know, just telling us to go use them isn't, you know, do I need to call legal when I do that.

John [0:21:03]: You know?

John [0:21:03]: So it makes sense to be a governance kind of thing.

John [0:21:06]: At the same time, I guess, the hard part is we'd love for the That bottom up initiative in addition to our overarching and governance.

John [0:21:14]: Because I think I'll bet you're finding something similar, whether there's a handful of people that are just really embracing things, trying things, and they're even you know, the sort of grassroots leader in their organization, They're helping leadership to define where they're going or even just get interest in just trying the alone, you know, implementing.

John [0:21:35]: And then there's the whole process of how do you decide as a group.

John [0:21:39]: How you're going to approach these things.

John [0:21:41]: What's appropriate?

John [0:21:42]: Because you do get very different sentiments of, you know, I've done this for years, and I want to do this the way I've always done it, and others are, like, we'd love for you to get this done in half the time.

John [0:21:54]: With this new scale.

John [0:21:56]: And so it's a really interesting challenge for us to evolve.

John [0:22:00]: My last thought is just, you know, there were three...

John [0:22:03]: I guess that I really liked there's a infographic by this group board of innovation that's such a nice one that says, you know, the first thing you're gonna think about is maybe there's time savings with these new tools.

John [0:22:13]: And then there's maybe better quality, like, things I couldn't do before, like multi wool things.

John [0:22:18]: And then there's, like, a whole new way of thinking that isn't just faster horse, but is, like, a fundamentally different way of doing things.

John [0:22:26]: So kind of what level you want on that curve of thinking about how you're transforming your research.

John [0:22:31]: And, you know, what that means for everyone.

John [0:22:34]: So for example, why why do we have that insights team over here in marketing?

John [0:22:38]: Doing branding stuff, and then we've got the innovation team doing research over here, and we've got the product teams doing research here.

John [0:22:44]: Someone might consider could those things come together.

John [0:22:48]: So they're interesting days of of taking a big step back to your point, R, but having that pause thoughtful pause?

Maria [0:22:55]: Yeah.

Maria [0:22:55]: I think, John, like, you know, your point around, there are these different advantages to, you know, exploring Ai And, unfortunately, the last year.

Maria [0:23:03]: I think most organizations have been focusing on the efficiency aspect.

Maria [0:23:07]: But you're point Right.

Maria [0:23:08]: Like, there are lots of really interesting new applications of Ai that are supporting better quality.

Maria [0:23:15]: I mean, even thinking about, like, tools that almost have support for people to create a research plan with Ao.

Maria [0:23:23]: Right?

Maria [0:23:23]: It might not be perfect, but it's definitely be better than what they would produce on their own without any oversight.

Maria [0:23:30]: So, yeah, I think, you know, that's a really important aspect.

Maria [0:23:33]: It's not just can we work faster, but can we actually work better?

Maria [0:23:37]: And what does that look like And what would be where are we sort of struggling right now in terms of quality?

Maria [0:23:42]: So I think, yes, This will definitely be a a very hopeful at conversation for organizations to mo on in twenty twenty six.

Ben [0:23:50]: John, I love that framework from the board of innovation thinking about the modality changes, the channel changes, the efficiency changes, because when I've played with Ai moderation tools, for example, it makes me think a bit about, like, a classic grid question in a screener Like, wow.

Ben [0:24:05]: This mh.

Ben [0:24:06]: Not that somebody invented this.

Ben [0:24:07]: But when tools started supporting them and like, wow I can ask five questions with the same liquor type scale, great.

Ben [0:24:13]: That is efficient...

Ben [0:24:14]: It's not holy changing research.

Ben [0:24:16]: And so similarly, like, Am moderation tools like, cool.

Ben [0:24:19]: I can ask, like, earlier this year, I interviewed product managers about collaborating with researchers, and I was able to interview about a hundred of them in a week.

Ben [0:24:28]: I could have never done that, Bend the researcher you're working for his User Interviews Even with a great access to a network and a a bunch of coffee.

Ben [0:24:35]: Now, of course, it has its limitations on the back end, at least I experienced such, you know, from organizing and making sense of and checking my work.

Ben [0:24:42]: But, John, the last bit, unlocking and uncovering new opportunities.

Ben [0:24:47]: That is what I wanna transition to now because both you and Maria have talked about composite or synthetic users and we had a question in the chat about what is.

Ben [0:24:55]: So a brief operational definition.

Ben [0:24:57]: A synthetic user or a composite user is a fake synthetic, not real actual user, human person, user of data, but it is based on existing information or as sometimes just the case synthetic information.

Ben [0:25:12]: So you have some corpus of data, and you might query it and say as John give an example here it.

Ben [0:25:17]: Imagine you are a, give a couple of demographic information pieces to an Ll, for example, either in house or out of house, and then you'll have it run through some paces.

Ben [0:25:25]: John, walk me through those moments when you like the synthetic user?

Ben [0:25:29]: And how might you recommend others to use it?

Ben [0:25:32]: Are there things they should be thinking about when they're embarking on the composite or the digital twin sometimes

John [0:25:38]: Right.

John [0:25:38]: It's.

John [0:25:38]: So I know it's so hard to have a solid definition.

John [0:25:41]: But I guess first, remember, think of it as just a new tool.

John [0:25:45]: So if I was the branding team for this, I wouldn't have called it synthetic users.

John [0:25:48]: It sounds nails on a chalkboard, took me about six months to get over that, but, if it was, like, persona come to life.

John [0:25:55]: You know a persona isn't perfect, but it, like, you know, gives you an impression of maybe what our directional personality.

John [0:26:00]: So you can see where I'm headed.

John [0:26:02]: I don't necessarily quote unquote data, but yeah.

John [0:26:06]: So the times when we found this interesting.

John [0:26:10]: So for example, an early innovation, I've seen groups that have used these things where they actually have a synthetic idea, remember you can have an expert as well.

John [0:26:21]: Right?

John [0:26:22]: So we're can have synthetic idea id are coming up with a product ideas and then have a a bank of synthetic users that are responding to that, cycle through that like, a thousand times and see what bubbles to the top.

John [0:26:33]: And maybe those are worthy of human exploration.

John [0:26:36]: Right?

John [0:26:36]: So just to give you an example of something you're probably not doing today.

John [0:26:40]: And I'm not even saying maybe that's the thing you have to do, but more think about what could be possible and then what's appropriate in your circumstances.

John [0:26:48]: I think that it is a nice tool in the cases for things that I just can't get.

John [0:26:53]: So we had a person in our class who did ultra high net worth people that with their specialty.

John [0:26:58]: And so offering two hundred three hundred dollars to come for an interview for an hour is not going to be appealing.

John [0:27:05]: But they had us a body of data they did have about these folks.

John [0:27:09]: So is there any way they can represent that and bring it to life so that they could still have a chance to query, you know, what about this possibility would this be helpful.

John [0:27:17]: And what would be a perspective of a person like that?

John [0:27:21]: What's a first step at imagining what that's like.

John [0:27:23]: Another are sort of inevitably, we we never get to interview as many people as we want.

John [0:27:29]: Right?

John [0:27:29]: So there could be marginalized groups, and we could we could represent There could be extreme so you doing a car with all wheel drive break.

John [0:27:38]: What are extreme ice climber like?

John [0:27:40]: And what that they need that's staying I love windsor first.

John [0:27:43]: They have different gear.

John [0:27:44]: You know, what?

John [0:27:44]: So you're probably not gonna make it to those kind of things.

John [0:27:47]: So what are is some of the ways that you might get an inkling of the things that you wouldn't normally be able to research otherwise.

John [0:27:54]: So if I've got dead zero, maybe it's better than zero.

John [0:27:57]: Right?

John [0:27:58]: There's plenty of nails on a chalkboard for this replacing what all of us do.

John [0:28:02]: And I think we're probably pretty sensitive to that.

John [0:28:04]: But thinking about what could be beyond what we could normally be capable of, might be an interesting thing.

John [0:28:11]: So I try to describe it as inspiration or a way to open your vantage points when we always use those to test our interview guides before we talk to humans so that either a, we can tell what might be confusing or we might have answer we're not expecting, and we might be more prepared for it when it comes from the real humans.

John [0:28:31]: So you can see that just...

John [0:28:33]: It's a...

John [0:28:34]: I don't know.

John [0:28:35]: I, you know, when we're learning how to use this tool.

John [0:28:37]: And it just...

John [0:28:38]: At as the last note, it probably you have a word in Maria.

John [0:28:41]: I remember the days because, you know, I got my degree when dinosaurs around the earth that we had this debate about, well, there's no way you could possibly use this new angle tool called Webex.

John [0:28:51]: That's crazy.

John [0:28:52]: You know, no one should be doing interviews remotely that's insane.

John [0:28:57]: Yeah.

John [0:28:57]: I can't get any of the data in nuance I would.

John [0:29:00]: So just remember, it's a tool.

John [0:29:02]: And so what is this tool potentially useful for, and let's put it to the test and that's to, you know, inform decision making on it rather than never or absolutely, it's the new wave.

Maria [0:29:15]: Yeah.

Maria [0:29:15]: I would agree with that, John.

Maria [0:29:16]: I absolutely like the reference to a tool, and I think like, Ai is a tool that you have at your disposal and how you use it is kind of dependent on what the job is.

Maria [0:29:28]: Like, whether this is the right tool of for the job.

Maria [0:29:30]: So you should always be asking yourself.

Maria [0:29:32]: You're, like, what is it that I need to learn and is synthetic users a avenue for that?

Maria [0:29:37]: Can I realistically learn what I need to learn through this approach?

Maria [0:29:41]: One thing to want to caution is that So that aesthetic users is an Ai moderate interviews entities like these are all att methods of data collection and, you know, design research really...

Maria [0:29:52]: We put a lot of emphasis on observing behavior.

Maria [0:29:54]: And so this is something you just can't.

Maria [0:29:56]: You can't observe a synthetic user.

Maria [0:29:57]: Right?

Maria [0:29:58]: Going about their day to day life, Right?

Maria [0:30:01]: Making purchases online or researching something online.

Maria [0:30:04]: Right?

Maria [0:30:04]: They're are not real humans.

Maria [0:30:05]: So it's really important to recognize where it can be useful certainly in like, market research, getting attitudes preferences as John's saying, like, testing out maybe some of your screener questionnaires and seeing whether there might be some misunderstanding there.

Maria [0:30:19]: That's certainly an interesting use case, but they're not going to be a full replacement for all types of of research.

Maria [0:30:26]: And certainly won't be useful in those more behavioral sort of spheres where, you know, you really just need to see someone in their own context,

John [0:30:34]: Yeah.

Maria [0:30:35]: Doing something to be able to make some good informed decisions about what would be the best solution in that space.

John [0:30:42]: No.

John [0:30:42]: If I may, just the interesting thing here is, just remember all of these things are the worst they'll ever be right now.

John [0:30:49]: They're only getting better.

John [0:30:49]: So I do tell people in my class, like, just do the thought experiment of supposed the representations of these things got much better, What might we do then.

John [0:30:58]: And I'm not saying they're like that today or they're not, But if it's more, you know, that's for all of you to decide for yourselves.

John [0:31:03]: But there are interesting ways that that...

John [0:31:06]: So the way these tools work right now, the sort of professional level ones is they have a layer of specialized say, behavior information that they have...

John [0:31:16]: So you've got the base, we can decide about the biases that are in those.

John [0:31:19]: And then you've got whatever your specialized collection is.

John [0:31:23]: So like, Vi introduced three million people a year or User Interviews has looked at all the major psychological experiments that are out there.

John [0:31:31]: And some conscious Ai has, like, behavioral science stuff they've tried to factor in.

John [0:31:36]: And then on top of that, it maybe the layer of all the things you know about your specific clients.

John [0:31:41]: So it's possible that there might be interesting things when you get that sort of higher level of sophistication there.

John [0:31:47]: And they do...

John [0:31:48]: You know, this is another one to try.

John [0:31:50]: So I think we're is, first of all, right.

John [0:31:53]: But secondly, it's intriguing to see you can basically do a day in the life or a diary study with a synthetic user today and user...

John [0:32:02]: And sorry.

John [0:32:02]: Synthetic user dot com.

John [0:32:04]: And so Again, I won't defend it to the end of the earth, but I am saying, you know, to continue to explore and see what makes sense for all of us.

John [0:32:14]: And there will be things where people can say, I can plug this in.

John [0:32:18]: That doesn't mean that's a good idea to do it.

John [0:32:20]: So there were tools today that you can use that say, yes, I'd like to interview ten or it like You Ai to interview ten of these people, and I'd also like fifty synthetic users to be tested in the very same thing.

John [0:32:33]: So I synthetic testing synthetic.

John [0:32:35]: And then give me a report and give me a report of the humans and give me a report of the other or altogether or...

John [0:32:40]: And I'm not convinced that that's a good idea altogether.

John [0:32:44]: Right?

John [0:32:44]: So...

John [0:32:45]: But we're learning.

John [0:32:46]: So I'm...

John [0:32:46]: I guess, I'm just saying So we want you to know these things are out there and we've gotta have a somebody's gonna ask us, so we wanna have an informed position.

Erin [0:32:55]: Awkward interruption.

Erin [0:32:55]: This episode of Awkward Silences like every episode of Awkward Silences is brought to you by User Interviews.

Carol [0:33:02]: We know that finding participants for research is hard.

Carol [0:33:04]: User Interviews is the fastest way to recruit targeted, high quality participants for any kind of research.

Carol [0:33:10]: We're not a testing platform.

Carol [0:33:11]: Instead, we're fully focused on making sure you can get just in time insights for your product development, business strategy, marketing and more.

Erin [0:33:20]: Go to User Interviews dot com slash awkward to get your first three participants free.

Ben [0:33:26]: I spoke earlier this year to some professors from Penn and Columbia about digital twins, which is the synthetic user of the market research space, and we had some questions about the accuracy.

Ben [0:33:38]: It varies as so much happens.

Ben [0:33:39]: The article I dropped was in the Harvard business review, and it's doing exactly what Maria and John are talking about.

Ben [0:33:47]: These leaders are thinking about it as one tool in the toolbox, especially when you're doing branding or marketing or early early stage squishy.

Ben [0:33:56]: We're not sure what we don't know.

Ben [0:33:57]: It can begin to give you a signal.

Ben [0:34:00]: It's not often the signal that these leaders make a big decision on or put resources behind.

Ben [0:34:05]: But just like we would sort of gut check if we didn't have money to recruit.

Ben [0:34:09]: We would check with our family or our colleagues, not those are real people, but it's it's kind of like that.

Ben [0:34:14]: It's sort of just like John, your example where you have lots of simulations to get a sense of what topics might we then get our human researchers to start playing around with and playing within and learning.

Ben [0:34:24]: And so the question around the Benchmarking I have seen some accuracy levels is better than half.

Ben [0:34:29]: But again, it depends on how many psycho social and behavioral metrics you're going after, how niche are you?

Ben [0:34:35]: What are you asking them to Maria's point about versus behavioral?

Ben [0:34:39]: Do you require observational?

Ben [0:34:40]: It all comes back to what these two folks were saying earlier, which is play with it.

Ben [0:34:44]: They're gonna be able to give you their experience.

Ben [0:34:46]: You should try these things.

Ben [0:34:47]: You should try these things You should try them out in a sandbox where you're not revealing any Pii or or violating any sort policies, but you should be trying them out to Maria advice earlier in the show.

Ben [0:34:57]: Maria, I don't know if wanna add on to anything there.

Maria [0:35:00]: Yeah.

Maria [0:35:00]: I mean, I think in twenty twenty six, we'll start to see more results of teams that have been trying these things out.

Maria [0:35:05]: So in some of my interviews with practitioners, like, they're in the very early stages where they're team is like, we are right now building digital twins, and it's based off our dataset.

Ben [0:35:16]: Yeah.

Maria [0:35:16]: Didn't you have any results yet.

Maria [0:35:17]: We're still kind of experimenting.

Maria [0:35:18]: So I think those organizations who are kind of at the forefront playing with this and, like, you know, seeing what they can do.

Maria [0:35:25]: We're gonna start talking about, you know, success stories or maybe not not so success stories, right, from what's worked what's not worked and we'll probably learn a lot more.

Maria [0:35:34]: So I think It was still in very early days.

Maria [0:35:37]: It's still very theoretical in terms of, like, what these tools could do, what, you know, digital twins could do where they'd be useful.

Maria [0:35:44]: Obviously, we have, you know, singular case studies from people in terms of like, well in this context.

Maria [0:35:49]: But I think, you know, yes, play with it.

Maria [0:35:52]: But also, I'm sure we're gonna get more answers next year as to, like, where Where teams have actually found it to be Beneficial.

Maria [0:35:59]: And then the thing that I'm little bit worried about as probably many people on the call Are worried about too is you know, do people get the wrong impression of what these things are and how they should be used.

Maria [0:36:09]: So that's again something we'll want to, you'll want to be thinking about as well as you play with these tools It's like, how can you kinda of communicate people who back to your first question, Ben, Like, what is research.

Maria [0:36:20]: Like, people who don't really truly have a definition for what research Is, and John was sort of making this point.

Maria [0:36:26]: Like, if they're using in his tool, is it research.

Maria [0:36:28]: Right?

Maria [0:36:29]: So that is something we certainly will, like, obviously have more to say on as the as the data emerges about whether or not these things are useful in what context.

Maria [0:36:38]: So, yeah, It's very early days try it.

Maria [0:36:40]: Be skeptical too.

Maria [0:36:42]: I think it's okay to be skeptical.

Maria [0:36:43]: Right?

Maria [0:36:44]: I think...

Maria [0:36:44]: Yes.

Maria [0:36:45]: It's really good to be excited and try things out.

Maria [0:36:47]: But also, like, you know, really test these things and anger to try and find their limitations, but don't just go off what people tell you, you know, and I anecdotally, like artists things useless, and it has no use cases, like, make the opinion like John say for yourself.

Maria [0:37:00]: I think the distinction between synthetic use and digital twins is interesting too.

Maria [0:37:05]: I think we'll hopefully, have a better definition next year, Like.

Maria [0:37:08]: I see it, you know, as sort of being almost, you know, digital twins being of more of that secondary research.

Maria [0:37:14]: Tool, like, you're already kind of, mh asking questions of something that already exists within the date of that.

Maria [0:37:21]: Right?

Maria [0:37:21]: Something that you've collected through another means.

Maria [0:37:23]: So I think it could open a lot of doors in terms of people asking for, like, research insights if it's curing...

Maria [0:37:30]: Well.

John [0:37:31]: To what Maria said, so first all, yes.

John [0:37:33]: And, actually, I've got a a graph Show people and when I'm presenting these things, and it's a totally made up graph by my thoughts.

John [0:37:41]: So here's take number one, maybe, you know, the one thing about synthetic users and asking the questions is is is incredibly low cost.

John [0:37:47]: Right?

John [0:37:48]: If I try something.

John [0:37:49]: So if I'm some big brand, and I'm like, what if we were to offer this discount, and you know if you did that with real humans or you did it in some public fashion, There might be a big uproar aurora they're gonna do this.

John [0:38:01]: And they're giving away our no more starbucks bonus at or whatever it is.

John [0:38:05]: And I guess my point is it's sort of a very contained place where I can try things safely.

John [0:38:10]: And also, it basically has no cost in comparison to recruiting participants and doing analysis, not saying that makes it that much better, but I'm just...

John [0:38:18]: So, I would anticipate there's so many questions that go unanswered it through a product innovation cycle of so product person or designers.

John [0:38:27]: Like, wonder if they'd like this or this.

John [0:38:28]: And I feel like in those moments, I would argue that they're going to because they'll start to get access to these things, that'll be one of the things, but I'm might as well try this and see what that says.

John [0:38:39]: Let's see what Bob the synthetic user.

John [0:38:41]: Thanks.

John [0:38:42]: Yeah.

John [0:38:42]: And for that matter, you know, I could have the one just to be clear.

John [0:38:45]: There's enough to be one.

John [0:38:47]: Right?

John [0:38:47]: You could have one that represents your...

John [0:38:49]: So we worked with pet parents, which was a fun study.

John [0:38:51]: And so, like, pet parents in New Zealand, pet parents in Sweden, pet parents in Paraguay.

John [0:38:56]: And so what might be the different perspectives and I can really quickly maybe get an inkling.

John [0:39:01]: Again, I'm trying to be very gentle some data, but maybe I can be more sensitive because we've got a collection of data that is potentially powering or influencing what that synthetic user's result is.

John [0:39:13]: So about eighty percent of our clients are asking us for a synthetic user as a result, not a Powerpoint tech.

John [0:39:19]: Right?

John [0:39:19]: So...

John [0:39:19]: And which is maybe three quarters terrifying in one point quarter exciting.

John [0:39:23]: But so as we also give guard rails alongside that.

John [0:39:27]: Right?

John [0:39:28]: Very, very clearly.

Ben [0:39:30]: So So each of us is in the education and training and development, you know, adjacent sort of world is addition to being research practitioners.

Ben [0:39:37]: And so I think of that, John as a part of our role.

Ben [0:39:40]: I mean, when when we roll out a usability democrat amortization practice to our designers, they need to be taught what the heck usability is.

Ben [0:39:49]: Do you use a metric.

Ben [0:39:50]: If so which one, when in house?

Ben [0:39:52]: Similarly here.

Ben [0:39:52]: Like, okay, we are going to create for you this sandbox, which is powered by what's called a composite or synthetic user.

Ben [0:40:00]: Here's how you ought to interpret the data, or, you know, here's when you need to tag in a helper or the researcher, you know, like just like you said there where there are different kinds of pet parents and how does a person identify as a a pet parent.

Ben [0:40:12]: There's so many research moments or moments when a researcher or someone trained and research can step in and sharpen, clarify or make applicable.

Ben [0:40:19]: So it's going to happen I, you know, to Maria And John's point.

Ben [0:40:22]: It's here.

Ben [0:40:23]: It's not coming.

Ben [0:40:23]: It's here.

Ben [0:40:24]: It's a matter of how much we're willing to massage it, organize it, govern it, make it work with our processes.

Ben [0:40:32]: That's...

Ben [0:40:33]: I think that's so important.

Ben [0:40:34]: John, I don't know if there was something you wanted to add there.

Ben [0:40:36]: Sorry.

Ben [0:40:36]: I mean this step over.

John [0:40:37]: Oh, no.

John [0:40:38]: I'm read at a set point earlier about, great question and how they are helping people, for example, with test plans and things like that.

John [0:40:44]: There's actually really interesting ways that, like, when I talk to the folks there, and there's another tool called ur, there's all these tools.

John [0:40:52]: But key point is both of those had the approach of, how can we help non researchers make better questions, make better test design and sort of gently guide them the right direction and not say, you know, why is this feature so great and why should we build it?

John [0:41:06]: So, you know, maybe we can have an even better, you, scientifically down question than that?

John [0:41:11]: And so it is interesting how we can use these things for good as well.

Ben [0:41:15]: Yeah.

Ben [0:41:15]: Yeah.

Ben [0:41:16]: I wanna shift before we get to a few pre, for audience questions.

Ben [0:41:21]: Maria, you and your team over in Energy think a lot about developing, ups skill, growing in this profession?

Ben [0:41:28]: You also think a lot about hiring first time, researchers broadly defined.

Ben [0:41:32]: What advice might you share for either a job seeker or someone who's looking to hire their first researcher.

Ben [0:41:38]: Are there new profiles or skills that as a job, seeker you ought to be looking to get or or not a get.

Ben [0:41:45]: On the other side of the point, are there things that you would advise jobs job, those hiring to stop doing or start doing?

Ben [0:41:51]: Yeah.

Ben [0:41:52]: That's a good question.

Ben [0:41:52]: Just whatever comes to mind reed jobs in the in the new Year.

Maria [0:41:56]: Yeah.

Maria [0:41:56]: I think there's a great questions, and I think a lot of the audience here and might be curious about this, you know, especially since they've been a lot of layoffs, and people are trying to enter the field and it's incredibly difficult market right now, especially for junior hires.

Maria [0:42:08]: So my one piece of advice to hiring managers is considered do you really need five years experience.

Maria [0:42:13]: Can, you know, can people show, you know, relevant experiences in different places that can, you know, and you can spot the talent that they're gonna grow quickly within your organization without too much, you know, with the right coaching, but also work too much, you know, hand holding or, you know, ramp up time.

Maria [0:42:30]: I'm that's essentially what hiring managers are looking for right now.

Maria [0:42:33]: It's like, they can hire someone who can they can just place in and start be efficient and productive and work fairly autonomously.

Maria [0:42:39]: If you are looking for your first role, I would say, just really try and show that you've mastered the basics in terms of, like, communication.

Maria [0:42:48]: So, like, that your writing is really concise and clear, that, you know, using Ai is super helpful in that respect the amount of, like resumes or, you know, case studies that I've read that clearly haven't used Ai to kind of help them with their writing.

Maria [0:43:03]: It's kind of evident.

Maria [0:43:04]: And it's almost disappointing because these tools are like, freely available for that purpose.

Maria [0:43:09]: So really trying to kinda make sure that when you present yourself, you're communicating super clearly what your strengths are.

Maria [0:43:16]: Any case studies are written, you know, very logically, that you not don't just go beyond go beyond the method, also talk about the thinking process behind some of the choices that you've made.

Maria [0:43:26]: Hiring managers are really looking to see that these people, the people that higher, like, are critical thinkers, good problem solve.

Maria [0:43:33]: That they can be dropped in with a team and that those that individual can kind of figure it out, which is what a lot of us have had to do, you know, sure.

Maria [0:43:41]: As we've joined different teams.

Maria [0:43:43]: So I think that's gonna be even more critical than ever in this new hiring soft cycle and this this last year and and the next year going forward.

Maria [0:43:51]: Experiment with Ai, try it with synthesis with, you know, helping you craft research plans or, you know, screen or questionnaires, see what its limitations are.

Maria [0:44:02]: Right about it.

Maria [0:44:04]: If you're a great writer, get a subs, get medium, you know, start to kind of write your meetings on what you're trying and include those things in your resumes.

Maria [0:44:11]: And I think, you know, obviously, you want as much relevant experience as you can get whether that through internships or volunteering with various organizations where you can do a little bit of Ux work right and demonstrate that.

Maria [0:44:24]: Those are all going to help you out a lot.

Maria [0:44:27]: It is a very tough market right now.

Maria [0:44:29]: So course.

Maria [0:44:30]: Everything you can do to kind of showcase that you will be ready and willing and you're able to kind of, like, get stuck in and figure things out on the goes is going to be important.

Ben [0:44:39]: John?

John [0:44:40]: Yeah.

John [0:44:40]: I mean, so first of all, Yeah.

John [0:44:42]: There's no question you're right on Maria.

John [0:44:44]: With Ai stuff, I think that also saying, you know, we have debates on our classes off and where, hey, us senior people can see, you know, poor summaries or something and kinda catch it, and we've got that sort of gut sense.

John [0:44:59]: And we're like, wow, what is the junior peterson gonna do with this?

John [0:45:02]: So are they going to just accept it?

John [0:45:04]: Or will they have that critical taste to be able to pick it up.

John [0:45:09]: And so it...

John [0:45:10]: I'd actually say that one piece of the puzzle here is if you can say only, I've tried these tools.

John [0:45:14]: But I'm and and a you said this, Maria, but, you know, really sort of hammering this idea that I can identify their strengths and weaknesses, I can evaluate their appropriate so that we can scale as these tools get better.

John [0:45:27]: So being the person who...

John [0:45:29]: Even though you are being hired can essentially help to hire the tools.

John [0:45:33]: Right?

John [0:45:33]: Because they're gonna be your kind of...

John [0:45:35]: You're gonna orchestrate them?

John [0:45:36]: And so we try to talk about going from being a doe to an orchestra of both yourself and your teammates or an now Ai things.

John [0:45:45]: And so how can you put those things together to make yourself sort of powered.

John [0:45:49]: So I can interview three hundred people over two days given the right circumstances, You know, I can analyze these things and I...

John [0:45:58]: And look out for the gotcha with Ai as it is today.

John [0:46:02]: So first of all, the basics of, like, great communicator figuring it out yourself, you know, having understanding that you've gotta be nuanced understanding your politics of where you are and how to navigate that successfully.

John [0:46:16]: So just to let you know, we're talking about synthetic users, we often make a synthetic user of the person where the big boss we're presenting to and say here's the things we're going to say, what are their first objection gonna be.

John [0:46:29]: And gosh, I wonder when you're interviewing?

John [0:46:31]: If you might want to consider the possibility to make a synthetic user of that person, just they've got a Linkedin profile?

John [0:46:37]: And then say, what do you think are the things they'll wanna talk about the most or what would be the thing that would be most upsetting to them Or, you know, That doesn't even mean the right, but you're sort of prepared for the neighborhood.

John [0:46:47]: Right?

John [0:46:48]: Even if it gives you confidence and it's completely off.

John [0:46:51]: And we are surprised at how many times our stakeholders ask the questions that are are synthetic users last of us, the day form presentation.

John [0:47:00]: So not that our bosses aren't very creative.

John [0:47:03]: I'm just...

John [0:47:04]: Anyway.

John [0:47:05]: Yeah.

John [0:47:06]: So, I guess that was one piece and the other...

John [0:47:08]: So it was just really understanding what can And I can't work, so you're now empowered with all these tools and knowing when not to use them.

John [0:47:16]: You know, we should not be using this in the circumstance.

Ben [0:47:19]: I wanna on the line for any job seeker listening.

Ben [0:47:21]: You didn't hear Marina John say you definitely need to know Python or If you're not using multiple regression then they're not talking...

Ben [0:47:28]: I mean, that might be a part of the role you're hiring for?

Ben [0:47:31]: It's...

Ben [0:47:31]: Can you recognize the context of the business.

Ben [0:47:34]: Can you speak to how your skills work or help that business do whatever it wants to do.

Ben [0:47:39]: And can you have discernment taste as John described there.

Ben [0:47:43]: I love that from each of you.

Ben [0:47:44]: It's...

Ben [0:47:44]: Because it's not easy.

Ben [0:47:45]: I don't mean to suggest that writing which which each of us do or communicating on a daily basis, which each of us do spend a lot of time refining is easy.

Ben [0:47:52]: It's not and it's so important to be able to say to a hiring manager.

Ben [0:47:57]: I can go into the product org, and I can help them think better about the customer.

Ben [0:48:01]: Whatever that looks like, surveys, diary studies on the ground, Id i's, Fine.

Ben [0:48:08]: I'll select that with them.

Ben [0:48:10]: But I'm able to and you speak to that level of why is the business there?

Ben [0:48:14]: Or why is your customer if you're working in a consultancy or agency?

Ben [0:48:17]: What's their problem, John and re each deal with leaders who have this sort of sin thing that keeps them up at night.

Ben [0:48:23]: So I'm so grateful that each of you, you know, said what you did because I think it's important for jobs seekers, especially to hear.

Ben [0:48:29]: Okay.

Ben [0:48:30]: We're running...

Ben [0:48:30]: Oh, go ahead, John.

John [0:48:31]: Oh, Because I I tried to ask, what's their boss's boss's?

John [0:48:34]: Bonus.

John [0:48:35]: Structure.

John [0:48:36]: Right?

John [0:48:36]: It might...

John [0:48:37]: At that point it might be public information.

John [0:48:38]: Right?

John [0:48:39]: So trying to make them...

John [0:48:40]: Even mentioning that you're trying to make then the hero, you know, indirectly.

John [0:48:43]: I can't imagine it hurting.

John [0:48:44]: So...

Ben [0:48:45]: Yeah.

Ben [0:48:45]: Yeah.

Ben [0:48:46]: Anything to add to that.

Maria [0:48:48]: No.

Maria [0:48:48]: I think in addition to, like, you know, communicate what you can do, like, also just be a researcher and ask questions.

Maria [0:48:55]: Yeah.

Maria [0:48:55]: Like, if you're in a hiring, you know, if you're in an interview and a hiring manager is saying, like, what can you do for us, like, I would wanna know, like, tell me about your team and what you think you're struggling with.

Maria [0:49:05]: And like, I can tell you some possible approaches that I might take.

Maria [0:49:08]: But I probably need to, like, do a little bit more, you know, chatting to folks organization or a bit internal research to figure out what is the best approach?

Ben [0:49:16]: Yeah.

Maria [0:49:16]: I think that demonstrates your research, you know, skill.

Ben [0:49:20]: Absolutely.

Ben [0:49:20]: Yeah.

Ben [0:49:20]: Alright.

Ben [0:49:20]: So let's imagine we have this conversation again, December seventeenth twenty twenty six.

Ben [0:49:25]: Maria, I'll start with you.

Ben [0:49:26]: What do you hope researchers or the research industry has done or How has it evolved?

Ben [0:49:33]: What does it look like?

Ben [0:49:34]: What do you hope it looks like or make a prediction about what the state of things might be just one year from now?

Maria [0:49:39]: Yeah.

Maria [0:49:39]: I hope that we'll see a little bit more uniformity across, like, research platforms in terms of capabilities.

Maria [0:49:46]: I think we've got a lot of tools out there right now that are, you know, very new to the landscape, some of those sadly will die.

Maria [0:49:54]: And some of those will get acquired.

Maria [0:49:55]: Yeah.

Maria [0:49:56]: And we'll start to see some more standardization of, like, these Ai interviewers or, like, how...

Maria [0:50:01]: I don't know, dia platforms are incorporating Ai or Ai in synthesis.

Maria [0:50:05]: And so that will hopefully be much easier people to learn and to figure out, like, how they can use these tools.

Maria [0:50:11]: So I'm hoping that kind of the tool landscape sort of settles a little bit.

Maria [0:50:16]: And Expecting that to be the case as, you know, the kind of Ai hype that's a that's that's dry up a little bit.

Maria [0:50:22]: I think also, I think in December twenty twenty six, we'll have more of those answers of things that we spoke about today that we don't you know, really have a lot of true and tried and tested case studies, like, how some of these different, like, digital twins are being used successfully in organizations.

Maria [0:50:37]: It's definitely one of my missions to get more answers into that for energy and for our audience.

Maria [0:50:41]: But I certainly think there'll be a lot more sort of agreement across the board in terms of where Ai is actually useful and where maybe it's gonna we're probably gonna forget Antibiotics.

Maria [0:50:54]: It's not going to be that useful, like, people aren't gonna be talking about, like, these long prompts that they had in documents anymore.

John [0:51:00]: Yeah.

John [0:51:00]: So I'm hoping

Maria [0:51:02]: that's gonna be the case by December twenty twenty six and that there's a little bit more of a strategy going forward.

Ben [0:51:07]: John, How about you.

Ben [0:51:07]: You've been doing a lot of thinking about all these various mach combinations of user research, what might the landscape look like or what questions do you think we'll be asking next year this time?

John [0:51:17]: Right.

John [0:51:17]: Well, as a person who's interviewed forty founders of tools now I realized.

John [0:51:21]: So I'm delighted to think like Maria on Might, there'd be a little more integration of these things.

John [0:51:27]: And even actually, these individual platforms don't talk well to one another.

John [0:51:31]: Sure.

John [0:51:32]: So there's a little bit of radical common sense to go on there too.

John [0:51:34]: I actually, I guess I'd love for us to have a better articulation as, like, research ops, people as leaders, here's how we'd like to use these things.

John [0:51:46]: Here's a framework for governance that we all we're an knee towards as a body of researchers and how we'll decide on, on, you know, the appropriate ethics and the appropriate, you know, usage of these kind of things.

John [0:51:59]: And then I think the other is, I'm not sure how much it'll stay where it is in in the sense, you know, r is exactly right that there are these crazy prompts people build and I'm guilty too.

John [0:52:09]: We've been doing a lot of stuff with agent work and I there's so much agent agent to agent.

John [0:52:13]: But the key point is you can have a really interesting sort of aerial system.

John [0:52:18]: Right?

John [0:52:18]: Where it could be like I'm gonna go look for quotes in this interview as an Ai.

John [0:52:22]: And then I'm gonna be a person that verifies are those really quotes.

John [0:52:26]: And then I'm gonna have another one that's like, is that really strategically relevant.

John [0:52:29]: And see you see where I'm going is we kind of argue, oh that thing was lousy, it doesn't work?

John [0:52:35]: But, you know, really good intros intersection of how are we really, really, really doing this or...

John [0:52:40]: What is an insight?

John [0:52:41]: How are we generating?

John [0:52:41]: And is there an opportunity for these tools to actually support us in some of that work in different ways?

John [0:52:48]: And granted.

John [0:52:48]: Things like making interview guides or things are more straightforward.

John [0:52:52]: Yeah.

John [0:52:52]: That's better suited.

John [0:52:53]: But on the cutting edge, is there an opportunity for these things to sort of up their game.

John [0:52:59]: So they're not just sort of fun toys, and they might be really interesting strategic partners.

Ben [0:53:05]: Yeah.

Ben [0:53:05]: Perfect place to end because I do think it is a moment for us as an industry to, and as a community and and disciplined of practice to mature again.

Ben [0:53:13]: Know we talk so much about what's the maturity levels of our organizations.

Ben [0:53:16]: Are they ready to kind of accept the gift of our research and I think to John Ray point.

Ben [0:53:21]: This is...

Ben [0:53:22]: Certainly that still needs to happen companies and organizations and teams and executives, especially as they get enamored by Ai being able to...

Ben [0:53:29]: Solve a lot of problems on using air quotes here, we'll need to continue to mature, and I think it's a moment to John Maria point that we ought to as a disciplined, mature how we're thinking about these tools.

Ben [0:53:39]: Well, John dropped his Youtube channel at Brilliant Experience.

Ben [0:53:42]: That's where you can learn more from him.

Ben [0:53:43]: NN/g and Maria eyes on Linkedin.

Ben [0:53:45]: They're both on Linkedin.

Ben [0:53:46]: They are not smart.

Ben [0:53:47]: Here.

Ben [0:53:47]: They are smart everywhere as it turns out.

Ben [0:53:49]: So if you had to drop earlier or you're tuning it on demand will link to a whole host of resources if you wanna find these folks.

Ben [0:53:55]: Maria and John, thank you so very much.

Ben [0:53:57]: What a great way to end the year.

Ben [0:53:58]: I'm so great some of the time in the insight.

Maria [0:54:01]: Thanks for having us.

John [0:54:02]: And of course, and please Falling at energy.

John [0:54:04]: I do too.

Ben [0:54:06]: Yes.

Ben [0:54:06]: They're best in the business.

Ben [0:54:07]: It's nice to have peers who you can say, like, gosh.

Ben [0:54:09]: I just hope I could someday right like that.

Ben [0:54:11]: So looking forward to finding out what Brilliant Experience and Energy does new your thank you all for spending a bit of your year and week with us, and we are wishing you a great holiday on a wonderfully insightful twenty twenty six.

Ben [0:54:23]: Bye, I'll stay well.

Erin [0:54:31]: Thanks for listening to Awkward Silences brought to you by User Interviews.

Erin [0:54:34]: Theme music by Fragile gang.

Erin [0:54:36]: Hey there, Awkward Silences listener.

Erin [0:54:48]: Thanks for listening.

Erin [0:54:49]: If you like what you heard, we always appreciate a rating or review on your podcast app of choice.

Carol [0:54:56]: We'd also love to hear from you with feedback, Guest topics or ideas so that we can improve your podcast listening experience.

Carol [0:55:02]: We're running a quick survey so you can share your thoughts on what you like about the show, which episodes you like best, which subject you'd like to hear more about which stuff you're sick of and more just about you.

Carol [0:55:12]: The fans have kept us on the air for the past five years.

Erin [0:55:15]: We know surveys usually suck.

Erin [0:55:16]: See episode twenty one with Erica alcohol for more on that.

Erin [0:55:19]: But this one's quick and useful.

Erin [0:55:21]: We promise.

Erin [0:55:22]: Thanks for helping us make this the best podcast it can be.

Erin [0:55:25]: You can find the survey link in the episode description of any episode or head on over to User Interviews dot com slash awkward survey.

Erin [0:55:32]: Promise