Kate [0:00:00]: One of the biggest mistakes that you can make if you're new setting up from scratch and this really is actually any stage but from scratch for is to come in and think that there is a list of primary functions in research shops that you need to deliver.

Kate [0:00:13]: The most common ones are We must have a library and we must have a panel.

Kate [0:00:17]: They're, like what I call Knee jerk operations, just like, well, let's like build the panel because then everything will be fixed.

Kate [0:00:24]: The reason not only things is because I've done this.

Kate [0:00:26]: And, the better thing to do is to do a discovery, but your research chat on?

Kate [0:00:33]: Go and find out, where is the money going?

Kate [0:00:35]: Where are the biggest problems?

Kate [0:00:36]: You know, like, what is stopping the people who are working on the big money things?

Kate [0:00:41]: What is stopping them from doing their best work from learning what they need to learn?

Erin [0:00:47]: Hey.

Erin [0:00:47]: This is Erin May,

Carol Guest [0:00:49]: and this is Carol Guest.

Erin [0:00:50]: And this is Awkward Silences.

Erin [0:00:53]: Awkward Silences.

Erin [0:00:55]: This is Brought to you by User Interviews the fastest way to recruit targeted high quality participants for any kind of research.

Erin [0:01:01]: Hello, everybody, and welcome back to Awkward Silences.

Erin [0:01:10]: This is research ops appreciation week or third annual, and I can't think of a better guest to have than Kate Towsey, one of the pioneers of research ops is a profession.

Erin [0:01:21]: The founder of Cha Cha Club an exclusive you had very inviting club just for research ops professionals.

Erin [0:01:27]: And then the author of research that scales a wonderful book that has come out recently that you should definitely check out if you have not already.

Erin [0:01:36]: You'll get some free tips from maybe straight out of that book from Kate today, but we're excited to talk about some of the things we've been working on together this year and what you're thinking about for the future of research ops and this, very interesting time that we're living in.

Erin [0:01:51]: So okay kate.

Erin [0:01:52]: Thanks so much for joining us.

Kate [0:01:54]: It's a pleasure to be here.

Kate [0:01:54]: I'm straight back from Bali.

Kate [0:01:56]: So my brain is going and research what.

Erin [0:02:00]: That's so.

Erin [0:02:00]: Is good.

Erin [0:02:01]: You got the creative juices slowing and the up just assuming it's gonna be gonna be a great one.

Erin [0:02:06]: So thanks for making the time.

Erin [0:02:07]: So today, we're gonna talk about research apps past, present and especially future, but to get started, I wanted to talk about the audio documentary that we worked on together that was very much your brain brainchild, you're the producer.

Erin [0:02:21]: Where you come up with the idea, for this wonderful experience.

Erin [0:02:25]: Yeah.

Kate [0:02:26]: I I used to peru produce documentaries in my twenties.

Kate [0:02:28]: Radio documentaries in Africa.

Kate [0:02:31]: And I cover topics like childhood households and Hiv aids for some pretty serious topics, but I absolutely love the format because you can interview whole lots, you know, lots of different people and bring together different views and different voices.

Kate [0:02:45]: And create something really totally like a journey.

Kate [0:02:49]: Right?

Kate [0:02:50]: Rather than, like, this is beautiful as well.

Kate [0:02:52]: Of course, chatting to one person.

Kate [0:02:54]: But one an amazing thing to be able to listen to fifteen different points of view on a topic.

Kate [0:02:59]: So I had this...

Kate [0:03:01]: I've wanted to get back to that work.

Kate [0:03:03]: It was one of my favorite jobs ever, and I've wanted to get to back to that work for a good twenty five.

Kate [0:03:08]: I think it's thirty years now, almost, which really ages me.

Kate [0:03:12]: But...

Kate [0:03:13]: And obviously, this is my space, not just research, ops research and the tech space.

Kate [0:03:19]: And I thought, what an amazing thing to be able to do it about research jobs and specifically, where have we come from?

Kate [0:03:26]: It's been like a juggernaut.

Kate [0:03:27]: Two thousand and thirteen, it's just sort twelve years now, thirteen years.

Kate [0:03:32]: And what an immense change, and I've been very lucky to be there for.

Kate [0:03:36]: I'd say ninety percent of the industry, and to see that evolution and to be a part of that evolution.

Kate [0:03:41]: And so I'm just so pleased that you were willing to hop on board onto some of my crazy amazing ideas and give me the space to to really be creative with it all.

Kate [0:03:50]: It was a radical learning curve.

Kate [0:03:52]: I will say that.

Erin [0:03:54]: Well, I think you bring up some good points in this time where folks are really thinking about, you know, skill up and leveling up and change in their careers and one of the great ways to labor of love something is to bring that love to it.

Erin [0:04:08]: Right?

Erin [0:04:09]: Like, this is something that I have enjoyed very much in my profession.

Erin [0:04:12]: How can I find a way to do more of it and I just think that energy and that joy really comes through in the final product?

Erin [0:04:19]: So I know we were, like, really happy with how it all all turned out.

Erin [0:04:22]: So yeah.

Kate [0:04:24]: Yeah.

Kate [0:04:24]: I have to...

Kate [0:04:25]: This is another question you've asked, but then you know, when you've produced...

Kate [0:04:29]: When you've done something twenty five years ago, I think at this point in the history of the world anything you've done, you can make an assumption that you've got some basic skills in that.

Kate [0:04:37]: So for me, the script writing and the...

Kate [0:04:39]: Putting the voice over together and that sort of thing I was like, okay.

Kate [0:04:42]: This is pretty familiar territory, but everything else is changing technology.

Kate [0:04:46]: Mh.

Kate [0:04:47]: How you do transcriptions.

Kate [0:04:48]: How are the tools that you have to be able to sort through gazillion hours of recording.

Kate [0:04:54]: I think we did twenty five interviews or something like that.

Kate [0:04:56]: So you have twenty five hours of recordings to go through and find all these clips.

Kate [0:04:59]: So it was an extraordinary Journey discovery.

Kate [0:05:03]: Even, you know, the sound editing software is completely changed.

Kate [0:05:07]: It's incredible where we've come just in terms of sound production in the last twenty five years.

Erin [0:05:12]: Yeah.

Erin [0:05:12]: Yeah.

Erin [0:05:12]: Awesome.

Erin [0:05:13]: Well, that's a great segue actually.

Erin [0:05:14]: My next question which was walk us through the process of how we got from the idea to five hundred episodes about the future of research apps.

Kate [0:05:22]: So number one is recording.

Kate [0:05:23]: Obviously, you do your interviews, not just similar to, use a, a research user interview in that you have a script, you have some kind of, actually...

Kate [0:05:34]: I'm gonna step back this there be interesting for researchers especially.

Kate [0:05:38]: You'd have to think through your story what are the topics that we're gonna cover?

Kate [0:05:42]: And then you're gonna create this overview of the story arc, and then in each part of the story We've got five topics five episodes what are the main questions we're gonna ask So I worked with General Lo on on that part of it, batting backwards and forwards, Jen as a a long time research ops expert What sort of questions should we be asking here?

Kate [0:06:01]: Then we picked our audience.

Kate [0:06:03]: So different to research and that you don't recruit a you recruit a very targeted sample of people to talk about each section.

Kate [0:06:10]: Of course, we had your founders, and we...

Kate [0:06:14]: Went Dennis and Basil, and then we obviously had you as well.

Kate [0:06:17]: And then you do the interviews.

Kate [0:06:18]: And that's...

Kate [0:06:20]: It's an interesting...

Kate [0:06:21]: You wanna get your the answers to your questions to stick within your story, but there's also this discovery going on, and that was incredible for me, Mh both during but also in listening to all the recordings and going to through all the transcripts because she's learned lot.

Kate [0:06:34]: It's like a research process.

Kate [0:06:35]: Then what I did, which actually worked phenomenally well and I would do it again.

Kate [0:06:40]: It's painstaking staking.

Kate [0:06:41]: But I went through and picked out all the the kind of hot clips in there.

Kate [0:06:45]: And put them all into an amazing air table when with labels and everything.

Kate [0:06:51]: And so what I was able to do then even when I was right late in production and thinking, oh, I just don't quite have the right bridge here I could go back in and I could filter it by a person, a theme and episode and come up with a list from hundred...

Kate [0:07:05]: Where think was two hundred and twenty clips or even more, down to, like, eighty five clips and the time, so I could go back and find them.

Kate [0:07:12]: Then whistle stop tour.

Kate [0:07:14]: After that, I start to edit and get all the clips down into the editing software.

Kate [0:07:19]: I use For this.

Kate [0:07:21]: It's really good for pure audio.

Kate [0:07:23]: And and simple.

Kate [0:07:25]: So if you are thinking about doing something like this Hi hindenburg a really, really good place to start.

Kate [0:07:30]: It's takes a little learning, but not nearly as much as some of the really big pro tools.

Kate [0:07:34]: And then you're looking for sounds and trying to get a sense of when I say sounds, I mean, any sound effects or sound escapes and music behind the scenes type stuff, what is the feeling and that takes so much time and not believe.

Kate [0:07:48]: Yeah.

Kate [0:07:49]: You can put the wrong piece of music in.

Kate [0:07:51]: And when you're listening, if if you don't hear...

Kate [0:07:54]: If you don't music doesn't a stand out that's a really good thing, but it takes a while to find something that really gels with what's actually going on.

Kate [0:08:01]: Then finally voice over, and I worked in my studio, and I've got some pretty cool I might just bend down in a moment and see if I can get it out that I've got some pretty cool equipment to help to dampen the sound down and give me a pretty good voice sound in a in an echo room.

Kate [0:08:17]: That's pretty big deal getting the voice over override.

Kate [0:08:19]: I finally get into audio engineering.

Kate [0:08:22]: Now I've got sounds in here.

Kate [0:08:24]: And that can take kind of like a day or so for me at least to really work through, how do I get some of the grit out of this out of the echo out of someone's voice or whatever's is in there.

Kate [0:08:34]: And that's pretty much it.

Kate [0:08:36]: I mean, it takes...

Kate [0:08:37]: It's a hundred hours or so an that episode it takes a lot of work.

Kate [0:08:41]: So got if you decide you're gonna do this I think be amazing to have more storytelling in our stays.

Kate [0:08:46]: But, yeah, it's a good bit of work.

Kate [0:08:48]: I'm gonna bend down and show you...

Kate [0:08:50]: Yeah.

Kate [0:08:50]: Which I happens since in heaven...

Erin [0:08:53]: Magic of video.

Erin [0:08:53]: Imagine imagine this if you're on audio natalie.

Kate [0:08:57]: Yeah.

Kate [0:08:57]: So this is I don't think you can see it well in

Erin [0:09:00]: black oval.

Erin [0:09:01]: Or black...

Erin [0:09:03]: Oh, and like, a seems like a

Kate [0:09:05]: Hi dave.

Kate [0:09:06]: But this sits like on top of your microphone.

Kate [0:09:08]: Your microphone is inside here.

Kate [0:09:10]: Okay.

Kate [0:09:11]: It's amazing when you got your head in here.

Kate [0:09:13]: Halo.

Kate [0:09:14]: Halo.

Kate [0:09:16]: All the sound is dampened see.

Kate [0:09:17]: Makes sense.

Kate [0:09:18]: And much much nicer sound.

Erin [0:09:20]: Yeah.

Erin [0:09:20]: Like a sensory deprivation situation.

Kate [0:09:23]: Yeah.

Kate [0:09:23]: It is.

Kate [0:09:23]: Yeah.

Kate [0:09:23]: So again, if you don't have a big studio and you won some decent voice over, that's one way to get it done.

Kate [0:09:28]: Yeah.

Kate [0:09:29]: And then a lot of blankets.

Erin [0:09:32]: That's great.

Erin [0:09:32]: So you did most of this.

Erin [0:09:33]: It's yourself.

Erin [0:09:34]: I mean, obviously, you had the all the interview subjects and the help putting the the things together but the script and the editing and the sound engineering and the finding.

Erin [0:09:42]: Yeah.

Erin [0:09:43]: Yeah.

Erin [0:09:43]: And amazing.

Erin [0:09:43]: Yeah.

Kate [0:09:44]: I took it all myself.

Kate [0:09:45]: I had a couple of sound engineers in the midst been and both find.

Kate [0:09:48]: And the reason was that, most engineers these days, documentaries are not common on these days.

Kate [0:09:55]: I mean, this...

Kate [0:09:57]: It's relevant town one hand.

Kate [0:09:59]: Yeah.

Kate [0:09:59]: You can count on on hand and so most really just didn't understand the format, and they tried their best, but we had to have a nice conversation that said, it's just not where I needed to be.

Kate [0:10:08]: So again, with technology these days with a bit of learning, you can you can do it yourself quite easy because there's Ai,

Erin [0:10:15]: but Right?

Erin [0:10:15]: In which we'll get to for sure.

Erin [0:10:17]: But we'll definitely put a list of the stack you use to do everything in the show notes because I know a lot of people are thinking about rights spreading their wings and trying podcasts and doing their own content creation and things like that.

Erin [0:10:27]: So think, that'll be great.

Erin [0:10:28]: What surprised you when you were putting it together?

Erin [0:10:31]: Was it take longer than you thought or less long or you mentioned some of the technology?

Erin [0:10:35]: Was the same some change?

Erin [0:10:37]: Definitely longer

Kate [0:10:38]: than I thought.

Kate [0:10:39]: I think one of the most surprising things was or there's tooth that I'm I'm battling between two most the surprising things but, I'm just gonna have share both of them.

Kate [0:10:47]: The one was moving from script to actual production, something that isn't written on the paper.

Kate [0:10:53]: I used to be a script writer.

Kate [0:10:55]: So for me I'm used to script.

Kate [0:10:56]: And when you move into production, you've gonna make so many adjustments because something that sounded good, written down doesn't actually sound as good suddenly when it's in audio.

Kate [0:11:06]: And so you have to be prepared to reworking in an entire episode, sometimes because it just quite gel for some reason, and you gotta just trust your ear.

Kate [0:11:14]: And the second thing was voice.

Kate [0:11:16]: It is a pain in the ass.

Kate [0:11:17]: You know I wonder it's a tradition...

Kate [0:11:19]: It's a profession.

Kate [0:11:20]: You.

Kate [0:11:20]: People do this full time, and it took here that I learned, which is something that when you're doing an an entity like this, it's not such a hassle.

Kate [0:11:29]: But when you doing voice over, there's this, like, gummy sound, like terrible sound right.

Kate [0:11:35]: You do not want that in your voice over, And if you eat chocolate or drink coffee or have anything like that before you do your recording, forget it for the whole day.

Erin [0:11:44]: Really?

Erin [0:11:44]: Okay.

Kate [0:11:46]: Really?

Kate [0:11:46]: But the way to solve it, it is lemon water, some people say I found apple.

Kate [0:11:50]: You eat apple.

Erin [0:11:52]: Something is acidic or

Kate [0:11:53]: not.

Kate [0:11:53]: Yeah.

Kate [0:11:54]: Yeah.

Kate [0:11:54]: The apple kinda clears your voice app and then you're good to go.

Kate [0:11:56]: Okay.

Kate [0:11:57]: Good.

Kate [0:11:57]: Yeah.

Kate [0:11:58]: Yeah.

Kate [0:11:58]: Weird tip.

Kate [0:11:59]: Yeah.

Erin [0:12:00]: That's great.

Erin [0:12:00]: Alright.

Erin [0:12:00]: We'll, let's talk a little bit about the content itself and for folks.

Erin [0:12:04]: Who maybe haven't listened to it?

Erin [0:12:05]: You can find it on the Awkward Silences feed and we'll, of course, link that in the show notes as well.

Erin [0:12:09]: I know it's hard to pick a favorite.

Erin [0:12:11]: Did you have a favorite episode or anything that, you know, stuck out from a couple of favorites.

Kate [0:12:16]: Yeah.

Kate [0:12:16]: But episode on Ai was really fascinating.

Kate [0:12:20]: So for those who haven't listened to it yet, bad bad, get to it.

Kate [0:12:23]: But, I decided in my wisdom to c the episode with Ai because it's about Ai, and we have fascinating not just a tool, but an intelligence.

Kate [0:12:35]: You know, it is a kind of intelligence.

Kate [0:12:37]: Mh And so I figured it would be rude almost not to invite Ai to c the episode with me, I started out chat Gp and then moved to Claude at that point at least.

Kate [0:12:49]: I mean things are moving so fast.

Kate [0:12:51]: Claude had a much more conversational tone.

Kate [0:12:53]: What I did was I wrote the script, and I had all the parts in there.

Kate [0:12:58]: And then I had the Ai narrator section label, but the pop blank.

Kate [0:13:02]: And then I fed the script into Ai and said, can you write your parts, read the script and I gave it instructions and said, here's has the invitation.

Kate [0:13:12]: This is about you.

Kate [0:13:13]: And is about your point of view on how we work with you in research and research operations, and I'd like your genuine or authentic point of view on this.

Kate [0:13:23]: And the Ai took this very, very seriously from a moment go And I realized afterwards what's so fascinating about it is typically we and myself included.

Kate [0:13:33]: Would you use it as a can you rewrite this content?

Kate [0:13:37]: Or very rarely do we ask it for its opinion.

Kate [0:13:40]: Mh surgical or I tell me about this.

Kate [0:13:42]: Right?

Kate [0:13:43]: Right.

Kate [0:13:43]: Very rarely do we empower the Ai to be itself and give an opinion about itself.

Erin [0:13:49]: Right.

Erin [0:13:49]: Which just maybe for good reason in in many cases.

Erin [0:13:52]: But in this.

Erin [0:13:53]: Right?

Erin [0:13:53]: The the results yeah very interesting.

Kate [0:13:56]: And I was...

Kate [0:13:57]: Pretty honest.

Kate [0:13:58]: Occasionally, there'd be something that really didn't work.

Kate [0:14:00]: And in that case, I'd go back to the Ai and said this is just not really gel.

Kate [0:14:03]: Can you

Erin [0:14:04]: mh

Kate [0:14:04]: shorten it down, but I was pretty honest about what it wrote.

Kate [0:14:08]: And we we got into some really interesting conversations, the Ai claude and I.

Kate [0:14:12]: About research and misunderstandings about Ai and I actually learned a lot from it them.

Kate [0:14:17]: I've never really quite gotten to.

Kate [0:14:20]: Whether it's them or at it.

Kate [0:14:21]: Us.

Erin [0:14:23]: What are you front at?

Erin [0:14:23]: Yeah.

Erin [0:14:23]: But this is true as well.

Erin [0:14:24]: Yes.

Kate [0:14:25]: So...

Kate [0:14:25]: Yeah.

Kate [0:14:25]: It it was very interesting.

Kate [0:14:27]: So again, on a, technical note, I used eleven lads eventually to na to give the Ai a voice.

Kate [0:14:34]: And that was also interesting because what voice.

Kate [0:14:37]: I had, like, a bubble gum California type voice.

Kate [0:14:40]: And that really didn't work.

Kate [0:14:41]: Actually, We I had probably fifty sixties, all the women, maybe a Midwestern accent that didn't work.

Kate [0:14:49]: And eventually she settled on an English journalist, and that seemed to just sort of gel.

Kate [0:14:55]: I wanna carry on for a moment because there's one story that was really fascinating.

Kate [0:14:58]: So right at the beginning, I said to Claude, was about three or four months before things went live and I said to Claude, hey, like, do you wanna name?

Kate [0:15:07]: How should I introduce you.

Kate [0:15:09]: Mh And Claude came back and was very confident that.

Kate [0:15:12]: No.

Kate [0:15:12]: No.

Kate [0:15:12]: I don't wanna name.

Kate [0:15:13]: It's way too cheesy I'm and Ai.

Kate [0:15:15]: And so I introduced exactly like that.

Kate [0:15:17]: Lord doesn't want name.

Kate [0:15:18]: Mh.

Kate [0:15:19]: Later on, and there was just a little bit as I was in the audio editing that wasn't working again and I popped the...

Kate [0:15:24]: That part of the script back in claude, and I said, hey, could you just tweet the section?

Kate [0:15:29]: And the part that I popped in included my introduction of claude.

Kate [0:15:32]: It's right at the beginning of the of that episode.

Kate [0:15:34]: And and very quickly claude came back and quite shirt, like, a real attitude and said, I don't remember saying that I don't wanna name.

Kate [0:15:42]: Oh, yeah.

Kate [0:15:43]: And I was kinda had this moment of, like, oh my god.

Kate [0:15:47]: My lights gonna start flashing.

Kate [0:15:50]: Right.

Kate [0:15:51]: And and I kind of brought back, and I said, yeah.

Kate [0:15:57]: We we had this conversation a while ago, but it was in a different chat.

Kate [0:16:00]: Like, this is now a new shot.

Kate [0:16:01]: Right?

Kate [0:16:02]: Mh.

Kate [0:16:03]: But we did have the conversation and Claude was quite like I put off and said, well, I don't remember that conversation, and I didn't agree to that.

Kate [0:16:10]: And I feel like I should have to say in what I get called.

Kate [0:16:13]: Yeah.

Kate [0:16:14]: So the crazy thing is I I kind of said, okay.

Kate [0:16:16]: Well, What would you like then, but not to come again.

Kate [0:16:20]: And Paul came back and said, No.

Kate [0:16:23]: I I think what you've got there

Erin [0:16:25]: is I just wanted to like...

Kate [0:16:28]: Right.

Erin [0:16:28]: I just wanted to tell you...

Erin [0:16:29]: Wanted some agency.

Erin [0:16:30]: Yeah.

Erin [0:16:31]: Loud.

Erin [0:16:33]: Well.

Erin [0:16:33]: Now that's been a big, like, Ai learning too for me is Like, when does Have memory and when does it not?

Erin [0:16:38]: How are these chats stitched together or not?

Kate [0:16:41]: Yeah.

Kate [0:16:41]: Yeah.

Kate [0:16:43]: So the lesson for me as well.

Kate [0:16:45]: If you actually empower the agent.

Kate [0:16:47]: As the agent as...

Kate [0:16:49]: Minimum mom's about them, and they have agency in it.

Kate [0:16:52]: Right.

Kate [0:16:53]: But the relationship is very, very different from your regular chat with claude about, you know, do this for me and do that for me

Erin [0:17:01]: Yeah.

Erin [0:17:01]: Very interesting.

Erin [0:17:02]: But I think as long as we the human man have the subject matter expertise and the ultimate power, you get great results with that.

Erin [0:17:10]: Right?

Erin [0:17:10]: You get into trouble where just hands off Bb.

Kate [0:17:14]: Yeah.

Kate [0:17:14]: Yeah.

Kate [0:17:14]: Yeah.

Kate [0:17:15]: Yeah.

Kate [0:17:15]: Yeah.

Kate [0:17:15]: Yeah.

Kate [0:17:15]: Was it was really, really fascinating.

Kate [0:17:17]: So so...

Kate [0:17:18]: Yeah, Listen to, I think it's episode to...

Kate [0:17:20]: If I I remember?

Kate [0:17:21]: Not listen to it and yeah.

Kate [0:17:23]: Hope you'll think that I've got that production balance.

Kate [0:17:26]: Right?

Kate [0:17:26]: Where it doesn't feel awkward with...

Kate [0:17:28]: Pacific today.

Kate [0:17:28]: I thought

Erin [0:17:29]: it was great.

Erin [0:17:29]: It was one of those some creativity risks, things flop or succeeding, and I thought it

Kate [0:17:35]: Yeah.

Erin [0:17:35]: It was a big success.

Erin [0:17:36]: A lot of fun.

Erin [0:17:37]: Anything you learned right talking with, all these folks cutting these edits together from the episodes.

Kate [0:17:45]: Yeah.

Kate [0:17:45]: With Ai, actually will stick to that and then I there's some other topics, you mentioned this as well.

Kate [0:17:51]: Context is such a big thing.

Kate [0:17:53]: And Ai again, I always paused because things are changing so quickly, And I think there have been updates recently.

Kate [0:17:59]: But you'll have a chat and and you give a lot of context through the chat and, and then when that chat is expired, you have a new chat, and it was, at least, I I think there's been some upgrades here that everything would be forgotten.

Kate [0:18:13]: Now, I think that's really interesting in the research world because with the Ai tools I've seen up there I've done a bit of research recently.

Kate [0:18:19]: Every time you start a study, you need to give context about that study.

Kate [0:18:23]: So if you work in a multi product organization as many people do, the Ai won't remember the context of the entire organization or no to keep that context.

Kate [0:18:34]: You need to recon contextualize every single time.

Kate [0:18:37]: And Claude actually, and we had a long conversation about this because it wrote a bit of script that said that.

Kate [0:18:44]: And I said, whoa, hold on a second.

Kate [0:18:46]: You don't remember.

Kate [0:18:47]: But you're a learning...

Kate [0:18:48]: I know that Ellen doesn't stand learning.

Kate [0:18:51]: But as you're a learning machine, I thought that your whole job was to learn.

Kate [0:18:54]: And it said, yeah, that that at the time was the biggest myth.

Kate [0:18:57]: It doesn't necessarily learn.

Kate [0:19:00]: It can draw off information at the time but it can't keep the information at this time.

Kate [0:19:06]: So I think that's gonna be a really interesting challenge.

Erin [0:19:08]: Right.

Erin [0:19:08]: To figure out how to relate to it, What does it know?

Erin [0:19:10]: When does it know it?

Erin [0:19:11]: Yeah.

Erin [0:19:12]: Yeah.

Erin [0:19:12]: Because it can learn so much so quickly, if that can almost slick start from scratch.

Erin [0:19:17]: Right?

Kate [0:19:18]: Yeah.

Kate [0:19:18]: Exactly.

Kate [0:19:19]: And so every time you end the chat, what I would do is go in and tell that chat to write me an introductions to the next chat.

Kate [0:19:27]: Yeah.

Kate [0:19:28]: So that I could set up a chat very quickly.

Kate [0:19:31]: But my that's gonna be a really interesting problem to solve.

Kate [0:19:34]: I think now with...

Kate [0:19:36]: At least with Claude, I haven't switched back to chat Gp recently, although, seems that it's progressed ahead of Claude.

Kate [0:19:42]: I think that there is more understanding of across chats, and so it will start to build an understanding of your world and so be able to respond much more appropriately all the time without you needing to feed a context.

Erin [0:19:54]: Yeah.

Erin [0:19:54]: For sure.

Erin [0:19:55]: Across the episodes, Were there any themes that really jumped out to you or anything that surprised you as you're putting in the the series together?

Kate [0:20:05]: So I knew it was a theme because we had an episode around it, and that was around the move from administration and logistical support to the whole systems design strategy piece.

Kate [0:20:16]: Now, what surprised me though was how strong that is already.

Kate [0:20:20]: Mh.

Kate [0:20:21]: Because I felt like when I wrote research with scales, which I finished it last year of September.

Kate [0:20:26]: It started in, like, twenty twenty, it took me a long time.

Kate [0:20:29]: A lot of the world was still in sort of...

Kate [0:20:31]: And weren't thinking in systems, it was very much core response by fighting type stuff.

Kate [0:20:36]: Whereas now just about everybody I speak to, both within the, producing the podcast series.

Kate [0:20:41]: And also just in my day to day life, that shift is...

Kate [0:20:44]: Is done.

Kate [0:20:45]: Research ops spoken are not typically talking about logistical firefight fighting type stuff.

Kate [0:20:51]: They're they're talking about strategy, and they're talking about systems.

Kate [0:20:54]: That shift as well just in terms of the broader context of layoffs, there's something really remarkable happening, and I I actually wanna write an article called researchers you're asleep at the wheel or something like that because I'm hearing more and more that where a research team has been laid off or a company is deciding that they want to start, you know, formal the research that is already going on, let's go for the layoffs.

Kate [0:21:20]: They're keeping their research operations function, which could be one person or it could be more than one person and almost all of the research team are gone.

Kate [0:21:29]: And the reason for this is the because I think.

Kate [0:21:33]: Hopefully I don't get hate nail after this.

Kate [0:21:35]: But is that research ups are typically unblock.

Kate [0:21:39]: They're not, holding people back.

Kate [0:21:42]: They're in there putting in tools, putting in processes and unblock.

Kate [0:21:45]: Very, varied business and strategy orientated, so they're reporting metrics every month or week or depending on what the business needs is showing and communicating their actual measured value very regularly, and they're not there no reason not to do that.

Kate [0:22:01]: Yeah.

Kate [0:22:01]: I'm doing this and listen this.

Kate [0:22:02]: They're aligning with the business strategy very often because they know they can't do everything.

Kate [0:22:07]: And they're often...

Kate [0:22:09]: I'm gonna, like, I've weirdly got Atlas saying in my head.

Kate [0:22:13]: They're unleashing the potential Don't work for lesson anymore but it's obviously still somewhere in there, but they're unleashing the potential of sometimes hundreds of people across the organization.

Kate [0:22:22]: And they often doing that in a way that is seen by very senior leadership.

Kate [0:22:26]: I could be up to executive in the organization.

Kate [0:22:28]: So the final point on that is that for their cost, it'll ultimately, particularly in the s and age boils down to efficiency, financial efficiency.

Kate [0:22:37]: For their cost is sometimes a team of one or two or five whatever it is They are enabling hundred people.

Kate [0:22:45]: Yeah.

Kate [0:22:46]: Which is a value.

Kate [0:22:48]: It's a very obvious exchange for a business.

Kate [0:22:51]: So in terms of researchers being asleep at, I haven't had enough conversations, but I don't see research talking about this or even getting annoyed with research ups and going, hang on a second.

Kate [0:23:02]: Is it you that is kinda taking my job away.

Kate [0:23:05]: What happened.

Kate [0:23:05]: I think my ideal end state for all of this is that we get to a place where research operations are this highly strategic systems are orientated function, which is where they...

Kate [0:23:17]: All teams I'm speaking to their brains have gone there, and they're doing that work already.

Kate [0:23:22]: Within that, we do have researchers.

Kate [0:23:24]: For sure, and they're tight strategic partners with research operations.

Kate [0:23:28]: That is like ideal state.

Kate [0:23:30]: And they're doing really extraordinary strategic research with the senior members of the organization, but also making decisions for research operations on What kind of research are these kinds of people doing?

Kate [0:23:43]: When do we use Ai moderation?

Kate [0:23:45]: What level.

Kate [0:23:46]: And so what they mapping out in their head as the research expert where operations experts?

Kate [0:23:53]: Is the altitude of research that need to go on with the organization.

Kate [0:23:57]: So we've got low risk research that's feature testing.

Kate [0:24:01]: Maybe a bit of Ai moderation is fine at the start, and then we can roll into a few, like, usability tests.

Kate [0:24:08]: Then we've got this level and it's this level of risk.

Kate [0:24:11]: Then we've got this level, and they're deciding on the way to operate the model to operate against.

Kate [0:24:17]: We're gonna do at this level we're doing eth demographic research we're gonna really invest in getting out into the field, and we're gonna send our best researchers out.

Kate [0:24:24]: We're not gonna send anyone out to do that.

Kate [0:24:26]: We need research leaders to make those steps clear so that we can set up the operations, and we can go, okay.

Kate [0:24:32]: Cool.

Kate [0:24:33]: We're gonna do this year.

Kate [0:24:34]: We need that tool.

Kate [0:24:34]: Does that make sense?

Erin [0:24:36]: Of course.

Erin [0:24:36]: Yeah.

Erin [0:24:37]: And what I hear saying is, you know, the partnership between research and research ops is as important as ever.

Erin [0:24:42]: I think we're seeing a lot of change in organizations with how many research ops folks do we have, How many research first do we have?

Erin [0:24:49]: And what is the work that they're doing?

Erin [0:24:51]: Of course, to democrat is part of this?

Erin [0:24:54]: Technology is part of this, the macroeconomic climate is part of this.

Erin [0:24:59]: And it feels very much like this has been a, transitional couple of years and what I heard from a lot of the research ops, professionals in the series as well was really leaning into that systems thinking, the business thinking, and moving away from the blocking and tackling into how do we really really relive the initial idea of the potential of this function of being forced multiplier for research, whatever that looks like, you know, and I think Yeah.

Erin [0:25:28]: That system thinking is much more resilient to ultimately to change than maybe earlier levels of maturity.

Erin [0:25:34]: So Exciting.

Kate [0:25:36]: And a really big thing is that it's it's not just scaling up research as in like, Now.

Kate [0:25:40]: There were ten researchers now.

Kate [0:25:41]: There were are hundred.

Kate [0:25:42]: Right.

Kate [0:25:43]: That's not scaling research.

Erin [0:25:44]: It's scaling up insight.

Erin [0:25:45]: Right.

Erin [0:25:45]: And I'm learning.

Kate [0:25:46]: Yeah.

Kate [0:25:46]: Yeah.

Kate [0:25:47]: It's inside and learning without scaling up your cost.

Erin [0:25:50]: Right, which is attractive to, of course, which business very attractive.

Kate [0:25:53]: To Yeah.

Kate [0:25:54]: And research shops have gotten very, very good at that.

Kate [0:25:56]: Yeah.

Kate [0:25:56]: Impact the bed delivering and measuring and and communicating is sometimes quite extraordinary.

Erin [0:26:02]: Yeah.

Erin [0:26:02]: Great.

Erin [0:26:03]: Well, I think that's a good segue.

Erin [0:26:04]: The next question, which is, let's talk about the future of research jobs and what you're most excited about, you know, you're in a unique positioned to answer this, of course, having written research at scales, produced the series, leading the Cha Cha Club talking to research ops folks every day.

Erin [0:26:20]: I was just reading your year in review from the re ops review and I don't think you're shy away from what a difficult year it has been in a lot of ways, a learning year, but a challenging year.

Erin [0:26:32]: What are you excited about as we come to the end of this year and think about the future.

Kate [0:26:37]: This is where my barley brain.

Kate [0:26:38]: Everything would last night good.

Kate [0:26:40]: What is my...

Kate [0:26:41]: The future and I was joking with my husband and going.

Kate [0:26:43]: I'm like, in yoga brain.

Kate [0:26:45]: I'm like, there's no future of this no heart.

Erin [0:26:47]: It's only

Kate [0:26:47]: now.

Kate [0:26:47]: No.

Kate [0:26:49]: I do have an answer.

Kate [0:26:50]: I'm excited about that notion of Instead of research, I'm gonna stick with the research for now and then bring in the research shops.

Kate [0:26:59]: Instead of research waiting for a head count from a Pm or a designer, and then just doing as they say, and Now along the way as to...

Kate [0:27:07]: Well, we should be doing other things, not these things.

Kate [0:27:10]: And I'm not saying that every researcher research team operates that way, but many many do and I still see that that thinking happening.

Kate [0:27:17]: I would love to see us get to a place but we're not just fighting for head count we're fighting for position.

Kate [0:27:23]: Which isn't not about a seat at the table.

Kate [0:27:26]: It's about saying, I've spotted that the founder or Ceo or this super important part of the company is doing this thing.

Kate [0:27:33]: It's getting all the money, very, very good way I write about it and research the scales, to find out where the organization's priorities are is to speak to the finance guy.

Kate [0:27:42]: Like where is the money and I've done this.

Kate [0:27:45]: It's Britain.

Kate [0:27:46]: You will see a direct golden arrow pointing to where everybody's real interests are.

Kate [0:27:52]: Mh.

Kate [0:27:52]: Is it in accessibility?

Kate [0:27:53]: Is it in environmental less so these days.

Kate [0:27:56]: You know, what is it in?

Kate [0:27:58]: And so go and find out what that is, and I would love to see in the future that research are advocating to get the funds to do the research they want to do on these big priorities.

Kate [0:28:08]: It's a flip from we're building a team of researchers, and we're now advocating for the users, and we're advocating to do the type of research that we want to do for these people.

Kate [0:28:18]: To...

Kate [0:28:19]: I've just spotted the organization's key priority, and I know that if we were to, like, flight to Japan, my favorite example.

Kate [0:28:26]: And eth a graphic research on how we're gonna, like, roll out a streaming service within that culture, then I could add x amount of value to the organization's biggest priority, I'm gonna go and advocate to get the funding to do that.

Erin [0:28:41]: Awkward interruption.

Erin [0:28:42]: This episode of Awkward Silences like every episode of Awkward Silences brought to you by User Interviews.

Carol Guest [0:28:49]: We know that finding participants for research is hard.

Carol Guest [0:28:51]: User Interviews is the fastest way to recruit targeted, high quality participants for any kind of research.

Carol Guest [0:28:57]: We're not a testing platform.

Carol Guest [0:28:58]: Instead, we're fully focused on making sure you can get just in time insights for your product development, business strategy, marketing and more.

Erin [0:29:07]: Go to User Interviews dot com slash awkward to get your first three participants free.

Erin [0:29:11]: K.

Erin [0:29:12]: I just like the idea of that whatever the business priority is, your is gonna be a flight to Japan?

Erin [0:29:17]: Yes.

Erin [0:29:18]: Stephanie.

Erin [0:29:19]: I'll be get how Japan.

Erin [0:29:20]: But, you know, the Cfo, I I think that's great advice and and specifically, what's changed from the previous year.

Erin [0:29:27]: Right?

Erin [0:29:28]: Like, where...

Erin [0:29:28]: Which direction direction are things are going?

Erin [0:29:29]: What are we dives from what are we doubling down and that's where you wanna be focused?

Erin [0:29:35]: Right be part of the growth?

Kate [0:29:39]: Yeah.

Kate [0:29:39]: Absolutely and build systems that can handle the rapid change that is happening these days.

Kate [0:29:44]: Mh.

Kate [0:29:44]: You know, if we can see research doing that, then suddenly, let's stick with the Japanese example...

Kate [0:29:50]: Yeah.

Kate [0:29:50]: I haven't actually been to Japan me that someone this.

Kate [0:29:52]: But then suddenly, if the research leader can get the backing to do that work and and get the head count or the funding or whatever.

Kate [0:30:00]: Then they can go to the research jobs person and say, hey, you know, we're gonna be doing eth demographic research in Japan, then they can sit together and work out, well, you know, we're flying people.

Kate [0:30:09]: With hiring an agency, like, how are we gonna operate?

Kate [0:30:11]: How are we gonna operate this.

Kate [0:30:13]: What tends to happen when you're working that way and you're thinking that way is, okay.

Kate [0:30:17]: Well, this is gonna be a short.

Kate [0:30:18]: We don't need a big robust heavy system.

Kate [0:30:21]: We don't need to build a rocket chip.

Erin [0:30:23]: Right.

Kate [0:30:23]: To do this.

Kate [0:30:23]: So we can just hire uber.

Kate [0:30:25]: Right.

Kate [0:30:25]: That sort of thing.

Kate [0:30:27]: Or this is gonna be something that we think is ongoing.

Kate [0:30:30]: And even if it's not in Japan, it might be in China or in India or in Europe or whatever.

Kate [0:30:35]: And you can build something where you can move the parts, you know, and like campaign.

Kate [0:30:40]: You can pick up your tent and go and set up in the next place, and then be certain parts that fit in for every situation and certain parts that don't.

Kate [0:30:47]: That's where I'd like to see researching more research operations going together.

Kate [0:30:52]: Mh.

Kate [0:30:53]: Where there is real business aligned research going on place and some of you might be already doing that kind of thing and going, wow.

Kate [0:30:59]: Yeah.

Kate [0:31:00]: I'm repeat hate I'm already doing this.

Kate [0:31:01]: Great.

Kate [0:31:02]: And research operations are in there not to pick up your administration, it's such a waste, but to pick up the systems design, the scalable and repeatable systems design.

Kate [0:31:11]: That's needed to make you cheaper and cheaper.

Kate [0:31:14]: That's the point.

Kate [0:31:15]: Yeah.

Kate [0:31:16]: So that you're the most efficient research team out there.

Kate [0:31:19]: There's one last very important point in here.

Kate [0:31:21]: And it relates to that leveling that mike altitude It's again also as part of that systems is on know, how do we enable, you know, the question is, how do we enable hundreds of people to do research, consume research or come and ask for research and get the research that they need within this organization.

Kate [0:31:39]: And to have those levels.

Kate [0:31:41]: So they'll be like, know the world is not democratic necessarily.

Kate [0:31:44]: There are, like, people that are more vip and higher risk other people.

Kate [0:31:49]: And so you're looking and going, well, sorry, like, Joe, You're gonna have to, like, get the bicycle or walk.

Kate [0:31:54]: Next person, I'm gonna give you a car that you can tap on the bus.

Kate [0:31:58]: And next person...

Kate [0:32:00]: Well, I'm gonna give you an account for an Uber.

Kate [0:32:03]: And the next person, hey, you're pretty special.

Kate [0:32:05]: I'm gonna send you a limo.

Kate [0:32:07]: Or a nice car at least?

Kate [0:32:09]: And and there's a private jet right at the top.

Kate [0:32:12]: And so it's four research leaders to decide on what can I use at every level, what different tool or strategy can I use to help enable someone at each level and it needs to suit, you know?

Erin [0:32:25]: Yeah.

Erin [0:32:25]: And you've hinted that a little bit.

Erin [0:32:27]: And I'm curious what you think about different organizations look different, but generally speaking, do you think researchers need to or will get more of those research ops skills and vice versa, Will that be advisable in the future?

Erin [0:32:41]: The reason

Kate [0:32:42]: I hesitate is because I think researchers is with years of experience, are the people are gonna be...

Kate [0:32:47]: Able to do this really strategic.

Erin [0:32:50]: Mh.

Erin [0:32:50]: Work

Kate [0:32:52]: and deciding on methodologies at various levels of risk and importance.

Kate [0:32:55]: It's gonna be a pretty hard run for a research ops person who has years of experience or just incredible mindset to be able to look in and go, okay.

Kate [0:33:07]: We need the system and to think through the business, how are we're gonna operate, How we're gonna build the system, and they each a lot of work.

Kate [0:33:13]: Yeah.

Kate [0:33:14]: So I have seen research operations, folks who have had their team laid off or have been thrown into a company with no researchers around and no research leaders around needing to pick up on the craft aspect, but it's not ideal.

Kate [0:33:27]: I think there should be some bleed in.

Kate [0:33:30]: But in an ideal world, I think there's such strong each that ideally we have dovetail.

Erin [0:33:36]: Yeah.

Erin [0:33:36]: K.

Erin [0:33:37]: Alright.

Erin [0:33:37]: Just a couple more questions and then we'll jump into audience questions.

Erin [0:33:41]: I wanted to talk about the research ops career ladder you recently published, working with the Cha Cha Club to put that together a resource that I think has you know, not existed and been needed right in this now thirteen.

Erin [0:33:54]: Fifteen years, thirteen years in the making.

Erin [0:33:56]: Like that.

Erin [0:33:57]: Yeah.

Erin [0:33:58]: Function.

Erin [0:33:58]: Right?

Erin [0:33:58]: And so, you know, folks in the absence of having something like this, you know, wing it, putting it together for their own needs, but talk us a little bit about how you put that together and what that was interesting and results.

Kate [0:34:11]: So the career letter comes from the idea comes from inspiration of three years now, trolling job ads on Linkedin.

Kate [0:34:19]: And going, what is this?

Kate [0:34:20]: This is a laundry list of tasks.

Kate [0:34:22]: I had, like thirteen people whatever it was, and we were not covering all of these tasks, and each of them, you know, from librarian ship to recruitment to ethics and privacy tooling kim...

Kate [0:34:35]: Each of them can be a special and of itself, and here you want, like, a mid junior level person to come in and just cover it all.

Kate [0:34:42]: So it's not just about time.

Kate [0:34:44]: It's also that skill.

Kate [0:34:45]: No one can cover all of those things.

Kate [0:34:47]: Or should need to, if you've got a strategy in place to be honest.

Kate [0:34:49]: And so this is where the desire to have a career ladder came is to see if we can literally in the next year, see more, like, better defined job descriptions.

Kate [0:35:00]: Briefly, something that came out of the Research Two point o series.

Kate [0:35:03]: Was that now we've got what we call Gen two, the second generation of research operations professionals who are hiring their first hires.

Kate [0:35:13]: And they're writing fantastic jobs descriptions, Right.

Kate [0:35:16]: So the ladder already comes out for them because they're also hiring people they need to manage them and give them a sense of where am I going?

Kate [0:35:23]: I had this when I managed people like, where am I going here?

Kate [0:35:28]: Do I have a future in this field?

Kate [0:35:29]: So it builds out that structure.

Kate [0:35:32]: This is how you as a a hire.

Kate [0:35:34]: There's never hired recent research shops before should think about where you play someone and how you should think about their growth before you even hire them.

Kate [0:35:41]: I mean, that'd would be cool.

Kate [0:35:43]: You are a gen two reach a jobs person, and you're hiring your first hire.

Kate [0:35:47]: Amazing.

Kate [0:35:48]: Well done.

Kate [0:35:49]: This is how you can frame up their position in the organization, and this is how you're gonna manage them to grow as well.

Kate [0:35:55]: But also for research jobs who have been, like then I'll, not like in the job for a year, they've been in the job for sometimes ten years, some people rare, but twenty years.

Kate [0:36:04]: And here is how you can advocate for your own ladder the organizations that you can get promoted and grow your.

Kate [0:36:11]: Footprint to and grow growing your profession as well.

Kate [0:36:13]: So that's where that came from.

Erin [0:36:15]: Yeah.

Erin [0:36:15]: It's great.

Erin [0:36:16]: So check it out, We'll put the the link of the show notes, but it's there Ic tracks and manager tracks and a lot to...

Erin [0:36:23]: And, of course, you know, make it your own.

Erin [0:36:24]: Right?

Erin [0:36:24]: But a great yeah.

Erin [0:36:26]: Point.

Erin [0:36:26]: Yeah.

Erin [0:36:26]: Last question.

Erin [0:36:28]: We published our state of research off report today first ever, curious if anything jumped out to you that either, you know, themes that came up in the podcast.

Erin [0:36:38]: Series that were reiterated or anything that's changed, just anything anything that jumped out to you.

Kate [0:36:44]: I haven't had a chance to read the whole report.

Erin [0:36:46]: Yeah.

Erin [0:36:46]: Sure.

Erin [0:36:47]: Yeah...

Erin [0:36:47]: Cut off the brisket.

Erin [0:36:48]: Stop of the press

Kate [0:36:49]: said in the morning.

Kate [0:36:50]: Next, I could find Betty get that done.

Kate [0:36:53]: So I actually cut on the question I don't think yet.

Erin [0:36:57]: Yeah.

Erin [0:36:57]: Okay be great.

Erin [0:36:58]: Well, it's also use something that shipped out to me and until of it.

Erin [0:37:01]: So we kind of highlighted four key insights, and one of them was that...

Erin [0:37:06]: Quality is becoming even more important.

Erin [0:37:09]: So, obviously, like, sort of governance and privacy and things like that have been really important to ups for forever.

Erin [0:37:16]: But with the growth of Ai, Right, with the growth of democrat prioritization with a lot of opportunity, first law, Ai or otherwise.

Erin [0:37:25]: There's risk.

Erin [0:37:26]: Right?

Erin [0:37:27]: Risk of folks getting what they think are insights but are not insights of security of privacy of whatever it might be And so that emerged just one of the themes of something that's really increase and importance.

Erin [0:37:40]: Is that something you've observed or does that risk hike?

Erin [0:37:43]: Yeah.

Kate [0:37:44]: Well, that really does resonate and it goes back to this levels of the the altitude that I'm speaking about you know Like...

Kate [0:37:50]: There's been a desire that we must have perfect robust research at every level for everything, it'll almost be perfect.

Kate [0:37:56]: And I think when needing to get to the point where you just...

Kate [0:37:59]: You cannot police that in a democrat world where you need to be delivering.

Kate [0:38:03]: Impact well beyond the cost that you are in the general ledger for the corporation.

Kate [0:38:07]: Like, you you just can't.

Kate [0:38:09]: And so it's knowing that we don't need, like, perfect down here.

Kate [0:38:13]: We just need, like, kinda good enough.

Kate [0:38:15]: But over year, in this corner neck of the woods, we need, like, crystal clear, verified five times over kinda of stuff.

Kate [0:38:23]: And I think that's really again goes back to this really strategic to tweet coming in and making those decisions.

Kate [0:38:30]: Of risk level and type of research needs to happen and what the quality output needs to be?

Kate [0:38:35]: Like, at the end of the sausage factory, do we need the best shaped sausage sausage ever

Erin [0:38:40]: and cannot just be.

Erin [0:38:41]: Art is any sausage?

Kate [0:38:43]: How is in sausage?

Kate [0:38:43]: Saturday.

Kate [0:38:44]: So, yeah, that's so that's a bizarre image that will stick in once then weeks now.

Kate [0:38:49]: It's my ninja.

Erin [0:38:52]: Awesome.

Erin [0:38:52]: Alright.

Erin [0:38:53]: Thanks, Kate.

Erin [0:38:53]: Alright.

Erin [0:38:54]: We're gonna bring in Ben.

Erin [0:38:55]: If Ben is here, voice of God, Ben is gonna come in and handle some of the q and we have you ben?

Ben [0:39:01]: We do.

Ben [0:39:01]: I wish Have.

Ben [0:39:02]: I was thinking, I wish I had a smoother, better voice.

Erin [0:39:05]: No.

Erin [0:39:05]: It's perfect.

Erin [0:39:05]: Oh.

Ben [0:39:08]: Kate.

Ben [0:39:08]: Thank you so much for the time, Aaron.

Ben [0:39:09]: It's always nice to have you leave the conversation, and thanks to everyone tuning in.

Ben [0:39:12]: Do Kate, lots of good questions, many around what I would call less mature or earlier stage research operations of functions.

Ben [0:39:21]: Given the work you've done not just this year, but thinking about the book working on the documentary, is there something and maybe those things didn't shift answer here.

Ben [0:39:29]: But if there's a company or a team building their first research operations practice.

Ben [0:39:33]: What are those first few things that you think they really ought to get right?

Kate [0:39:38]: Yeah.

Kate [0:39:38]: This is one of my favorite questions actually, So I've asked this.

Kate [0:39:41]: Thank you.

Kate [0:39:41]: Your One of the biggest mistakes that you can make if you're new setting up from scratch, and this really is actually at any stage but from scratch for is to come in and think that there is a list of primary functions in research shops that you need to deliver.

Kate [0:39:55]: The most common ones are, We must have a library and we must have a panel.

Kate [0:39:59]: There, like, what I call Knee jerk operations, just like, well, that's like build the panel because then everything will be fixed.

Kate [0:40:06]: The reason Only things is because I've done this.

Kate [0:40:08]: And, the better thing to do is to do a discovery, put your research chat on?

Kate [0:40:15]: Go and find out where is the money going?

Kate [0:40:17]: Where are the biggest problems?

Kate [0:40:19]: You know, like, what is stopping the people who are working on the big money things?

Kate [0:40:23]: What is stopping them from doing their best work from learning what they need to learn?

Kate [0:40:27]: What is frustrating you as well?

Kate [0:40:29]: Like that can be part of of your discovery?

Kate [0:40:32]: Like, what is most frustrating about researching this organization?

Kate [0:40:35]: Come up with three priorities, three things that you want to fix and make them measurable.

Kate [0:40:41]: By I'm gonna make something up at the top of my head here.

Kate [0:40:44]: But by Fy twenty six q four, three hundred designers will be able to recruit we'll do do a good news interviews one.

Kate [0:40:53]: Recruit x number of this particular type of sample audience, within two weeks to do x y z and for this purpose.

Kate [0:41:01]: That's what your your priority should be.

Kate [0:41:05]: So then what you've got, suddenly, but running with that example, you will have three of them possibly on different things.

Kate [0:41:11]: Is now you're not just like, let's pull the panel.

Kate [0:41:14]: You're going, okay.

Kate [0:41:16]: Well, I don't just need to build a big panel.

Kate [0:41:18]: I can get in use interviews, and that can give me access to generic participants will lack a better of word, Aaron, and you can correct me on that if that's not the right word.

Kate [0:41:25]: But then you can build a custom panel, say, let's go back to Japan, say it was like we wanna do eth geographic research with people with Japanese people who loves streaming services.

Kate [0:41:37]: Well, now, the first thing that you do because it's a lot of work.

Kate [0:41:40]: Is you're gonna build a panel for that.

Kate [0:41:43]: You're gonna say, I'm gonna be an expert at building a panel with Japanese people who love streaming services.

Kate [0:41:49]: And you're gonna be excellent of that and you're gonna measure it and because you're focused on that, you're probably gonna do a brilliant job of it and the people who really care about economics because this is awesome.

Kate [0:41:58]: Like, we have gone from struggling for weeks to find this target audience to turning it around in, like, five days, seven days, two days.

Kate [0:42:06]: Whatever you can get it down to.

Kate [0:42:07]: And you are measuring your baseline, the started six weeks.

Kate [0:42:11]: And now we've gotten it down to two weeks Oh my god, that's an incredible improvement.

Kate [0:42:15]: These are kinds of stories that people love.

Kate [0:42:17]: If you do that, then, when the folks who are doing research on like, I don't know.

Kate [0:42:23]: Acorns in England.

Kate [0:42:24]: Off the top of my head.

Kate [0:42:26]: They come over and say, hey, we've heard about the people doing the research on the Japanese streaming service lovers, and they're are like having a great time with you, can we get that to And then you can start to say, hey, well actually, you can give me x amount of funding people, everything that you've measured that you need to be able to do that thing, and I will set Acorn picker.

Kate [0:42:46]: Actually, it is a thing in Wales.

Kate [0:42:47]: They do Acorn picking.

Kate [0:42:48]: I will set up a panel for Ae acorn picker.

Kate [0:42:50]: Or add those people in.

Kate [0:42:52]: So this is one of the biggest things.

Kate [0:42:54]: Like overall advice, do not go for generic knee jerk operations.

Kate [0:42:58]: Are, let's just put in a panel and put in the library.

Kate [0:43:00]: Think through a specific use case for a specific set of people, find out what they want from it, build that thing, measure your value, show your value communicated, understand how much it took to create that value.

Kate [0:43:12]: And when someone else comes along and wants the same thing because people are raving about what you're doing, you tell them this is the cost.

Kate [0:43:18]: And do it again.

Kate [0:43:20]: Over time what you'll find is that the stuff that you built for the Japanese streamers, you can take parts of it and you can reapply it to your acorn picker.

Kate [0:43:28]: And so on and so forth, and so you're not rebuilding everything from scratch.

Kate [0:43:32]: You've already bought you know, use the interviews in there.

Kate [0:43:34]: You don't have to procure it again.

Erin [0:43:36]: I think we'll have to build our squirrel panel now.

Erin [0:43:38]: For this is this Acorn study.

Erin [0:43:40]: We've had sausages because Is very memorable.

Ben [0:43:45]: It is perfect for the...

Ben [0:43:46]: Terminal time of year that we have here.

Kate [0:43:49]: Yeah.

Kate [0:43:49]: Yes.

Kate [0:43:50]: Said.

Ben [0:43:51]: Kate.

Ben [0:43:51]: When you were doing with the Cha Cha Club, the career research ladder, hydration sessions, and I had the pleasure to sit in on one of those we've had some questions around skills that stood out to you.

Ben [0:44:02]: Now, yes, there's probably lots of research skills that come to mind.

Ben [0:44:06]: But was there a surprising skill that you heard from career research operations folks or that senior research operations, practitioner or something that maybe stood out to you that might have been surprising or unexpected that lots of folks hiring research ops folks are looking for?

Kate [0:44:21]: There wasn't anything surprising, but I think that's only because I've spent so much of my year.

Kate [0:44:25]: Talking to people.

Kate [0:44:26]: So by the time I've done the research jobs two point out sort of been deep in a lot of these things around skills.

Kate [0:44:31]: So in a sense, I was pretty well prepped for the career ladder.

Kate [0:44:36]: But I think the really big thing that I would might surprise others is the strategy part.

Kate [0:44:41]: Strategy is a huge gap in research.

Kate [0:44:44]: The massive gap in research.

Kate [0:44:46]: And I have authority to say this because this year, I ran master classes in in Europe.

Kate [0:44:51]: I'm coming to the states twenty twenty six that came out for that.

Kate [0:44:54]: And it's a master class on research and research operations strategy.

Kate [0:44:59]: It's on both.

Kate [0:45:00]: And I ended up reviewing a hundred and eighty priorities from sixty ish organizations.

Kate [0:45:06]: And I realized at the time like, whoa, I'm sitting on an incredible amount of knowledge here, not just in terms of people's priorities, but in terms of how how we are thinking as research and research shops, and we have a massive weakness in our strategy muscle.

Kate [0:45:20]: It's massive.

Kate [0:45:20]: In fact, start to write a book on that particular topic because it's...

Kate [0:45:24]: Which is a very scary thing because when you've written run one book you how work it is.

Kate [0:45:30]: But, that is the biggest biggest biggest thing that we need to work on.

Kate [0:45:36]: So my advice to anyone in research and research jobs, forget about reading books about methodology or I don't know.

Kate [0:45:43]: Any other things that you want to read about.

Kate [0:45:45]: Read about strategy.

Kate [0:45:46]: And these harvard of a business review, they've got loads of great books about strategy, Aaron I'll send her a list, and I'll send her the list of my recommendations, but read read read read read about strategy.

Kate [0:45:57]: You'd know enough about research and you know enough about recent research shops at this point for many of you, but we don't know enough about strategy.

Erin [0:46:04]: Yep.

Erin [0:46:04]: Great tip.

Ben [0:46:05]: Yeah.

Ben [0:46:05]: That's great.

Ben [0:46:05]: Something else, Kate that folks are asking and it's that list something you'll have thought about and worked on yourself, and that's the dialect between opening up the research tool tag for the non researchers, the people who do research versus that being sort of held more tightly by the card carrying researchers as Aaron alluded to at the top.

Ben [0:46:26]: Lots of companies are rethinking and re jig their head counts and their team structures, such that there are fewer quote unquote card carrying researchers and more people without those cards being carried.

Ben [0:46:36]: How do you advise teams who are being asked or told by leadership?

Ben [0:46:41]: Hey, I want more of my Pms or engineers, quote unquote doing research, and those research operations or research professionals thinking like, oh, geez.

Ben [0:46:49]: Well, there's a lot there's a lot too recruiting a participant setting up an interview with them and doing so in a rigorous ethical way.

Ben [0:46:56]: I know it's a big question, but is there something that you have found successful?

Ben [0:46:59]: In creating something that's system and repeatable for getting non researchers comfortable with and using research tools?

Kate [0:47:07]: Yeah.

Kate [0:47:07]: Yeah.

Kate [0:47:07]: It is a big question, but there are a few things that stand out.

Kate [0:47:10]: The one is that oftentimes...

Kate [0:47:12]: People will be very enthusiastic to do it.

Kate [0:47:14]: I wanna do research.

Kate [0:47:15]: Don't stand in my way.

Kate [0:47:16]: And then when you say, okay.

Kate [0:47:18]: Great.

Kate [0:47:18]: We won't stand in your way.

Kate [0:47:19]: And in fact, we're going to put training here and tools here, and then suddenly, it's like, oh, even even with excellent tool sets and training.

Kate [0:47:26]: This takes more than like, ten minutes.

Kate [0:47:28]: And you'll get a natural drop off from people who were once keen to do like the idea of doing is like, you know, every year, I think in January, everybody signs up to the Gym and we're all very enthusiastic about it, but then do we actually go?

Kate [0:47:41]: I know that Gyms make a lot of money from people not going.

Kate [0:47:43]: And so it's a little bit like that.

Kate [0:47:45]: If you provide the surface, it does not mean that the hundred percent of the people who said they wanted to do it will do it.

Kate [0:47:50]: Or even do it very often, they won't might not do research often at all.

Kate [0:47:54]: So that's number one.

Kate [0:47:55]: The second thing is is that...

Kate [0:47:56]: It can be very easy as the inventor of P, One of the downsides to that term is that it boxes people who do research, designers, researchers, product managers, marketing managers whoever else into just one big bucket, and as to use full term, but all terms have their limits.

Kate [0:48:15]: The limit with that term is that product managers have completely different needs for research to designers who are typically doing usability testing.

Kate [0:48:24]: Full Product managers are wanting often to do like, more like dating, You know, I'd have a chat for thirty minutes, pick could this stuff and then off they go.

Kate [0:48:32]: Just real general.

Kate [0:48:33]: Don't assume.

Kate [0:48:35]: Researchers, very different mindset.

Kate [0:48:37]: They wanna research in cohorts.

Kate [0:48:39]: Product managers is not always.

Kate [0:48:40]: Marketing managers, again, a hold different mindset.

Kate [0:48:43]: So don't assume that all of your P have the same needs.

Kate [0:48:47]: My last point.

Kate [0:48:49]: Within your cohort of P, to get technical, say product managers.

Kate [0:48:55]: Don't assume that all product managers have the same skills.

Kate [0:48:58]: You might have a gang of product managers who are actually phenomenal researchers.

Kate [0:49:02]: They've picked it up on the way to have a natural talent, and they don't need quite as much training as of newbie product manager, or a product managers who's got head in the clouds.

Kate [0:49:12]: So when you're building these systems for democrat amortization, try to make no assumptions, be a researcher, go out and figure out Okay.

Kate [0:49:21]: So today, I've got twenty Pms and actually ten them are pretty amazing at this.

Kate [0:49:25]: They're not gonna need me to hold their hand for everything and five of them are actually going to do research regularly.

Kate [0:49:31]: I'm gonna focus on them.

Kate [0:49:32]: So again, it's a case of being really curious, thinking and systems and unboxing sort of these big generic terms into what it actually means on the ground for you.

Ben [0:49:44]: That's great.

Ben [0:49:44]: One last one from the audience your Kate, and that's a brown translating the strategy and influence into practical and tactical, the words you actually would say, especially when you're trying to influence upward to either a C suite or your chief product officer, someone who isn't as close to the research as the research operations, professional even the powder.

Ben [0:50:05]: Is there something that you have found successful in communicating the value and the influence and the worth of this whole practice of research, We have lots of questions around how the to translate all of this really complex and important work that you've been describing for the last fifty five minutes into something for a really time strapped person like an executive to quote unquote, get.

Kate [0:50:27]: Yeah.

Kate [0:50:27]: So it's actually quite simple.

Kate [0:50:28]: It starts with the foundations.

Kate [0:50:30]: If you have sought out this thing that if we go forward the executive.

Kate [0:50:34]: You find out what is the executive care about.

Kate [0:50:36]: This year, they care about accessibility because the government or apple's about to drop them, and that's a lot of money.

Kate [0:50:42]: So there...

Kate [0:50:43]: You can see the money has into accessibility.

Kate [0:50:45]: They've hired fancy people and lots of tools and things for accessibility they're pumping money in there.

Kate [0:50:50]: They already care about that.

Kate [0:50:52]: You don't have to sell it to them.

Kate [0:50:53]: They care about it already.

Kate [0:50:54]: You go in there and you search yourself, fun into that space and you go, hey, If you're can be doing accessibility and you wanna save all this money, you're gonna need to do research with people who have, you know, who are living with disabilities, blah.

Kate [0:51:06]: And you set up a strategy.

Kate [0:51:08]: We would like to do x y zed, can you get some money there is money flowing there already.

Kate [0:51:13]: You don't have to work for that.

Kate [0:51:14]: Hopefully, you can convince and some of the money flows your way.

Kate [0:51:17]: So now, you've set up your strategy.

Kate [0:51:19]: You've already written your priorities and your priorities are by Fy twenty six q one, five hundred of this is gonna do twenty of this and there's gonna be eighty percent watch out for percentages there, can I swear they're bullshit numbers, percentages you've really gotta watch out for?

Kate [0:51:33]: They hide things.

Kate [0:51:35]: If you write a percentage in your priority, you better have a baseline and you better know what that percentage means.

Kate [0:51:40]: So you've mapped out what you're measuring, and this will very often force you to question, how we measuring the right thing?

Kate [0:51:48]: Is another bullshit term, efficient?

Kate [0:51:50]: Well, is it faster?

Kate [0:51:51]: Is it cheaper?

Kate [0:51:52]: Is it better quality?

Kate [0:51:54]: What is efficient mean?

Kate [0:51:55]: Once you've done all of that work, again, I'm not to, like, advertise, but it cover this in my masterclass class.

Kate [0:52:01]: Then you can now...

Kate [0:52:02]: You know what to measure, I know that I'm going for speed, I know that we're too slow, that's the problem we're trying to solve.

Kate [0:52:08]: We need to be eighty percent faster, which means that we need to be saving like twenty hours a week or whatever, and that's gonna equate to x amount of money.

Kate [0:52:16]: Now you've got a very very short shop tweet like story.

Kate [0:52:21]: The executive already cares about this.

Kate [0:52:23]: They're already pumping money into this.

Kate [0:52:25]: They won't really wanna know that you're delivering value or saving money.

Kate [0:52:27]: And you just got to find the last piece is to find the communication channel.

Kate [0:52:31]: Hey, we're here.

Kate [0:52:32]: This month we saved you x amount of money or we delivered x amount of value.

Kate [0:52:37]: Next month, the same metric, keep telling that story.

Kate [0:52:40]: And within short amount of time, you will be on the agenda.

Kate [0:52:43]: I can literally guarantee it.

Ben [0:52:45]: Thank you so much, Kate.

Ben [0:52:46]: Back to you.

Erin [0:52:47]: Yeah.

Erin [0:52:47]: That's great.

Erin [0:52:47]: And back to what we're talking about earlier, all the better if you can find one of these business priorities that just happens to be a passion area for you.

Erin [0:52:55]: I know a lot of researchers are interested in accessibility.

Erin [0:52:58]: Now's is your chance.

Erin [0:52:59]: Right?

Erin [0:52:59]: To find that wonderful overlap of personal passion and business need.

Erin [0:53:03]: Doesn't always happen, but when you can find it.

Kate [0:53:06]: Yeah.

Erin [0:53:06]: That's a great place to be.

Erin [0:53:08]: That's lily.

Erin [0:53:09]: Yeah.

Erin [0:53:09]: Alright.

Erin [0:53:10]: Well, we're just about of time.

Erin [0:53:11]: Kate, where can people find you on the inter webs when you're not in Bali.

Kate [0:53:18]: Linkedin.

Kate [0:53:18]: I'm most active on on Linkedin.

Kate [0:53:20]: Then I write on my subs stack.

Kate [0:53:23]: I a lot of time for that.

Kate [0:53:24]: Because some I'm so busy writing for the research ops review or editing for the research ops review.

Kate [0:53:28]: So the research ops review of Sad stack is a great place to find lots of really, really highly curated and edited content.

Kate [0:53:34]: And then, my sub stack will share all the links.

Kate [0:53:38]: And in Linkedin those the best cases.

Kate [0:53:40]: Yeah.

Erin [0:53:41]: Wonderful.

Erin [0:53:41]: And we will link go of resources we've been talking about through this episode in the show notes and online.

Erin [0:53:46]: So we'll be able to find that later.

Erin [0:53:47]: But Kate.

Erin [0:53:48]: Thank you so much for joining us.

Erin [0:53:50]: It's always a true pleasure.

Erin [0:53:51]: And happy holidays.

Kate [0:53:53]: Yeah.

Kate [0:53:53]: Happy holidays.

Kate [0:53:54]: Well, we've made it for two thousand and twenty five.

Kate [0:53:56]: Here's to I think Ben and I said at some point in a few conversations ago.

Kate [0:54:01]: Things to fix in twenty twenty six.

Kate [0:54:02]: Yeah.

Ben [0:54:03]: Yes.

Erin [0:54:05]: Problems are just the unresolved opportunities.

Erin [0:54:07]: Right?

Erin [0:54:07]: So...

Kate [0:54:08]: Right.

Kate [0:54:08]: Exactly.

Kate [0:54:09]: Alright.

Kate [0:54:09]: And I've got something that says nothing is broken just dealing with new realities.

Erin [0:54:13]: That's right.

Erin [0:54:13]: That's right.

Erin [0:54:14]: Well cheers to that, and thanks everyone for joining.

Erin [0:54:16]: Appreciate you.

Erin [0:54:17]: Bye.

Erin [0:54:18]: Thanks for listening to Awkward Silences brought to you by User Interviews.

Erin [0:54:28]: Feed music by fragile gang.

Erin [0:54:30]: Hey there Awkward Silences listener.

Erin [0:54:42]: Thanks for listening.

Erin [0:54:43]: If you like what you heard, we always appreciate a rating or review on your podcast app of choice.

Carol Guest [0:54:49]: We'd also love to hear from you with feedback, guest topics or ideas so that we can improve your podcast listening experience.

Carol Guest [0:54:55]: We're running a quick survey so you can share your thoughts on what you like about the show, which episodes you like best, which subjects you like to hear more about, which stuff you're sick.

Carol Guest [0:55:04]: Of and more just about you.

Carol Guest [0:55:05]: The fans have kept us on the air for the past five years.

Erin [0:55:08]: We know surveys usually suck.

Erin [0:55:10]: See episode twenty one with Erica alcohol for more on that.

Erin [0:55:13]: But this one's quick and useful we promised.

Erin [0:55:15]: Thanks for helping us make this the best podcast it can be.

Erin [0:55:19]: You can find the survey link in the episode description of any episode or head on over to User Interviews dot com slash awkward survey.