#171 - Summer Throwback: Kate Towsey on Starting a ReOps Practice
ResearchOps is anything that's going to support your research team logistically or otherwise.
Erin:This is Erin May.
John-Henry:I'm John Henry Forster, and this is awkward
Kate:silence. Silences.
Erin:Welcome back to Awkward Silences. Thanks for joining us again this week. We have Kate Tousy with us, and she is, I wanna say, the living expert on research operations. And so that's what we're gonna talk about today. We're gonna talk about research ops.
Erin:Jh is here with us too.
John-Henry:Hey, how's it going?
Erin:Kate, thanks for joining us.
Kate:It's great to be here.
Erin:2019, the year of research ops. True or false, Kate?
Kate:So I would argue that 2018 will kind of historically, which seems like a crazy thing to say, be seen as the year of research operations. It was really the year where we did I formed the ResearchOps community and we did the What is ResearchOps or WhatIsResearchOps initiative that sought to understand what is this thing called ResearchOps and what do researchers need from this thing that could be called research operations? So I reckon if someone had to write a Wikipedia page about it, which someone probably might do at some point, they would point to 2018 as the emergence of research operations.
John-Henry:This is like the band. Like, I like the early stuff. Maybe you were the more mainstream consumers and you're the insider.
Kate:Yeah. For sure. Or like
Erin:the product adoption cycle where, you know, it really hit its moment last year. But I think that the trend's gonna continue, but we'll see.
Kate:Well, I hope so. I think there's still a lot of conversation to be think there's a ton of conversation to be had. There's a ton of things that are on my mind now that I'm in an organization and building out an ops team and actually doing the stuff. So if it's got anything to do with me, the conversation's only just getting going.
Erin:Great. Well, let's let's have some of that conversation and talk about some of what's on your mind right now being being in the the thick of research operations every day. So tell us a little bit let's start from the beginning. What does research ops mean? What is it?
Kate:So, well, it's this, I guess, two answers. The one is the framework, which I do actually happen to agree with. And this is the framework, a graphic about what are all the different elements that could sit within research operations. And this came out of WhatIsResearchOps initiative last year that I mentioned, where we involved it ended up being like a thousand something people around the world, people who do research and mostly researchers who were sharing what are their main challenges in doing research, what do they think research operations is, what are their triumphs and also what they wanted from a community of people talking about ops for researchers. And we took a whole lot of that data.
Kate:It was a tonne of data. We had done a survey. We had run 17 workshops around the world. Was it 17 countries and 30 something workshops? I need to refresh myself on all the numbers again, but a lot.
Kate:And did a bunch of analysis. Could probably have done a whole lot more, but, you know, I get analysis paralysis. So came up with this framework of these are all the areas of research operations. So to answer your question, research ops is anything that's going to support your research team logistically or otherwise. The kinds of things that we work on are research participant recruitment, looking at governance structures, working with legal team, either internal or external, to develop appropriate consent forms, how you store those consent forms, how you store all the data that's produced in doing research and that's both for finding it and for compliance.
Kate:The kind of knowledge that comes out of all this research you do, how do you share and store that? Do you even share and store it? Our questions, We look at stuff like tooling and thank you gifts and all these bits of logistical stuff that sit around research and aren't the research itself.
John-Henry:Yeah, have actually have the graphic up that you guys produced out of all that work. And just looking at right now, there's the main, you know, node of research ops in the middle and just did a quick count. There are then 12 kinda like other primary nodes off of it with a bunch of other little, you know, blips off of that. So it's it's a ton. Have you had experience, like, when a company or an organization is thinking about, like, hey, this is something we need to formalize, like, how do they even go about, like, approaching it or starting it?
John-Henry:It seems like there's so many different layers to it. Like, is there a common entry point that teams have?
Kate:It's a really good question. And again, because things are so nascent, can speak from We've kind of got few case studies to look at it, there are case studies. And one of them is mine. I'm leading and managing a research operations team at Atlassian. I've been at Atlassian for eight months, moved to Sydney for the job.
Kate:And I left a twelve year consulting career to work for an organisation so that I could speak with experience. It's always gonna be N equals one experience, but still experience and share and engage with people around the world who are doing similar things. So I also run a workshop on how to get started with research operations. I'll be doing one shameless pun, but I'll be doing one in Toronto in June at Strive. So once you get into the organisation, it's really understanding how the research team is being run, what the strategy is over the next year or even coming years, and working very closely with the head of research to understand what their vision is for the team.
Kate:Because putting in research operations is not always cheap. It's usually fairly permanent. I mean, nothing's permanent, but it's usually fairly kind of It's infrastructure. It's pretty permanent things. It's teams, it's people, it's hiring people.
Kate:It's getting tooling in place that could be on a two year contract. So you want to be making really good decisions that are valid over the next year or two. And that's where your engagement and working with the head of research who has hopefully got a really robust strategy and vision for where they're going and how to implement it and buy in from executive or whoever to make that happen is absolutely pivotal to the work of operations. So in Atlassian, and this is pretty common across big organisations or any research space, the biggest problem is research participant recruitment and also the biggest opportunity. So we've started really working very hard on building out a team around participant recruitment, on looking at various vendors that we're using to do participant recruitment on our Atlassian research group or our research panel and the tooling and compliance around how we manage that and thank you gifts.
Kate:And there's like, I mean, just the participant piece is a job in itself. And I'm always saying that when you look at those 12 bubbles on the framework, each one of those is a business that needs to be staffed and the budget needs to be managed and it needs to become a profit centre for the business. So when people think about, well, I'm just gonna do some ops on the side. It's it's not possible. That's like saying, oh, I think I'll just open 12 businesses on the side to get this right.
Kate:And it just, it's not gonna work.
Erin:When you talk about the process of kind of starting to build out an ops function and talking with the head of research to figure out the strategy, I start thinking about you're really doing internal research. Right? Do you ever feel like I'm doing my own kind of ethnography to figure out how to build this team? Or what sorts of methods do you kind of employ to figure out what's gonna make this organization successful from an yeah, from an ops perspective?
Kate:A 100%. And at one point, I became a little bit known as the researcher of researchers. I spent many years when I was consulting for government in The UK for government digital service on researching some of the best researchers around. And at the time I remember massive imposter syndrome because I actually wasn't a researcher. I had come into GDS to look at the content that researchers produce and see if I could get towards building them a research library to house that content and ended up having to learn from these amazing, this team of kind of 40 amazing researchers as to how to do research.
Kate:And the irony was that when I left, when I went into other roles after I'd done that work at GDS, ended up being three years with a few other clients next to it, people didn't wanna hire me as a content strategist. I never went back to content strategy and they're all offering me jobs as a researcher. Eventually I started taking on those jobs because I had to eat and needed to be very honest about my kind of my fairly basic, robust but basic level of research And yes, I still use that to this day. I think it's much more organic now. I don't spend a lot of time interviewing.
Kate:I'm embedded in the research and insights team. I spend a lot of time engaging with them on a daily basis via Slack or in conversation or over lunch or whatever to really start to understand where and in team meetings to understand where are our gaps, what's going on. But then also our, again, you can talk about problems, but also opportunities are so fundamental because I'm walking into a space where there was nothing really before and we just need recruitment and we need someone to store our audio visual. And I've got enough background knowledge at this point to get going with something so that I can learn more about what we need to really look after our unique situation.
Erin:So sort of no matter what, we know we're gonna need a certain number of things in any situation. What are some of those things that any kind of organisation might need?
Kate:Yeah, exactly right what you said. Participant recruitment, it doesn't matter what kind of research you're doing. The kind of participants you have has a massive impact on how well your research goes. It's the beginning of the pipeline of good research. So now in terms of operations, the framework is like the sort of spread of bubbles and I'm starting to look at the research pipeline as being well, step as an operations, we step in well before recruitment even.
Kate:We will be offering our research education team support in setting up their education workshops and boot camps and office hours and various initiatives they've got, which are there so that people across the organisation who are doing research, whether researchers or PMs or designers, are doing really good research. So as an operations team, we step in well before recruitment even so that the requests we get for recruitment are good quality requests. And we are then providing good quality participants to them because they've been professionally sourced by our in house team. And then after that, we're looking at, okay, so you are in a research session and you need a consent form. You might need various types of consent forms for the context.
Kate:And perhaps even you're doing ethnographic research and we don't look after this now and we wouldn't probably for another couple of years if we need to. Looking at travel bookings and helping if you've got a lot of researchers going out in the field, how are they getting there and who's helping them coordinate all the travel around that? So consent forms and the research happens, but it's being recorded on audio visual content. So the next piece in the pipeline is where is that AV content stored compliantly and so it's findable and trackable and auditable. And the tooling that happens all along the way of now needing to do analysis and transcriptions and and and and And then right out to the other side where you've done good quality along all the way and now you've got good insights and now it finally goes into a research library of sorts and what does that look like and how much value does it deliver.
Kate:And then the next piece, which is something I am actually working with now with two people on the team, is engagement. So you've done hopefully all this good quality research, you've been trained really well and at the other end comes some really, really valuable insights. And how do we get those insights under the noses of the people who need to notice it and be close to it and make sure that it's delivered in a way that is timely and understandable and applicable to what they're doing so that the research that's been done both from research and insights, but also from designers and PMs across the organization has real value and impact.
John-Henry:I wanted to just quickly go back to something you said earlier when you mentioned, you know, all of these different disciplines within research ops are almost jobs under themselves, and they take a lot of investment to do well. And I'm just picturing the situation of maybe a startup or a company that's grown to a 100 or a 150 people and they're kind of at that point where they probably have a mentality of they like to work iteratively and they like to try things before they buy, but they're probably doing enough research where they need help and they need some of the research op stuff. So that seems like a place where it might be hard to go out and get a bunch of full time, you know, racks to bring people in and really specialize in this. Like is it better to start by having somebody who's trying to spread themselves thin across a couple of these areas or they just pick one area and really try to own it to prove the value and then kind of spider out into the other areas that research ops gets involved? Does that make sense?
Kate:It does, makes absolute sense. And in some ways the answer is it depends. That's the first answer. The second answer is we're still figuring this out. And the third answer, which is my more opinionated version, is that then you're going in and you're needing to basically look at one thing because you're not going to have the time to look at more than one.
Kate:And if you do, you are spreading yourself too thin and not really able to deliver necessarily a lot of value. And I've seen this in organisations who have hired, and even my predecessor, there was one person trying to do some form of operations. And in the end you really become an administrator because you can't be doing the admin and the strategy and the planning and the implementation. It's just not possible. So depending on the circumstances, you're gonna need to find what is the most pressing thing right now.
Kate:Usually it's participant recruitment. Everyone's favourite one, which would include thank you gifts as well. We call them thank you gifts because we don't pay our participants and we don't incentivise them in a sense. It's now become a habit for me to use that term. So one of the strategies we've got, because we are looking after 20 researchers and then up to 300 people who do research or PWDRs as I respectfully shorten the two.
Kate:So one recruiter can handle 15 requests at any one time. And Surit, who's magical and running our research recruitment desk, she's got around 35 requests on her Kanban right now, which is not a manageable amount of work. So you look at that and you think you have to show the need in order to get people to scale and how do you manage that growth, which is any startup's problem. So what we've done is looked at external vendors like user interviews, like any one of the other vendors and also just like Forage Research or Askable here in Australia, various vendors that we can be saying, well, one person cannot handle all these requests, but what they can do is be handing requests out to external vendors. We can be partnering with other parts of our business so that we are triaging as much as actually doing the recruitment.
Kate:So there are always means and ways and strategies for being able to coordinate an aspect of research operations. You make the most of the resources you do have right now so that you can show value to grow. If you feel like you're never going to be able to show the value to grow because it's such a small team, there's just no way you're gonna get another person on your team. Your best bet is to focus on one thing and do it as best as you can. There's one caveat to all of that and that is that every single one of those bubbles on the ResearchOps framework are related to every single other bubble.
Kate:And so if you touch one bubble, you're probably going to be touching every other bubble. So in research participant recruitment, you are going to be touching compliance because as soon as you build up a list of people, if you do decide to build out a list, you are going to be storing people's data. And so now you have to engage with legal. And I mean, I won't labor the point further than that, but in some ways you have to touch every little piece of it, whether you like it or not.
John-Henry:Yeah, that's fair. So you want to specialize and find the area with the most leverage in the short term, but it's hard not to get yourself woven into the other areas as well. I wanted to ask this a different way. I feel like we've talked, you know, research ops is very broad and seems to include a lot. If we flipped it around and said like, what is definitely not part of research ops?
John-Henry:Are there anything that comes to mind as, you know, being very clearly outside the discipline?
Kate:Yes, this is one of my favorite topics. So I think what happened last year, 2018, and the conversation that we drove around research ops is that for me, at least towards the end of the year, there was a confusion about what is ops and what is not ops. And in fact, I ran a workshop at DesignOps Summit in New York. And a lot of that workshop, it was a full day workshop, was really helping people understand that the conversations that researchers are having about being more organised are not the conversations that operations people are having with each other. So to illustrate that or be more clear about it, research operations is not methodology.
Kate:It is not what kind of research you go out and do. It is not research strategy. So in my situation, Lisa Raekels, who is the head of research and insights at Atlassian, She will be deciding what kind of research we're going to be doing and how the team will be shaped and what kind of people are on that team. How they're to be spread out across the organisation or not. Are they centralised or decentralised?
Kate:Are they focusing on usability testing or a certain proportion? Are they doing rapid research? All these kinds of questions that anyone who's got any kind of noggin for running a research team of any size is going to be thinking about, dreaming about and writing notes about in the early hours of the morning, most likely. Those are the kinds of things that she is dealing with and coming up with and engaging me in understanding and working towards supporting. So I'm not saying that this is happening at all, but it's my favourite example because it's the most illustrative.
Kate:If she were to wake up tomorrow morning and decide, you know, we have to get out in the field. I need all of my researchers to be travelling around the world, spending time in people's offices, living under software engineers' desks, whatever her kind of vision was, the operations that I'm delivering now would change significantly. The tooling would change. Suddenly my participant recruitment would be geared not towards remote research, which is a lot of what we do now, it but would be geared towards finding people who are happy to have a researcher come and live under their desk. The consent forms that I'm offering would be and the NDAs would be different because it would be people going into another person's space.
Kate:The kind of technology I offered would be different because it would be labs in a bag and kit that allows researchers to record out in the field. I would be now looking at travel bookings. So you can see that having a head of research or a manager or a research leader who has a very clear vision, the vision is their job, the methodology is their job, the strategy is their job to decide on how the research is done and the quality of that research is their job. My job is purely creating the infrastructure that allows that vision to come to life.
Erin:I'm interested in this dialogue, right, because the vision, the strategy, that belongs to research. If the researcher says we are gonna do the under the desk thing, which I'm getting a great visual image of that sounds like really interesting research. We're gonna do that. Are you gonna say, hey. You can do that, but it's gonna cost you, like, a lot of money.
Erin:Like, is there a dialogue there? Is there a pushback or an interplay? Or, what's that kind of look like in terms of how you interact with the strategy side effects?
Kate:Again, another one of my favorite topics at the moment because and this is where having come to Atlassian and left consulting where I'd kind of dive in and build a lab or a panel and then leave and touch one of those bubbles and walk out and had not properly appreciated that every and I learned this through consulting that I would deliver one bubble, but really it touched every other bubble and I would never address those bubbles. And was I doing this terrible thing by leaving them with an unfinished ecosystem? So what's happening in Atlassian is I'm able to be seeing how all these pieces sit together and what the results are of that. So centralising all the costs of research, because now I've centralised research participant recruitment to varying degrees. I'd be lying if I said it was entirely centralised, but we're getting there.
Kate:Means that I've also centralised the cost and it means that I've now got very specific overview of what all the different things are costing us as research. So I think it's fascinating in that it's not that I've created new costs necessarily. The recruitment we're doing might be more expensive now because we're doing better quality recruitment and more compliant recruitment. But essentially it's taking us probably as much time and I would say hopefully less if I'm doing my job well. And we're able to see, oh my gosh, we're spending so much money every quarter on thank you gifts and swag boxes and e gift cards and that kind of stuff.
Kate:And this much money on all the various recruitment vendors. It's very easy for founders, CEO, heads of etcetera, etcetera, to be saying we must be close to customers. We must spend a lot of time getting to know our customers. And to be fair, the argument over the last seven years from heads of research of people, seniors in research has been we need to do more research. We must be doing more research.
Kate:Hey, let's do more research because it's been something that is needed to be sold in. And I think it's been sold in now. We've got massive teams across the world. It's not unusual to find a team of 10 in an organisation and you've got teams of hundreds and hundreds of researchers in the very big organisations. So when you look at the cost of research and you're able to represent that because it's centralised, I suspect that as this happens more and more, it's gonna be a greater conversation around, okay, well, what are we getting back for the spend?
Kate:There's so many hundreds of thousands of dollars being spent on research every year. And where is the real quality coming from out of that?
John-Henry:I have to ask just because I'm curious now. What is in your mind the distinction between a thank you gift and an incentive?
Kate:Oh, yeah. Thank you gift is I realised when I was saying that that kind of e gift cards, swag boxes, Atlassian university vouchers or free trials or whatever, various things that we can offer or charity donations, not something we've got done yet, but we're working on. Those are all under the banner of thank you gifts. The reason that we use thank you gifts, why I use thank you gifts as my terminology is because from a legal perspective, we don't pay people to take part in research. Incentive is fine to say internally, but my problem is that if I'm saying incentives when I'm speaking to researchers and people who do research, I say incentives to them and then they say incentives to their participants.
Kate:And we really want to get more towards the point where we're saying thank you for your time as opposed to paying you or incentivizing you to share with us. So it's a minor thing, but it's become such a habit for me now, thankfully, after some months of training that I just use Thank You Gift.
Erin:I love it. Words matter. You're saying that word?
Kate:Words matter. Yeah, words matter. Well, in The USA, if you pay someone or give them above $600 there are legal implications to that and tax implications. Another thing around tooling is tracking how many times you've sent someone some kind of monetary incentive, some kind of monetary gift and how that adds up over time.
Erin:So the value of you talking about centralization and I can see everything. You know, I can see what we're spending. I can help things get used, the engagement team. Right? A cost potentially could be, hey.
Erin:This is slowing me down. Is that a push and pull you've seen, or how do you kind of mitigate any feelings of bureaucracy or moving too slow or anything like that that might It's come along with
Kate:a massive theme and it's something we're battling with. Again, it goes back to proving that your service, the service that you're offering is a valuable one and that's needed. And then at the same, and not having the scale to meet the demand and then needing to meet the demand to prove that you need someone to, you know, it's that push and pull that you'll have in any business that you're growing. We have got another person joining, Sarita, next week in fact. So we'll double our capacity and over time we are looking to go up to four people, which is minuscule if you compare to people like Google who have, I think it's up to 50 people working just on participant recruitment.
Kate:That's it. That's all they do. And other teams where there's five, six, seven, eight people working just on recruiting participants. But there is a kind of a concern in an open organisation where people are used to doing their own thing that you're going to bottleneck me. And the way that we're working with at the moment is to be very clear in communicating where we can help, where we can't and not shutting someone down if they have to do their recruitment on their own.
Kate:We like to discourage it, but it's always within the caveat of we have an ambition, which is to be able to look after every single research participant requirements in the organisation, but it's unlikely we're ever going to be able to meet that because you would just need a massive team to be able to do that. And is it valuable for us to be doing that anyway? Do we rather want to be focusing on high quality recruitment for high quality research where we really, really are needed as opposed to just feeding the beast of being with customers and finding stuff out about them. My language is not clear on this because it's really We're working through this and trying to figure out how do we distinguish between these two things. Should we distinguish?
Kate:What do we support? What do we not support? And we had this great image yesterday, which was the little shop of horrors. Do you remember that movie?
Erin:Very much, yes.
Kate:Yeah, with a little like plant, Seymour. Was it Seymour? Seymour the dentist. I forget now. Seymour was the dentist, right?
Kate:Yes. And kind of this little, little cute little plant that you start feeding with your we'll just do participant recruitment for all of you. And it just gets bigger and bigger and bigger. And I was saying to Sarita, hope that you're not Seymour and you don't get eaten by the plant. So we had a couple of GIFs about that yesterday, which was really funny and the image sticks in my mind.
Kate:So yeah, it's communication and just not blocking people who are wanting to do their their own recruitment. And yet the other side of it is one of the reasons that we're centralising is because there's only so much Even with a big organisation, there's a limited number of people who are going to be interested in taking part in research, who have specific demographics and are the type of person who might take part at particular points for particular projects. And if everybody across the organisation who is not aware of compliance and is storing details all over the place is contacting people for research that's just not compliant. At Atlassian and I hope you'll excuse my French one of our values is literally don't fuck the customer. That is how you say it.
Kate:And we want to look after customer data and make sure we're doing a good job of that. So and Atlassians understand this and then want to play ball with us because if we centralise, we can set up processes that are compliant, that we know we're looking after customer data well, that we're not contacting people a gazillion times a year to take part in research, that we're being respectful of their privacy and their time. And that can only happen when you've got centralised systems to make sure that we are connected up with marketing and the number of times they contact someone is linked up with the number of times we contact them and the same with support. So I would be like, you know, we're not near there but we're working towards there and it might take us a year or so to get to a full fledged version of that. But I think when people understand that's where you're aiming for, you're not aiming to bottleneck them or to steal their power, they really do get on board and wanna play with you.
John-Henry:Yeah. Let's do the complaints in the other direction. When you get a bunch of research ops people together at these events and these meetups and conversations you've had, what do they complain about in terms of, like, their struggles they face or their organizations not understanding pieces of it? One caveat, I'm gonna take not enough resources off the table because I think if you get a group of people from any department together, they'll complain about that. So like are there trends that come up as frustrations or things that is really top of mind for the research ops community when they all get together and chat?
Kate:It's interesting because there aren't a lot of people actually doing research ops in the world. And this is where I can be a little contentious. I have a distinction between researchers being organised, which is a wonderful thing. And we can learn from each other. We will continue to learn from researchers as ops.
Kate:You're our customers as researchers and we constantly want to be in touch and learning not just from my own organisation, but from the broader kind of community of researchers. But the conversations that we have are very different. So whereas a researcher who's being organised will talk about their favourite recruiters and getting in touch with them to get a brief, I'm talking about service desks and about scale and about staffing and about how to manage the staff and whether those staff only do recruiting or do other parts of research jobs so that they don't go crazy. And then vendor relationships and tooling and all sorts of different kinds of workflows across the organisation and how I work with our data holders and our marketing team. So that is a very, very different set of tasks and a very different conversation from a researcher saying, have got a few vendors I really like using or several of them I like using And I've gotten really good at briefing and understanding screeners.
Kate:But when I hang out with research ops people and going back to the kind of core of your question, we have like I was very happy to have lunch a couple of few months ago, I guess now with Tim Toi, who heads up the Airbnb's 10 person research ops team. And we had a fantastic lunch purely because we could sit and moan about various vendors that weren't working for us and share praise for other vendors who were really working for us. And basically like a big gossip fest, I hate to say, about who was because it's a small industry at the end of day. Who was working for us, who wasn't working for us, what kind of team structures were working within his team and the kinds of structures I was at least at that point with my team of one, me and one, hoping to form and the kinds of workflows that we're working and we're not working in recruiting participants. Those are the kind of conversations that we have as Ops people.
Kate:And we'll talk about tooling and costs around running a library, for instance. So it's very exciting conversations we would have and conversations that might blaze other people over because I would get excited about, oh my gosh, you've got someone who looks after your quantitative tooling plus your lab tooling plus your booking systems. And I'd get excited about that as a concept, which is maybe not interesting to a lot of other people. Very unique crowd.
Erin:Back to the band example, you're like fans of the world class band.
John-Henry:Absolutely.
Erin:I wonder if it's to hear you talk, it almost feels like do you feel a certain camaraderie with other ops teams, right, that are supporting other kinds of teams, DevOps, sales ops, ops ops?
Kate:Yeah, it's such a great point because I was talking to Lou Rosenfeldt about this the other day and I'm doing some collaboration with him now with the design opsresearch ops community. And I don't mind being put under design ops because I feel like there's a really great conversation to be had between first, it would be in my mind, research ops and then design ops and then DevOps and really a singular work flow that goes through all of it and how as ops we all meet up and we don't duplicate and we dovetail along the way. And that conversation hasn't started up yet. Quite a few people have said, hey, why are we not having this conversation? And I just don't have the bandwidth right now to have another kind of big conversation about something like that.
Kate:But even in Atlassian, I think we're hiring in a new design ops person and I'm very curious to start to talk to them. And it's funny because I go with my team into a restaurant and we'll be sitting there and going, hey, like, you know, with those restaurants where you can see the kitchen and we'll end up in these geeky conversations about, I wonder how they do this. And I'll look and say, do you reckon we should work in operations? Just curious about how things work and how they fit together and what the workflows are in general. So I think there's a massive crossover in looking at other ops within the organisation.
Kate:And I've been very curious just even looking at courses on regular business operations. What do these people talk about and think about? And it's all the same stuff, just that our focus and our unique talent is knowing enough about the research side of it that we don't have to be taught all the time what researchers are thinking and why they're thinking it because we have a kind of a learned knowledge around that.
John-Henry:So research ops, I think we've established, is a thing, but it's still a new thing. What is like the biggest mistake that a team should look to avoid if they were getting more serious about this going forward?
Kate:The biggest mistake is to think that you can do it as a side job. If you could, as a researcher, if you could do it on the side or as a head of research, if you could do it on the side, you would have probably done it already. And head of research has got so much work to do just in strategy and methodology and craft. The biggest mistake is to underestimate the size of and the effort that needs to go into even delivering one element of research operations. And I think that's kind of independent of scale.
Kate:Even if you've got a team of 10 researchers, still there's a fair bit of recruitment to do there. And you might be able to and this goes to your earlier question around kind of where to start. You're going to be doing a relative amount of recruitment for 10 people and you could probably dip into a couple of other things, but achieving anything of significance is going to be hard work. It's going to be difficult. Building a research panel and actually getting that going and doing a proper job of it, it's gonna take at least two or three days of your week full time, like two or three days full time, if that makes sense.
Kate:A lab is about the same thing. You know, I've built labs in the past, research spaces, and that's at least three days of my time getting it off the ground and then still needing someone to manage it and make sure it's clean and organised and the booking systems are working. That's a two or three day a week job. So all of these things, they need people power behind them and the costs shouldn't be underestimated either. So in some ways I look at it and I think I'm working in a big organisation who is investing in the space through Lisa's leadership.
Kate:And I wonder, and this is a question in my mind as opposed to statement, is research operations something that is really needed in the larger organisations, the larger teams where it can be expressed fully or can it be done in environments at a much smaller scale? Can it even be done by one person? I'm dubious because in my experience it doesn't work, But I'm very open to kind of exploring what are the various levels of research operations? What does that look like?
Erin:Yeah. Is there a tipping point? I'm sure it depends on a lot of things, but you have X number of researchers. You mentioned 10. You know, how many researchers do you need to need a research ops person?
Kate:It sounds like a weird specific number, but eight seems to be the number. When I've spoken over the years, sort of talking about half a decade now in speaking to people across the world with teams and when they feel they need some kind of support, it seems to be at six or seven, you're of feeling like you're okay kind of mucking along as a team. And then you get to eight and you're like, oh, I'm feeling a little bit of a squeeze here. And by nine or ten, you're now feeling you've got more team meetings and the demand across the organisation's more on the researchers and where do they store their stuff? And those sorts of questions become more prevalent.
Kate:So yeah, it's kind of one of those stick your finger in the air kinds of answers, eight ten feels like when those conversations become something that comes up more and more often.
Erin:So don't take the job as the tenth researcher without a research ops person.
Kate:Yeah, probably, unless you want to be carrying a lot of that weight. So I have to add to that, that for instance, with our scale and we don't have enough people working on participant recruitment at the moment to handle the sort of service we're offering right now. But what we've done is said, okay, we're using various sort of cloud recruitment tools and we allow the researchers to self-service through that. So we support them in certain ways and we've supported them in setting up the tool, in procuring the tool, in making sure there's money in the tool, but actually doing the recruitment is their job. And same with our quantitative recruiting.
Kate:Here's your vendor. I manage the relationship. I fund it, but I don't have the bandwidth to actually do the engagement with the vendor and actually doing the briefing and everything, that's over to you. So there are kind of various levels, I guess, in that sense of how you can deliver your ops. It might not be the full expression of I'm going do everything for you and you won't need to lift a finger.
Kate:It might be a little self-service at points.
John-Henry:I have one last question. As a person who does research myself, who's not a dedicated researcher, on the op side, what's that experience like when a person who does research comes to you guys? Is it easier because they're maybe less fussy and have less of a strong opinion than a true researcher? Or is it it's all over the place and you need to hold their hand a lot more? Is there a point of view on how that tends to go?
Kate:Yeah, very much. So I think one of the things with designers people who are not researchers doing research, it goes back to that old story of research is not easy and it's not just talking to people. It's a skill and it's a practise skill that needs to be constantly practised and developed over time. And that goes right from even knowing what kind of methodology you're going to use. Is it the right methodology for this particular problem?
Kate:Defining your problem and what you want to learn. Full pipeline is dependent on each part of it has very specific skills. So when someone who is not a skilled researcher and is sort of wanting to try and do something but not getting it right all the way comes along and their brief is slightly off and then their demographic is slightly off and the kinds of participants they get might not be quite right for their research because you've basically had to go with their brief. The whole kind of thing just maybe their interview skills weren't quite right. Just the whole thing doesn't end up great at the end.
Kate:So we find that all the way through the process, there's a bit more handholding and trying to figure out what does this person actually want? And hang on a second, why are they doing this research with this technique? Maybe this is an interview and not a survey or maybe this isn't a diary study. Maybe this is something else. So we've got to do much more of that thinking because we're also questioning their capability.
Kate:Whereas when a known researcher who you know knows what they're doing, when they come to you and they ask you for five of X, you those people for them and things go a lot more smooth.
John-Henry:Cool, good to know. I'm used to being told that maybe I should take a sharper look at some stuff. That sounds good.
Kate:Yeah, No, it's a Yeah, we love working with researchers. And so the dream is to basically be servicing just really, I mean, from an ops perspective, if you want an easy operations, just say, please just give me 90 researchers to work with and all of them must be amazing researchers and I'll be in heaven.
Erin:We're at an hour and we got a couple good awkward silences in, and I think we covered most of what we wanted to cover. I always like to ask Kate, is there anything that you wanna tell us, any closing words everybody should know about research ops that we didn't talk about?
Kate:This is where I'm always like, you know, the duck with the legs under the water and I'm like, what can I ask? I'm really interested in how the conversation progresses over the coming years. I suspect that it's I wonder and suspect that perhaps the conversation will be a little smaller than it was last year in 2018 because researchers are going to carry on being interested in research and craft and so they should. But it means that there's a lot less people who are truly interested in the operations conversation for operations. And I think that's very honest and fine.
Kate:But over time, it's going to be interesting to see the questions around what kind of people do you hire into operations? Because there aren't people You can't advertise and say, I want someone who's got a lot of experience in research ops because there aren't many of us out there. So it's going to be very exciting over the next year or two, three, four years or whatever, hopefully beyond that, to see how we shape up and how the conversation grows from an operations perspective and not from a craft perspective.
Erin:Well, we'll put this in the archive and make sure to check back.
Kate:That'd be great. That'd be really Thanks
Erin:for listening to Awkward Silences brought to you by User Interviews.
John-Henry:Theme music by Fragile Gang.
Erin:Editing and sound production by Kerry Boyd.
Creators and Guests
