Brad Orego [00:00:00]:
I don't think I would have been able to say yes. Like, I don't think I could with, like, do I have those answers at webflow right now? No. But am I confident I can build a team that can provide those answers? Yes. And I don't think I knew enough about how research can impact the business. I don't think I knew enough about how research can work across the entire org. And like I said before, work with marketing, work with customer success and sales as well as product and design that we all sort of know and love. Like, if I didn't have the experience and have the knowledge to say like, yeah, I can actually do that, we wouldn't be here. Like, I wouldn't be building out this function as I am.

Erin May [00:00:39]:
Hey, this is Erin May and this is Carol Guest. And this is Awkward Silences. Awkward Silences is brought to you by User Interviews, the fastest way to recruit targeted, high quality participants for any kind of research. Hello everybody and welcome back to Awkward Silences. Today we're here with Brad Arrigo. Brad is the head of research at webflow, one of our favorite products here at User Interviews. And we're really excited to be talking about how you are building the UX research team of tomorrow today. We've had a lot of experience working at a variety of companies, different stages, different verticals and different kind of research team setups and so able to bring all of that to bear into how you're thinking about building this team.

Erin May [00:01:30]:
So, so excited to have you here, Brad. Thanks for joining us.

Brad Orego [00:01:33]:
Yeah, glad to be here. Super excited. I mean, I'm always happy to talk about research and the current and the future and all that. Yeah, I think there's some cool stuff that we're working on at webflow that I've done at some previous companies as well and always happy to share.

Erin May [00:01:46]:
Awesome. We've got Carol here too.

Carol Guest [00:01:47]:
Hey everyone. Glad to be here and yeah, really excited to learn more about how you're approaching use it research as you're joining a new company.

Erin May [00:01:54]:
All right, well, to get things started again based on your years of experience in different contexts. For me, it's always fun to jump in and you get a fresh start and you've joined a company that seems to have a lot of buy in for research, which is of course always wonderful. Lots of interesting things to explore. So you're jumping in. I'm sure there's a million things you could prioritize. How do you kind of jump in and think about, all right, how am I going to build it and update my team, my practices, my strategy. What am I going to do here? How do you approach that at webflow?

Brad Orego [00:02:23]:
Anytime I'm joining any company and this is not news or rocket science, like I'm not the person that created this, but like you gotta start with the listening tour, right? You have to go around and talk to if you have an existing team that you're inheriting, chat with all of them and see what's been going well and what some of the challenges they've faced. I mean you'll get some of that from the interview, but I think people are a little bit more polite when you're interviewing versus like, okay, like now that we're on the inside, like the ink is dry, let's talk business. Go on your listening tour. I think I take a bit of a unique approach in that you'll talk to product, you'll talk to research, you'll talk to some of management. And I'm over here talking to sales and customer success and marketing. Like I cast the net very, very broad. And I do that for a few reasons. I mean one, just like philosophically, I think research should be working more broadly and I think that we'll talk about that a bit more.

Brad Orego [00:03:14]:
When you join a company, you have a unique opportunity to establish relationships and to build rapport. And you come in and you say, hey, I'm new, I want to chat and I want to know how we can work together and how I can help you. It's really good to build that network and to start to find those allies, right? Find the people that will help you. It's been three months at webflow and like I haven't really worked with our go to market teams but I'm like having these conversations with folks and saying, hey, I would love to work more closely with marketing, work more closely with customer success. Like we all have some stuff that we're focused on right now, but especially as we head into planning for next year, let's continue these conversations. So you start with the listening tour and yeah, I think you have to pay attention to why are you there? Like why the company hired you for a reason, right? So like what are you going there to solve initially and then sort of what are the longer term plans that you can start to flesh out as you get more information?

Erin May [00:04:06]:
Talking with stakeholders, that is a listening tour is sort of new into a company is always going to be important. But one of the things I've learned over the years is that it's so important for researchers, right, to just have A strong understanding of the stakeholder ecosystem.

Carol Guest [00:04:20]:
Is there anything, when you're getting into these listening tours, curious as a researcher, if there are any sort of questions or angles that you find are particularly useful to explore?

Brad Orego [00:04:28]:
It's a really good question. I mean, I think just naturally, as researchers, we pay attention to things that other people don't pay attention to. This is not necessarily related to the listening tour, but having the casting this wide net, having these ongoing sort of conversations with folks. I say, I use this phrase with my team all the time, where it's like, this isn't any official announcement, but I'm kind of reading the tea leaves here where it's like, as researchers, we're really good at looking at a lot of noisy signal and finding the through line there. So I'll be having conversations and hearing some things over in marketing and talking to some people in engineering and talking to some people on product, and then I sort of think about it. I'm like, huh? If I triangulate all the things that these people are saying, this is what I think is happening, or this is what I think we need, or this is what sort of is a priority. Even though no one's explicitly saying that. I don't necessarily have really fancy questions that I ask.

Brad Orego [00:05:19]:
It's quite literally the three questions are like, hey, help me understand you and your team and your function. How can research help you? How can I help you do your job better? And how can you help me? Right, like, it is a very straightforward for the listening tour, a very straightforward script. It's just like three questions. Let's do some intros, let's talk about our backgrounds, let's talk about some of the challenges that you have faced recently and like the things that you are working on. Let's do some brainstorming of how research might be able to answer some of those questions for you or get you access to things that you may not have access to and then sort of wrap it up with like, great, we'd love to help you. Here's some things that we're going to be working on and how can you help us and who else should we talk to at the company as well? That's a really good one. When you're a new on, like, hey, I'm new here, I've talked to these people, like, who else is on this list or should be on this list?

Erin May [00:06:06]:
So you do your listing tour, get all this data, you're starting to find some signal, then what, what do you do with it?

Brad Orego [00:06:11]:
Essentially you have to figure out, like, what's being asked of you, right? Like what is the unique ask of you? Talk to your leadership team, you talk to other folks at the company and it's like, all right, what is the thing that they're, they're asking you to do? So at webflow, one of the sort of big things that I was brought in for is like, everyone talks about research democratization and self service and all that. We're working on doing that. We're actually working on doing that in service of freeing up some of the time of our research team. Because I inherited a team of four researchers and one research Ops person. And it's like, they do great work, they're a really strong team. I get all the time people at webflow being like, oh, your researchers are so great. And I'm like, thanks. I didn't have anything to do with that.

Brad Orego [00:06:49]:
I got very lucky inheriting this team. But we're trying to free up some of their time so that they can be more proactive, they can be more forward focused and strategic. I have our executive team coming to me and saying, hey, can you help us with this three year vision? Right? Like that's a very unique charter, like for an INSIGHTS function and for a research team, like usually it's just like, cool, help people deliver a product. And now I have this wonderful executive team that's like, hey, yeah, we're thinking about this thing a year and a half from now. Like, can you help us think through that? Can you go do some research and like provide some clarity here? So you sort of distill everything that you hear and you're like, okay, this is what's being asked of us. What do we currently have? Like, what is actually the current team? What are the tools? What are the capabilities? If I'm being asked to deliver this thing, how much of that can I deliver right now? And sort of, you know, step three is then like, what, what are the gaps? Right? So to be a little bit more concrete here, as I mentioned webflow, like I inherited this really, really good team of researchers. The researchers are aligned to the product org, right? So there's like four product pillars, we have four researchers, one per pillar. That is the current structure and that's kind of how they operate.

Brad Orego [00:08:01]:
There's a lot of support, as I mentioned, a lot of like interest from senior leadership in doing research and like being more proactive. I asked our chief product officer, I was like, hey, hey Rachel, how should I divvy up my team's time between like reactive, like responding to Requests and this like proactive forward focus thing. Right now it's basically 95. 5. Like almost all the work that they do is taking in requests and answering those requests. And I was like, what's a realistic target for like Q4 and then heading into FY26 and. Cause she was like, let's try to get to 7030 in Q4. And I was like, so we're gonna go from 95.

Brad Orego [00:08:37]:
5 to 70 30? And she's like, yeah, yeah, then we can get to like 5050 next year. And I just like, okay. That's a lot of change really quickly to have that support from senior leadership and from product leadership to say, yeah, people should be like, we're doing research enablement, research democratization because people should be doing their own research. And that's a very. It's different to have a company that has that support from leadership and really just has a lot of like when people ask how things are going at webflow, I'm like, yeah, yeah, yeah. All companies say that they're customer centric and they care about their users and all that. It's usually there's like some degree of like bullshit in there, right? Like there's some degree of like, okay, you say that, but then you actually start talking to people. And that's not the case in webflow.

Brad Orego [00:09:18]:
It's probably more so than I thought it was. So like, everyone wants data, everyone wants research, everyone wants to. People are very. The level of maturity is very high. It's a very savvy company when it comes to UX and design and research. It's very well respected across the Org. It's a really interesting thing to assess and to pay attention to and to sort of contrast this with previous companies like got to zero. Like needed to quickly scale up a research practice.

Brad Orego [00:09:45]:
They had, like, I had support from product leadership. I don't know if the rest of senior leadership was really on the same page, but the sort of level of maturity across the org, like there was essentially no product or design leadership until three months or six months before I got there. Like, it was just like all of this was brand new. So getting all the PMs and designers aligned and then for me to come in and say, hey, you're all doing your own research now, that was a pretty big lift at auth0. Whereas Webflow, it's like a lot of that groundwork's already been laid for me.

Carol Guest [00:10:15]:
I was going to say this sounds like the dream. I think that many researchers would love to work at this organization. And this might be a question that's hard for you to answer, but I guess I'm curious, do you have any sense of how webflow got there? Like, how was it that they're so interested in researchers for strategic and not just product work?

Brad Orego [00:10:33]:
There's a few things. I mean it really has been part of the culture from day one. Vlad, Sergey, Bryant, the co founders have been like, okay, we're building this thing, but we really need to pay attention to what people are saying and what they want to do. And like, is part of the culture and has been part of the culture from day one. And every startup kind of says that, but that like laid the groundwork in a lot of ways. And like webflow is a very active community. We have a lot of people who are very vocal, so we pay a lot of attention to what's happening there. And I think the sort of other thing that has happened is that the company has grown a lot recently and we're sort of on this like road to IPO or this like scale up stage of company's life cycle.

Brad Orego [00:11:12]:
And there's been a lot of change in leadership. Right. Like the folks that have got us to this point have for a variety of reasons decided, you know what, like, I'd rather go back to doing stuff that's more fun for me. And like building and scaling a company maybe isn't necessarily that fun. So we have our chief product officer started at the beginning of this year. Linda, who is in the CEO role, was previously CEO but took over in like April or so. I think our cfo, I should know when Craig started, but there's just been a lot of turnover at the executive level. So you get a company that has this cultural foundational belief in empowering people to web development, superpowers, whatever our catchphrase is.

Brad Orego [00:11:51]:
But it's like that's built into the ethos. And then you bring in a new group of fairly young and fairly savvy execs who this might be certainly not their first company, but just it's a really cool company because everyone is very down to earth and is, you know, we're not getting like stodgy old execs in here. And they just care. They just care so much about the product, they care so much about the community and they're willing to think about things differently. And that's really cool.

Erin May [00:12:17]:
So you came in and there's an appetite for change, there's an appetite for quick change. You're like, oh, that's kind of quick, but let's do it. Now it's Q4. How's it going?

Brad Orego [00:12:26]:
Yeah, it's good. A lot of the first three months or so have been trying to understand like get the lay of the land, settle the team down. Right There was, as I mentioned, a lot of change, sort of at the exec level. My hiring manager who was leading the insights function which like by the way, research is part of this insights function. At webflow, which is kind of unique, we have data science, data analytics or like business intelligence, machine learning and research all sitting in an org together and that's reporting up through product. So my hiring manager was like senior director of Insights. They decided to leave the company. So I like joined and was like, okay, cool.

Brad Orego [00:13:02]:
I will be working directly with the cpo, which is great. Working with Rachel is awesome. But we had like earlier today had a meeting with the other two managers to be like, okay, let's talk about our FY26 strategy because we're starting to do that planning and we are interviewing for the backfill for a VP of Insights. But in the meantime it's like, hey, we're just going to move forward because we have to and we can't sort of wait around. So it's been a lot of that and it's been forming an opinion of what the next three years essentially look like, not only for webflow but for this insights function and for my team, my research team in particular. So this unique challenge that we have, right, where it's like we've got product teams and people sort of delivering what I characterize as like a zero to six month roadmap where it's like, okay, we've got a pretty good idea of what we're building. We don't necessarily have all the answers and we don't really know the order or what it should look like. So we need some research to support that.

Brad Orego [00:13:55]:
Right? And this is most people, when they talk about user or UX or whatever research, like they're actually talking about product research. They're not talking about market research typically they're not talking about like corporate strategy or anything. And it's like it is, how do you deliver a product roadmap in this sort of zero to six month time horizon. But then of course I have the CFO and the CPO and the CEO asking me to help figure out the long term strategy and do like we're in on the due diligence conversations when it comes to acquisitions and the build buyer partner discussion for adjacencies. We're looking to Move into. Thankfully, I have some experience working in corporate innovation and business Dev from I spent about a year with American Family Insurance in their corporate innovation team. So I'm not afraid by m and A talk. I think a lot of people would be scared by that.

Brad Orego [00:14:45]:
Yeah, we just have this unique challenge where it's like, okay, we need to answer the 0 to 6 month question and the 6 to 36 month questions and like I have difficulty traversing that time span myself, let alone like someone actually heads down doing the research. It's like, yeah, here, help us figure out how to design this interface and also should we build by our partner in this adjacent. Like that's just a very different way of thinking and it's honestly a different, not necessarily a different skill set, but a different orientation. So what we're actually trying to do right now and like the current plan is to more or less spin up to like parallel research teams. Right. We'll have the product research team aligned to the product pillars. Looks like any other research function that we all know and love. And then we'll have this kind of new idea.

Brad Orego [00:15:27]:
And I'm not the first person to come up with a strategic research team. Big companies have this, but they don't have them in the same org. So you have all of your corporate development, all of your corporate strategy that's way over in one part. And then you have your UX researchers and your product and design people over here. So it's really unique given how small the company webflow is going to put these things together. And yeah, they just are going to operate completely differently. So it's. We'll see how that works out.

Brad Orego [00:15:54]:
But the skill sets are different, the cadences are different and the deliverables obviously are going to be different.

Carol Guest [00:15:58]:
And how are you thinking about. Okay, so you have a product research team today. You said you have four researchers and then you're standing up a new strategic research function. How are you thinking about that transition? Like, is it very different people that you would hire for strategic research? I imagine that there are some people in product research who often want to do more strategic research. So how are you navigating that piece of it?

Brad Orego [00:16:18]:
Yeah, it's a really good question and I don't think we have complete clarity on it, to be honest. I don't think we necessarily know how projects traverse that space from one to the other. The current sort of line of thinking. And this is. I've had conversations with our director of product strategy, obviously with the executive team as well, that essentially the way the strategic research team is going to function is that we'll have qual researchers, right? But we'll also have market researchers who have some quant support from data science and that team's going to be responsible. So we're standing up like a voice of the customer program. They're going to manage that. And really their job is to keep track of, I don't know, 10 or 15 or so different topics that we're interested in and give us like a monthly pulse check, right? Like, okay, are things trending upward? Are things trending downward? We're going to need to assess willingness to pay over time because we're trying to figure out at that 6 to 36 month time horizon which direction we need to start to move the company.

Brad Orego [00:17:13]:
So we need to like pay attention to these signals. Say, hey, something came out of the VOC program that we weren't expecting. We're going to investigate that. Or we've been tracking this thing over time and all of a sudden we're getting more and more signal there, so we should probably start to move in that direction. And then there's sort of a formalized process to take everything that we've learned and pass that over to somebody in the product org. Right. We'll decide obviously where it makes sense based on what the topic is. But there will be this, hey, we've done enough research, we've done enough investigation, we've proven the business market sizing is a whole nother thing, right.

Brad Orego [00:17:47]:
That's going to go into that. The market researchers will say, this is what our user base thinks and this is what the entire sort of market thinks and we'll deliver this whole package over to the product team and then researchers on that side will pick it up. And of course things can go the opposite direction. Right. If we're doing some interviews, we're doing some usability tests and we keep hearing the same thing that's not on the radar. That's something that you run up to the strategic team and you say, hey, are we thinking about this? Are we tracking this yet? What's going on here? So there's an opportunity to, for some collaboration. That's why I'm really excited to have these two teams together. And I think from an individual level, like part of, if we go back to like, why are we investing in research democratization? It's to free the researchers up to be a little bit more forward thinking, to be a little bit more proactive and like thinking about the future.

Brad Orego [00:18:32]:
So they may not necessarily be doing this 36 month time horizon type of work. But if they're looking at, I've started an open question doc for the research team where it's like, hey, all of these like deep existential questions about our product and our customers and all that that we don't really have time to dig into right now. Just start writing those down. If you're having a conversation with a customer and they mention something like, put it on this list. Because as we start to free up more of your time and as we start to spin up and scale up this strategic research team, I want to be the team that is bringing these opportunities to the business that no one's really thinking about and no one's really talking about and maybe they haven't spent the time to investigate. Like that's the role that I see this research function as a whole filling and that's something that's going to be. Both the product team and the strategic team will be involved in that.

Erin May [00:19:20]:
So it sounds like the strategic research is very much a continuous sort of ongoing thing as opposed to like time for a three year plan, let's do a big sprint, right? It's the signals are coming, you're exploring them as there's time and then sharing them on a continuous basis. Is that right?

Brad Orego [00:19:35]:
That's the plan. I'm sure there will be spikes, I'm sure there will be something will come up and we'll have to do a bunch of work to learn what we can learn about that. But I want to avoid that. I want it to be more of a like long term sort of long tail approach and that as we see trends shift over time that's an indication because I really don't think most companies or teams are doing that. I don't think anyone is looking at a three year time horizon and actually like benchmarking responses and like running the same surveys and doing the same interviews or whatever it may be over time, like these longitudinal studies we certainly do in academia, but in the industry I don't think it happens a lot.

Erin May [00:20:09]:
And then for the product researcher, are you keeping that same sort of one to one with the one researcher to one area of product? Are you adding more, figuring it out?

Brad Orego [00:20:18]:
Any given product pillar has four to six individual like PMS and product teams. So having one researcher trying to span all of that is. It's a challenge, right? Like we Q4 is the first time as far as I know in webflow history that we're going to take it. Like we're going to write down all the requests that we've gotten and we will decide, okay, some of these are self serve, some of these we will own. That's a new thing. But like just tracking capacity is kind of a new concept for the research function. So we recognize that we need to add more there. If we just look at pure ratios of researcher to product or researcher to designer, like we're just not.

Brad Orego [00:20:53]:
I have four researchers and I think 22 product teams or something like that. And it's like okay, they'd be better if we had a little bit more parity there. So actually the hiring plan right now is to add essentially one sort of senior or staff level researcher in each pillar to do a lot of the strategic, sort of, no pun intended strategic, but set the direction and say, okay, these are the research studies that we're going to support and these are the methods and the audiences that we need. And then to have someone that's more sort of like junior associate sort of base level that can be a little bit more hands on, a little bit more execution focused. And both of those researchers will be coaching folks who are self serving research as well.

Erin May [00:21:31]:
Great.

Brad Orego [00:21:31]:
Yeah.

Erin May [00:21:31]:
I imagine as you do your go through the capacity and what kind of requests are coming in, you'll figure out how much can we self serve and then how much support do we need for that self serve. So is that sort of powder the people who do research support coming through the product team or research ops or how does that come together?

Brad Orego [00:21:48]:
Together it's a combination of both. You know our research ops team, I think of research ops a lot of different ways, but I don't see research ops as this support function. I think a lot of people are like oh, we have a research ops teams and they recruit and schedule interviews for us. No, we have calendly for that. Right. We have software tools that can do these things. So when I think about the role of research operations I think I like split it out into like three or four main things. We on the hiring plan somewhere maybe next year is to bring on a research practice manager whose job it is is to make sure that we have guidelines and we have standards and we are doing ongoing education and training.

Brad Orego [00:22:26]:
We are making sure that people are following the best practices. Because one mistake I certainly made at auth0 is like you can write out all the documentation you want, you can have all the like how to videos but if no one's consuming it, it doesn't actually matter. And that is a full time job if you have 20 product managers and designers. So it's 40 people plus your research team all doing research differently like it's a nightmare and like the outputs get messy, the quality goes down, the timelines get all out of whack. So like having someone who is dedicated to being like, okay, this is the process and I'm going to make sure everyone knows it and I'm going to make sure that our training and our all that's up to speed. That's one job. Having an insights librarian in research Ops who was responsible for maintaining the Insights repo and I call it not even just maintenance, but activation, right? Like how do we make sure that we don't just put a bunch of stuff in a repo and it sits there forever, but people are actually accessing this. People are weaving this into their product plans.

Brad Orego [00:23:27]:
I had a conversation earlier today of like, how do we make sure that Insights get out ahead of the roadmap and are not just like consulted at the end of like, hey, this is what we're doing, give us some data to justify it. It's like, no, no, no, no, it needs to be the other way around. We need to say here's all the data that we have, we should react to that in some way. So yeah, research ops, like some of the self serve stuff, it's going to be PMs and designers, but it's also going to be product marketers. It'll be folks we have someone from our education team is doing some self service research right now because they're like, hey, we're going to make some changes to our content and to videos and I want to understand what we need to maintain and what are things that we can sort of let go as they try to scale up more of the product education.

Erin May [00:24:11]:
Awkward interruption. This episode of awkward silences, like every episode of Awkward Silences is brought to you by User interviews.

Carol Guest [00:24:18]:
We know that finding participants for research is hard. User interviews is the fastest way to recruit targeted, high quality participants for any kind of research. We're not a testing platform. Instead we're fully focused on making sure you can get just in time Insights for your product development, business strategy, marketing and more.

Erin May [00:24:36]:
Go to userinterviews.com awkward to get your first three participants free.

Carol Guest [00:24:42]:
A lot of individual jobs. I'm wondering is your plan that these are. Is this three or four new roles? Are you imagining it starts hybrid and then splits out over time. How do you staff up a team this big? Research ops.

Brad Orego [00:24:53]:
For Research Ops in particular, yeah, I mean we have one right now. We have Heather who's done great work, has been the Only reason Research Ops person for like a year and a half or two years or so. Poor Heather is trying to do everything we need to scale up the, the one thing I like, I'm trying to get out of the community and we have not seen yet. I was like, okay, we have ratios of like PM to designer to engineer to researcher. Some people have ratios of research ops to researcher, but we don't have ratios of research ops to powder, right? Like we, I have yet to see any company say okay, this is how many research Ops staff we have and we're supporting this much self service research because like yeah, you just need more people if you're going to have, as I said, if we're going to have 50 people doing research and not all those people are trained researchers. Like there's just a certain amount of unlocking that much potential for the team comes at a cost. And if we don't want to, like I all the time talk about responsible democratization, right? Where it's like okay, democratization's gotten a bad rap because everyone thinks you just let people do whatever they want. That's not actually responsible, that's not actually going to drive the right outcomes.

Brad Orego [00:26:03]:
You need to have checkpoints. So like our self service process, it's mostly like I'm very thankful I have a team of researchers because at Auth0 it was me doing all of this. But it's like there are concrete checkpoints for every single self serve project. So you're over here like okay, write your research plan, get it reviewed, right? You're not allowed to even get into recruiting until we know we're asking the right questions to the right people and we have concrete objectives that we're trying to drive, right? So like there are checkpoints there. There's checkpoints when you get to recruit and screen if you need to run pilot studies. That's something that the research team will decide. If I look at a research Ops function that needs to support this like sophisticated process, you need a program manager or it's like some sort of, you need someone managing the operations strategy, right? So like we're hiring a research program manager, eventually that person would move into managing the team director of research ops. You know, sort of down the road we need people that are like more technical.

Brad Orego [00:27:01]:
What we're calling a research Ops specialist who's going to handle all of our automation, all of our tooling, making sure that things are talking to each other. As I mentioned, the insights librarian, the practice manager. Like I think research Ops is at a point where the field has matured and we can start to sort of build these subspecialties. Whereas if we go back a few years, it's like Research Ops was barely even a title, was barely even a thing. It's really exciting to see this movement and I try not to think of myself as a thought leader, but I think I do kind of push the envelope in a lot of ways. If you want to have a well run research practice and a well run democratized or self serve research practice, you just need more people.

Erin May [00:27:43]:
Yeah, it does feel like that ops role gets more important as you have more people who aren't researchers doing research. For sure. You mentioned a little bit about tooling and AI and I wanted to talk about that a little bit more. I don't think you could think about the research team of tomorrow without having some kind of point of view on AI and tooling and how those things are enabling democratization. So where are you with that, Brad? How are you using AI? How do you want to use AI? Do you want to. Is there a mandate to use AI? Have you purchased AI native tools? How are you thinking about it?

Brad Orego [00:28:15]:
We just adjusted the top level corporate goals or whatever. Like what are totally blanking on the term. I don't know if we use okrs or whatever, but whatever. Like our top level things cut down half of them because we were trying to do too much. But one of those is like investigate AI. Like for the second half of this year it's like we should be looking into AI, we should be building AI solutions into our product. We're not just like throwing AI at everything but it at a corporate level is something that we're looking into and like the engineering team has started to use more like code copiloty things. So it's top of mind for webflow.

Brad Orego [00:28:50]:
It's not something that we've made a lot of use of thus far. So we, I mean we have Sprig, we have Dovetail, we have some tools that have some AI stuff built into it and we're using those with some trepidation. As much as I love Dovetail and like the Ask Dovetail feature they just launched, it's like sometimes it's just not, it just doesn't give you the. I'm not confident in a non researcher interpreting those results is what it comes down to. At the end of the day I trust my team, I trust anyone that is a practiced and trained researcher. But I'm always thinking about how can we use these tools to amplify democratization and amplify self service. So if I'm not confident in someone who's not a researcher doing this, then it just doesn't pass the bar for me. So a lot of our use of tooling, it's not necessarily AI focused right now.

Brad Orego [00:29:37]:
We do a lot of automation. I just set up our entire self service process runs through jira. And when you move your ticket from one step to the next, it spits out all the subtasks with links to the documents that you need to read or fill out or whatever. There's a lot of little things that you can do if you have access to a zapier or an if this, then that or workado or whatever. Like we, Rodrigo, our ops person at auth0 did magic with workado. We had so much cool automation there that automation 1 decreases the amount of like the opportunity for someone to do something wrong, right? Like if it's a process, and it is, you will have this task created for you. Or here's a template that is filled in with your stuff. Like automagically, you're decreasing the amount of risk there, but it's also of course making everyone more efficient and you're getting more time for your OPS people, more time for your researchers and your powder.

Brad Orego [00:30:29]:
So automation I'm like always a big fan of. That's why I've carved out a role specifically for tooling and automation. And when I think about looking to the future and things that I hope in very short order we're going to start to. It's a good reminder for me as I like work on our FY26 plan. I have this idea part of the reason I came to webflow. There's a lot of different reasons, but I want to experiment with AI empowered democratization where if we can train an LLM, like train a little bot, a little assistant to, as you're writing a research plan, say, hey, that's a leading question, maybe you should rephrase it this way or to recognize something like ordering effects to say, well, if you ask this question up here, then you ask this question down here. Maybe you should switch the order of this. Or just like pull in our best practices and our style guide and say, hey, we typically use this language instead of that language when we're talking about a feature or specific set of users or whatever it may be.

Brad Orego [00:31:24]:
So the more I think about how can we leverage AI not only as researchers but as research ops people and in sort of this democratized world, can we get to a first draft that is way better than it would be without it because that's going to decrease the cycle time and how much back and forth you have to do with your research partner. If you. I've reviewed plenty of research plans and I deeply appreciate everyone trying their best and like putting something together, but there are times where I just have to like throw it out and start over. Right. Because they're not researchers. They don't know. I'm not expecting them to know what we know. But if we can avoid that, if we can avoid some of those cycles with an AI assistant, I think that's super cool.

Brad Orego [00:32:04]:
I think I've told this to Basil and some other folks of like, can I just get a research tool that does all of the like participant selection, scheduling and rewards delivery for me that I can just say, hey, I want to talk to 20 enterprise customers with less than 500 customers in the next two weeks. And it looks through our participant pool, reaches out to people, looks at my calendar. If I just could do that. Because the biggest point of friction in any research process, in any company I've ever worked with or talked to is recruiting. Recruiting is the biggest slowdown. It's the biggest point of friction. It gets worse when you're doing enterprise B2B SaaS, all of that. Like there are complexities there, but we've kind of solved Scheduling and X AI launched a scheduling AI assistant 10 years ago.

Brad Orego [00:32:57]:
So if someone can figure that out, like shut up and take my money.

Erin May [00:33:02]:
Yeah.

Carol Guest [00:33:02]:
When you think about, I'm curious when you think about some of the enablement that you're doing with the research practice manager and some of the automation, how much of what you're trading people who do research on is sort of guidelines specific to our company. Like this is how we do recruitment here and this is maybe the language we use about our products versus sort of more general. I'm mostly curious, like is there an opportunity for some more broader education on powder that's not specific to a company or is it really pretty targeted at how you do research at webflow?

Brad Orego [00:33:32]:
It's really not. So there's definitely like if anyone is particularly motivated, any individual could go online and take a course and it'll get them 80 or 90% of the way there. Then there's just some nuances to our process, our toolkit, our how we handle recruiting, what our philosophy is on different research methods and how we structure questions. But people can go out and like learn this. My experience is that most people don't. Right. Like they already have a full time job if they're not that interested in doing research. Like, if they understand that it's the means to an end, right.

Brad Orego [00:34:04]:
That they're like, I have questions about the design I'm building or the product I'm trying to launch or whatever it may be, they understand that research is important to that and that it's faster and easier for them to do it themselves than to try to get our time and attention. If folks wanted to be researchers, they would be researchers, and they don't. But it is. What's really interesting about this is I mentioned that person from the learning team, from the education team. They were like, oh, yeah, I've been part of research democratization at previous companies. So they, before even talking to me, got on our Confluence page, read all of our documentation, took our research plan template and filled it out. Once you get a little bit of familiarity and you kind of understand the rough shape of a research process, that's skills that are transferable and every company is going to do it a little bit differently. But I could sit down and list out the six phases of research and it's going to be 98% the same across different companies.

Erin May [00:35:04]:
So we've talked a lot about how you sort of did your listening tour. You learned the current state, the desired future state, building out this new org structure, processes, tooling. I'm curious how that's different than maybe what you would have done two years ago, three years ago, five years ago, both with what's changed, with what you've learned as a researcher and just the environment of research in general.

Brad Orego [00:35:29]:
Yeah, I think there's, there's a few things here. There's a topic that comes up often of, like, what's the ROI of research? Right? Like, how do we prove the value of research? And I. There's two ways I typically respond to that. The first way is that who's asking you? Right? Like, is someone actually asking you to prove the value of research? And if they are, where are they coming from? Like, what part of the business are they in? Because you need to have a different conversation based on who's asking. If it's someone from finance, great, let's pull some numbers. Let's look at, you know, the average time it takes to execute designer or engineering with or without research. Right? Like you can calculate ROI if it's someone from, you know, product, maybe you talk more about the impact of the roadmap, but if no one's asking you to prove the value, then don't do it like it's a waste of time. People are like, no, we need to justify our existence.

Brad Orego [00:36:22]:
Like, if no one's asking you to prove that you're a worthwhile investment, they already believe you. So it's kind of a waste of time. That being said, you can bake this into the work that you do. And you do this by, like, making sure you're doing the right research. Right. Okay, so how do you do that? This is where this, like, how we're tying this back into this question is that I've got a set of four questions now that every single request that comes in, if you can't answer these four questions, we're not doing the research. And we're not doing it because we have other stuff we could be doing. So the first question is like, okay, why are we doing this? What is the context? Like, what's the background? Where is this request coming from? The second thing is, what are we hoping to learn? What are the research objectives? Like, how do we sort of structure the research that we're doing? The third question is, what will we do with the results? So, like, what changes are you going to make to your strategy or your design or your product or whatever? And the fourth question is, how does that impact the business? What KPI are we trying to influence that this change will make? And if you can't answer those four questions, we're not doing the research.

Brad Orego [00:37:31]:
I didn't have those four questions five years ago. Right. If we go back to, like, you know, 2018, I quite literally went through my notes preparing for this chat and I was like. I was like, I know it's evolved over time. Three years ago, I guess, four years ago at this point, when I joined Auth0, it's three questions. That last question of what KPI does it impact? It's not there. I just hadn't thought of it yet. I hadn't matured enough as a leader and as a researcher.

Brad Orego [00:37:55]:
So it was just like, why are we doing this? What are we hoping to learn? What will we do with the results? And that gets you most of the way there. I will probably have a fifth question in another five years. And if we go back further to 2018, 2019, it's only two questions, why are we doing this? And what are we hoping to learn? So as I have seen, the research that I've done, maybe not have the desired impact and not get the sort of gravity that I think it should have. Like, I've started to add more and more questions and we've started to say no to things or force people to clarify. So that we can make sure we're doing the right research and we're delivering the right results and insights like that is one thing that's certainly changed. I mean, I think Research Ops maturity. I was like the hundredth person in the Research Ops slack community way back in 2018. There's now like 14,000 people in that space.

Brad Orego [00:38:42]:
Right? So just like Research Ops over the past six years has completely blown up. Not that it didn't exist before, but we didn't have names for it, we didn't have titles. I still reference the eight pillars of Research Ops and that's five years old or something at this point. Like it's such a great work of analysis and synthesis and all that. If we go back two, three years and I at Auth0 joined in 2020, built an ops first research practice. We didn't have an insights librarian, we didn't have a practice manager. Right. Like we had Research Ops specialists and that was it.

Brad Orego [00:39:19]:
And they kind of did a little bit of everything. And certainly one person would focus more on knowledge management and one person would focus more on automation. There were some subspecialties, but we didn't. Like, I couldn't justify dedicating someone solely to that because, you know, the field just hadn't matured a lot. And I think the other thing is, if I, if I had joined a company two, three, five years ago and someone from the executive team came to me and said, hey, can you help us with this three year vision? Like, we're trying to figure out if we should acquire this company or if we should partner or build it ourselves. Like, can you help me, you know, provide? Like, I don't think I would have been able to say yes. Like, I don't think I could with, like, do I have those answers at Webflow right now? No. But am I confident I can build a team that can provide those answers? Yes.

Brad Orego [00:40:08]:
And I don't think I knew enough about how research can impact the business. I don't think I knew enough about how research can work across the entire org. And like I said before, work with marketing, work with customer success and sales as well as product and design that we all sort of know and love. Like, if I didn't have the experience and have the knowledge to say, like, yeah, I can actually do that, we wouldn't be here. I wouldn't be building out this function as I am. Because I would have been like, no, sorry, I don't think that's something that we should really do. And the opportunity wouldn't be There it's.

Erin May [00:40:41]:
Cool to see your evolution as a research leader and the research maturity of the entire ecosystem parallel. And you can really see everything you just talked about in your plans that you're building at webflow with the emphasis on reops, with the emphasis on long term strategic thinking and really asking that question, should we be doing this research at all? I think when you talked about the obsession almost with ROI at times of research, it's a dollar in, $3 out, and the relative value and the prioritization, which sometimes you have to do. But sometimes the best question is, should we do this research at all? Which is always a great place to start. So that's a great summary. Should we move to our rapid fire section? All right. Okay. Brad, two or three resources that you like to recommend to folks?

Brad Orego [00:41:27]:
Yeah, so the first one I mentioned obviously is the a pillars of research. Really anything coming out of the research ops community. A lot of really, really talented people there. A few books have come out of that community as well. Obviously. Kate Tausi, I have not read it yet, but it is on my desk. Kate's recent book, Jacob Burghard's working on a book as well. So there's like some cool stuff that the research house community has been around long enough, so definitely check those out.

Brad Orego [00:41:51]:
And big fan of learners as well. I think all the content that they produce, all the stuff that they're doing. Yeah, really, really good resources there.

Erin May [00:41:59]:
Yeah. Great. We'll link all those in the show. Notes for folks listening.

Carol Guest [00:42:03]:
Yes. Any favorite research questions, favorite interview questions?

Brad Orego [00:42:07]:
I don't know if I have any favorites. I mean the gist of it is when I describe to like people who are non researchers or maybe not even tech, what we do. Like, all right, our job is to figure out what sucks about something and tell someone else about it. Like tell someone else to do something. And the way that you do that, like this is the easiest. Like this is discovery research in a nutshell. Hey, talk to me about this topic. Talk to me about this thing.

Brad Orego [00:42:33]:
How do you solve this problem currently and what sucks about your solution? That's about it. Like that's all you need to do is like help me understand this problem space. And then what problems do you have with the way that it works right now? Because then you'll get all these opportunities that you need to go investigate and figure out which one you want to build. But at the end of the day, like that's two questions. That's all you need. There's nuance to it, obviously, but that's it?

Erin May [00:42:55]:
Yeah. I love that people like to talk about. Well, depends on how sensitive the topic is of course. But you can get people talking if you ask them what sucks. Oh yeah, yeah, that's a good one. Where can folks follow you on the Internet or otherwise?

Brad Orego [00:43:09]:
Yeah, LinkedIn. I don't post super often there, but usually when events or things are happening. Always happy to connect with folks. I update my own website, bradygo.com with all of my like talks, all of my articles and whatnot. I update that every now and then. I don't have a newsletter. I'm really. I've had several people tell me I should like you should put a substack or something together.

Brad Orego [00:43:30]:
I'm technically writing a book as well, though I've been dragging my feet on that for a while. So I write for a great question as well. So like you'll see my face pop up on their blog every now and then. But yeah, just find me on LinkedIn.

Erin May [00:43:43]:
Awesome. All right. Well Brad, great to finally get you on our podcast. Thanks for joining us. Really appreciate it.

Brad Orego [00:43:49]:
Yeah, always happy to chat. It took us a while to get here, but happy to be here. Happy to share.

Carol Guest [00:43:54]:
Great having you.

Erin May [00:44:01]:
Thanks for listening to Awkward Silences brought to you by User Interviews theme music by Fragile Gang hi there Awkward Silences listener. Thanks for listening. If you like what you heard, we always appreciate a rating or review on your podcast app of choice.

Carol Guest [00:44:26]:
We'd also love hear from you with feedback, guest topics or ideas so that we can improve your podcast listening experience. We're running a quick survey so you can share your thoughts on what you like about the show, which episodes you like best, which subjects you'd like to hear more about, which stuff you're sick of, and more just about you, the fans that have kept us on the air for the past five years.

Erin May [00:44:45]:
We know surveys usually suck. See episode 21 with Erica hall for more on that. But this one's quick and useful, we promise. Thanks for helping us make this the best podcast it can be. You can find the survey link in the episode description of any episode or head on over to userinterviews.com awkward survey.