Erin May: This is Erin May.

JH Forster: I'm John Henry Forster and this is Awkward Silences.

Erin: Silences.

Erin: Hey, everybody. Welcome back to Awkward Silences. We are here kicking off the new year, it'll be 2019 when you listen to this, with Maggie Crowley. She is the Director of Product Management at Drift, conversational marketing. We are so happy you're here, Maggie. Thanks for joining us.

Maggie Crowley: Thanks. I'm super excited to be here.

JH: Welcome. Welcome.

Erin: We wanted to talk to Maggie about doing research in the context of product in the context of working at Drift. Drift, as you probably know, who are listening, is a company kind of really built on human connection and being close to customers and helping their customers be close to customers. They, as a company, want to be close to customers and user research is about being close to customers, so a great guest we have with us today. Tell us about that. Why is user research important to product, and what does that look like, and what does it look like at Drift?

Maggie: Sure. For me, it's hard to separate out the concept of research from what a product person does all day long because I think about, as a PM, one of your jobs is to understand the problem that you're trying to solve for a customer. Fundamentally, in order to understand a problem, you're doing research in some way, shape, or form. To me, research is really about, how can I better understand my customer and how can I better understand what they're trying to accomplish in their job?

Maggie: Specifically, at Drift, we're all about conversations and scaling conversations, and so we are ... We have the side benefit of being a chat platform, so we're constantly talking to customers through our own software. But, in general, because we're always trying to have conversation in real time, we're ... User research is sort of part of the conversation we're having with customers, so I think a good example of how that plays out in reality is we have some shared Slack channels with some of our biggest customers.

JH: Oh, cool.

Maggie: Yeah. What that does is it allows you to have ... You build this real person-to-person connection with a set of customers and then they're sort of with you all along your product journey. Because you can talk to them all the time, you get to know them. You get to know what problems they're having. You know when all the sudden they have a big quarterly review coming up because the Slack channel starts firing and they're like, "I need that number. How can you help me do this thing?" Then you start to get a feel for who they are, and I think, for me, that's part of that research process and part of how conversations contribute to it.

Erin: I love that...

JH: That's cool.

Erin:... Slack is a way of doing kind of informal longitudinal studies.

Maggie: Yeah, it's amazing because I think about ... For me, I'm trying to find out, what outcome is my customer trying to get for themselves and how do I help them get there? Being able to hear them constantly talking about their job is a really useful way to get that because it's sort of hard to figure that out in a very specific setting.

JH: The composite of a user is something that kind of builds up over time. Right? You add on from here and then if you have this kind of constant stream you can kind of put it together and have a better of view of what different personas look like and all of that.

Maggie: Right.

JH: One thing I always think about with research as a product person is ideas come from everywhere. Right? Some ideas come from the top and you can ... There's snide comments about those are always the bad ideas or whatever, or they're from sales, or from users, or from marketing, or from your own product team, or from the data, right? I think everyone comes at them, consciously or not, with a different bias depending on where the idea came from. Then research is really that layer that kind of is the equalizer of, let's try to actually pull our bias out of who came up with the idea and instead get to the baseline of, do people care about this?

Maggie: Yeah, I totally agree. I think that we have the benefit of having really experienced founders who do have a very strong opinion about product, so we do often have that founder idea come through. But, I think, again, what's interesting is that pull four or five quotes from direct customers on a certain topic and that's always going to win.

Erin: You talked about how you're working to be very outcomes focused and how you're pushing building products and how research is a tool to do that. Can you tell us a little bit more about what that actually looks like?

Maggie: Yeah. I think, again, I'll just give a specific example because I think that's going to make it easier for the audience to understand. For me, the products that I work on at Drift are focused on marketers and helping marketers be more successful. When I think about outcomes, the thing I'm thinking about is, what is my marketer judged on in their job? They're judged on bringing more leads in the door and qualifying them and getting them to their sales team. For me-

Erin: You know that from talking to marketers? Is that how you arrived at that?

Maggie: Yeah.

Erin: Yeah, right.

Maggie: I mean, you guys probably have a marketing team. We have a marketing team. I just go and talk to them, and I know, what does their monthly review deck look like? What metric are they on the hook for in their business? It's bringing business in the door, so if that's who the person I care about what they're doing in their life, how can what we're building impact that number? Right? Because rather than saying, "Oh, my marketer has a problem with X," how can I help my marketer achieve their goal in their job? That's always going to be more successful for us. In terms of building a product. When I think about outcomes, that's the type of outcome I'm trying to figure out how to get to because I think if you can help your customer achieve what they're trying to achieve in their own life, they're going to love your product.

Erin: Then are you using research primarily to kind of validate the solution that gets to that outcome, or to prioritize the outcomes that matter the most, or the context in which outcomes matter, or how does it kind of play together?

Maggie: Yeah, I think it's really all of the above because ... It also depends on the type of research, right? I think, for us, and for me in particular, I'm, at any given time, doing sort of generative research of just, how can I understand who my customer is better and what they care about? Maybe we're doing specific ... I think Pluralsight calls it customer confirmation testing. Figuring out specifically whether the thing that we want to build is going to achieve the outcome that we wanted that specific feature to have. I think there's a range of testing and research you might want to go through for a given product. All of those different pieces sort of ladder up into prioritization, which I think that's a whole other podcast in itself. How to pick a feature, I think there's probably a million different ways to do that.

JH: Yeah, yeah. For sure. Just a side question.

Maggie: Yeah.

JH: Do you guys embrace the meta-ness of your product and company in the sense of you have a marketing team, you have a sales team that I'm sure gets a lot of benefit out of your tool?

Maggie: Oh, yeah.

JH: Because we have some of that as well. It's like, "We do research too and we use our own stuff."

Maggie: Yeah. Having now been in a company like that, as a product person, it will be really hard, if ever, to go to a place where that's not the case because it makes your life so much easier because you have that first ring of dogfooding that's your own company. But I also think, on the other hand, if your own company isn't using your feature or isn't seeing success with your feature then that can also be a little challenging loophole that you have to think about.

JH: Yeah, yeah. It's a red flag. The question I was going to ask is, do you think with that dynamic, the whole quote of, "You are not your user," does it make it more important to go out and make sure you're getting outside perspectives or less important because you have some internal crutches to lean on?

Maggie: I think it makes it more important because when it's your own products the team gets really used to using it the way that it is and they're less able to identify problems with it. I think you start to hear things like, "Oh, yeah. How you do that is X and we have a workaround that's Y," and then you start to just ... You close off, I think, opportunities that you might otherwise see if you're just talking to a customer who doesn't have as much built-in understanding because your team picks up some of the technical challenges or they know, "Oh, that piece of code is really old and I know it's really hard for the team to fix, so I focus over here." I think that part can sneak in and make it a little bit dangerous to trust just your own use case.

JH: Yeah, totally.

Erin: Now I'm thinking about all the times of whether as a customer internally, it's like, "I'm a user. I'm having this issue," and then you get the product line, "Well, that's really hard because the database is structured this way." It's like, "Tell it to someone who cares. Fix it."

Maggie: Right. Yeah. My customer does not care at all that we have a weird database. That doesn't matter to them. They just want to use the thing.

JH: Yeah, yeah. Do you guys use ... You mentioned the Slack channel is a valuable place to kind of pick up what these people are thinking about day-to-day and what matters to them in their role. Will you use that ... I would, I guess, typically call that kind of passive insights. You're just kind of collecting stuff as chatter happens. Will you also use that channel for active research and going out and being like, "Hey, will anybody talk to me?" or send out a screenshot and see what people think?

Maggie: Yeah, absolutely. I think, A, we can listen in on the channels. I will say also that there is a limit to how many channels you can be in, obviously, and there is a risk associated with being in the channel because then you also have to be there for your customer and they might have needs of you that you might not want to be providing all the time. It's not always amazing. I think it's a great tool for recruiting for a more formal user test. Because I've always felt like that's one of the, as I'm sure you guys know because you sell for this, one of the hardest problems of doing research is getting the people to do the research. Being able to have that channel and that direct line to someone makes it so much easier because then you cut out so much of the time that you have to spend waiting for someone to give you feedback.

Maggie: We very often will do things like, "Hey, customer, we're thinking about this problem. Does anyone want to jump on the phone in the next couple of days?" Usually, someone will raise their hand and say, "Sure. I'd love to." Or we'll throw in an envision link or we'll throw in a picture, and we'll just say, "Give us your quick thoughts on this." That's a good way to kind of, A, indicate where we need to be more testing, but, B, get some quick feedback.

Erin: How is that structured in terms of who has access to those Slack channels or how does that information get shared with people who might benefit from those insights?

Maggie: Yeah, it's something that our customer success team does for some of their clients. If it's a really important client or a certain tier or whatever, I don't know how they do this, but if they have one set up and they think that's an easy way for them to communicate usually a product member might get tagged in because they're having a bug or they're curious about how a feature works. Then we just kind of stick around and then depending on our relationship with that customer, we can pop in whenever is appropriate. Again, I think it's ... I'm not sure we have a very formal sense of when we should and should not do that, but it's just a channel that we have that I've found to be really useful.

JH: How do you kind of balance the scrappiness of something like that and the immediacy and the ease of use against the approaches that are a little bit more formal and deliberate about making sure you get coverage from all your personas or don't hit the same customer too many times so they don't overrate the sample? I have a real bias towards done is better than perfect, so I kind of have a soft spot for the scrappiness. I'd rather have a conversation than not because the overhead of doing it the other way was too much. I do think there is a real ... There's a real challenge there of a bias of the loudest customers are always the ones who are willing to hop on a call and so I only get feedback from them and I'm actually missing stuff. Is that something you guys think about as a team?

Maggie: Yeah, we do, and I'm not sure we have perfectly solved that problem. Again, I think I agree that we would rather get feedback full stop than spin our wheels trying to make sure that we have the perfect level of feedback from the right mix of personas, like you mentioned. It's just really hard and time-consuming to do that. I think if you have a good understanding of your customer and you have good intuition you don't always need that.

Maggie: I think, again, it would really depend, for me, on the type of feature that you're working on, right? If it's a really complex, new, high-risk thing, I would really want to lean into making sure we got a better selection of feedback versus something that we know is going to be the right thing and we just need some quick feedback. Again, we just hired or just started doing more formal user research as a way to balance that ad hoc feedback that we're getting. We started as a test because, again, it was sort of new for us, and we're really seeing a ton of value in having this more regular feedback, more formal testing to balance out the quick stuff that we usually do.

Erin: What does formal testing mean? What's that look like?

Maggie: Yeah, so I think we have biweekly user tests that we're doing with a user researcher in our office. Ideally, when something comes in and she runs them through a structured test, and it's sort of ... If you have a design ready or if a feature is in QA or you have something ready, she can get it on her list and she'll go through and try to understand the questions that you have about whatever feature it is that you're interested in testing.

Maggie: Again, it's super new. We've only been doing for, I guess when this podcast goes out, a month or two, so we're still trying to figure out the best way to digest the learnings from those sessions. Just having those videos ... We're sharing video clips out to the teams. Having those videos and having someone tag, "Oh, at minute 25 they talk about this topic," has been so useful for all of us and so much more efficient.

JH: I mentioned before of where different ideas come from. Within our team, we all have our pet ideas. One of mine is definitely recurring sessions of, like you mentioned, every other week let's just make sure we always talk to a set user and then as the sessions approach we'll figure out what to use them for. Do the opposite of the don't at me. Like, "If you're a user researcher please at me and tell me if you like the work that way or not because I would love to get more feedback on if that's helpful for other teams." But it makes a ton of sense to me and it's cool that you guys are having some success with it.

Maggie: Yeah, I think I should've spoken to our head of design. I think one of the things that's amazing about it is that if we just know that we always have that test then we can work around it as a product team, but also, it's so much easier for our user researcher, Freya, to recruit for those tests, right? Because she always knows that this is this cadence that we want to do them on and then she can come up with her own process for doing that. Then we always know that it's going to happen, so it just makes it much more efficient for, I think, everyone on the team.

JH: Yeah, yeah. The fixed costs are getting atomized, right?

Maggie: Yeah.

JH: Because you...

Maggie: Yeah, exactly.

JH: Yeah.

Erin: Freya sits on the design team? Design sits on the product team? Is that ...

Maggie: At Drift, when we talk about the product team it's product, engineering, and design all together. There is a design org, and engineering org, and a product management, I guess, org, and we're all sort of equals.

Erin: Cool.

JH: You mentioned risk at one point earlier in the conversation. Do you guys factor that into how much research you're going to do? If it's a feature ... I tie a lot when I think about risk to how expensive it is to build. If it's something that maybe it might flop but we can do it pretty cheaply, I'm a little bit more bullish. I'm like. "We can try it for real and if we have to throw that code out or eat that investment maybe that's okay." In other cases, if the build cost or the uncertainty is high enough the diligence on the research pays back much more. Is that something you guys think about in terms of how much you will research a given idea or ...

Maggie: Yeah. We have a similar process, although we sort of frame it differently. The way that we think about it is sort of we commit to shipping a new feature basically every single month and that means it sort of rotates through the different teams sort of who's up for the big ship of the month. We think about those as sort of customer launches and then market launches. Customer launches would be follow-ups on features that we have or big, new improvements, V2s of things that we know are just going to continuously improve a feature. Then we might have some market leading launches that we think are going to be big, new things that are going to drive our business forward sort of step change wise versus just a follow-up feature.

Maggie: For those bets, we do a lot more research for those ones, obviously, because we're still ... It's stuff that we would want to understand more about, we're not as familiar with in the first place. I think as part of that, we do a lot more research, but we also have a process of breaking those things down into very small chunks so that even when we're shipping the first piece of that it's still super scrappy and something we could throw out.

JH: Yeah, so...

Maggie: Yeah?

JH: How did you guys land on the monthly launch process? We talked to ... Not for the podcast, but in a different conversation with Hiten Shah a few months ago, and he mentioned something very similar of loving that approach. Just curious how you guys landed on that as the way to do it.

Maggie: Yeah, I think he's actually an advisor for Drift and he knows our founders David and Elias so I'm actually not sure how we landed on it. That happened ... I've been at Drift for about a year and that was in place by the time I got here, so I think we just believe in making sure that you can always ... Let me put it a different way. I think you can always keep your product moving forward if you can commit to something like dates. Committing to dates is really important and I think a lot of product teams struggle with getting features out the door. The way that we've figured out how to ship products is such that we focus on dates and then we work backwards, and so that's what allows us to hit those monthly cadences. Again, it requires you to be really ruthless in how you approach what scope has to be involved in a feature, if that makes sense.

JH: Mm-hmm (affirmative). Yeah, absolutely.

Erin: How often does research not make it to the cut or is there always because you're so scrappy and kind of minimum viable research is something always happening?

Maggie: Mm-hmm (affirmative). I don't think we have done anything with research in some fashion. I don't know if there's ever been ... If there has been, I would guess they've been the features that have failed. I would imagine the ones that didn't hold water and didn't stick around for very long. The pet features I feel like never really make it. Yeah, we try really hard. Again, we have sort of an unfair advantage in having a chat tool, that it's so easy to talk to customers that it would be sort of wild to me that we wouldn't speak to at least a handful before something went out the door.

Erin: Yeah. Rough estimate, how many customers have you talked to since you've been at Drift personally?

Maggie: Oh, my gosh. I'll give you a weekly answer because I talk to two to three customers a week probably, if not more. Over the course of the year, math, whatever, a lot.

JH: Yeah, that's awesome.

Maggie: Yeah.

Erin: What's been your favorite or most interesting part of that, doing so much customer talking so regularly?

Maggie: That's a good question. I think, for me, it's the way that we approach our customers. Again, being conversational and readily available means that our conversations with our customers are much more candid than I think they had been for me in the past at different companies. The customers really know who I am. They know what my email address is. They know how to find me on Twitter. They know they can talk to me all the time, and so you can hide behind some of the more standard product management language around whether or not you're going to do a feature.

Maggie: I find that I have to be much more candid and honest about what we're doing and why, and that's been the biggest change because you talk to the same customers over and over again and you form relationships with them. Then all the sudden so-and-so, John, is going to call you and say, "Hey, remember we had that conversation. Why didn't you build that feature the way I wanted you to?" Then you're sort of ... You owe it to that customer to explain yourself, so, I think, for me just being open and figuring out how to bring your customers along on the research journey that you're on has been the biggest change.

JH: It feels like you guys have a very effective and progressive marketing team. Do they participate in the research as well to kind of know how people are talking about this or do you just kind of share what you learn after the fact or does that kind of fit in anywhere?

Maggie: Yeah, they're incredible. Working with a marketing team like that just make our lives and our jobs way easier because they're so ... They hype customers up and so customers are excited to be part of the Drift community and then they want to be involved in all of this, which also makes user research a lot easier. That's one thing that they do. They get our customers super excited about being part of the brand, which is awesome. Then we have a product marketing team that's also along. They sit with us, so right around where my desk is we have Cody and he sits with me and he's listening in to all these conversations, and so they're part of the product building process. I think sort of organically we're all starting to use the same language about a certain feature.

Erin: When you say you use the same language, is that ripped from the customers? You're using the language that they're using or what's that like?

Maggie: I try to. I think my feeling is that the best way to describe a feature is in the words of a customer. If they're describing a problem in a certain way, that's how I want to describe it. If they name a metric a certain way or if they describe a page a certain way, I just want to mirror that language because then there's no cognitive overload. They know exactly what it does. Their expectations are properly set. I think that's really hard to do a lot of the time, especially if it's a feature and you have some sort of internal slang for it, but I think the more we as product people can do that just the better everyone's life is going to get.

JH: To switch gears a little, I feel like product is this weird mix of different disciplines of you need a little engineering but you're not an engineer. You need a little design, but you're not a designer. I think you also need a little research, but you're not a researcher. At least for me, when I was starting out my product career, there was definitely the imposter syndrome of you really want to make it seem like you know what you're talking about and being persuasive and kind of championing your ideas. I think that's a little in conflict with the admitting we don't know and I need to talk to people to learn what's right and this and that. Do you have advice for people who are maybe earlier in product and how they should approach research or things they should be aware of?

Maggie: Yeah. It's an interesting question because, to me, the more you can admit that you don't know the more powerful your ideas can be. Because when you understand what you don't know then you know what to research, then you know where to focus your metrics and your quantitative analysis and who to talk to. Then when you do come to a resolution or an insight you have all the backup research that will help your team understand why you made the call that you did.

Maggie: To me, I think a superpower is actually saying, "Hey, I'm not technical. Help me understand what you're doing in words that make sense to me. Hey, I'm not a designer. Help me understand why this design is going to help the user accomplish X or why this feature is actually going to move this business metric," because that's the magic of the project manager is the person who's sitting in between all those pieces of the business and saying. "How is this going to get us to wherever we're going?" If you are too knowledgeable in an area or if you don't know what your blind spots are then I think you're much more likely to miss something along the way.

JH: Yeah, the transparency, I think is very underrated. Especially just in your ideation. I was talking with somebody else earlier this week and it's like if you are only bringing your final, fully-cooked ideas to even internal teams and really champing them, then over time it looks like you were very opinionated and you always have to have your way because you're always showing them your best idea that you've thought a long time. If you're bringing them along throughout the process, where you're bringing up ideas, you're killing ideas, you're bringing up ... It shows that you're iterating and you're learning and it's not just like you think you know everything. I think people sometimes try to keep that part hidden and then just present the final product and it's like, "No, no, no, no. Get more people involved throughout. It'll help trust and everything else."

Maggie: Yeah, I 100% agree. We have a thing that we call story time that we do. That's at the beginning of a project when the PM has sat down and done some research and has defined a job to be done. There's a framework that we use, which I'm sure you've heard. They define a job to be done and pull some information together, and they write a one pager. Then the full team, so design, engineering, product, and whoever else is interested or has specific knowledge about the thing get in a room together and then the goal of that meeting is to understand the problem and come up with research questions.

Maggie: That's the start of every single feature that we build. You have to do a story time and that means that every single person on the team is involved in uncovering what we don't know and then figuring out how we can answer those questions. That's part of how we make sure that everyone is involved in research and everyone is approaching it like, "Hey, we don't really know what the solution is. Let's make sure we understand all the different pieces before we move forward."

Erin: This has been great. I feel like I have a decent picture of what research looks like at Drift and how it fits into shipping product so much. One thing we didn't talk about and I'm curious about is the quant side. You have analysts. Who's look at that to bring that to story time or to other parts of the process?

Maggie: Yeah. We're in a really interesting transition on that front right now. Before, when we were small enough ... When you only have a certain number of customers and you're an early stage startup, you don't really need, in my opinion, a ton of that because you can just talk to everyone. Right? It's relatively easy to get a sense for what's happening with your product. You don't need serious product analytics if you don't have a ton of customers.

Maggie: As your business grows, obviously, that changes and the burden for metrics becomes higher, and so now we're moving into we've grown really, really fast, we're scaling really fast, and we need to have a better handle on those types of things. We're ramping up on the quantitative side. For us, the way that we're starting to think about that and the way that we want to implement it for all of the stuff that we do is making sure we have a target for what a feature's supposed to accomplish and being able to measure whether the feature accomplishes that target. We also want to have a timeline, so how long is it going to take us to move that number from here to there? What are the steps that we're going to take in between those two things? Then all of that would roll up to the area of the product that you're working on would also have sort of a north star metric and all the features should ladder up to that one thing.

JH: I feel like I'm going to-

Erin: Anything else?

JH: ... steal story time.

Maggie: Yeah, story time is one of the absolute best things that we do. We all love it and it really ... I've seen ... Especially for engineers who've worked at other companies who haven't had a chance to be part of that discussion, you can just see them light up when it's like, "Hey, you're going to be involved in this thing. We're not just going to tell you what to do. You're going to be part of the entire process."

Erin: Do you have lots of story times? That's the kickoff. There's one story time per feature?

Maggie: Yeah, we story time. We go back and do research. Designs are happening, research is happening, and then we get to a point where we're ready then we kick it off.

JH: I feel like I would take this paradigm to the extreme and then be like, "Now we have nap time and now we have ..." Just go fully all in on it.

Erin: Snack time is a good one.

JH: Yeah, yeah.

Maggie: Yeah. We've definitely once story timed our way straight into a kickoff, but I think I would probably get in trouble for that one. But we just realized that we had all the answers that we needed and so we felt like we could just go for it.

JH: Yeah. I think there's a lot of value in people in different departments don't need to be experts of other departments, but having a little acumen and a little empathy and a little awareness for what your colleagues are up to and how they approach things, and it goes both ways, but I think there's just a ton of value that comes out of that in terms of how teams operate and the process at large.

Maggie: Yeah, absolutely. I would never ... I love the fact that because of marketing and sales our customers, we have strong ties with them within Drift, and so we really do feel like one team rather than like, "Product won't build that feature that I need to sell this deal."

JH: Makes sense. Makes sense.

Erin: Awesome. Maggie, what didn't we ask you that we can artfully splice in the middle of the episode?

Maggie: I don't know. I feel like we really covered how we think about talking to customers here.

Erin: Awesome.

Maggie: Which is cool.

Erin: Well, you've been a great guest. Thanks for joining us.

Maggie: Yeah.

JH: Yeah, thanks.

Maggie: Thanks for having me. I hope there's enough awkward silences.

Erin: Thanks for listening to Awkward Silences, brought to you by User Interviews.

JH: Theme music by Fragile Gang.

Erin: Editing and sound production by Carrie Boyd.

Maggie: Great. I'm glad I could-

JH: Oh, I wanted to ask. What was it like to be in the Olympics? That was going to be my warm up.

Maggie: Oh. I was like, "Wow, I escaped without someone asking me that question."

Erin: Do you like talking about it?

Maggie: It's been 12 years and I still don't know how to answer that question in a succinct manner because I was 19 and it was absolutely wild. I still can't wrap my mind around the fever dream that was being at the Olympics. Yeah, it was really incredible. I got super lucky with a lot of different things to allow me to make that team and have that experience.

JH: Cool. Cool. Did you meet a super famous person? Am I asking all the basic questions? I feel like I'm being so annoying.

Maggie: No. I get asked, because I was a speed skater, if I know Apolo Ohno.

JH: That makes sense.

Maggie: I did. We were on the same team.

JH: Checks out.

Maggie: Then people ask me whether the Olympic Village was as crazy as it seems. I have to say I was a super nerdy 19-year-old who was also in college, so cannot confirm. Yeah, and then some people ask me what it was like to be in Spandex in front of billions of viewers, and just as awkward as it sounds.

JH: Makes sense.

Maggie: Yeah.

JH: Cool. Yeah, I think we probably hit everything, right?

Erin: Yeah.

JH: All right. Cool. This was a fun one. I think this is going to be good.

Maggie: Good.