Erin: Hello, welcome back to Awkward Silences. We are here today with Laura Powell who is a specialized generalist at Appcues, and she is the co-creator of User Test Fest, formerly known as Drunk User Testing. And we're going to talk about the rebrand of the event, how to make events and user testing more accessible to users within your company and to testers outside your company. So, so excited to have Laura here today.

Laura: Thanks guys for having me.

JH: Nice to meet you.

Erin: JH is here too.

JH: I am, I'm back as well.

Erin: Drunk User Testing, User Test Fest, how did we get there?

Laura: Yeah, how did we get there? Great question. We had been doing the event for two years, I think we when start to realize and have conversations about the name of the event. We got a lot of feedback from companies who were not comfortable with the name. And public companies, for one, being like this is not something I can take to my team. This is not something I can have my name on. Those seeds started getting planted, and I think for us it was certainly a very easy event to market. It was not a hard event to sell out attendee-wise. Companies were harder to sell but they knew they'd be getting a lot of value out of it.

Laura: And so we kept having conversations about it and finally this year we kind of all reached the same conclusion, which was that no one was really beholden to the name. And it was becoming more and more not what we wanted. And we realized that maybe actually in casting this wide net and having this name, we're targeting the people we don't want to target and have at the event. And inherently making this event exclusive, it was something that I think we qualified for a long time because we're hosting it and we're not bro-y, we're not like that, we're not these people who are just looking to get blasted at a tech event. But at first blush if you're seeing this without having any context, how do you not have that reaction? So I'm really happy that that conversation ended where it did. I'm so much more happy that the name matches what the event is, which is bringing user testing to more people, and that's really where we're trying to focus.

Erin: Yeah, I think that's one of the cool things about once you start diving more into making things accessible, there are all these layer to it and the event is meant to make user testing more accessible but we actually need to make the event more accessible and here are some ways we can kind of do that.

Laura: Absolutely, there's certainly things that I think as an expert in your own product even, that you think, oh I've thought of everything or I've considered every outcome for the user. I know how this goes and I know the most about it because I made it. And I certainly felt that way about the event and it took me talking to a lot people, to say oh my gosh, actually that's not the case and we need to iterate on this because that's my advice to everybody. How do you get better? Iteration and feedback. It's the same thing for my event, that's the only way we keep producing such a successful event. And having the enthusiasm we do around the event is because we're taking that feedback seriously.

JH: That's awesome. Do you guys have a sense for people who attend, they're not setting up a booth of anything but they just want to be there and be a part of it and test some stuff? Like the background, are those people who work in product and design, and they have some familiarity with this stuff? Or is it people from all over the spectrum, and they just want to see what it's like? Or, do you have any sense for that?

Laura: Yeah, so we don't collect any demographic data from our attendees, we grab their job title if they have one and the company they're with. Most of our attendees are coming from tech, because that's where we're promoting the event. So we'll see a lot of people in product design, in UX, in products. We'll also see a bunch of people in marketing, sales, customer success.

Laura: And kind of as a weird byproduct, not surprisingly, but surprising to us when we started doing this event, it became very popular for networking, and people who are looking for jobs, people who are looking for product designers to meet. Appcues actually, I think we've hired four people we've met at the event in the last year. So, it's just a great event.

Laura: I think also, as I mentioned earlier, there's this feeling a lot of people who come the first time, we have I think a really, really high rate of people who keep coming back the event. Because they know how fun it is, and they know what it's like to feel validated as a user, even if it's just for that night. Even if they're not a paying customer of our product, to say hey I have input, and to know that that's taken seriously and could have an impact is such a high.

JH: The question that comes up, or a theme that comes up in a lot of these conversations, is kind of this notion of scrappy research, just get it done, any means necessary. And then people who are a little but more formal research, like really go through the right methods and practices to do it as best as possible.

JH: This would seem, you're on the scrappier end of the spectrum of it's probably not the target user in all cases for people who are giving the feedback. It's noisy, it's distracted, you're doing quicker sessions than you'd preferably do, so there's some things you'd qualify on that side. But do you see that the people who are running the tests, they're really happy with the feedback they're getting, and it's meaningful? Or is it more of just a fun way to celebrate it and get a couple thoughts? Do people get meaningful insights out of it?

Laura: Absolutely, yeah. And I think it's both. I think it's in that Venn diagram, that middle where it happens to be a really fun event but also people are getting a lot of value out of it. So if it's purely a numbers game for people, yeah you'll do 20 to 30 to 40 even, user tests in a night. Which is a lot of bang for your buck, but in addition to that, there are people who don't have the means to do formal user testing or don't have the time, smaller teams who are getting ramped up. Or even a team who's like, I just want to test this tiny piece of it, and they'll confirm that hypothesis at user testing. I'm really proud to have, I think our average NPS score for testing companies is 97, from all of the events. So I'm pretty happy with that.

JH: So there's one company who's on your bad list probably?

Laura: Yeah, there's a few companies who probably aren't familiar with the event or always say three hours is a long time to test. Absolutely, it totally is. Or they'll make suggestions like, hey what about having a full bar? And I'm like, maybe not. So I think it's incredibly valuable, and I think certainly if it wasn't valuable for teams, I wouldn't be doing it.

Erin: Awesome. I know you've taken it sort of on the road, and it failed over the years. Are there tips you've learned in preparing companies to make the most out of the time, that are maybe applicable to companies outside of the context of the event, for how can you kind of speed up the process of getting a lot of feedback quickly.

Laura: Yeah, definitely. I think the thing when we're coaching testing companies about preparing for the event, the number one thing for me is plan and rehearse. There's a lot of companies who will say like, oh this seems like a pretty casual event. Or it seems like it's fun, ergo I don't need to do that much prep work. If you want to succeed, obviously rehearsing and preparing is the best possible mechanism for doing that.

Laura: I also think, on the flip side, don't overthink it. Right? Test one thing, make it pretty small, understand that. For a lot of people this is their first time testing at all, and so I think it's pretty easy to get overwhelmed. And certainly the environment's really high energy and you can get overwhelmed by that. And so I think it's about having fun, not overthinking, but planning because it is, at the end of the day it is something that you want to get value out of.

Erin: And what kind of tests are people doing? Are they doing basically qualitative usability tests? Are people getting into tree tests, or more specialized tests? What kind of tests do people run?

Laura: We see a lot of A/B tests at the event. They'll be pretty simple redesign tests, companies who are like, we're in the middle of this, we want to see if this part of it is going to be successful. A lot of it is usability though, primarily. The tests are pretty short, they're five to seven minutes, so it's not going to be a long in-depth user test where you would sit down for an hour and delve deeply into a flow. It's pretty snappy.

JH: You mentioned obviously the rebrand which is a pretty big change in evolution and iteration and all that. Are there other things on the road map or plan? How do you continue to see this event evolving, is it just more frequency? More locations? Or other kind of fun wrinkles that might come into it?

Laura: I mean, I think anything is possible. We'll always kneel to the feedback and see what happens. People have no shortage of feedback when it comes to this event, as with anything. And I don't expect that to change. I think for us in terms of programming, locations will definitely be interesting. We want to bring the message of user testing to everybody and get everybody excited about it. I think something we've considered is, do we do any content around it? Should we have a speaker who talks at the beginning of the event? Is that something that would be a draw or a value to somebody, and stuff like that. You know, a year ago we had Keytar Bear at the event for like an hour and that was a huge draw for people, it was funny, it was fun. He told me it was his favorite event, no big deal.

JH: For non-Boston listeners, we should probably qualify what Keytar Bear is, because it is awesome but it might not be familiar. I think Erin had a look, where's she's like, what is this?

Laura: Yeah, it's very contextual. So Keytar Bear is a bear, or at least a man dressed as a bear, who plays the keytar. And he's a local legend, you'll see him playing at T stops and his sightings are a very big deal. But obviously very contextual to Boston, I would never fly Keytar Bear to San Francisco and expect it to have the same impact. People would probably be like, I mean okay.

Erin: On a scale of zero to Gritty, where is Keytar Bear?

Laura: Oh, Keytar Bear versus Gritty, that is ... wow, that's a comparison I have not made yet. You know, Keytar Bear has a softer face, I would say, and a softer expression. But just as much grit as Gritty does, in that the costume has never been washed. Keytar Bear rides a moped, so yeah.

Erin: Okay, yeah I'm Google imaging him and it looks like he's wearing a variety of outfits but these might not be the go-to.

Laura: Yeah, so the outfit changes, it's just the hands and the head that stay the same, and the keytar.

Erin: He does look well loved.

JH: Did he do any testing? Did he hang out afterwards and test some products?

Laura: He did not do any testing, but I have to tell you JH. There was this moment that everyone know makes fun of me for. Where it was super warm in the room and when he arrived, he's in the suit and he's even wearing a leather jacket. Which I'm like, oh my God. And he looks at me and he goes, can you get a fan? And I just looked at him and I was like, well we're all fans. It was not my best moment, but he was just like, like a fan fan. And I was like, right, right, right, right. So yeah, that was my brief run-in with Keytar Bear, now I'm famous.

JH: That's awesome. You mentioned it might be someone's first time running a test or being involved in research. Do you see that people who are doing the testing at these events because it's a little more on the informal side, and kind of fun. Do they use it as a chance to get more people on their team exposed to research and pull them in, and have the typical researcher, product manager involved to oversee it but let some other people get some at bats in a lower pressure environment.

Laura: Absolutely, It's definitely something we try to do on the Appcues team where the last event we just had, three of the five people on the team testing were new Appcuties, so they've never been to the event. They're barely into the products and we kind of throw them in, baptism by fire. Obviously they've got weeks to prepare but it's really fun for them to meet with our users or just meet people in the wild and use their product and say, oh my God I can make eye contact with you and give meaning to this interaction. So we encourage a lot of the testing companies to do the same. Like hey, don't just make this a product event. If you have engineers that want to come, invite them. If you have customer success people that want to come, invite them. If your CEO wants to do testing, we've had this happen, come on down. Please, I encourage it. I think the more you can get out of the building, the more successful your product will be.

JH: Agreed, yeah it seems so important. Erin can speak to it better but we've tried to do, we call it research at UI, where we try to get the whole team involved and it's usually for real research topics that we need to get insights on. So it feels a little higher pressure, but it's also the element of, the more you do something the more normal it becomes. So it's kind of like you can just push through and get developers and get people in support and other roles talking to customers and doing these sessions. The first ones maybe are a little bumpy or a little nerve-raking for folks, but it kind of gets easier. It's cool, the event to have the support and seeing everyone doing it together probably goes a long way too.

Laura: Yeah and I think it's good to build that muscle memory too with your team, right? There's lots of things you can do. At Appcues we have user testing day every month, so people are meeting with users actively outside of that. People will rotate support weeks, but I think this is a very interesting opportunity to be an uber advocate for user, and say, oh my gosh okay well I might work in marketing where I'm not touching these things on a regular basis. Or I'm so hyper-focused on advertising and promoting a specific angle of our product that I forget who's actually using it and what their days are like. We're being used for 30 minutes in a day with a whole slew of other tools, we can't expect people to have the same level of expertise in our product that we do.

Erin: Well you mentioned, get out of the building which I love as a metaphor too, right. Because you can't always get out of the building, if you're a remote company, there is no building.

Laura: Right.

JH: Get out of the house still applies though.

Erin: Just for a variety of reasons, just get out of your dwelling.

Laura: Right, just get out of your comfort zone, or like the circles.

JH: The companies that come in to test, are they pretty progressive on the user testing side of things? They do this with some frequency, or they're talking to people? Or are there some people who are like, we know we need to be doing research and this event is going to be our catalyst to get it in motion and we'll figure it out from there. Is there any trends on that side?

Laura: Yeah, it's really funny to see actually. I would say, by and large all product people, all UX people I talk to always want to be testing more. Always, hands down. There's never someone I don't talk to that's like, oh actually we don't have time, or this isn't a need we have right now. Yeah, but I will say that it depends on where the company's at, and unfortunately something that I've seen trending is that product people or UX people aren't usually the decision makers for getting this approved or getting the budget, or whatever. I think that can be really frustrating, if these are people who are evangelists about it and very excited to do it, how do we make sure that they're empowered because if you're not putting money into user research, what are you doing?

Erin: Yeah. That's been a huge learning for us in the research at UI stuff JH was talking about was, there are all sorts of benefits of research, but if you're doing research for product development purposes it's useful to have the bandwidth through the idea that you might take action on the insights. Uncover, right?

Laura: Absolutely.

Erin: It can be a frustrating thing to know you have a problem to solve, to even know what the solution might be and not be able to solve it. So that's been useful for us and seeing the impact of the research, right? It's like step one, do more research, step two, use the research.

JH: Right, you need the virtuous cycle. The comparison that comes to mind, maybe because it's January of 2019 but it's people always say that they want to go to the gym more, they want to eat healthier. It's very similar to research in my mind, in the sense of everyone says they should be doing it more. So it's like this innate known thing that it's good, but it's the behavior change and actually sticking with it and seeing the benefits that's the hard part. Especially when the alternative, in the case of eating healthy is junk food, which is designed to be delicious. The alternative in product development or design or whatever it is, trusting your own instincts and gut, which is also pretty appealing, like I know what I'm doing, I'll just follow my own instincts.

JH: And so, events like this where you have a tribe outside your company to go sync up with and keep you accountable and stuff, I think is an important piece of it. It's harder to behavior change stuff solo, you need somebody to help keep you accountable, and if you can expand that to be outside your organization I think that's even more powerful.

Laura: Yeah, 100% I think you're totally right about that and I think that aspirationally this event seeks to be a community for people who work in product and UX and certainly I think that's what it is in Boston now. It's people who test, know each other and get to know each other through this event. And I think the biggest thing you can do is, like I said, just keep testing, as hard as it might seem. There are a lot of people who are like, I don't even know what to test, I don't even know where to start. Okay, well we'll go through that process, I'll coach you through it. It's not as scary as it seems.

Laura: I think a lot of products don't mean to be deaf to what their users want. Obviously no ones intends to be a terrible UX, but when you get in the habit of relying on feedback from one source, current customers or current users or tickets and ticket sorting. Or just what the product team goes and goes to the desert and drinks iowaska and comes back and it like, we have the vision now. And you're like, oh okay cool. You know, it's a good way.

Laura: And so I think for us, it's been really fun to see very small teams who don't maybe have a product team who are like, yes let's get excited about this. And even really large teams like Enterprise, grades software platforms where you're like, I can't even imagine what it's like to figure out what a product looks like or this is how we reinvigorate the team, this is how we get back on track.

JH: Right, yeah, there's a very motivating piece that comes out of it. And especially as you mentioned you can hear from your own existing people too much. I think a piece of this too, in the sense of maybe a lot of these folks aren't your exact target market, but fresh eyes are valuable, right.

JH: I forget the study that was popular a couple weeks ago but I think it was Amazon did some testing in India, or another market, and they realized that nobody knew what the little search icon on the search bar meant. The circle with the line on it. I forget what people thought it was, but they had a totally different association with it. It's one of those things where you're not going to learn that from people who use the product already since they already know that you click that button to get your search results. And so that's an extreme example but in other cases, having a complete outsider who's probably never going to use or buy your thing look at it, might kick up a couple things that get overlooked by people who know what they're doing.

Laura: Yeah, I think that I saw a thread yesterday about product design and designing for safety and I think that everybody has to learn a whole lot about their product, right? I have to be taught about this event from so many people even though I'm, let's call it the foremost expert on User Test Fest. I expect other people to teach me about this event and what it's missing, and what I haven't designed for. And in that case I think that's totally applicable to a lot of products that we're just like, oh well that's what that is duh. And you're like, oh no, no.

Laura: I think the thread I saw yesterday was about Google Drive and how there's a tendency in some products in technology to ignore safety, specifically when it comes to relationships between men and women and what that's like. And I think, oh yeah that's interesting and something you would never know unless someone brought it to your attention to say, oh hey this is actually a problem for me, I have someone in my life who's toxic, who still has access to my Google Drive. How do I resolve this?

Laura: I think that's definitely applicable to the events. I'm not going to know how to make the event more accessible until people teach me. Not necessarily, I think obviously there are things I can do. And that starts by widening the circle and asking people, I want to hear from you, I want to hear your feedback, I need to hear your feedback in order for this to succeed.

Erin: You've mentioned your killer 97 NPS, which is amazing, do you have any specific success stories? Not to put you on the spot but people are learning things that are improving their products, you know.

Laura: Certainly, I won't name people specifically, I don't want to put them on the spot. We had an event last year in San Francisco where we actually had two competing products testing at the event. That was really interesting and both learned a lot, and I would say one definitely is more usable than the other. I think the people who have the more usable product were validated by that. I think the people that didn't, now know what they can work on. It was a huge night for learning.

Laura: We had another team at the event in New York who have a product that's amazing and very accessible and really fun, but they're getting ready to maybe raise money. And so they have a few little polished things that they're trying to work out before they go to their potential investors and say, hey this is where we're actually focusing on, and hey this is who our users are, and hey this is how they behave. It was incredibly awesome for them. They had hypothesis before that night and they were like, this sealed the deal, we know now. We feel ready, we feel prepared.

Erin: I always like to ask what we didn't ask. What else do you want to tell us?

Laura: Um, let's see. I think a lot people ... do you guys get this a lot? A lot of people will ask me, how do I get involved in UX? How do I get started? Where do I start? People are never looking for junior user researchers. And so what I tell a lot of people that come to this event especially looking for jobs, it that everybody can be a user researcher. You don't have to be in product or UX to know who your users are, and know what it takes to love your product more.

Erin: Yeah, I think we're about to publish our state of user research report, and one of the more interesting tidbits for me is looking at all the various degrees, undergraduate and graduate, that folks have that end up practicing research. It's all over the place, I mean there's a lot of anthropology, and people science, then there's architecture and design, and just anything you can think of. We had several nursing degrees, I mean it's all over the map, but I think having an interest in people and then having the ability to turn qualitative data into useful insights.

Erin: And then a hugely important part of it is being able to evangelize the importance of that internally. So it's really a question of interest and skills, but I think it's useful to know that there are so many paths that people are already taking to get to that role that wherever you are, you're in an okay place in terms of getting there. There are all these opportunities that we've already been talking about to get involved with talking to users, so find those or create those. Start sharing what you're learning internally and see what happens. So that would be my two cents.

JH: Yeah, I think it depends. Big company, small company, right? I think if you're at a bigger company there's probably, if you are doing a good job in whatever your current role is, you typically have opportunities for lateral moves where teams will train you and on virtue of you being smart and liked in the company they'll happily take that on. So kind of networking internally and stuff, I think you can find things.

JH: I think if you're in a smaller company or looking to make a bigger change career-wise, I think lots of start-ups and other places. Or anybody who really writes or talks about user research online is so friendly and so engaging, that if you like something they wrote and reach out to them and say, hey I loved this, what are some other great resources on this? You'll probably start a conversation.

JH: The way that designers will sometimes make a fake portfolio where they go through and do redesigns of existing products or services. If you were to do a mock interview where you write the guide and you talk to five family and friends for free and synthesize the results for a real product, like an app you use or something and throw that up as a portfolio item. I think people will respond to that kind of stuff, and it's not a huge time commitment, and it's good practice.

JH: I think there's a lot of stuff, if you're clever just by virtue of the people ... as Erin mentioned, the people who do use the research, like people and they're interested and they're curious. And so if you're doing interesting and curious things and getting under their radar. I think opportunities are going to start to open up for you, so you just gotta kind of be clever and find ways to engage.

Laura: Yeah, and sign up for user interviews as a participant. I mean, honestly.

JH: For sure.

Laura: Go see how it's done.

Erin: Participate in that research or, it's all good, just sign up.

Laura: Yeah, just sign up. I love it. I did two tests, I told Erin, I did two tests last year. It was so much fun. Just to go see what other people are doing, and to give feedback obviously, which I am addicted to doing.

JH: Yeah, it is fun being on the other side. That was one of the names, was Other Side of the Table, because we're interviewing interviewers. But yeah it is really fun being on the other side.

JH: The last idea, I'm just going to throw out since we were talking about how other people can do these types of events. If you do work at a big company and you have people who come in in onboarding classes of like a dozen people, part of that onboarding should be, let teams internally go do user testing with them. They're going to work at the company, they'll meet people, you'll get fresh perspectives on whatever you're working on. You can do these kind of Test Fest things anywhere you have a bulk of people, right? So if you're at a huge company and you're like, we're never going to be able to go to this event, I won't be able to get cleared for it or whatever. Just do your own and make it part of onboarding, it'll be a cool thing. I think people can find lots of ways to do more of this lower stress, kind of fun testing that gets things in motion.

Laura: Yeah, 100%, 100%. Do you guys have a product that you would like to see at the next User Test Fest? You would nominate? Not necessarily because they have bad UX but because you're just like, oh it would be fun to give feedback IRL. Or maybe they do have ... I don't know if we want to bad mouth any products but-

Erin: Oh, of our own? Or just like something out there in the world?

Laura: Just yeah, like a product you would love to see.

Erin: Oh gosh, any kind of insurance or government product just needs all the UX in the world.

JH: I'd be fascinated-

Erin: Airlines have come a long way. Sorry what were you saying?

JH: No, go ahead.

Erin: I was going to say something nice about airlines, which I never do. But the Delta app has gotten really good.

Laura: There you go, shout out to Delta.

Erin: They check you in automatically now. It's amazing.

JH: I was gonna say, spaces that are kind of crowded where you sort of have to be opinionated. I honestly think to-do list type stuff, like on Macs there's Things, Todoist, Asana, all these different solutions. And they all kind of functionally do the same stuff, of you make lists and you check things off of it. But they all kind of have their own flavor or system they put on it. And I'd be really fascinated to know how they hear user feedback and factor that in versus being like, no our differentiator in the market is we put this twist on to-do's so we have to stick to some of our core principles and not get homogenized. I don't know if there's anything there but I'm just always astounded by how many to-do apps there are. So I'd be very curious to know how they go about making themselves unique.

Laura: Yeah, definitely. What's their differentiator?

Erin: Similar to that there the whole Airtable versus Google Sheets, or maybe that's not a good one. But it's the create your own software versus we have a point of view for you that you can use or choose to leave. Seeing a lot more of that develop I'm curious how many people really go deep and taking some of these products that are super, super flexible to the full capability of what they can do versus use one or two features. I guess anyone who makes any app has this experience, right, where a certain portion of the audience is really power using and everybody else is using 3 features.

Laura: Yeah, of they're a total edge case and they're like, oh we're using this for this one really obscure thing and it works really well for it, sorry. Yeah, I love it when that happens.

Erin: Yeah. Cool.

Laura: Cool.

JH: Cool, well that was super helpful and it was really fun chatting with you Laura and we appreciate you taking the time and hopefully I'll be able to attend the next User Test Fest. I haven't been to one in a little while, so excited to get back out there.

Laura: Yeah, we'll see you there.

Erin: I'll be first in line for the remote User Test Fest.

Laura: Perfect.

Erin: Awesome. Thanks Laura, thanks so much.

JH: Thanks.

Laura: Thanks guys. Bye.

Erin: Thanks for listening to Awkward Silences. Brought to you by User Interviews.

JH: Theme music by Fragile Gang.

Erin: Editing and sound production by Carrie Boyd.