Erin: [00:00:34] Hello everybody. And welcome back to Awkward Silences. Today we're here with Danielle Smith. She's the Senior Director of Experience, Research and Accessibility at Express Scripts, a Cigna company. Today, we're going to talk about the experience research team. So what are the challenges and opportunities of a multi-disciplinary approach to user research?

Danielle, thanks so much for joining us today.

Danielle: [00:00:59] Thank you. I'm happy to be here.

Erin: [00:01:02] We’ve got JH here too.

JH: [00:01:04] Yeah. We've only talked about this one at a high level, so I'm super interested to learn what disciplines have been combined together here. It feels like a Captain Planet sort of thing.

Danielle: [00:01:13] Totally is.

Erin: [00:01:15] So Danielle, starting from the top. I know that this idea of this sort of multi-disciplinary approach is something that you've come to after years of experience. So could you tell us a little bit about what led you to thinking this might be a great way to think about UX research, organizationally in a company.

Danielle: [00:01:37] Sure. Earlier in my career, I've worked in different parts of it, a lot of different companies. And I've seen situations where, the market research team may do a study that's redundant with something that the UX research team has done or the analytics team is planning an AB test.

And not really considering usability findings, in informing that hypothesis. and coming to Express Scripts in 2016, when we were a standalone company, we didn't really have those disciplines at all. So there was no one really to conflict with it, to do redundant work, but we still needed to do the work.

I'm not saying that we do market research per se. There is definitely a place for that in a brand and market research. but by combining a few different disciplines, we can get an end-to-end view of the user experience, throughout the life cycle. And that's something that we talk about a lot as experience designers, where we want to be involved with end to end, but in traditional UX teams and UX research methods, a lot of the times we can't be.

And that was my experience before where we spent a lot of time and effort on concepts and prototypes and early, and even high fidelity mock-ups. But once things got launched, we were onto the next thing, just because of the way that the teams were designed. And so at Express Scripts, I had the opportunity to create something that was a little different and look from end-to-end and look at those problems as they manifested.

JH: [00:03:12] At Express Scripts. Did you like, did you have the responsibility of figuring out how to design that team and pull it together? Or is it something you worked on collectively or how were you able to, evolve it?

Danielle: [00:03:23] So a mix of luck and responsibility and me not really believing everything that was in the job description. When I got here, I was explicitly hired to build the UX research team. Express Scripts was in the middle of a technology transformation. you'll probably hear that from a lot of companies, especially in healthcare.

Just moving off of old tech to new tech. As part of that the leadership really wanted to invest in user experience, which was great because healthcare experiences aren't necessarily easy to use. I just realized I should probably back up Express Scripts is a company that people may not be familiar with. We are what's known as a pharmacy benefit manager, as well as a home delivery pharmacy. So for people who are covered by some sort of health insurance, that express scripts services as a pharmacy benefit manager, you can also get your maintenance medications delivered via mail, home delivery mail.

Our mobile apps and mobile website are really for everyday people to go in and learn about their benefits. What prescriptions are covered, which pharmacies are in network and sign up to get some medication delivered to their home. If they're on a medication that qualifies to be delivered and we covered at the time about a third of the United States.

So even though that may not be a brand name that you hear a lot about. We touch a lot of people's lives in a very kind of intimate way in terms of being, and managing their medication. So that said, in 2016, we were so suit, we were trying to grow our emphasis on user experience. so once I got my arms around building the UX research team and worked with my partners in content and UX design to kind of build that partnership that we are all used to.

We started asking more questions about web analytics. So what's being used today, how's it working? And we had a tab on the website that allow people to leave feedback and rate the site in terms of, Net Promoter Score. And none of those disciplines were really being considered as part of the design process and they weren't also, they were also needing to be transformed.

So you think about how I started with the idea of technology transformation we needed to also modernize our analytics and our kind of, what's just, and we call it a voice of the customer program. So I, built the case of kind of rolling all of those things together as part of an end-to-end experience program.

And my leadership at the time was like, great, nobody else is doing it. Go ahead. So, um, We did have a couple of people spread across the organization who were responsible for analytics and a couple of people who were responsible for the net promoter score programs and they got rolled in into my team.

And so while I didn't set out to build this end-to-end view, it became pretty obvious, pretty early on that it was that missing piece that we needed to know, how to design. Really, design more efficiently and more effectively, especially given our context. and I do want to put that caveat out there, like in healthcare and in a lot of industries, but specifically in healthcare, it is really difficult to determine like what your core use cases are.

We do as best as we can to with some foundational research. but once we launch something, we pay a lot of attention to the feedback that people leave on the website, because there's always another use case out there that is either a stress case or, just something that was just out of the realm of our knowledge And so we take that and give it right back to the product team and roll it back into the product really quickly, given our current, the way that we're currently set up.

Erin: [00:07:40] so you came into this job. In healthcare, like every industry is unique, but one of the things you hit on is that it's so personal. And you knew that it was going to be really important to understand the user journey and the feelings and emotions and jobs to be done.

And so it seems like you found this opportunity to bring those disciplines together under your guidance, such that you would have this kind of fuller picture of what's going on with the users.

Since that came to pass, what have been some of the challenges and successes of doing that?

Danielle: [00:08:20] Um, well, Practically, some of the things that have worked really well is, having a better, like a more robust recruiting practice in our user research team. So with, with having direct peer relationships and regular meetings with our analytics team. And we also have, what we call it population data science, where we really monitor our user segments in terms of demographics.

Our user research team can ask. Okay. What kinds of people might typically be doing these kinds of tasks on the website? What kind of prescriptions might they be using so that we can recruit representative users, and also populate our prototypes really well with real data or not real data for representative data.

I say not real data because personal health information is

JH: [00:09:16] Right, right.

Danielle: [00:09:17] You know, So like from a day to day basis, that has been like a very quick win, low hanging fruit. Like all those words you would use. some of the bigger successes that we've had is we've worked across the team to build a persona set, and that was a longer term project that involved every one of the disciplines on the team, to really look at, kind of high level segmentation of our users, then layering on like typical NPS survey feedback and then going and looking at, analytics to see what sorts of interactions they typically have, and then having those in-depth qualitative conversations with representative groups that took a long time, but it was something that is we're really proud of.

And we constantly iterate based on new data as it comes in, because it involves everyone on the research team. To make sure our kind of persona set that we're using to guide business operations and product decisions are grounded in the reality of our users. Some other interesting things that have happened is that we've noticed that, There are people across the team who are really interested in career development. And and I say that kind of facetiously everybody's interested in career development, but by having this broad set of skills on one team, they can explore other areas of research, analytics, data science really easily without having to switch companies.

So it's only team collaborate on projects a few times a year. It's not a, it's not a mandate or anything, but by them being on one team, if, if I hear about, let's say a usability researcher looking at, quality by qualitative comments on why people might want home delivery, then I will say, Hey, you should work with this NPS researcher.

And see what they have, and maybe y'all can work together on building a survey. And that gives the UX researchers some exposure to more complex survey methods and the NPS researcher exposure to the actual user experience of interacting with the prototype. And it's been really helpful to have them all work together like that.

We've had similar collaborations with data science. So we have a couple UX researchers who are learning R and little bit of SQL, not a lot, but just a little, just to do some more complicated analysis and vice versa.

JH: [00:12:03] and how often would you say like the team across those different roles and disciplines, like on a given effort or project? Like are collaborating across them versus Hey, this is just a pretty standard kind of usability thing. So just the user researcher is going to be involved in it. Is it like always like the full set or is it more situational or.

Danielle: [00:12:20] it's more situational. So the user researchers are re are aligned to, a traditional kind of design and product and engineering, engagement or team informal team. the rest of the team is more, I guess a more Your special powers. They interact on an as needed basis, but we do have regular team meetings and regular interlocks, but there are some projects that are just the traditional usability engagement, and we've done some things to help make the teams more self-sufficient.

So like we have an internal recruiting panel that. If we want to do some sort of special participate, some special study where we need really specific participants, then the user research team can do that themselves because the data science team has already built out that panel for them. So we've done things like that.

but I will say that is one of the challenges of having this many disciplines on a team. Is that. When they, when things get busy, folks get heads down and want to get things done. And it's hard for them to, to recognize when they should reach out and ask for help or, bounce some ideas off of some other folks to get them involved, from a different kind of data perspective.

Erin: [00:14:29] Is that challenge more acute on a cross-functional team or is it a function of the size of the team? is it helpful to have, even as things get busy people that might be able to help within the same team or sure.

Danielle: [00:14:44] yeah, it's I would say it's more of a symptom of a cross-functional team. Um, Yeah, I saw it happening when we were eating when we were tiny, but we just still had all of these functions. it's hard for, and that's one of the things that I didn't realize is that I, as a leader, needed to be really more, explicit about talking about how I saw all these pieces fitting together.

As the team got to be 10 people, there were new people there that just thought we just so happened to all be on one team. Not that there were things that would be stronger if we work together. so I did have to do a lot more communication about. The mission of the team and how I saw them working together and providing some encouragement. When I saw folks really getting tunnel vision on their specific area.

Erin: [00:15:43] one of the things you mentioned earlier was, in the beginning, it wasn't an issue of people duplicating work because you need more people to, there's more things to do when we need more people to do them, but you do get to the sort of more people you add to an organization or to a team.

This kind of on the one hand, you want people to collaborate across roles for synergy and shared understanding, and everyone has something to contribute. And all of that, on the other hand, you don't want to be duplicating efforts and you want to be having some focus across those different roles.

So how have you struck a balance there in terms of having all these different kinds of functions? Some of them. multi-functional right. you know, finding that right balance of staying focused and yet, working collaboratively as well.

Danielle: [00:16:34] It's an ongoing, it's a learning process and we have a couple of. collaboration meetings. So we have two research meetings a week, one that's for everyone, but the other one is optional to dive into methods. but outside of that, you can pretty much be heads down focusing on your thing.

But one thing I've had to do, one thing I learned and that I've had to do, one thing I learned is that I do need to have regular check-ins with pretty much everyone on the team.

And the cadence of that may vary. but it is really an opportunity for me to. Help them talk through any confusion that they may have, or, like what I've noticed is that, people who are really strong specialists, they have ideas about how they can co collaborate. But if they haven't done it before, they're, they aren't sure they don't want to waste anybody's time to your point.

everybody's busy. Everybody's. Super smart. and so we in these one-on-ones, really just bounce ideas off of me. I'm like, yeah, that might be a good idea. Or, maybe not right now, cause we were really busy. But another thing that we do that I think has been really helpful in Knowing when to focus and when we have time to do something big and collaborative, or strategic and collaborative, this might sound silly, but we, we use here at we, we are, we do our work in agile sprints. We have, back, we have, ceremonies and cadences and each team has their own, but.

The level of visibility, like we have some aggregated dashboards or whatever that you can see, who's doing what, when, and if you notice that, someone's doing something similar to you, then that's another mechanism of having those conversations. I won't say that we discover a lot that way.

Cause you have to, you have to be looking for it, but. me and my leadership team are able to see what folks are working on some sort of regular basis that way.

JH: [00:18:40] This, uh, this is all sounded very like harmonious. But are there ever cases where somebody who's maybe a more traditional analyst? Is like advocating that their skillset would be the better way to get an answer on something and maybe a user researcher is advocating for their thing. And there's actually more of a debate of what method to use or how to come to a decision on something or what data to gather or do people generally see eye to eye and collaborate through it.

Danielle: [00:19:05] I would like to say that people are very harmonious all the time. Um, But what I see more often is, if someone's like traditional analyst, they may want to just use the survey data and any sort of quantitative metrics coming out of a usability test and do math and create some sort of product.

And that has caused more consternation than anything because depending on how your data are collected your sample size, your question, like it's not something you can just really consume at the same way, you can consume some sort of operational analytics. And so that has caused a little bit of friction.

The other things, I think we've largely avoided by trying to have one team, is that we don't get a lot of people, even in the enterprise saying, Oh, we don't need a usability test. We just need to AB test because the user researcher is usually part of our product team conversations and they might speak up and say, this usability test might take us longer to do because of X, Y, and Z.

I'm comfortable just doing this as an AB test. Let me pull in my partner from that team and they'll help you set that up.

Erin: [00:20:30] Right.

Oh, you don't have different insights teams fighting to prove their own value against each other to solve a, like a business question. You have a team who's equipped to answer any question with the right tool. And then, the central body that can figure that out, which does seem really powerful.

Danielle: [00:20:47] Yeah, it's been really powerful. I think with, like I said, the biggest problems have come up with like unattended data analysis, but, that was also a learning exercise for all of us were, We have to learn to communicate more about different studies or document them better so that more folks in the enterprise can use the data.

JH: [00:21:10] if you found yourself in a new situation and you had the chance to build up a team like this again, are there things you would do differently or things you would definitely repeat? Cause they worked really well or, any like lessons learned in that regard?

Danielle: [00:21:21] So many lessons learned. I think that the thing that I do struggle with is because it's, it is very multidisciplinary. we do have to have a lot of meetings. I personally meet with everybody on the team. One of the things that I I might emphasize more is to have more, mixed methods experts on the team.

And we have that now, but in the beginning I was really focused on, I guess I was the only mixed methods person. and I was really focused on building out individual disciplines and. Nah, and letting them mature organically. And I think earlier on, I should have tried to hire, recruit people with mixed methods experience to help draw those connections across the team.

Erin: [00:22:14] Yeah. Yeah, because in a way it's counterintuitive, right? If you have. A larger team, a cross functional team. You want specialization, but on the other hand, if your team is growing and the business needs are shifting, you also have a lot more option value. If you have people who can shift around a little bit, as the needs of the business, as the kinds of questions, the businesses asking change.

You can probably also speak better to the other cross-functional folks on your team. If you have, a somewhat of a variety of aptitude across these methods as well.

Danielle: [00:22:51] Yeah. And I'm definitely not saying that. we definitely do need specialists. I'm glad we do have specialists. I just think that having more than just me who can see, and that's not the case now, obviously, but in the beginning, this idea that someone who knows how to do, more complex statistics and, run usability research or user interviews, would have been a good addition early on.

Erin: [00:23:17] what about your advice to someone else looking to do something similar and every organization's different, right? So given that, what's a good way to think about building out a cross functional insights team.

Danielle: [00:23:30] I would say, start with your product life cycle. If you're building something that has a kind of low cost of launching and like launching a product and then making changes later, like most digital products, I think of them that way then. A team like this would probably be very helpful for building strong relationships with teams like this across the org where y'all have, peer sync ups, or informal working sessions to build those relationships.

If you can't build it all in one team, I would strongly suggest this. If you have a product that has a. Kind of higher level of releasing. Like you have to have things really tight and buttoned up before you launch, then having a lot of product like infield product feedback rolling directly into your development process may not be worthwhile.

So I would definitely consider like how you're. Your product and your experiences exist, for services, this would, this actually makes a lot of sense in my opinion. the other thing I will, the other bit of advice I'll give is, your capital or your, Expense for tooling and software is a lot different.

If you have a team that looks like this, the tools that data scientists use are different. some of them are expensive. The tools that, like I said, we manage the analytics team. There's a lot of licensing that goes along with that. So think about how things are budgeted in your org and, where.

Like where those costs should go. I'd hate for you to be surprised by some of that. and the other big piece is like, thinking about the skill sets you need. talk to folks who manage teams like this, what kinds of, people and skills do you, what kind of people do they look for? What kind of skills, do they, recruit for and.

What kind of development path do they have? You don't want to build this team and people see no opportunity for advancement because one of the benefits of a team like this is for long-term, long-term learning and retention. So you want to keep folks engaged and try to retain them for a while.

Erin: [00:25:59] Good advice.

JH: [00:26:01] Yeah, I'm super curious to see if we'll, we'll see this become like more of a trend. It seems like there's a lot of benefits, so it'd be cool to watch this space, as they say. Let's see if, see, see how other teams evolve going forward.

Danielle: [00:26:12] I think I like it. I know a lot of, your ex. UX researchers out there, may not be a fan of the client and maybe Oh, look, the call over the client, but it's all meant to provide a really in-depth picture of your users and their context and their journeys. And I do think it works well together.