[00:00:00] Andrew: It's always important to be keeping like a pulse on who the ideal customer profile is because it can really shift quite rapidly as well when different market conditions maybe present themselves or new technology emerges. There's many, many different reasons that can evolve. Who targeting at any given time?
[00:00:21] Erin: This is Erin May.
[00:00:23] JH: I'm John-Henry Forster, and this is Awkward
[00:00:27] Erin: Silence Silences.
[00:00:35] Erin: Hello everybody and welcome back to Awkward Silences. Today we're here with Andrew Michael, who is the CEO of Avrio. Today we're gonna be talking about how you can use research to identify your best customers, your ideal customers, uh, which is something I'm very excited to talk about cuz we are doing just that right now as we like to redo from time to time.
[00:00:58] Erin: So selfishly, uh, this is gonna be a fun one for. So thanks for joining us, Andrew.
[00:01:04] Andrew: Thanks for having me. It's great to,
[00:01:06] Erin: Awesome. We got JH here too.
[00:01:08] JH: Yeah. When we talk ideal customer profile, icp, I can't help but think Insane Clown Posse, so I'm gonna skip that outta the way every time. Focus back on the topic of hand.
[00:01:17] JH: Yeah. .
[00:01:18] Andrew: It's one of those acronyms as well. Like I, I have a slot at some point as well. It's like, Just this like cartoon character peeing on the wall and it's like ICP as well. Cause it's . Yeah, it's a funny in SaaS who use like way too many acronyms.
[00:01:31] JH: Very important. An important one too though. So yeah.
[00:01:33] Erin: Yeah. So just to reiterate, so you were talking about ideal customer profiles and none of those other ICPs. Yeah. Awesome. So we just discussed what is an ideal customer profile. We know what the acronym spells, but like really what are we talking about when we talk about an icp?
[00:01:49] Andrew: Yeah. So I mean, I think from my perspective, it really depends on the stage of growth and where you're at.
[00:01:55] Andrew: And I think like this is maybe often looked in the beginning, but ultimately I think in the early stage startup, it's really like your ideal customer profile. Who's ever gonna give you money at that point in time? Because like, you literally just need to survive and you need to make things work. But then I think as the, the company grows and as the product evolves, Your understanding and your definitions should be evolving constantly.
[00:02:15] Andrew: And like you say, you're revisiting the work now and you're revisiting the research. I think this is just something that you should be doing all the time. And also another thing I think on this is that. When we think about our ideal customer profile, if we start looking at our customers who we have today, they're a direct reflection of the product we've built in the marketing we've done up until now.
[00:02:33] Andrew: But they don't always necessarily represent the best ideal customer for the business or for where the market is moving. So always thinking about these things and thinking it's like is a constant evolution that you build on top of with your understanding over time. And it's an evolving research process.
[00:02:47] Erin: I. In the beginning, your best customers are the people who will pay you. I imagine that doesn't ever really change, right? But it changes like, you know how you wanna think about those opportunities.
[00:02:59] Andrew: Absolutely. I think, I mean in the beginning you accept pretty much any customer, and I think from my perspective, like shows of churn of him, we talk about like churn in the context a lot and I think in the early days, like when you just accept anybody and everybody as a customer and you just try and acquire a new one and everyone, what ultimately ends up happening is you end up seeing that impact on churn and retention ultimately, which ends up like training your company's resources like support ends up spending.
[00:03:23] Andrew: Time serving customers who aren't the ideal fit. But at the early days, you really, you need that for growth and you need that to sustain the business. But then over time you can really start to realize, okay, the time and effort, uh, that we spending serving these customers doesn't make sense for us and we're better off.
[00:03:37] Andrew: Getting a better grasp of who we should be serving better, and then focusing on that audience so there are times when it makes sense to fire customers.
[00:03:45] JH: Yeah. Yeah. That phrase always is a good one. Yeah. , how do you think about it as you're an early team and you're kind of going through this transition of, we just gotta find customers and then we have some critical mass, and now we can start to be a little bit more intentional about which of these customers are the best fit?
[00:03:58] JH: Like where does that happen? Is that, is there like a certain stage or scale where that tends to come in or it varies or how do you know when you should start thinking about this? I guess is maybe the question I'm trying to.
[00:04:07] Andrew: I think you should always be thinking about it. So, uh, for, I, I'd use a, as an example, like, I think before we started the business, we already had an idea of who the ideal customer profile was, and that started as a result of early customer research that we were doing.
[00:04:21] Andrew: So really just trying to dig into the pain point, into the problem we're solving. We started to formulate like which group and which audience resonated best, understood the problem most had the biggest pain. We started building the product with this ideal customer profile in mind. But then as you said, like as you start to grow and as you start to acquire new customers, you start to notice new segments and new audiences that you perhaps didn't consider.
[00:04:43] Andrew: And we are busy going through this evolution now as well, is starting to look at some of these other adjacent use cases, adjacent audiences, and trying to see, okay, like. Does this make sense for us to be serving this audience now? And it's something that I think you're always doing as part of your business, but really it starts like before you even get started building product.
[00:05:01] JH: That makes sense. Yeah. Yeah. So it's, it's not that you don't do it early on, it's that you don't have enough signal and data and probably until later on to be more confident in it. But it's something you're always thinking about in refining and having hypotheses around.
[00:05:11] Andrew: Yeah, and I think this was something that changed, like my perspective from our time at Hot Jo actually.
[00:05:17] Andrew: When I joined, hotshot came from a previous startup that I'd founded. We were VC backed, so we had this like heavy reliance on like we needed to share data and numbers and statistics when it came to how the business was operating and running. And when I joined Hotshot, actually, like I found it for the business that had scaled.
[00:05:34] Andrew: It was heavily, heavily reliant on qualitative research and just speaking to customers and almost nothing on data alone. And. The reason for this is that at early stage, the data that you have is more like noise than signal, and only when you start to hit a certain size and scale, like can data become really reliable and your best source of information, it always is your customers, but even more so at the early stage, like you really need to be speaking to your customers day in and day out and never lose that pulse.
[00:06:03] Erin: So I think, we'll, we'll go back and forth between sort of the, the what and the how and the why here, right? So what is an i p and you know, why does it matter and, and how do you use it? So maybe we can jump into the, how do you use it for a minute? Because you know, one of the things that we talk about a lot is how a lot of artifacts you get out of research, such as an I C P, ideally can be used by a.
[00:06:24] Erin: People in the company, a lot of different departments, maybe in different ways, but curious how you've seen, uh, an ICP actually be used by different departments and to the point about sort of different stages, your ICPs gonna look a different way, that might play out different ways in different stages as well.
[00:06:41] Andrew: Yeah, so I think what's interesting is always like most companies go through this evolution where a lot of teams have what they think is the understanding of the ideal customer profile, and they'll go to marketing and they'll have one sort of a definition. They'll go to customer success or sales and product.
[00:06:57] Andrew: But at some point in time, like someone in the team says, Hey, we need to. Gets a grasp to this. We need to have a common understanding of who we're going after so that when we are using what we deem to be ideal customer profile, we have like consistency across the board. And by, that's like the ways you can use it, is pretty much every team in the company can use the ideal customer profile in some form or manner.
[00:07:19] Andrew: If you take marketing, for example, like understanding who you're selling to, really then starts to think about the communication, the ad network channels that you're gonna go through, how you're gonna distribute content, thinking about the product and positioning, like that's how marketing can really use the ideal custom profile with.
[00:07:36] Andrew: Insights in mind, and then it, it just flows down the chain. So sales, they, instead of spending their time trying to sell anything and everything, you give them really a good fit profile. So depending if you have the luxury with the high inbound amount of leads, like you can filter, then using your ICP or when you're doing your outbound, So it pretty much like going throughout the organization, you can find different use cases for the team to be able to use it effectively.
[00:08:00] Andrew: And then what it end ends up happening is that at the end result you get a higher ratio, higher proportion of the ICP that you're acquiring with non icp. And overall, like what you saw IST over time is the significance that it has on, on the bottom line and the revenue for the.
[00:08:15] JH: You mentioned a little bit like on inbound stuff, you know, if you can identify who's an icp, you can route them differently and, and do importance of their, kind of filter them. Does that mean that when you're defining an ICP you need to do it off, like characteristics that are easy to identify or like publicly available? Or should you also be thinking about like more proprietary information or data you're gonna have on users as being a factor too?
[00:08:35] Andrew: So I think it's, it's always gonna be a combination of both. And, uh, so I can tell you a little bit about how we did it at Hot Show in the early days and then how that understanding evolved, uh, over time. So when we first started looking into like identifying idea, customer profile, I think the very first thing that we just did was we took a look at our own internal data and we said, let's take a cohort of users.
[00:08:58] Andrew: Been with us or customers that had been with us for longer than 12 months paying and had a certain level of activity. So the activity component was the own proprietary data we were looking at, like how active were they within the product and how regularly were they using us? And then we, we just use a service like Clear Bit or any one of these data enrichment services that took a look at, okay, who is our, uh, overall demographic?
[00:09:20] Andrew: And then started breaking that down about various thermographic and demographic properties. So company size, age, Alexa, rank, uh, role of the individual. And from there you could really start to see some patterns, uh, emerging from different, uh, segments and different properties, individual properties, the.
[00:09:38] Andrew: And what we tried to do in the beginning, the first uh, step of this was we just said, okay, like, I'm just random things now, but let's say like mid-market product managers, product designers, and like one or two different use cases. This was the, the profile and what we slowly started to realize like over time was like it served us well from the beginning.
[00:09:55] Andrew: Like that's the data we had at hand. Like we needed some formal definition that we could go to and we could draw to, but as we grew our understanding and our learnings as well grew with. And we slowly started to realize it was just a really like narrow view of looking at only our existing customers.
[00:10:11] Andrew: Because again, like as I mentioned in the beginning, there was a direct result of the early product that we had built and, uh, the, the market we had. And it also has like survivorship bias in it of the existing, uh, user base. So we then slowly started to, to layer on different levels of complexity. We also.
[00:10:27] Andrew: At some point did some, a lot of like pricing and packaging research, which a big part of that is like understanding who the buyer personas are. And through that we actually started doing a panel study. So this was like the next layer of evolution was like not only looking at our own audience, but introducing like hotshot to a new panel who didn't know our product to service and then tried to understand, okay, what were those demographic or demographic properties that showed interest that were, had the highest likelihood to buy, or the most willingness to pay?
[00:10:55] Andrew: And we then could say, Did this audience align with our previous understanding? Are there maybe some audiences that we haven't attracted well enough yet through our product or through our marketing that we could be going after? Because there's a big opportunity in the willingness to pay or the likelihood to buy, and then it just like, Every stage.
[00:11:11] Andrew: We started layering on a new understanding of our customers and, and new learnings. Uh, not to go on a long monologue, but I think what it end up, uh, getting to in the end, what we realized was that having like a fixated definition of, uh, like this is who they are. They're like, uh, product managers. This role that is not necessarily the best way to go about using the ICP because you end.
[00:11:34] Andrew: Like neg and ruling out a large portion of the audience, it could be a good fit. So what we ended up doing was looking at building together a scoring model to understand to what degree do they fit the ideal customer profile. And through that research, what we ended up doing was like, let's say you had, we found that product managers perhaps were like much more engaged.
[00:11:55] Andrew: They stuck around for. They would get a three points in the scoring model, whereas maybe let's say digital marketer, some in the medium range, they would get a two and then like other roles like customer success would get a zero or one. And we did this for each one of thermographic properties that we had and each one of the things.
[00:12:11] Andrew: And we based it not only off of, we based off a panel, we based it off retention. We based it off recent, recently acquired customers. So what ratio made up the recently acquired and we also based it off. Conversion rates. So we, we took a cohort that was six months old and we looked okay, out of everyone that signed up there who ended up converting and were there any sort of firmographic or demographic properties that stood out that converted better than most?
[00:12:35] Andrew: And again, that could have been fed back to marketing and say, Hey, like, There was only a small amount of these, uh, this, these properties that we got through in this cohort. But they had really, really strong, uh, signals and they were really strong conversion, uh, rates at the end. And then we took all of that data and ultimately what we ended up having was the scoring model.
[00:12:52] Andrew: And it was, it was quite beautiful. And then because you could see like as the score. Increased, so did net retention and so did overall revenue that we're generating from these, which is ultimately the end goal that we're trying to achieve.
[00:13:03] Erin: There's a lot to dig into there, which is great. I have so many questions for you, but one of them was, you know, if you think of the ICP as a binary or not, and it sounds like no, that the scoring allows you to think of it on a spectrum of let's say zero, to much more than zero, and, and so I'm curious, how do.
[00:13:21] Erin: Use that score. Right. So, you know, presumably you would want to, you know, market to and find people with higher scores, give them more of a sort of velvet rope experience once you've identified them. But how do you think about, you know, breaking up how you address these sort of different tiers of score or different ranges of score?
[00:13:42] Andrew: So even though you still look at it as a range, you still do form groups within those ranges, but it's, it's not to say that if somebody doesn't fit all five criteria, that they're not an ideal customer profile. So I think that's sort of one of the mistakes we made in the beginning was just like, okay, they had to fit these five criteria.
[00:13:59] Andrew: If they didn't, like, if they weren't more than five years old, they're not a good business fit for us. They might have been like extremely fast growing startup and uh, they would've made an extra customer, but cuz they didn't have one of the five or one of the 10 or whatever it was. So one is like, you still definitely wanna have this, uh, understanding of who the team thinks is the ideal custom profile, like who they are as, as a general, as a whole.
[00:14:21] Andrew: But then having them as a range in the spectrum then allows you to sort of qualify leads a little bit better. One of the first ways we really started using it was in our lead qualification scores for sales team. So the scoring model then allowed them to say, okay, uh, if the score was below five, like we are not going to try book demos with these companies.
[00:14:40] Andrew: If it was between five to 10, we would, uh, and then if it was like 10 plus, like we really wanna be pushing to try and book demos with these c. So this really allowed them then to have clarity on like, to what level of degree do we want to, to be going into driving sales. The same thing again on marketing as well is the score itself.
[00:14:59] Andrew: You could end up seeing okay, like which campaigns, uh, were driving higher acquisition of certain levels of scores. And then you could then dive into and say, okay, like what was driving that score at the end of it as well. But, so I think there's many, many ways that you can filter then that down as well into support into other areas of the business.
[00:15:17] JH: Cool. And on the scoring itself, you kind of gave the example of, you know, maybe this job title gets a three, this job title gets a two and you know, zero one and stuff. How much of like coming up with the like weighting and the score scale and stuff for all the different criteria is like an art versus science thing?
[00:15:30] JH: Like is it something where you just kind of have to get started and be like, We kinda have a gut feel that, you know, job title's pretty important or you're doing something more like a stricter analysis to come up with like the scales and the weighting of the different factors.
[00:15:41] Andrew: It's a combination of both. So it's strict analysis in the, in terms of, so what we would do is, let's say we're looking at retention, uh, now, and we would say, wanna say like, who has the highest retention? We would just take a property in c k, how, uh, does company size impact retention? And we'd do like solo people, like one man companies.
[00:16:02] Andrew: Two to 10, 10 to 50, 50 plus, and then we see what does retention look like for those individual segments. Then we'll take the same thing and we would look at conversion rates. So like we take the cohort and we see out of that cohort the size, like one person company or two to 10 thing. How many of those converted look at recently acquired customers?
[00:16:21] Andrew: So we would take these different and then the panel study as well, and then you, from there, you get to start. Pattern and a trend like which ones end up emerging better than others across all four vectors. And then from there it's like giving a scoring. And the scoring then is like based off of the data.
[00:16:36] Andrew: But you need to somehow like use your intuition and gut feel sometimes because there may be a little bit of conflicts between what you see maybe on retention and what you see on the acquisition fronts. And then, Those dials and those levers you play with. But you can ultimately see, like if you've done things right at the end, is like when you map out your entire user base against the scoring model and then you map it against verticals like revenue and uh, retention.
[00:16:57] Andrew: And you ideally wanna see this nice looking slope that increases as score increases.
[00:17:03] Andrew: All right, A quick awkward interruption here. It's fun to talk about user research, but you know what's really fun is doing user research and we wanna help you.
[00:17:11] Erin: We wanna help you so much that we have created a special place. It's called User interviews.com/awkward for you to get your first three participants free.
[00:17:23] JH: We all know we should be talking to users more, so we've went ahead and removed as many berries as possible. It's gonna be easy. It's gonna be quick. You're gonna love it. So get over there and check it out.
[00:17:31] Erin: And then when you're done with that, go on over to your favorite podcasting app and leave us a review.
[00:17:40] Erin: How often, you know, you talked about updating your ICP and your scoring sort of regularly or constantly. What is your recommendation in terms of how to do that? Is it, do you pick up the pace as you scale or,
[00:17:53] Andrew: yeah, so I think like this exercise we probably revisited every year. At Hot. And I think it depends on the stage and growth. Like I think for our stage and early stage now, like we constantly evolving our understanding of the market. Like we religiously reviewing it and we we're updating the understanding of it. I think as you start to hit scale, you have the luxury of data. Like things get a little bit more predictable in the system.
[00:18:18] Andrew: Uh, that's when you can slowly. You don't, it doesn't need to be as frequent like as we are now, but at least like I would say a yearly to twice a year. It needs to be something that needs to be revisited, that needs to be edited, but it should rather be seen as more of like a living organism that evolves as your understanding evolves over time and having a central, uh, way for teams to be able to contribute new learning.
[00:18:41] Andrew: To it. So sales is on customer conversations day in and day out. Like they're learning new things, the market's evolving, the climate's changing, like, so it's always important to be keeping like a pulse on who the ideal customer profile is because it can really shift quite rapidly as well when different market conditions maybe present themselves or new technology emerges.
[00:19:01] Andrew: Like there's many, many different reasons that can evolve who you're targeting at any given.
[00:19:07] JH: You had called out the survivorship bias of like being something to be mindful of and that was kinda where my mind went from an early part in this conversation. I'm curious how you think about that. So you have the situation of these customers are doing great, retaining them really highly and stuff.
[00:19:20] JH: These people are churn out. So, you know, you start to fixate on the ideal people who are maybe, you know, the highest retention and stuff, but like there are probably things you could do or consider prioritizing, investing in to, to change that churn behavior of this different cohort. And like who's owning that part of it?
[00:19:35] JH: Is it like product needs to identify those opportunities and make the case, and then if you see the behavior change, it comes into the icp? Or like, what's the order of events and how do you make sure that you don't. You know, get too narrow minded?
[00:19:45] Andrew: Get too, I think that's actually where the panel comes in really helpful, is that you're presenting your product or service to a totally new audience, and then you're trying to understand, okay, is there an audience that we're perhaps not serving today that we could be doing a better job of?
[00:20:01] Andrew: And then like if you do see the opportunity, then, then yes, there's definitely the role of user research, the role of product, to really dig into those different use cases, to dig into those audiences and try to understand is there a bigger or better opportunity for us out there. And another area I think as well is also good to look is even just taking a look at some things like Google Trends.
[00:20:19] Andrew: I think we did this, uh, as well at hotshot at some point and really try to understand. Where was the market heading? Like, uh, how did we see things evolving over time and were we serving the right segment for the way that the market was evolving at that point? So I think there's, there's different factors that you can look to, uh, when it comes to trying to understand, okay, are we serving the right audience?
[00:20:39] Andrew: And, uh, is there perhaps maybe a big opportunity that. We've just neglected or maybe our marketing hasn't spoken to or our product hasn't been built for yet.
[00:20:47] Erin: Yeah, and this obviously it gets into business strategy, right? Cuz anyone at any stage, no matter how big or small, is dealing with finite resources, right?
[00:20:55] Erin: And, and part of what an ICP allows you to do is to focus, right? We can't serve everyone. We can't serve. The entire market of every human being on earth. Um, so we need to focus either a lot or a little bit within, within that. And yeah. I'm curious how you use some of this data or the ICP to make a call between say, you know, we're not at a hundred percent saturation in our identified market, or I c P, we could get more there.
[00:21:22] Erin: Right. But we see this emerging. Other audience that we think we could serve better and making those sort of decisions of, you know, going deeper versus expanding. Is there research here that you do that can help you make those calls? Or is that just a business decision?
[00:21:39] Andrew: I think those are incredibly hard decisions to make and they always feel like very big risky gambles.
[00:21:45] Andrew: Uh, when it comes to like, it's like do you double down on something that's working or do you try and go after something new? And I think like, You can build conviction through research and, uh, through speaking to customers like, uh, continuously, like getting a pulse and feel what's in the market. But I think ultimately that's just like, it comes down to business decision that needs to be made.
[00:22:06] Andrew: It's like, do we wanna double down or do we like, wanna continue playing in the space that we're in today?
[00:22:11] Erin: Yeah. And then once you've made that decision, you can use this sort of work to then really identify those audiences and, and the hard work of understanding what they care about and how they buy and all those sorts of things.
[00:22:21] Andrew: Exactly. Yeah.
[00:22:22] JH: Yeah. Cause there is a kind of, um, or I guess actually I may ask you a different way, is how much do you think about like, the total addressable market as part of icp? So like, did we have these customers? They're great, we retain 'em at a high rate. They pay us a ton, but there's not that many of them. Like should they be our icp? Is that, is that part of it or is that a separate exercise.
[00:22:40] Andrew: No, I think that's definitely a part of it as well, is like you wanna understand how big is the opportunity, because if there aren't many of your ideal customer problems, then it's not ideal. So you ultimately wanna be ensuring that as well.
[00:22:52] Andrew: That when you're defining your ideal customer profile, that the segments and the audience is big enough and there's a big enough market for you to go after for it to make sense. Because my parents said like, you really need to pick focus. And you say, okay, this is who we're gonna be serving. This is who we are.
[00:23:07] Andrew: And if you're gonna invest all that time and company's resources and energy, you need to be sure that okay, there is room to grow within that space. And it's not gonna be something where like six months or a year or two years down the line you're scratching your heads and saying, okay, what now? And on that as well.
[00:23:20] Andrew: I think at some point, like hot show we did next, we had defined our icp. But what we also did like for focus to go in further and something that we, uh, we got from Reforge, they've got some really great, uh, courses and is the idea of this anti persona. Who are you not serving and who is a bad fit? Uh, so just as much as like you wanna understand like who's the ideal is, it's also really helps the team gravitate and like set an understanding of who they're going after, when they know who they shouldn't be going after as well, and who they shouldn't be, uh, spending time and on, because then ultimately like, If you have both sides, you understand who we are serving and who we shouldn't be serving.
[00:23:57] Andrew: Like everything else you do, like the way you handle support or the way you produce content or the different channels, like everything just becomes so much more clearer and you're able to then double down on what's really gonna work.
[00:24:08] Erin: Awesome. We talked a little bit about this before, but. You know, are saying you're kind of grouping together the different scores and the different levels. So I'm curious like what you've seen this look like in, you know, companies you've worked with in terms of, do you have like tier one, tier two, tier three, and are they sort of groups of. You know, person level personas combined with firmographic companies or like, what's the, the artifact you might end up with here in terms of this is our ICP or constellation of ICPs and anti ICPs, and then we're gonna build out some, some playbooks, some, you know, ways that we're gonna work with these different groups once we've identified.
[00:24:48] Andrew: Yeah, so I think the first thing is you would end up getting to like a, a generic icp. So you're gonna say like it's mid-market SA businesses between two to 500 employees and maybe a certain like specific vertical industry that you're going after. So I think you still get to that end result. Uh, then you can say, okay, like this is our key, like our main ideal customer profile.
[00:25:11] Andrew: And you might have maybe. One or two others, uh, that, uh, run tangental to that, that fits in, uh, with it. So you still do the exercise of like the grouping and you say, okay, this is who we're going after, but you're mindful that you're not like fixated, that you need to hit every one of those criteria in order to be qualified and, and to meet the needs of the things.
[00:25:30] Andrew: So it, it just starts, uh, like that to begin with. And then you use the scoring as a way to see like, to what degree, uh, do they fit and do they meet the needs on that.
[00:25:40] JH: One thing we like to poke on usually with this stuff is what do you see when teams do this poorly? Like they go out and they try to do ICP and they end up with something that's not very impactful or useful, is it that they, you know, make the wrong inferences off the data?
[00:25:52] JH: They overcomplicate the scoring. They don't come up with an artifact or a way of referring to it internally. Like what are the things where this can go wrong and not be useful despite good intentions?
[00:26:01] Andrew: I definitely think over complicating is one of those ways that you can, like analysis paralysis. You spend way too many, much time, you have way too much data at hands that you don't know what to do with it, uh, afterwards.
[00:26:14] Andrew: And I think the best ones in, in like projects that I've been involved in as well, working with other companies with identifying ideal custom profile is really like when you can get to like three or four characteristics that clearly define like what's gonna make a good fit. And then from there, those three or four tend to be, uh, really actionable as well, so you can get to the end of it.
[00:26:34] Andrew: So like one of the, the big indicators we found early on at Hot Show was like the single biggest indicator if a customer is gonna become successful with us or not was their Alexa rank. And it seems very obvious. It's like if a comp, if a company has a really low Alexa rank, they have a lot of traffic and uh, they made a good fit.
[00:26:52] Andrew: But it was such a simple metric that allowed, like for optimization to begin with. And in the early days we used different services and. We tried different tools out there that analyzed all the different parameters and then tried to put together like an overly complicated scoring model that ultimately didn't produce much results.
[00:27:08] Andrew: But that was like one of the very first instrumentations of the scoring model was we just looked at. What was their Alexa rank? And that's something that it's very, very, uh, powerful because it's very easy for the team to understand. It's very easy for everybody to actually action and to use when it comes to like analyzing who they are, where they're coming from, was something that you could easily go then and take a look at.
[00:27:30] Andrew: Okay, what are the top 1 million Alexa sites look like? Uh, what are the industries? What are they serving like? So it really adds a lot of value from that perspective. So I think. The thing I've seen is like when we tried to like overcomplicate things, like that's when it's hard for the team to understand like, what does all this mean and how do we use this?
[00:27:51] Andrew: But the simpler, uh, you can get these things, I think the better because. They need to be adopted by everyone in the company. And the worst scenario you can be in is when like you've over complicated things and then people just gone with their own roots anyway, and they end up having their own individual definitions and classification systems, which ultimately like gets people fighting against each other instead of working together.
[00:28:12] JH: Yeah. Yeah. It's sort of the, the GA's law phenomenon, right? I don't know if you all are familiar with that one, but the, uh, a complex system that works is invariably found to evolve from a simple system that worked and then a complex system designed from scratch, never works and cannot be patched up to work is like the maximum.
[00:28:27] JH: And I think that's very true for a lot of things, but it feels like kinda what you're getting at is you need to have something that's working and gets traction, and then you're gonna revisit. You know, regularly. So it'll, it'll get more complex, but don't jump in the deep end.
[00:28:36] Andrew: I've been extremely guilty of like, over complicating things, uh, to start and then like learning the hard way. They just like the simplest solution always. And then iterates from there. What did you say was the law again?
[00:28:48] JH: Uh, GOs uh, GOs Law. Yeah. John GOs. It's like a famous systems book. Yeah, it's, uh, it's a good one. It's true for a lot of product work, but I think this thing also lends.
[00:28:56] Erin: Cause I think it, you know, the problems always seem very complicated, right? It's like, ah, how are we gonna double our revenue this year? And so it's like, clearly we need a complicated solution. But that's not the case.
[00:29:06] Andrew: Not the case.
[00:29:07] JH: Well, I think you want. The making it complex makes it feel more like valid almost. Right? It's almost like there's like some theater to it. Look at how much work and effort went into this. It, it's gotta be good. Right? And so there's that trap too.
[00:29:18] Andrew: Exactly.
[00:29:19] Erin: So what do you, you talked about this a little bit, but, so the, I think the idea of the anti ICP is really interesting. So what, what do you do with. Those people you just like put a big X on your website when they try to create an account or spear really mean to them, ignore them. Like what do you do with folks that kind of fall outside of the folks you wanna focus on?
[00:29:41] Andrew: Yeah, so I don't think you ever wanna be mean or .
[00:29:46] Erin: Not good business practice. Yeah,
[00:29:47] Andrew: yeah. Alienate people coming to you. But I think it just helps give clarity on high use of, uh, these customers. And, uh, so like an example, uh, that came for example, like let's say at Hoja, we had one of the cases in the early days, Nancy persona was engineers, this was, this evolved over time and change, but it wasn't gonna be a tool that they could use for debugging, like if their app had broken. So what that meant was that like when we thought about features, then like we weren't gonna prioritize any feature requests we got. That meant like, how do we debug, or when we produced support documentation, like this was not gonna be a use case that we wanted to let people know how they could do it.
[00:30:28] Andrew: And so it just means. Further deprioritizes those use cases like, and it gives clarity to the team. Like, okay, well I get this in support requests a lot. Like what should I do with this? And then the easy answer is, sorry, this is not a use case that, uh, we currently support, uh, hot. And if it does change in the future, we can let you know.
[00:30:48] Andrew: So there's ways that you can still like, uh, come to the table with like trying to be supportive, but ultimately like just being transparent with the end user and saying, okay, this is not what the, the product is being built for. If you figure out a way, like, let us know, we'd love to, to hear any suggestions, but it's not something we have immediately on the roadmap or something that we are gonna be support.
[00:31:07] Erin: Yeah, and that's a great point because you know, people might very well stumble upon, especially with product led growth or just easy to use products, and they stumble upon and like, great, this product's gonna be great for me. And they start having a really bad experience. And for someone to come and just tell them, it's like, look, sorry we didn't build this product for you is, you know, it's a really good service to that user.
[00:31:27] Erin: Maybe they can go find something better for their use case, or like you said, maybe eventually they will be the use case, but not doing anyone any favors. Keeping bad fit customers around, trying to bend it into their will.
[00:31:41] Andrew: Exactly. And that ultimately, like what ends up happening is it, does it backfires?
[00:31:44] Andrew: Is that like people, then the word of mouth goes and like this product is bad, word of mouth. Like I used it and it doesn't do anything. And like, yes, but it wasn't built for you to begin with, but we just didn't explain that to you clearly, or we didn't communicate that effectively. So, uh, yeah, definitely think is all, and as you said, like in product led businesses where.
[00:32:03] Andrew: Anybody can just get in. They can try like the, the switching costs or the, the ability to like kick the ties is very, very easy. Being upfront with that and the way you communicate really helps like avoid that negative word of mouth that you really don't wanna be.
[00:32:16] Erin: What are some of the best things you've seen happen in companies who have done a good job with building and innovating and iterating on their ICPs?
[00:32:25] Andrew: Retention, uh, goes up, uh, I think this is like a big, big thing as well, like through Chm. I've interviewed like over 180 guests now from some of like the fastest growing source businesses. And definitely like a key thing that stands out in all of them is. Have a really, really good understanding of who the ideal customer profile is.
[00:32:43] Andrew: There's strong alignment in the organization, everybody's building for their use cases, working towards it. So I think the ultimate end goal is that you have happy customers that end up sticking around with you for a long time. Uh, and don't leave because. You get to understand their problems deeply. Like you can empathize with them like entirely.
[00:33:02] Andrew: And then the whole organization really has this good in depth knowledge that they can serve them better.
[00:33:07] JH: You, um, you would call that a little bit. One method that can be useful is, you know, having a panel of like perspective users, non-users, getting that signal from them through conversations. Are there other, uh, research methodologies that prove useful when you're trying to like identify your ICP.
[00:33:21] Andrew: Yep. So I think there's always, the panel is typically done like with surveys from that perspective. So it's more like on the quantitative, uh, side, but you're still trying to collect, uh, quality of insights. There are different other mechanisms. Obviously always use interviews like number one, I think on my list as well for gathering, like really unique insights.
[00:33:41] Andrew: And then there's different channels I think that you can explore as well. So there's like really great select channels and select groups out there where you can gather insights and you can gather. I think you can also, like when you're looking to try and understand which segments aren't being served, uh, well, or you could be doing a better job of even sometimes like leaning over and looking at, uh, competitors and seeing, okay, what are some of the requests that are coming in through their product roadmaps and, uh, the thing, what are some of the complaints that you're perhaps seeing on sites like G2 Crowd and is there perhaps an opportunity where your product or service is doing better, or an audience there that's being neglected?
[00:34:18] Andrew: So there are quite a lot of different channels that you can take a look to, but I would say like nothing beats like speaking to customers and really trying. Like speak to as many people, even outside of your customer base as well, but that match the profile.
[00:34:32] Erin: We know how you can find people like that.
[00:34:34] Andrew: Yeah. I have an idea as well.
[00:34:38] Erin: Churn, uh, churn fm. That's your podcast?
[00:34:42] Andrew: Yeah, that's it.
[00:34:43] Erin: I've definitely listened to it. It's good. Why turn and not retention?
[00:34:47] Andrew: Yeah. , I think just playing on the pain a bit more. Yeah. So when I decided to start the podcast, like the idea was I wanted to start a business actually, but I wasn't sure what I wanted to do yet.
[00:34:58] Andrew: So I thought, let me pick a problem that I know if you ask me ceo, what they would pay to solve the problem. They would say pretty much everything in the bank because they know it's gonna pay back in compounding interests. So Uhhuh, Uhhuh, double down on the pain. I think that was another. Yeah.
[00:35:11] Erin: Makes sense. I find it very interesting when you hear, we hear that internally too, you hear a lot more talk of churn than retention and obviously they're opposite sides of the same thing, so
[00:35:20] Andrew: Exactly. And uh, the best way to solve, uh, churn is actually to focus on retention is not, it's weird because it's like the first place people say like, we need to soft churn.
[00:35:30] Andrew: And they're like, okay, let's go and interview people that are exiting and ask them why they've changed. Like, that's not really where you wanna be focusing. You're gone maybe . Yeah. What is making those that are successful with your product, like that's where you need to focus your time and energy, so Totally.
[00:35:44] Erin: Awesome. Well, any parting advice for folks listening on their ICP work?
[00:35:50] Andrew: I would just say like, it's a never ending, uh, process, and it's something that's like the earlier you can get your teams aligned and the working together to define it, uh, and understand that it's not just like a qualitative, uh, piece of work that needs to done or a quantitative like you need to.
[00:36:06] Andrew: The what with the why? And that's something we talked a lot about at Hoho. So like data's gonna tell you what's happening in your business, and then speaking to customers is gonna tell you why. So the, the exercise, it's important to have sales and success, uh, come to the party with us because they're the ones speaking, they're on the front lines all the time.
[00:36:23] Andrew: You need to have data, you need to have product. So an exercise, if you're thinking about doing it, it really needs to be like a company-wide initiative. And uh, there can be teams like user research leading in itself, but having this as a company alignment and bringing things together is like one of the most powerful things I think you can do in your business.
[00:36:39] Erin: Yeah. Fantastic. That tracks. Thanks so much for joining us. It's been a lot of fun.
[00:36:43] JH: Cool. Thank you.
[00:36:44] Andrew: Thank you very much for having me. It's been great chatting.
[00:36:49] Erin: Thanks for listening to Awkward Silences, brought to you by User Interviews.
[00:36:54] JH: Theme music by fragile gang.