Hosted By

Rabah Rahil
CMO at Triple Whale

Guests

Ash Melwani
Co-Founder & CMO of MyObvi
Dylan Ander
CRO & SplitTesting Expert

If You're NOT Split Testing You're LOSING Money

In this episode, we go over the power of Split Testing and how to use it to your benefit as an ecommerce brand.#Adspend

Notes & Links

📧Subscribe to Whale Mail for exclusive industry insights and in-depth marketing breakdowns: https://www.getrevue.co/profile/tripl...

🐦 Follow us on Twitter for Industry insights https://twitter.com/triplewhale

Follow the people featured in this episode here:

- Rabah's Twitter: https://twitter.com/rabahrahil
- Ash's Twitter: https://twitter.com/ashvinmelwani
- Dylan's Twitter https://twitter.com/Splittestingcom

Transcription

Dylan Ander (00:00:00):

Let's say you have two ad sets that like, again, you're just changing one thing or you're doing a formal split test on Facebook. Mm-hmm <affirmative> you can't trust Facebook's, you know, numbers there. If you're doing it that way, you actually have to plug it into a statistical significance calculator. Mm-hmm <affirmative> so let's say your CPA is $32 on one ad set and 30 on another. What is the probability that it's due to random chance you don't know? So although you think, oh, 30 is better than 32, it might be due to random chance based on your sample size mm-hmm <affirmative>. So you actually have to plug that into a calculator to determine if it's statistically significant enough of a difference to deem that is not due to random chance.

Rabah Rahil (00:00:41):

We're back for our first live episode in person with my partner in crime as always Ashani and Dylan Ander split testing.com the best CRO agency on the planet. Um, we're gonna go through a bunch of cool stuff today, creative split testing. Um, you have some really cool brands that you work with that will also talk about as well. Um, but you're actually a bit of a dark horse on Twitter. So for people that don't know you, why don't you give us some of your background and then, um, we'll just jump in

Dylan Ander (00:01:09):

Yeah, a hundred percent. So, uh, been in econ my whole life in college, I started a print on demand store for sloths. Like the number one seller was like sloth prints on the butt. It was awesome. And it was like, I'm lazy. So we were drop shipping back in the good old, uh, Facebook is an ATM days. <laugh>, you know? Yeah, good old. But, um, yeah, we started there. Um, I felt rich, but the thing is the reason why I did super well versus other people, there was a tool called split testing.com back in the day was a free split testing tool. We were not the creators of it. We, their domain expired and we inherited split testing.com. Wow. Full disclosure. I don't wanna pretend like we were those guys <laugh> but when I was in college, I was literally 17, 18 years old and split testing my headlines back in the day, cuz it just like made sense in my head before interesting split testing was much of a thing.

Dylan Ander (00:01:55):

Like Jeff Bezos had a quote, you know, uh, I, he said that he attributes 90% of his success to experimentation, meaning split testing. Sure. Right. So if you're not supposed to have your store, obviously do it. It's super important. So that was our differentiator where I would like 20% boost on a, you know, Lander and boom scale. My ad spin I'd feel like a rich college kid and it was awesome. So I carried that out and eventually, um, I don't know if you know, Insta that follow on follow software back in the day helped work on the API there. Oh. And I white labeled like 50 different people for them to go sell Instagram automation services. Wow. So I got my sales in, I got my services in, I kind of learned the ropes there and then I just had an agency cause a friend like needed some marketing help.

Dylan Ander (00:02:36):

I was like $5,000 a month. And I was like, I don't know, 20, 21. I was like, it's a lot of money boys. Let's do it. Yo. Anyone know how to do SEO. And we, we just figured it out that way. I don't know if you guys have started agencies before, but when you first start, it's like, you know, you literally, I don't know, Hey, someone needs Facebook ads. Oh yeah. We've got a guy like we'll do it. Of course. And like you start and do it and do it. And there was so that's next core media, which most people know me from. Yeah. Um, and next core media grew and grew at Geico and planet fitness. I was doing a massive SEO contract with Geico. Um, I was consulting for planet fitness on some of their new ventures. Amazing. Um, and then eventually I kind of let all my employees go, all my clients go cause I was just so miserable.

Dylan Ander (00:03:14):

Like dark time agencies rough. But now, so at, at that point, what I did, this was about three, three and a half years ago I pivoted the agency and I was like, well, like, you know, media, buying's cool. Like a two to one, three to one, five to one on your money. Where could I get people like a hundred X ROI? Like literally if I'm gonna go in on the services and that's CRO, which is pretty solid because one split test can make a global increase. I mean, things like your header, your footer, your nav, like, you know, like tons of things are all on every page in the site. So a tweak in there I just recently did a split test, um, for a company called Cosk. Yeah. Literally 1.2 million in new revenue, cuz we changed one word in the header. Like it's crazy what it does. So I just wanted to create that service that like creates long lasting wealth and huge improvements to all these clients and you know yeah. Um, and, and, and as ad costs only get more, more expensive. You need to convert better to, you know, compete on those CPMs as Ash can talk about. Yeah. For days

Rabah Rahil (00:04:10):

<laugh> I love, I love that and Tom's gonna kill me cuz I always say I love that. Um, first off though, I do wanna give a shout out to you for scraping yourself together to come to this. You were just recently at, uh, probably one of the best festivals worldwide.

Dylan Ander (00:04:22):

Bonnaroo baby <laugh>. Yeah.

Rabah Rahil (00:04:23):

So definitely I appreciate what was the best at

Dylan Ander (00:04:26):

Um, do you know churches?

Rabah Rahil (00:04:28):

Yes,

Dylan Ander (00:04:29):

Dude. They're

Rabah Rahil (00:04:30):

With

Dylan Ander (00:04:30):

A V right? Yeah. With a V they're a metal band live. Like they literally like, like it's crazy, really fun fact too. I've actually written three billboard, hot hundred songs. I used to be a fulltime songwriter in Nashville while I was running my econ bro. Who's

Rabah Rahil (00:04:44):

Guy, what are you doing with that? Who's this dish, what are

Ash Melwani (00:04:46):

We doing?

Rabah Rahil (00:04:47):

Who is this guy? I that's about all the, all the I have <laugh> um, but yeah, bringing it back to the metal. One of the interesting things, no pun intended, um, was we have a forecasting calculator in triple well. And the thing that blows my mind is if you can just move a conversion rate by one or 2%, it's like the, the tide that lifts all boats where it's just such an interesting, like, I mean, it's just absolutely massive impact. I mean, you guys really, you have a really actually interesting testing infrastructure. So I want you kind of take us through that cuz you kind of have this low fidelity fast moving and then you move into kind of a high fidelity. Yeah. And I want you to tear it apart or tell him what he is doing well or not.

Dylan Ander (00:05:28):

Well, I'm a new Yorker baby. Let's

Rabah Rahil (00:05:29):

Go. <laugh> shout

Ash Melwani (00:05:31):

Out NYC. Absolutely. No, I mean, I've kind of told the story, you know, over and over again. But I think the biggest thing that kind of helped us get through the iOS updates, um, was split testing a lot of things. Right. Um, we used to do a lot of big changes first and then kind of dial each winner and then split test, you know, the smaller things. Right. So, you know, probably August September-ish, you know, is when we saw that impact of the iOS updates, you know, costs were going up, um, we weren't testing enough creatives and we really thought like, okay, the platform is just being the way that it is. And, and then that, and that's, that's all we can do. Right. We just have to wait. Yeah. But you don't have to wait, you shouldn't wait. Right. It's like, you can go and start making four or five different landing pages and see if you can even just double your conversion rate, right?

Ash Melwani (00:06:17):

Like you like all of, I think the last two years we were using a single landing page and we made no changes whatsoever. And it got us to where we were, which was fine. But now things are harder. You have to go and literally test, I think multiple landing pages, at least quarterly. Right. You know, at least have a cadence where you're setting up four different landing pages structurally this way you, you know that like these big changes could probably move the needle way more than just split testing headlines. But that comes next. Right. So what we ended up doing is like, we, you know, we tested four to five different styles of landing pages found our winner. You know, there was a, there was a, there was one specific winner that just blew past everything else. And then that is where we started dialing in.

Ash Melwani (00:07:02):

Right. That's when you start split testing your headlines, um, moving the placement of your CTA buttons, you know, do we put the buy boxes at the bottom? Do you put it at the top? Where do you put social proof? These like these things is where now if you have like a baseline of like 5% conversion rate, how do you get it to six to seven to eight? Yeah. Right. So that's like the biggest thing that we've kind of utilized and like pulled as like the main lever of these like rising ad costs to really like just convert more people. Yeah. Right. Cost is gonna go up. Like we, we saw it last year. Our, our CPC was like on average 1.9. So a dollar 90 cents today it's like $3 and 20. Right. But we're converting double. Yeah. So that's how I think we've kind of like survived, but now getting past that like 20 K a day and spend, we have to improve the, the conversion rate a little bit more and then also balancing increasing AOV.

Ash Melwani (00:07:58):

Right. So that's kind of where I wanted to kind of learn from you where like, okay, where's that balance of increasing conversion rate. Cause you can do that. You could be like, Hey, buy this $30 product. There's nothing else available straight to checkout, boom bang. Like that's it mm-hmm <affirmative> but then like your C is probably still 30, $40. Mm-hmm <affirmative> so now how do you get that $60 AOV? How do I get that? A hundred dollars AOV with maintaining conversion rate? So that's like the biggest thing I think brands should be focusing on and wondering if you have any tips for, for everyone out there.

Dylan Ander (00:08:27):

I mean, uh, got a couple, there

Ash Melwani (00:08:29):

We go. <laugh> passed it up.

Dylan Ander (00:08:31):

So, um, first thing that I always said, I made a, a tweet that not a lot of people liked about a month ago where I said any brand that is even remotely profitable that rebuilds their website, absolute idiots. If anyone is considering a full site redesign and a rebrands I've literally changed a font and conversions tanked or increased. Right. So like things as little as changing, like maybe your dark logo versus your light logo, like in your, your header, all of it matters. So would you ever run ads without attribution? Hell no. Right. So would you ever change a whole page on your site changing like 20 or 30 things and not knowing which made the change? Yeah. So the thing about Landers, which I'll get into is in a second is like if you're testing different Landers, you have a baseline. It's almost like putting a bandaid though.

Dylan Ander (00:09:13):

If I'm being honest where let's say you test one new Lander in like a new structure, I don't wanna say atory. Let's say it's a, you know, tid for tie, you wanted to move the buy box, you're doing a URL, redirect test, that type of stuff, the incremental improvement. It, you need to understand each level that's changing. So if you change like seven or eight things in the Landers, mm-hmm, <affirmative>, you're not gonna know which of those seven or eight things happen. So let's say you had a net 15% increase mm-hmm <affirmative> there might be an element that has a net like minus 4% increase. Right. But it, so it could have been a 19% if you incrementally did each of them. Right. Versus, you know, like having no attribution of which change right. Actually made the change. So that's the one gripe with testing new Lander.

Dylan Ander (00:09:51):

So first off never rebuild your whole website. That's just doesn't work. Yeah. Um, so for different Landers, the thing I'll say about focusing on Landers is, um, where a lot of people overlook CROs the offer, right? So one thing that you guys do a great job, I always preach like the power of free mm-hmm <affirmative>, which is what can you give away for free? You can give away info products, you can give away content, you can give access early access, you know, app access. Like there's so many things you can give away. Like you guys give away. I think the free, uh, scooper, sugar

Speaker 4 (00:10:17):

Cup scooper, all that, all of those,

Dylan Ander (00:10:20):

All the, I, I, I made a thread, like, I don't know, also like a month ago and shouted you guys out cuz you did a good job. So yeah, it's in the offer itself. So the way that we start every campaign for true CRO is go and ask your customers what they want. Mm-hmm <affirmative> cause you guys like for triple well, for Avi, like you guys may think, you know, what your customers want. Right. But only they know. Sure. And like, if anyone says I would click this or I would do that immediately fire that team member, get rid of them or slap yourself in the face. Cuz you as one person, you're not necessarily the demographic of user. Right. I don't know if you own a E eCom story that you're not the demographic of user. So you go and ask your customers, customer feedback survey.

Dylan Ander (00:10:55):

Why do you buy from us? You know, what makes us different? What's the best part of the experience? Get a little bit of the emotional side, like for econ brands, like, you know, like, um, what are you looking for when looking for collagen or supplements, right? Like what do you hate about supplements? You know, all that type of stuff and you get it and you shape your offer. Mm-hmm <affirmative> and with the Landers, making sure that your offer is well shaped, like that's so overlooked that I would rather you take four identical Landers and just kind of change up the actual offer itself. Yeah. And then from there start testing the rest of those Landers. Right? So it's like get the meat and potatoes, which you can double your conversion rate by figuring it out. So mm-hmm, <affirmative> on, I also own a couple of supplement brands.

Dylan Ander (00:11:31):

So on one of them we added plus like, you know, get a free ebook. Right. And it smashed conversion rate. It was awesome. So huge increase like where you offer that little like free, uh, the little free product try, like plus a free, you know, um, one of the things for like, like, like hair loss is probably one of your biggest pain points for collagen. Yeah. Yep. So how to maintain 10, you know, an ebook of 10 ways to maintain positive hair, right? Smart. Like that's the kind of thing that you can add as an offer to increase conversion rate, fundamentally, forget the tweaks, forget anything like you have to obviously have to AB test this and you know, make sure that one offer's converting better than others, but that's really the meat and potatoes of it. So it's like ask your customers and formulate an offer. So it's like before you touch your Lander, before you touch anything, just go on a walk like deep breath, put yourself one thing. Shout out Nick Harris. He's uh, our new chief strategy officer split, te.com. Nick it's, uh, at Nick D to C with a two D two C shout out Nick. We love you

Rabah Rahil (00:12:25):

Just recently on mentor pass as well. Shout out Kenny,

Dylan Ander (00:12:28):

Shout out Kenny. I just joined actually I think like yesterday, amazing killing it. If you guys, uh, need some support and help, I'm here to help. Um, so Nick, he actually taught me about like, you know, he worked at a lot, a lot of nine figure brands before empathy mapping, which is every step of you have an issue with hair loss. You therefore go to goo and start Googling. What is your experience and emotions you have at that point, when you go to the next stage and you're doing your research, how do you feel, what emotions are going on there and literally mapping the empathy of the customer all the way through. And this also goes to retention, email marketing, text messaging, like literally empathy mapping, the entire process of like from before they find you to, when they're becoming a raving fan, right. When is weakest, you know, it's kind of like also a self analysis of cool.

Dylan Ander (00:13:11):

They take it 20 days in. They don't, you know, or like for example, like on, on supplements, like with my brands three days in, Hey, my joint still hurt this thing doesn't work. And I'm like, Hey, you need to give it like two weeks to really work. Especially collagen as well. Yeah. So now start on the empathy mapping, right? Where right after the post purchase emails, like two or three days in like, Hey, I know you're waiting, um, think long term two to four weeks. Right? So you're able to map every single stage. Right. Um, and getting that empathy mapping all along. So Nick turned us on to empathy mapping, which absolutely changed the game for acquisition. Because by looking at how they're gonna feel over here, you can market it other than just saying like, Hey, let's solve the problem. No, no, no. Like let's actually like, Y the problem in like solution is really well defined and like there's a pathway and it makes it so much easier to tell the story. Yeah. Make good ads, good creatives that teach it, you know, that they can really harp on. Like maybe like for, for collagen, like hair loss. There's like four situations. I don't know. Like you've a receding hairline. It's thinning thin, like, you know, there's a couple different angles. Maybe you have to go and test those subtopics and have you do it customer feedback survey. Yep. Da da. So it's like just literally map the USPS, that's it? Yeah. That's

Rabah Rahil (00:14:21):

Literally what we do. <laugh> yeah. The empathy map is really interesting cause I, I enjoy something called jobs to be done. And there's something similar where there's two ways that you can kind of identify the job to be done. One, you take, you basically have this essentially like buyer journey, but it's a little more nuanced than that. Or the forces they're it's same, same, but different. And the forces are interesting because you don't only get a purchase when you have, you have these pushing and pulling forces essentially, and you only get the purchase or the behavior change when, um, you get, they call it a tip in there. So it's, it's really similar in terms of that empty mapping. And I really like that. Um, so we're not talking over people's heads. Can you explain like your five, like explain to a five year old kind of what AB testing is? So people that aren't kind of hip to it and kind of know what we're talking about. Give 'em a little foundational knowledge there.

Dylan Ander (00:15:06):

Yeah, exactly. So I'm gonna kind of take it through the lens of how I do the CRO campaign, which is CRO is just conversion rate optimization. If you get a hundred people to your website, how many people purchase okay. That is known as your conversion rate, which, you know, most people know what the conversion rate is. Um, it's just the art of increasing your conversion rate is what CRO has become. It's more of an umbrella term where boosting your average order value. Like, you know, it's really just revenue per user. So if someone clicks on your website, how much is that worth for you? So he said, it costs about $3 and 20 cents. If you're making $4 and 20 cents like four 20 blaze, then like, you know, you're <laugh> then you're, you're making, um, you know, you're definitely making profit there. So understanding the revenue per user or conversion rate, they're a little bit, you know, same, it makes sense.

Dylan Ander (00:15:52):

Right. Same but different. So, um, you know, you go to your customer feedback survey, then you go to heat mapping, which is, you know, seeing how people interact with your website. Yeah. There's, you know, we use Hotar at our agency. So, um, we use all the click maps, scroll maps, see how people interact with the website. So that's another thing that we do of like qualitative data that we put in, cuz people are clicking on a photo that doesn't go anywhere. There's clearly a problem with how it's like presented, you know? Right. If people like, if there's like a Nagar that a nav bar that has like six things across and people don't click on two of them remove 'em, it's probably gonna help your convers rate. So there's things like that that will help. And then ultimately after we get to that, we do this massive audit.

Dylan Ander (00:16:28):

So we become like professionals in your brands of whatever brand we're working on. And then it turns into split testing. Got it. So split testing is, or another known as AB testing, right? AB testing. Like I like using split testing cuz you can have many variants, right? Yeah. So it's not just an a and a, B it could be a, B, C, D E F G. And then there's a whole bunch of multivariate tests where you're changing a and B and then C and D and they go against each other. Yeah. So, but the whole point long story short is that you're testing different things on your website. Button color is the most simple to someone who's just starting an e-commerce, which is you have a green button. If we get a hundred people to the green button or a thousand people for the green button and a thousand people for a red button, we can see how much revenue comes in for the green button. Yeah. And the red button. Yeah. And that's just the most simple example of cool. If the red button has more revenue per user, like revenue per click, then awesome. I'm probably gonna go ahead with the red button and implement it on the pages that we do so you can change literally anything. My developers have shown me that there's nothing that can't be split, tested on a website and it's been awesome. So, uh, does that kind scratch that?

Rabah Rahil (00:17:33):

It's incredible helpful. The, the other question I had, so there's everybody throws around a term statistical significance. So I have a question about that. And then the other question too, that I had is, um, you were talking about earlier, uh, I think the fancy term is controls. If you have, have too many changes to the page, you don't have a really good controlled experiment because you don't know, like to your 0.1 change might have been a plus 20% push, but then you also had this other change that was a minus five. And so another change that was a minus two, then another change that was like a minus three. And so net you're seeing a plus 10 increase, but if you'd only done that plus 20 thing and you would've done it in a more, uh, controlled manner, if you will, where you're only testing one or two variables at a time. Yeah. You get a lot more insight from the experiment. So I guess first question, can you explain kind of statistical significance? What are you looking for there? Is it like an impression rate? Is it something or impression threshold or traffic or what have you, and then also, um, do you think it's good ever to test a lot of variables or should you always be doing like one or two kind of max mm-hmm

Dylan Ander (00:18:38):

<affirmative> totally. So, um, statistical significance, I'm just gonna go stats SI. That's what we call it. So, um, if something is stats or not every single piece of marketing, whether it's email marketing, Facebook ads, like the best media buyers, sit there, plug it into a calculator to see if something's stats sick. Right. So what statistical significance actually means is the percentage that it change is due to random chance, right? Because if I flip a coin, which is, you know, 50, 50 on heads and tails, if I flip it a thousand times in a row, it can land on heads a thousand times in a row. Right. So it's the, you know, the random chance that like your version is so the, you know, you can never have a hundred percent statistical significance. Cause that means something as fact, but it could be 99.9 9 9, 9, 9 99. So, you know, in science like, and like medical studies, there's like, you know, a version of someone without the medicine and someone with the medicine sure.

Dylan Ander (00:19:35):

In the medical communities, if you have like a 99.1, that's considered like dangerous to put in mankind. Yeah. With us, we can go as low as an 85, 90% confidence interval. So for whatever you're doing it, you need two things. Right? So like a thousand people visited here and 28 purchased thousand people here and you know, 40 purchased, right. You just plug that into a statistical significance calculator to not make it fancy. So let's say, you know, a big gripe I have on media buyers is let's say you have two ad sets that like, again, you're just changing one thing or you're doing a formal split test on Facebook. Mm-hmm you can't trust Facebook's, you know, numbers there. If you're doing it that way, you actually have to plug it into a statistical significance calculator. Mm-hmm <affirmative>. So let's say your CPA is $32 on one ad set and 30 on another.

Dylan Ander (00:20:20):

What is the probability that it's due to random chance you don't know? So although you think, oh, 30 is better than 32, it might be due to random chance based on your sample size mm-hmm <affirmative>. So you actually have to plug that into a calculator to determine if it's statistically significant enough of a difference to deem that is not due to random chance because 30 verse 32, fundamentally, if you assume that it's always gonna be 30, you don't know for sure as a predictor of the, the future, but you'll know if something's statistically significant between right. Between two different ones. So

Rabah Rahil (00:20:53):

Love that stat CIG fans I'm hip to it, baby. Yeah. <laugh>

Dylan Ander (00:20:58):

How you look in stats SIG today,

Rabah Rahil (00:21:01):

Stat C amazing.

Ash Melwani (00:21:03):

Well, what does that, what does that number look like for you guys when you're running your test? Right? Mm-hmm <affirmative> is there, is there a minimum, like sessions per experiment that you guys are looking for to get to that? You know,

Dylan Ander (00:21:14):

So it's a hot take cuz there's a lot of opinions on this. Yeah. Because if you're a small store, like for example, um, I come to you and there's a test and I say, Hey Ash, there's an 84% ch like chance, right? Like based on the statistical significance, mm-hmm <affirmative> 84% chance that this change that we have here will make you an additional $40,000 per month. There's still a chance that it could be negative. Right. Mm-hmm <affirmative> like, you know, there's uh, yeah. 16% that I gave. So like there's a 16% chance that this is completely random. Right. And there's an 84% chance that it's gonna be a $40,000 increase. Would you like me to implement that on your site? Yes or no? Yeah.

Rabah Rahil (00:21:53):

You'd take that bet. Right?

Dylan Ander (00:21:55):

Bet. Most people would take that bet. So it's like that type of thing where a lot of people say 95% or nothing, right? Like when you learn in high school, it's like a P value. So if you subtract the statistical significance, P value is 0.05. So 95% statistically significant. So for us, if it's below 85, we kind of say to the entrepreneur like that we're working with, Hey, like this is kind of your decision. Here's the facts. Here's exactly what it is. It's below our desired threshold. Right. But the way that we implement it is if we see all signs, right? So let's say your conversion rate increase, cuz you have a KPI like key performance indicator of what you're trying to increase some split tests. We're trying to increase average order value. Yeah. Some were trying to increase conversion rate. Some were trying to increase, add to cart rate and don't care about the purchases. There's so many different like test types that you can run.

Rabah Rahil (00:22:40):

What have

Dylan Ander (00:22:40):

You, yeah. Termin, exactly.

Rabah Rahil (00:22:41):

Terminating event can be whatever you want type of

Dylan Ander (00:22:44):

Thing. Just like ads, you know, like deeper in the funnel is typically better. So, um, yeah. I mean the minimum is really like what we evaluate. So if you go and Google optimize, which it's free, guys, I recommend you use Google optimize, um, the drag and drop feature. Isn't always the best. You kind of need to know how to code. If I'm being honest

Rabah Rahil (00:23:02):

Or I've broken multiple sides of it, honestly.

Dylan Ander (00:23:04):

Yeah. So we use Google optimize, but no joke if we're swapping a photo, my dev like one of my lead devs was really good. Can take two hours to make that test because what about all the different sizes of screens? It's gotta look perfect everywhere. Yeah. Right. So a lot of people just swap in a photo hope, you know, from Shopify and they think it's gonna work. And it tanks it's like, well, this objectively is way better. It's like, yeah, because it's not dynamic. There's different mobile types. There's like, there's a lot to go on there. So right. It takes a bunch. And then last part of that is there's also secondary metrics. Mm-hmm <affirmative> so let's say our main KPI is like purchases. Right? Right. So we, or sorry, conversion rate, we care more about lowering our, our CAC than actual like ROAS. Right? Mm-hmm <affirmative> so like there's a difference between the two, right? You just want the lowest cost per new customer. So like net CPA, of course. Um, if you're working on that front end conversion rate, mm-hmm, <affirmative> take a look at your add to carts. Every time you get an add to cart with your abandoned email sequence and assuming you have a text message sequence, you get a value per add to cart. Right. So if that has gone up, but your conversion rate stayed the same.

Dylan Ander (00:24:09):

True. Maybe that's good to implement. Yeah. Let's say your average order value has gone up, but your conversion rate's the same. That's great. Might wanna implement it. So there's secondary metrics and let's say you're focusing on SEO and things like that. Dwell time, time on site. Yeah. Number of pages you visited all inside of Google optimize it. It links in like directly to Google analytics. So there's not a single stat that you can't see. Well, I mean maybe like per channel, Google analytics gets a little wonky. That's the only thing I don't trust Google analytics for. But otherwise if it's hard coded and set up properly, that's where it is. So people saying there's a number of like sessions I need and stats CIG and all of that, where 85 is my general rule. If like you had to like, you know, gun to my head, I would say if it's below 85, kind of not the best, but it's really advanced where you look at all the secondary metrics, everything looks positive, but your first needle mover didn't happen. Yeah. You may still wanna actually implement it. So it's case by case. But yeah. Anyone feel free to DM me with a Google analytics or Google optimized screenshot, and I'll tell you what to do. Yeah.

Ash Melwani (00:25:06):

But like, okay. So, so for example, right, say let's take the red button, blue button, right. Are you looking at, if, if for example you have a hundred visitors and obviously blue is in the lead, right. Let's just say, is that enough for you to decide or pull the plug on that? Or like, I feel like you need a cert, at least for us when we're doing it, we're doing at least 5,000 sessions per variant. Mm-hmm <affirmative> just so that we know that. Okay. There's no, there's no chance. There's no randomness at all. Yeah. So like what does, is there a set number that you guys go with or is it more so like, I guess it's kind of maybe up to the, to the owner as well. Like they want to cut it at some point, you know?

Dylan Ander (00:25:42):

Exactly. So, um, if you ask the data purists, right? Like, like statisticians, not marketers, like they would say we would need a thousand conversions

Ash Melwani (00:25:52):

For sure. Right. Thousand conversions

Dylan Ander (00:25:53):

Conversions. Got it. Right. So purchases. Right. So that's not how we go at our firm. Right. Because not every brand's in, I mean, we've got plenty of eight, nine figure brands that, you know, we've got a couple thousand per variant, you know, even if it's multi variant, like of like, you know, seven or eight versions. Right. Um, that's where it starts transparently. We used to work, you know, when we started out, we worked with smaller brands. We were smaller agencies of course. And we had a minimum of a thousand sessions sessions yeah. That we would need per variant. And we would never call a test. I don't care if it has to run forever. Right. I don't care if it has to run for three months, can't have less than a thousand sessions. Cuz if you think about it, if you flip a coin a thousand times, do you trust where it's gonna land? Yeah. I kind of do. I think it's gonna be close to 50 50, so right. A thousand is kind of, if you just think about it, I mean it's completely arbitrary. Right. But a thousand per variant is my bare minimum. We never even still today. Yeah. No matter what it is, cannot do thousand per variant. So if you have a five way test, you need a thousand in each one. But also I'm not condoning once you're at a thousand, leave it the more you can afford to run. Right. That's the whole point.

Rabah Rahil (00:26:56):

Right? That's that's how I feel where ultimately one, I think it is a function of the business size mm-hmm <affirmative> because not only do you not have the data, but you don't have the time, like you can't wait for like you need to like, and so if you can at least derive directionality from it and saying like, Hey, you know, like if you're a small business, like it matters where like, if you're a small business and you move your conversion right up like one or 2%, it's not really that big of a deal. But when you're an eight, nine figure brand like a 1%, one point bump on your conversion rate is like almost business changing. So I think there's a certain aspect of understanding the context of the actual impact of the changes where it's just like a hundred sessions probably isn't enough, but maybe 500.

Rabah Rahil (00:27:37):

Isn't like, you can, for me, it's almost like the sniff test of like, if you showed this to 500 people, what do you think? Okay, cool. Mm-hmm <affirmative> are you fine to go ahead with that? Okay. Let's go. Cuz like to your point, Dylan, like a thousand PE a thousand conversions might be a three month, a quarter period for some brands where it's just like, you know, so you have to get into that. Um, that's incredible question. What one, how does Google optimize or how do all these people do the holdouts? Is it cookies? Or like how do they bifurcate kind of mm-hmm <affirmative> the audiences that are gonna see these different things. Cause ideally you don't want people polluting each, like if you're in cohort a or cohort B or cohort C mm-hmm <affirmative> you don't want any kind of cross pollination. Correct.

Dylan Ander (00:28:17):

Exactly. So Google optimize it's set and forget in that regard. So if you land on a session ID, even if you come back, right. So let's say I visit a product page green button instead of like a red button. Um, if I come back to that same page, like a day later, if the test is still running, I will see that green button again. Mm-hmm <affirmative> so it's kind of like a, not like you kind of don't have to think about it if you're using, you know, any like the paid ones that are really expensive, like VW O AB tasty Optimizly they're all the same, but so is Google optimized. Got it. If they visit once there's a session ID and if they come back, you don't have to worry about it. They're still gonna see the same one. Got it. Right. But then if you cut the test and you're at something new, you know, you're back to your code base. Yeah, exactly.

Rabah Rahil (00:28:56):

Makes sense. Awesome. Um, what's the craziest test you've seen, like in terms of performance where you, cause we were talking a little bit offline about like you were changing like X, Y, or Z, and you were like, oh my gosh, we made a bazillion dollars for our brand. Like what are some crazy kind of weird tests that you have seen before where you're like, oh, that's weird or that's interesting kind of stuff.

Dylan Ander (00:29:17):

Yeah. So, um, one of my senior CROs, uh, at split testin.com, he, so like one big thing that I always say is above the fold on mobile, get your damn ad to car button above the fold on your fucking mobile

Rabah Rahil (00:29:32):

Device.

Dylan Ander (00:29:33):

Yeah. So one thing about it is when you go into Google analytics go to device and I believe it's screen type, right? Or there's a secondary filter for screen type. You can take an average of the screen resolution of the mobile phones that visit your website. So when you make mockups, you should use that to default, right? Because there's people of smaller phones or much larger phones. So optimize for the ones that your customers are. So that's a really good, deep tip that most people don't know. But again, go to that average and make sure that average at all costs that adds a cart button is above the fold on mobile. So one time, one of my senior CROs was like, well, Dylan, like, there's the quantity selector. Do you want me to remove it? And I was like, fuck it. Yeah. Why not? Yeah.

Dylan Ander (00:30:17):

AOV. And they're like, well, the AOV is gonna get hurt a lot. Right. Cause like, you know, cool. You give them the option. We looked at the heat map and two people out of 20,000 visitors <laugh> clicked on the quantity selector. That's amazing. And then I was like, wait, hold on. And then I went to three other clients, I don't know, 10 in 10,000, like literally point something percent of customers are using that quantity selector. So hot take quantity, selectors suck. <laugh> you know, you need to make your buy box, actually be the ones that work. So we remove quantity selector. And now what we started doing is we put the quantity selector last. Yeah. And then there's a variant of just completely removing it nine times outta 10, which is probably stats Zig there. So we actually, we actually have a killer database of over 1500 split tests.

Dylan Ander (00:31:01):

We ran. So like internally we kind of get to see what's working. Not yeah. The benefit. So yeah, we get that ads card button above the fold, no matter what. So even if there's quantity or not quantity selectors, but variance. Um, like for example, like yeah. There's variants. So whether they're colors for fashion or there're flavors for supplements, we take that add to car button and we've been adding it above that. Remember you said that. So, so we literally added above there. So it's like, wait, so you're saying the ads, a car button goes above, like my t-shirt size and I'm saying, yeah. So first off do not get anyone at home. Don't go and do this. Don't just, you have to test it. Say like, that's the whole point. This is not financial advice. This is not financially not. Yeah. So on this unqualified opinion and not financial advice.

Dylan Ander (00:31:48):

Yeah. Like we found that to be like a hot take test that like get it above the fold so that buy button in like you need high contrast. Right. Green is something you usually test. It's known to be one of the most inviting colors that like obviously green button gang. I kind of wanna make a hashtag on Twitter, like green button gang. Just, I tossed it a couple times. See merch shit. Put that on the shirt. Damn. That's great merch. Yeah. So yeah, getting it above the fold is just like crucial. So, you know, forget the, the quantities and all that. But the only thing with that is it, none can be preselected because what will happen, we found out on customer service is if that out card button is above the fold. Right. And they like it and they just click on it. Let's say there's strawberry banana and chocolate.

Dylan Ander (00:32:29):

Mm-hmm <affirmative> the default will be the strawberry. Right. So, but that adds cart button. They just like click it and go and buy. Right. They may have wanted the banana or the chocolate. So you need to make sure that like when you, you click the ads to card button, the little error message says, you need to pick a selection select. So make sure there's no preselected ones on there. As long as there's no preselected put that ads card button, like literally almost interesting. Like, so like, you know, product it like, like product photo. Yeah. Title, the price button. <laugh> like literally nothing before any content before any, anything. And it also helps price anger. So price anchoring is another thing that's huge. Yeah. So a lot of people say the larger, the AOV, the smaller, the text should be the, you know, smaller, the AOV, the larger, the text should be like the size of the actual price.

Dylan Ander (00:33:16):

Right. We found that to be like, not necessarily the most true. I'd give it like a 70, 30 that like that's wrong based on our like database of tests. Yeah. We find almost always giving that price larger makes a huge difference. Yeah. So the three things that you want them to notice right away. Well, four things really, which is the product, what it looks like, the name of it, um, the price of it and where they can buy it, which is that adds to car button. So those are the four things. So it's called price anchoring where if you're buying sneakers, which I know you hate. Yeah. Uh

Ash Melwani (00:33:45):

<laugh>.

Dylan Ander (00:33:46):

It has to be, um, if you're looking at like $750 sneakers versus $75 sneakers, you shop differently. Yes. So you, as the e-com owner, it is your job to make sure they know how to scan the website, which is price anchoring. So just running the test where you have this big fat, bold price, even if it's an ugly $5,000 price tag, which is a lot of money for most consumers, um, still want it nice and big. Cause it's your responsibility to make sure. So right. Those are two hot takes. I've definitely got some others, one that we've been doing recently as well, which is hard to test and optimize. If you don't have sick developers, shout out Keith, we love you. Um, the second photo being UGC in your car cell, you guys do that really well. Don't you? It was one of my CRO hacks. I don't know if Ash has been, uh, scooping off me on Twitter. Amazing. Little

Ash Melwani (00:34:30):

Bit, a little bit. He's ever this little bit he's yeah.

Dylan Ander (00:34:34):

He's he's on, watch him, watch him. I'm giving too much. Damn. I'm getting out of a gig.

Ash Melwani (00:34:40):

Well, one thing that you, so just from listening to you now, right? There's so many things I wanna go back and test, right? So now for people who are like watching and, and like writing down these notes, like all I gotta, I gotta change the font size price. I gotta move the at to car button. I gotta do this. I gotta do that. How would you, what's the best way to structure all these tests and kind of carry out efficiently for people. Totally. You know, with let's let's say like you have, I, I don't know, maybe, maybe give the structure for, you know, small to mid-size businesses so that they can kind of yeah. Carry out these task.

Dylan Ander (00:35:11):

So it's the same for everyone. Yeah.

Dylan Ander (00:35:14):

Top down. That is it. So if you go on hot jar, which you can download for free. So no matter what size business you are going, download hot chart for free. It'll take a while for sessions to collect, but there's click maps, which is where you see what people are clicking on and scroll maps. So one of the hidden tricks is looking at the scroll map because it obviously goes down, you know, less and less, right. So if you think about it, if a hundred percent of people are looking at everything above the fold and only 12% are looking at the footer, it, the, you know, above the fold will have almost 10 times the impact of what's in the footer. Right. So it doesn't mean things on the bottom. Aren't good. Right? Cause sometimes like you want a long form page and just the buy box at the bottom, but we always start top down because above the fold is what has most of the like massive improvements cuz the most people see it. Right. So if you're testing something, that's two or three, like we call 'em buckets down the page, right. If you're testing the third bucket on your page, you may only get 40% of people seeing that. Right. So you definitely want things that a hundred percent of people see right.

Ash Melwani (00:36:18):

Goes. Okay. Um, and then, so for example, within these buckets, right? Say you wanna do, like, let's just talk about the product page, right? The buy box section, right? There's like about three or four things that you can test there, right? Yeah. Mm-hmm <affirmative> are, are you doing one test at a time? Are you doing like a multi variant test? Um, is that if, if you're doing a test at a time, is that over like four weeks mm-hmm <affirmative> or you do it in a week? Like what does that, what does that structure look

Dylan Ander (00:36:42):

Like? Yeah. So the same way that we had a minimum of that thousand per variant, yeah. Two weeks is the minimum for how long a test has to run. Right? Never run

Rabah Rahil (00:36:51):

Tech, agnostic,

Dylan Ander (00:36:52):

Traffic agnostic. Interesting. Even if you get to a certain level and the reason is weekday traffic reacts differently than weekend traffic. So if you do it for one week, you may have a really impressive weekend or a sale or something like that. Try normal. So if you have two of them, exactly it normalizes two weeks gives you enough that like it is. So for our agency, you know, we've biweekly calls and you know, biweekly tests, that's the whole point. So it's like, you need to let it run for two weeks minimum. So I don't care if it's and the other thing too is if you're like a really nine figure brands, you can just select and Google optimize like 20% of your traffic for testing mm-hmm <affirmative> so think about it in terms of ads, like what percent of your ad spend is testing

Ash Melwani (00:37:30):

10%.

Dylan Ander (00:37:31):

10%. Yeah. So cool. So if you want to use, be really conservative. Yeah. Use 10% of your traffic for testing. Yeah. Yeah. If you're a smaller brand, you love to use a hundred, but that's a little more risky. Right? Right. So it's kind of risk reward. Based on your side, the larger you are, the more flexibility you have, which, you know, those

Rabah Rahil (00:37:46):

Sensors when I was at, uh, my old agency and we would do like the brand lift tests and stuff like that, that was always a challenge because they, you could only do it with this audience and then you have this holdout and it would be like, usually you want to test against like your best audience and it would, there's this kind of again, to your point, like this risk award of like, yeah, we're gonna get learnings, but at the same time, like by definition, the goal isn't performance, it's the learnings. And so you, you would have to kind of, uh, weigh the risk and reward, like you said there, um, do you ever cut off a test early? Like when you see it shit to bed and you're like, oh my God, I can't like waste any more traffic

Dylan Ander (00:38:19):

If it's negatively statistically significant. What

Rabah Rahil (00:38:23):

Is that? Whoa. No, <laugh> hold on. Hold on.

Dylan Ander (00:38:26):

So just like it

Rabah Rahil (00:38:28):

Real mouth,

Dylan Ander (00:38:28):

Hold on. <laugh> no, Bon is a different real <laugh> so the same way an increase can be statistically significant. Yeah. A decrease can be statistically significant. So it might be negative 95%, negative 85%. Got it. So what you can do is you can cut that test variant. So more traffic goes to the rest. Got it. So there will be a point when it's like, damn, we're like a week in, so I will test for a week. So a week is my minimum. No matter what, if you press a button a week is the bare minimum that you can go right at a week, mark, then you can cuz there's like, we call 'em shark fins. Cause it can vary day to day, just like Google analytics or Facebook ads. If you put it on a graph. Of course. So your daily shark fins, you know, at least a week will like remove the shark fin.

Rabah Rahil (00:39:11):

Got it. Makes sense. That is interesting.

Dylan Ander (00:39:14):

So you can cut after one week if it's really, really bad, but you can't promote a winner in a week. Must be two weeks.

Rabah Rahil (00:39:19):

Yeah. No, that makes a ton of sense. I actually like that. Do you find reductive or additive testing is better? Like do you think when you take away stuff is better or do you like to add stuff?

Dylan Ander (00:39:32):

Totally depends on the brand. Okay. And we have data to back that up. Yeah. Fashion brands. When you look at it like sneakers, a lot of them like there's like your, you know, you're above the fold stuff. Like your ads card, button, description, warranty, shipping and testimonials. And that's it. So something like supplements, what you put in your body is sacred. Consumables are like sacred. There's a lot of infos are sacred. So, well, I mean, relax math. <laugh> easy, easy. So yeah, it really does depend on the industry. So, but I mean like as, as an agency, I would say a lot of the times we come in and people are like, man, we've added this thing. We've like worked harder. That's one thing that a lot of like people at home think that like the physically larger the test on the screen, the bigger the test, right.

Dylan Ander (00:40:20):

We've changed small little things that like are physically small on the screen, but make a massive difference. So understanding that difference too. So whether you remove or add understanding that first off, removing a small thing, it might be a bright green, like, you know, like the Klarna after pay all those things, we've moved those around or sometimes removed those and nothing got hit. But the thing is, all we did was remove that and like 40% conversion rate improvement on a PDP because really it's that bright color. That's distracting them from everything else. So just cuz it's physically like small doesn't mean that it's not a big test. Right. So that's number one. And number two, when we start with new clients, we find there's a lot of bullshit on the site. Yeah. Just remove it. Like yeah. Usually what, like one of our first tests that we run on homepages, undoubtedly is removing elements.

Dylan Ander (00:41:07):

That's like remove elements, like, like we have a nomenclature that we use like internally. So like homepage remove elements is pretty much the first test for every single person got. So it's like we test removing different options. So like, it'll be like an ABCD test. Like we think this one could definitely go this and this could go this and this could go and maybe move this guy up and we can structure that as one test to be pretty bold and give yourself, obviously, if they have enough traffic, like now, like we've got a higher threshold of clients we work with, but you know, making sure that we can. So removing things typically is like our first go to cuz we can like, I mean you can code it in two minutes. Yeah. Like you can on Google optimize, you can right. Click remove. Yeah. So like you can remove elements real fast.

Dylan Ander (00:41:45):

Yeah. So like, I mean I can sign a client, you know, two minutes later do that. So that's kind of like some quick ones we get. Yeah. So, and the other thing too, shout out, Nick, again, he told me that I was always talking about it wrong, which is like, we'll find some, like even in the sales process, I'd be like to my agency owners out there, we'll find some quick wins, we'll get some, you know, big, big, early victories. Like it's like, no, they're actually compounding data points. Yeah. Meaning if I improve your homepage, conversion rate 7%, which is only globally gonna be like 1%, you know, something like that. But then we improve the next test. Like another 4%. You now have a, you know, compounding improvement effect. And then now your emails, if they're going to the homepage and something else, now we do another test like three months later that does a 20% increase on the homepage. From where you first were assuming your conversion rate stayed the same, you're looking at like a 50% increase, not a four plus like 11 plus right. Or seven plus 11 plus you know, 20 it's exponential. So the compounding data Fs crucial.

Rabah Rahil (00:42:43):

Do you look at like new and returning cohorts as different or

Dylan Ander (00:42:49):

Absolutely.

Rabah Rahil (00:42:49):

Because going back to your point of like changing the website, I got a really amazing, uh, audit done. Shout out, uh, audit bunch of amazing

Dylan Ander (00:42:57):

Canadians Sean, you're the man, Sean. He's amazing Taylor. Yeah. They're they're our biggest partners, honestly.

Rabah Rahil (00:43:01):

Amazing people, amazing humans.

Dylan Ander (00:43:02):

By the way, if you look at audit, you'll see that it's backed by splittin.com database.

Rabah Rahil (00:43:07):

Amazing. We, we only hang with the homies. Um, and they did this amazing audit and we actually overhauled, uh, a site. This is when I was at my agency and it was beautiful and everything worked out well, but we saw a little bit of actual degradation in the conversion rate. And my thesis was that, I mean, when we looked at the actual data, it was almost all coming from returning customers. Mm-hmm <affirmative> and because you get a vibe and you get like, oh, and you move stuff around, it's kind of like, you know, you, you change your own grocery store layout or something that you're really accustomed to and people. And not that it wasn't a great thing. It was just like, people can get really used to inefficiencies <laugh> mm-hmm <affirmative> and we removed those inefficiencies and yeah, it took a little bit, it ended up normalized and netted out, excuse me, in the positive.

Rabah Rahil (00:43:51):

But it was just interesting to me because everything made sense and like it all worked out and the new customer conversion rate was fantastic, but we did take a little bit of hit in terms of those incumbent cohorts. Um, how do you deal with that? Or how do you think about that? Or like, uh, cause I I'm with you on that where there's just overhauling a website is just, I'd rather just play with Landers all day and just put paid traffic that way versus overhauling the website. Cuz that in, in retrospect, I think that's probably what we should have done is just kind of eased people into a new aesthetic versus like, oh, Hey look at this beautiful new site,

Dylan Ander (00:44:24):

Whole thing. Yeah. So I've got a fat piece of advice for everyone at home. So, um, there's four main ones. So when you go to Google analytics, you have those filters. There are those two little boxes that are at the top. You can post production, you can take my fingers <laugh> um, and you can filter by almost anything. You can make custom filters, but just go mobile and desktop. So when we're choosing a test as a winner, your mobile code base and your desktop code base are literally different websites. Like they're not even the same, like Shopify has two different code bases for your mobile and desktop website. So a test can win on mobile and lose on desktop. That's interesting. So that's number one. So if you're, if you're a CRO firm or anyone else, like if you guys are split testing, internal, whatever it is, if you just go based on total, uh, you just failed interest because like I said, you can have different changes.

Dylan Ander (00:45:16):

So think of four and then, so other than mobile and desktop, there's also new verse returning and that's also pre-done filters. You just hit the drop down and, you know mm. And, uh, yeah. And it hits, you know, new and returning. So those are four things that you need to look at. So I always make custom filters of desktop, new mobile, new desktop returning and desktop new. Yeah. So I don't know if I, I messed that up, but you, yeah. You guys get what I'm saying. So it's like, you know, you have four custom filters that now that's how we really judge mm-hmm <affirmative>. So everyone at the bare minimum look at mobile and desktop because we've had, we've found a lot where like, we're marketers, we're working on the computer. So not a lot of people have this. So I actually like pay for my team members.

Dylan Ander (00:45:58):

We have a little clip that you put your phone right next to your computer, which highly recommends you have your phone here should like put it on silent so you can work. But like, when you're looking at your website, pull it up right here and look at your phone. Cause people don't do that enough. Mobile is 90%. So yeah, this means 90%. This means 10%. Yeah. So going back on the mobile and desktop, what's super crucial is, you know, looking at each of those cohorts. So if it's winning on, let's say like returning users, tanks, but new users, it's great. That's called personalization tracking. So yeah. So personalization means you're feeding different experiences to different people based on what is on the website. So new verse returning is a behavior. Um, mobile verse desktop is a behavior. There's also location based, right? Like if you go to Neil patel.com, like, you know, the SEO legend, it says it literally it's my favorite.

Dylan Ander (00:46:46):

It says, I'm looking for a business in Brooklyn, New York to improve. <laugh> like, is it yours? So it's like personalization obviously to like location. So yeah, you can feed and Google optimize and you press it permanent. So it let's say like, you look like literally, if you don't do this already, Ash, go back to all your historical tests. Mm-hmm <affirmative> and filter every single result for new verse returning mm-hmm <affirmative> and you may be like, damn overall, it was losing, but it crushed it for new new ones. You can go on Google, optimize, set up an experiment for personalization and press like go and permanently. And now for the rest of eternity on your website, only new users see that experience versus old. So it takes no code, no anything mm-hmm <affirmative> you can personalize after that. So there's mobile desktop, new returning. And, and that's like, I don't know, 80, 90% of like the actual experimentation that we do the results.

Ash Melwani (00:47:35):

So you're probably deploying, like, even if it's just like, all right, mobile is crushing for new, you'll probably just deploy that and can the rest see that? I think that's where people kind of mess up. Yeah. And it's like, cause we redid our entire homepage. Right? Yeah. And what we saw was the same thing, same thing that you said, right. Returning was a lower conversion rate, but new, new customers were, were higher. Yeah. Um, I think we made that executive decision cuz overall it was more of a, it, it, it just depicted our brand a little bit better than what it was before, which was like just collection and buy boxes and stuff. But our customers were used to that. Right. Mm-hmm <affirmative> so that's probably why it decreased a little bit, but I think over time, right. People, customers get used to it and they're like, okay, I can find things again. But to your point, it's like, what if we just made that entire experience a little bit more personalized to new, to new visitors and then not have to worry about reteaching customers where they find their favorite products. Right? Yeah. It's a great, great

Dylan Ander (00:48:28):

Point. So on the homepage, the ones that we find are the best is like I said, above the fold. So to contextually now, like, you know, the listeners can like kind of follow along the logic, the hero image, like, you know yeah. Above the fold and the headline are two of the biggest that are ripe for personalization. Yeah. So let's say like for gummies, it's like the best collagen in the world. Right. So it's like that's for new visitors, right? Yeah. Cause they're now meeting the best collagen in the world. Exactly. What about the new returning, like returning users, like ready to re-up on your, you know, thing. So if people

Ash Melwani (00:49:01):

Deal something. Yeah.

Dylan Ander (00:49:02):

Well, no, not the deal. This is a headline where you're just changing the messaging to returning users, which is like, you know, um, improving hair loss. Right? Like now you can drive the value proposition instead of like introducing them to a brand. Cause they already know you they've been there before. Right. Right. Right. So it's like not necessarily a sale or a discount, you can go that route. Like I've done that before where like 20% returning discount. But like the thing is, you don't know if they purchased or not, you can target target it that way. True. But you can change your messaging just a little bit of like, you know, meet your new, you know, plug for health. Yeah. Versus like, you know, continue improving your health. Right. So those are two very different headlines, very different headlines fundamentally. Right. So that you can have for new verse returning

Rabah Rahil (00:49:43):

Or even, I guess you're right though. You'd have to have some sort of, uh, tracking in terms of purchase, but even being able to, and I think I, I get we're saying, cuz it does start to get into the offer, but um, a lot of people never subscribe on the first purchase. Right. Mm-hmm <affirmative> but at the same time, like almost everybody's subscription is like the best economics purchase. Like all things being equal. Like you would love people to hit the subscribe and say, versus mm-hmm <affirmative> actually just one off purchase. And so there's some things you can get into there. That's so fascinating. I never thought of it that way.

Ash Melwani (00:50:13):

Probably. So would you, so would you say that you would be pushing these changes live just through like Google optimize or is this something that you can also like how, how do people do this? Like is it only through that? Is there apps that you can do that?

Dylan Ander (00:50:24):

No. So there's, there's a lot of relatively too. Um, yes. You can build native, like I'm like, so you can build anything native. Yeah. Right. Like, like especially, I mean think like 90% of our clients are on Shopify. Right. Um, you can build anything native. That's like, you know, there's if your developer ever says, no, it means you need a better developer. That's

Rabah Rahil (00:50:41):

What the money's for. <laugh> that's

Dylan Ander (00:50:42):

What the money's for. Holy Lord. <laugh> my team's been blowing my mind lately. I'm like, whoa, you were able to do that. That's crazy. Yeah. So, um, you can make it custom. Um, Google optimiz has a really like convenient setting, which is how you go into the editor and however you changed it. Right. You can just go in and instead of like, you know how there's like the AB test multivariate test personalization it's right there. So you hit that personalization. You do the test experiment and you just click go and that's it. So that's the easiest way to do it. Like the least amount of code, the easiest way it's free. Yeah. That's why I recommend Google optimize for it. So there's like other personalization, like if you Google personalization, like that's the future of CRO. Yeah. Is how personalized can you make your website to every single different user? The message for people who are coming through Google ads, if you got down to it, like some of our biggest clients they have like, you know, I don't know, like collagen dash one collagen dash two collagen dash 5, 6, 7, 8, 9. And they all are identical except for like the

Ash Melwani (00:51:44):

Headline,

Dylan Ander (00:51:45):

The headline. Yeah, exactly. So it's like to match that. So getting into crazy personalization is how you ultimately scale, scale, scale, scale, like, yeah. Ads are great, but like each of these personalized like channels, like email and SMS, you know, like scaling ads, it has an ancillary benefit by getting more people in, but you need to make sure it's personalized. So yeah. Google optimizes the easiest way. But if you Google personalization, you'll see like Rebi, Rebi is like a really cool Shopify app. Do you guys use it?

Ash Melwani (00:52:12):

Uh, we just started looking into

Dylan Ander (00:52:14):

It. Yeah. Cool. So would recommend, um, for re and, but only for people that have like many skews, right. Because like it's really good at personalization and like AI learning of like

Rabah Rahil (00:52:22):

That's the Amazon, uh, kind of mimic, right?

Dylan Ander (00:52:25):

Exactly.

Rabah Rahil (00:52:26):

Because Amazon, like everybody's like Facebook news feed equivalent of Amazon, like everybody's, it's a bazillion different homepages. Like everybody it's that Amazon homepage is essentially unique to you. And so this is the Shopify

Dylan Ander (00:52:39):

Thing. So this is a shop by plugin. It's called Rebi Rey. Shout out if you wanna reach out to us for a good partnership pod that's yeah, we do. <laugh> sponsor to the podcast. <laugh> yeah, it's really good. But the only thing that I do use it for is, um, recommended purchases. That's what I found that Reba has nailed it. So that's a different area of personalization. Yeah. Than the actual stuff like personalization, if you Google it, like how do I personalize my, you know, E eCom store? It's like, oh, recommended purchases. Or you may also like, that's where Reba's really good for you put it in. And their AI ultimately like figures out what is actually the best. So right. For you guys. I do think Rebi on the bottom of your PDPs, right. Would probably be solid. Right, right. Just a guess. Just not financial advice. Yeah.

Ash Melwani (00:53:20):

No, no hundred percent. No, it makes sense. Cuz I think we're doing it in a way of where we're doing that research of like, okay, what are some similar or even through triple, it tells us frequently bought together. Right? Exactly. So it's on these products, like, all right, you may also want this, this and this. Right. Which is similar to the Amazon, you know, at Al to cart. So I think that might be good cuz then at least there's some type of intelligence that's like looking at all the factors and then showing that dynamically. I think Shopify had that. Right. I don't know if it's any good, to

Rabah Rahil (00:53:48):

Be honest. I knew I only knew of the plugin.

Ash Melwani (00:53:50):

Yeah. There's dynamic, uh, product recommendations. I don't know if like there works.

Dylan Ander (00:53:55):

Well, you do need to be on Shopify plus for that. Um, so that's like a minimum two to $4,000 per month going not everyone can yeah. I'll belly ache of the GA four problem too.

Rabah Rahil (00:54:08):

Get me start on GA four. Yeah. I can't believe Google's just gonna blow up your UA. It's nuts. There's like no roadmap. There's no succession plan. It's just like, oh F you guys, you gotta use this horrible, horrible thing called GA

Dylan Ander (00:54:19):

Fours horrendous. So horrendous. Um,

Rabah Rahil (00:54:22):

We're talking about Google analytics for, for anybody. And when we say UA, it's universal analytics, which is like, mm-hmm, <affirmative> the dummies version of, you can just put it in an absolute works versus yeah. Google analytics four, you actually have to understand data pipeline and you have to put a nice data pipeline together. It is. Yeah. Atrocious. So like you have to be really high or from my experience, it's been like really bad to implement it's you have to be really high level.

Dylan Ander (00:54:44):

It's fucking worse. It's horrible. Google. We love you. And we also hate you. Thanks a lot for nothing. No it's so yeah. So luckily Google optimize now integrates with GA four. I actually have like a little focus group of a couple of fellow nerds who are like trying to figure it out. Believe I cracked the code.

Rabah Rahil (00:55:00):

Oh, look at this, reach out to my man, if you need that G

Dylan Ander (00:55:02):

Shameless plug, which is not available to the public yet, but we made a software that goes straight into the Google analytics for API renames everything and talks and looks, we we're we're considering like I'm talking to a couple e-com owners, like what they want. Theoretically. My goal is to make a universal analytics for clone.

Rabah Rahil (00:55:23):

Amazing.

Dylan Ander (00:55:24):

So that's an upcoming thing. So like just sticking on the CRO side, like, you know, um, I've got a couple SAS products coming out that like literally will automate a lot of split testing, like full machine learning. I wrote a multi-arm bandit that like gets to statistical significance faster other than just 1, 2, 3, 4 1, 2, 3, 4. Cause you can start autopilot pushing traffic to the better performing ones, improve it faster. So I've got a couple innovations coming up that, uh, you guys will be hearing about

Rabah Rahil (00:55:50):

This one door closes another opens. I love that. <laugh> um, a couple more questions. One. When you're talking conversion rate, do you like to look at how many people actually come to the site or are you going straight from PDP to purchase? Cause I know people sometimes have a content play and I, I always recommend depending on the context and I want to answer for you, but just as my viewpoint was when people are coming to read content, I don't con I consider that like them coming to my library, versus when people get into like a slash product or something, I consider that them actually kind of coming into my shop. And I actually like to measure that. Cause if not, if you have a huge content database, like, and you're getting a ton of content traffic, you might have like a 0.5 conversion rate and it like people aren't coming there to shop. And so it's like, it kind of can pervert that conversion rate. How do you think of that?

Dylan Ander (00:56:40):

I want you to answer that first. Yeah.

Rabah Rahil (00:56:41):

How do you think of that? Cause you guys do have pretty strong content

Ash Melwani (00:56:44):

Too. Yeah. I think the biggest thing is we'll never like exactly what you said, right? It's I'm I'm sending you there just for the purpose of education. Yes. Right. So for example, if I think we send out weekly recipes, right? Um, yeah. It's a way to kind of cross sell. So like if you've, if you purchased a fruity cereal and we're showing you a recipe of, uh, cocoa cereal, waffles, right. It's like, okay, you could still use your fruity cereal collagen for that. But then like, it kind of shows you there's other flavors that you can utilize it for, but it's tough because you don't, there's no, there's no call to action on the recipe page. Right. I want you to be there to either continue looking at other recipes. So like the CTA is go to the next one, not go to the product. Right.

Dylan Ander (00:57:26):

May wanna test that homie.

Rabah Rahil (00:57:28):

Well, and it be fair though, to just push back is like, you might already have the

Ash Melwani (00:57:31):

Product. Yeah.

Rabah Rahil (00:57:32):

So you've already bought it's

Ash Melwani (00:57:33):

True.

Rabah Rahil (00:57:33):

It is true learning how to use it and consume it.

Ash Melwani (00:57:35):

And so a lot of those, a lot of those, like for example, a lot of the text and emails that go out because of that, a lot of the conversions that happen after are view through not click through mm-hmm <affirmative> right.

Rabah Rahil (00:57:49):

Z

Dylan Ander (00:57:50):

Manager command. Yeah. It's like, like, so I do agree with you. So I have like three or four clients that just smash SEO. Yeah. And they're looking at the blogs and they're like, how do we convert to customers? How do we convert to customers? And in Google analytics there's pathways. Yeah. That you can see entrance pages and exit pages. Yeah. Yep. So are they coming from like the main homepage clicking to a blog and then just bouncing? Yeah. Are they first on a blog then go to the homepage product page and then drop. Yeah. So like, you know, you kind of figure out where the leaky hole is in the bucket. Yep. So with the content, there's definitely different KPIs like for SEO. I mean, it, it sounds crazy, but like I'm actually trying to help the SEO team further as like a CRO agency.

Dylan Ander (00:58:32):

Like let's get your time on page time on higher. Like let's get exactly like all those metrics that you get to improve and get better usable content. Yeah. There was a client today where we looked at their heat map and they did a little summary of the top in this article. You'll learn this, this and this. And they didn't have hyperlinks going down. Yeah. So we saw in the heat map that it was, and I was like, let's test this and boom, oh my God. The time on site, like went through the roof, it was like, um, like a 60% increase in time on site, like of the page and the clickthrough rate to other pages was like a bumped 30%. Mm. So you're a hundred percent right. That there's different, like intentions and, and also go back to empathy mapping. Yeah. That's the other thing too, of like understanding where they are in their journey with you. That's what it ultimately comes back to as a north star.

Rabah Rahil (00:59:15):

Makes sense. Powerful anchor links. Um, yeah, that makes a ton of sense. Think I just lost my train of thought.

Ash Melwani (00:59:21):

I had a really big question, I think. Right. Um, the biggest thing that I think we were talking about before we started, right. Is CR uh, conversion rate versus like a, uh, AOV. Right. Um, right now I think, you know, towards the beginning of the year, it's like, how do I improve my conversion rate? I just wanna acquire customers at the lowest cost right now. Right. Mm-hmm <affirmative>, which is, I think we nailed that. And I think we took out all the, the friction of buying. It's like, all right, you come to the website, you go through this landing page, you understand what the product is? All right. You can either buy one bottle or two and that's it. Right. Um, to, to some people's point it's like remove the two and just go all in on the one. And your conversion rate will probably skyrocket, but my C is still gonna be 30, $40.

Ash Melwani (01:00:03):

I'll probably move a little bit more volume. But like, how do I, I think the main question here is how do I balance between conversion rate, but then AOV as well. Right. Mm-hmm <affirmative> so like, my CAC is like, I can right now, like 20 K a day, my pack is at 50. Right. My AOVs at 60. Yeah. I can't, I can't get anything lower as I scale. Right. Mm-hmm <affirmative> so the only thing I can do, like I can make better creatives. I don't think it's gonna move the needle as much as moving the needle on CRO. Right. Right. So either I get people to buy more or just, I think that's my, my thing is I need to get people to buy more. Right. Mm-hmm <affirmative> so how do I do that without impacting converge? Totally.

Dylan Ander (01:00:43):

Totally. So, um, comes down to that original part of asking your customers what they want and just give it to them. Like, what is their goal is their goal to change their lifestyle. If it is you need three products, you need three different products in a bundle together to change your actual lifestyle or health or hair or anything like that. Mm-hmm <affirmative>. So it comes back to that offer that you give, dig in more to the offer, right. Find offers that are more money, more value. Like you get what you pay for. Right. And you can sell what they're paying for too. So that's my answer. Number one. Mm-hmm <affirmative> answer number two. Um, the default, um, of like where you see, um, you know, of like which bundle you see first, right? Of like what they click on. That's another way that like easy way doesn't really hurt conversion rate. Like make sure your default is like, if you have one, three and five, sorry guys, if you, if you have one, three and five in your buy boxes, make sure three or make sure three or five is usually, I mean, there's something we test, but make sure three or five is like your default that people go to first. Yeah.

Ash Melwani (01:01:40):

And it doesn't hurt conversion rate or you've seen it not hurt conversion

Dylan Ander (01:01:44):

Rate. No, it doesn't hurt conversion rate and boost say V a bit. So again, not financial advice. Go test, try it. That's another one. And then obviously post-purchase upsells. Yeah. So that's another thing that you do. Prepurchase upsells is something also that actually can hurt conversion rates. Yeah. If you continue selling. Yeah. Yeah. So agree free. So in the card itself, if you're selling it doesn't mean don't do pre-purchase upsells. Right. But post-purchase upsells. Meaning like to, to those who don't know at home, one click up sell as your Firestone, you have the best mastermind. We love you. Um, the game. Oh. As, as there's the man, um, you know, Zipify one click up sell. So like after they purchase, it's like, would you like to buy another? Yeah. So that offer right there. You can go crazy with those. Yeah. I have a trick that I use for supplement brands, which can, you can work for works best for consumables, but here's a little dirty secret. Mm. What does your upsell look like after people buy? I know you probably have like a couple of conditionals. Yeah. But overall what's your most popular upsell?

Ash Melwani (01:02:41):

Um, so that we have two, right? So for one of our main funnels, which is our burn, um, supplement, we get another one for X amount off. Right. Mm-hmm <affirmative> that, that has like a 16% take rate. Mm-hmm <affirmative> I tried a new one for our collagen one. So on the landing page, I'm testing, showing, uh, four flavors. And then in the post purchase, I'm showing four new flavors of like, Hey, these are limited edition flavors. You can only get it here that had a 20% take rate.

Rabah Rahil (01:03:07):

That's cool. Which crazy

Dylan Ander (01:03:08):

On one bottle

Ash Melwani (01:03:09):

On one bottle. Yes.

Dylan Ander (01:03:10):

Cool. You wanna hear something crazy? <laugh> are you ready? Yeah. Hold your pants. Try.

Rabah Rahil (01:03:15):

Nice, please.

Dylan Ander (01:03:17):

Upsell eight bottles.

Ash Melwani (01:03:19):

Jesus.

Rabah Rahil (01:03:21):

If anybody does, it's your community. I've seen some of those pictures.

Dylan Ander (01:03:25):

Yeah. It's crazy. So like so much if you're it's it's insane. Right. So it's like, here's

Ash Melwani (01:03:29):

The thing. I probably won't see these people for another year though. <laugh>

Rabah Rahil (01:03:31):

That's

Dylan Ander (01:03:32):

The one. Would you rather the money now or now? Or never's

Rabah Rahil (01:03:35):

A fair point. That's a fair point for sure.

Dylan Ander (01:03:36):

So, and there's a discount, but you're getting it now and you're getting guaranteed sales. So the thing is, um, we have a conditional based on like, if it's a first time buyer, second time buyer again, personalization. Yeah. Right. Um, I upsell way more bottles than they need. So for my supplement brands, my smallest upsell is five bottles. Right. And the take rate is around like nine or 10%, which is not much less than yours, but mine's worth damn a lot more. And also when you push more units together, your cogs go down. Yeah. Cost of goods go down for sure. Which is again, more margins. So it's like less

Rabah Rahil (01:04:10):

Picking all these things. Yeah,

Dylan Ander (01:04:11):

Exactly. So interesting. The post purchases be an animal. Like some people go really aggressive on email marketing, you opt into their like pop up. Yeah. Um, they literally send you four emails a day for the next three days. Yeah. You either get hell of unsubscribes or hell of sales. You get both. Right. So it's like a big burn strategy. Like, I mean, PE already got the sale. Why not upsell like six or eight? You know? Cause if people really like the product, they're like, ah, I'm gonna use this anyways. Like I can just put it in my house. Now I get a 40% discount. And with that many units, you can give a 40% discount and your unit economics are killer.

Ash Melwani (01:04:46):

Yeah. Makes sense.

Rabah Rahil (01:04:48):

That's amazing. Wow. I will caution people though about hitting email too hard. I had a, uh, agency that took over when I left and they were sending like 78 million emails a month and like the first month and a half, like the numbers looked amazing. But the growth in, like it was a negative net growth in his list subscriber. And like people are getting pissed off and like you can't for sure. Email something where you need to make sure that it's tactful. Like it can't just be a money grab and SMS is even worse or, uh, more sensitive where it's just, just such an intimate medium. So definitely again, whatever the goal is, like, if it is a money grab, you know, smash, email do thing, but yeah, exactly. Email should bet full play in my opinion. And you need to make sure that you're generating value for the user, not just using them as a bank account. Um, one last question and we'll wrap up. Yep. Shipping. You guys are messing around with shipping. How like important is free shipping, no shipping. Cuz you guys did some really interesting stuff with shipping, right. That changed.

Ash Melwani (01:05:42):

We're about to bit. We about to. Yeah.

Dylan Ander (01:05:44):

Well, so you tell me what your plan is and I'll tell you if I, uh,

Ash Melwani (01:05:47):

So the plan, um, customers, if you're listening, sorry. <laugh>

Ash Melwani (01:05:53):

So we wanna best college <laugh> so we wanna, so right now we offer free shipping. Yeah, right. Um, right now what we do, no threshold. No, there's no T there's technically no threshold, but when you do add to cart, it does say you've unlocked it at like $25. Got it. Just so that it's like, oh, I've earned this right customers. Um, you're gonna get it regardless. Um, so now what we wanna do is we wanna get these people who are buying just a single bottle to get one more thing. Yeah. So we we're thinking of raising that threshold to actually put it at 50 and then charging flat fee of four, $5. Right. Mm-hmm <affirmative> um, and then if, if that first threshold is 50 and then the next threshold for free gift is 75 and then a 1 25 and then 1 75. Right. So it's like one more product just to get to the next tier. It's not like two or three. Right. It's like just add one more thing. Right. Mm-hmm <affirmative> so we're thinking of using free shipping as that initial, like gamification like, oh, I just unlocked this. Well, wait $25 more. I can get a shaker go. Yeah. $25 more. I can get this. Right. So that's the test we wanna do. Um, just have to get around to it.

Dylan Ander (01:07:00):

Truthfully. That sounds like several tests in one I would stick to, you know, like if you want to gamify it, that's like a whole plan. Right? Like you would need a CRO rollout

Ash Melwani (01:07:09):

To confirm, well, it's all there. It's basically all there. Yeah.

Dylan Ander (01:07:11):

So, so here's a dirty trick. Everyone has done a countdown timer before an e-commerce. Right, right. Is it a really countdown timer? Sorry. Customers have a, it's not a real countdown timer. You can still buy after 10 minutes or whatever. Yeah. Um, there's there's only three units left by now, before they're out of stock. God, that's kind cool. So, you know, those have gotten you, especially on the shoes. So like

Rabah Rahil (01:07:35):

The shoes might be real. That'll be fair. <laugh> yeah, they are. That's

Dylan Ander (01:07:38):

True. Well, you never know. That's why you put it there. I, so, um, so what we've done before is keep your full free shipping mm-hmm

Ash Melwani (01:07:44):

<affirmative>

Dylan Ander (01:07:45):

But say free shipping at 50, at 60, 75 and a hundred, but it's always free shipping.

Ash Melwani (01:07:53):

Interesting.

Dylan Ander (01:07:54):

So people think it, and then they get to the cart and they're like, wait, hold on. I thought it was $50 unlocked. It's 30 click boom.

Ash Melwani (01:08:02):

Choose.

Dylan Ander (01:08:04):

Yeah. So this is so it's like an intentional typo or error. Yeah. That's in the customer's favor.

Ash Melwani (01:08:09):

Right?

Rabah Rahil (01:08:10):

This is really interesting. So I don't know if you guys' logistics, uh, can do this, but somebody was actually doing something similar where, um, basically it was like a two day shipping and you would get like the two day shipping for free, but like not for free, like, everybody was pretty much like their logistics were capable of two day shipping mm-hmm <affirmative> but you, I don't remember if you would pay a little more or you would hit a threshold and then you'd get the two day shipping, so you'd get it faster or what have you. But it was in the same vein of expedited delivery. That's that's what it was. Yeah. Yeah. In the same vein where like I get my stuff faster because I bought more. So anyways, I'm kind of in the weeds there, but that's uh, that's in incredible. Um,

Ash Melwani (01:08:46):

I'm gonna try that immediately.

Rabah Rahil (01:08:47):

Immediately. That's about a being

Dylan Ander (01:08:49):

Ash's totally DM me a bunch of times. Like, Hey man, I'm gonna implement this right now and get back to you. And he never gets back to

Ash Melwani (01:08:55):

<laugh>. I dunno. Some of these tests. Yeah,

Dylan Ander (01:08:59):

Damn. I'm like, man, you're reciting some CRO hacks buddy.

Ash Melwani (01:09:02):

<laugh> yeah. This one I'm gonna have to definitely test. Cause I, I was thinking about it last night. I was talking to Ron about it and I was like, all well, we're gonna have to, you know, change the shipping. And like you said, it's a lot of steps. So makes sense to just up the tier and just,

Dylan Ander (01:09:17):

Yeah, that's it. One other thing on this too, in the announcement bar at the top, there's this thing called, I think it's called AOV countdown or like AOV countdown time or something like that. And it says free shipping at like $75 and they add $25 to cart and it says 50 more dollars to free shipping. Like it's a dynamic Shopify app. Like it's a plugin.

Rabah Rahil (01:09:36):

We had something similar for, uh, I used to work for, uh, my agency. We had a beard care client and we had a bundle builder page cuz one of the things that he was getting crushed with was AOV like his, his Mo AOV was

Dylan Ander (01:09:47):

Anthony make

Rabah Rahil (01:09:48):

Box. No, ah,

Dylan Ander (01:09:49):

No

Rabah Rahil (01:09:50):

<laugh>

Dylan Ander (01:09:50):

No took a swing.

Rabah Rahil (01:09:51):

Um, but it was this bundle builder where it was essentially like this little sticky side bar and you had this collection of products and like, oh, buy this, that and that. And we controlled those products cuz you would make a collection to make sure that they were high margin cuz you didn't want people to buy like, you know, three low margin products. And then all of a sudden you get 50% off kind of thing. Yeah. And so you have to be, make sure the economics net out, but it worked out really well. Wasn't great for new customers, which would make sense mm-hmm <affirmative> but um, for our returning customers, we, we really smashed that. It crushed in terms of boosting, um, the AOV in terms of our new cus or our, uh, returning customers to get 'em to spend more money cuz you already like the product.

Rabah Rahil (01:10:28):

And then we would have like a brush and then these, these things that kind of like built well and then you get like 30% off and we get this higher AOB. So the, the acquisition cost was there. So, um, alright, one last creepy question and then we'll wrap up mm-hmm <affirmative> so this is actually, uh, I didn't do my homework. I've been traveling. I'm sorry folks. And, and Dylan actually saved me, but, and this is why he's gonna have a better answer probably than me and Ash. But the question is, is it more efficacious to dry off in the shower, out of the shower? I'm gonna go first. So I'll give you some time to answer and you already have the answer, so we'll get, we'll have you wrap up cause you probably have the best answer, but my vote is out of the shower because, because sometimes I'm a little loosey goosey with the towel and it's the ban of my, my existence when like the little corner hits a puddle and then I like get like some of the towel wet. So I'm, I'm an outta the shower guy where

Ash Melwani (01:11:19):

Uh, um, very similar. Yeah, yeah. Out the shower. Uh, but also you have the, uh, bath mat outside in it kind of. Exactly. So I'll I'll quick pat and then hop out. Yeah. Yeah. That's my,

Rabah Rahil (01:11:31):

I'll see your quick pad in the shower.

Ash Melwani (01:11:32):

Yeah, no, I have my, I have my towel like, like hanging so I could just grab it quick pat and then I'm out. Not fully.

Dylan Ander (01:11:38):

So you're an in the shower guy.

Rabah Rahil (01:11:41):

Well, yeah, I guess what we have, we have to define dry and off then, right? Yeah. How

Ash Melwani (01:11:44):

Long?

Rabah Rahil (01:11:44):

Because I think I am. I have, cause I'm

Ash Melwani (01:11:46):

Technically dry when I can I'm

Rabah Rahil (01:11:48):

In the shower pattern outta the shower. The other thing too, Tim shower,

Ash Melwani (01:11:51):

Pattern out, shower

Rabah Rahil (01:11:53):

Dryer. Well, the other thing I just realized too is the shower configuration is matters too. Like if it's a bathtub shower, configuration, it's a totally different strategy than if it's a standup shower for me. Yeah.

Dylan Ander (01:12:02):

It's true. That's why you can't take CRO financial advice. Like there's so much that

Rabah Rahil (01:12:07):

Shower test.

Dylan Ander (01:12:07):

Well, here's the thing. So

Rabah Rahil (01:12:08):

What's the, what's the, what's the right answer

Dylan Ander (01:12:10):

Here. The reason I did this, the right answer in college, we had a, um, a public speaking class and they said you can debate any topic, more ridiculous, the better. And this was our, our, our debate. So the, in the structured debate we won for in the shower, in

Rabah Rahil (01:12:23):

The

Dylan Ander (01:12:24):

Wow, because

Rabah Rahil (01:12:25):

I'm 0% in on that.

Dylan Ander (01:12:26):

Well, here's the thing. You're in Texas. We are in New York when it's cause you winter, you're gonna be freezing when you get outta the shower. So you're not gonna want that. That's number one. Number two, you save yourself water on the floor where you can slip and potentially die. But

Rabah Rahil (01:12:39):

That's poor people. Stuff who got bath mats. <laugh> we're

Dylan Ander (01:12:42):

Rich. Whoa, come on. You're you're gonna say that no water gets beyond the bath mat or is your whole bathroom, a bath mat, but

Rabah Rahil (01:12:47):

It's tile. Right? And actually what's

Dylan Ander (01:12:50):

Tile on water

Rabah Rahil (01:12:51):

To be fair. I lived in Indiana and our house had a bathroom in the carpet. Which how, how or

Ash Melwani (01:12:57):

Carpet in the

Rabah Rahil (01:12:58):

Bathroom. How ridiculous is that? That

Ash Melwani (01:12:59):

Type it's

Rabah Rahil (01:13:00):

Okay. Oh, I will give you the temperature control. You, you definitely get cold and you slap that. You grab

Dylan Ander (01:13:05):

The towel,

Rabah Rahil (01:13:05):

You close the curtain.

Ash Melwani (01:13:06):

You're telling me your bathroom. Doesn't steam up completely.

Dylan Ander (01:13:09):

It does.

Rabah Rahil (01:13:11):

What's the pen sometimes too. If you have a sketchy water heater and stuff, but the

Dylan Ander (01:13:14):

Temperature, you said rich people, shit. I don't have a sketchy heater. Exactly.

Rabah Rahil (01:13:17):

The temperature, the temperature control is a good vector. I didn't think of

Dylan Ander (01:13:20):

That. Well, here's one downside of in the shower.

Rabah Rahil (01:13:22):

What about the danger of the, do you do the, the foot up on the side or something like that? Cause what if you're in a standup shower,

Dylan Ander (01:13:27):

The hard to reach

Rabah Rahil (01:13:29):

Did the towel dip is like the ban of my existence. I can't do it cuz like you,

Ash Melwani (01:13:32):

I don't want it to touch. I don't wanna, you

Rabah Rahil (01:13:33):

Render like a corner of the towel useless cuz it's wet now. Like you can't draw off with a wet towel. So how do you get around that problem?

Dylan Ander (01:13:38):

Well, that is the

Ash Melwani (01:13:40):

Accidentally drop

Rabah Rahil (01:13:41):

The towel done. Right? I drop it outside. I don't know. But

Dylan Ander (01:13:44):

If you're outside bend over and like you're, there's like water coming out now. Like everything's wet. Nobody's on the floor and then you can slip and die.

Rabah Rahil (01:13:52):

I don't know. I

Dylan Ander (01:13:53):

Think, I think the safety risk

Rabah Rahil (01:13:55):

Outside

Dylan Ander (01:13:55):

I'm team inside. You're

Rabah Rahil (01:13:57):

Inside from

Dylan Ander (01:13:57):

Team

Ash Melwani (01:13:57):

Inside. I mean, I've been doing this for 28

Rabah Rahil (01:14:00):

Years.

Ash Melwani (01:14:01):

I can't change now guys. <laugh> so team outside, I get the points. They're all valid team outside.

Rabah Rahil (01:14:07):

Amazing. All right. Tweet at us. Are you team inside or team outside? Shower Dylan. This is such a blast dude. Thank you so much for coming on. Split testing.com. Tell people how they can get more involved. Do you, where can they follow you? This time is yours. My friends

Dylan Ander (01:14:19):

Split testing.com. That's uh, long. The short of it. It's what we do. Um, find me on Twitter. DM me, ask me questions. I'm genuinely here to support. I have, there's so many answers that I have now that I didn't have when I was either starting or in the middle of my journey. It's about CRO I've. I've also earned, you know, dozens of millions in ads and all of that. So if you guys just have e-comm questions, come to me, DMS are open. It is me. Uh, I am there to help. So you know, if you guys are brand doing over, I don't know, you know, couple million a year, definitely top we've found money back guarantee. It'll blow your minds. But uh, other than that, yeah. Um, I'm here to help here to support, follow [email protected] or split testing com without the dot on Twitter. And um,

Rabah Rahil (01:15:01):

Amazing.

Dylan Ander (01:15:01):

Appreciate it. Gentlemen,

Rabah Rahil (01:15:02):

Tell us about the vitamin shop. Tell us about the mentor pass. You know what to do

Ash Melwani (01:15:07):

Actually tweeted me today. I'm I'm very grateful. Uh, picture of, uh, shop. Well,

Rabah Rahil (01:15:11):

You guys were just out. Was it

Ash Melwani (01:15:12):

In Jersey? Yeah, just did a little popup, uh, in Jersey just to get some, uh, some, uh, taste, the table

Rabah Rahil (01:15:18):

Wrap looked good. The whole setups clean

Dylan Ander (01:15:20):

You size.

Rabah Rahil (01:15:21):

We pops it slaps

Ash Melwani (01:15:22):

For sure

Dylan Ander (01:15:23):

Slaps,

Ash Melwani (01:15:23):

But yeah, if you're ever buy a vitamin shop, check it out. Avi, send me a picture on Twitter. Um, on mentor pass right now and uh, just follow me on, uh, Twitter at Ash Mulani.

Rabah Rahil (01:15:34):

You've been dropping some thread heat.

Ash Melwani (01:15:36):

Yeah, recently

Rabah Rahil (01:15:37):

I feel a little jelly. Good. Isn't it putting some numbers down, stepping up.

Ash Melwani (01:15:42):

We're keep ride the momentum.

Rabah Rahil (01:15:44):

Amazing. You're really, uh, settling into your guru status here. Right?

Ash Melwani (01:15:49):

GU GU showing what's working for us and amazing buddy.

Dylan Ander (01:15:53):

I wish I could be a guru like Ash

Rabah Rahil (01:15:57):

Capital.

Ash Melwani (01:15:57):

We gotta wrap this up.

Rabah Rahil (01:15:58):

Amazing. All right, folks, we're headed to the, uh, New York whale road show. We are gonna be in Toronto, London and LA. Uh, it's gonna be awesome sauce. We have a bunch of cool stuff coming out. So if you wanna follow more [email protected]. If you wanna get more involved, we're actually fingers crossed going to triple whale.com and then the apps moving to app. I know that's good. Remove the try. Love it's cleaner that way. Thank God. A little social network reference. So, uh, fingers cross. We're gonna be pushing that out. So we're really excited there. And then what else we got? Yeah. If you wanna support my obnoxious sneaker habit, I on the LA, what was that? Kenny, Kenny was telling me that Shaq actually donates all of his proceeds to like animal shelters. And I'm over here buying egregious actual sneakers. So if you wanna support my obnoxious sneaker habit, go book a mentor pass. Um, they're really fun. I love it. Kenny's built a really cool

Ash Melwani (01:16:49):

Platform's

Rabah Rahil (01:16:50):

A lot fun. Been a really big blast. Uh, wait,

Dylan Ander (01:16:51):

Show him your shoes, like lift your leg.

Rabah Rahil (01:16:53):

I don't know if you can see these. Yeah,

Dylan Ander (01:16:55):

I'm not, no I'm saying lift your leg, man.

Rabah Rahil (01:16:57):

I'm not as do it, but I'll take we'll we'll

Ash Melwani (01:17:00):

Postproduction

Rabah Rahil (01:17:01):

Postproduction but I, I got some heat on today. It's right. They're in like a top, top two, top three, uh, proper heat, but we're going the fancy Mito of course. And then the fancy dinner afterwards. So, you know, that's it. Yeah. And, and the boss is in town. You gotta look good. You gotta look good. Of course. Um, awesome folks, if you enjoyed this, share it. Subscribe, smash that subscribe button and, uh, Dylan, thank you so much again for not only setting this up. This is our first in person in person last. I love it in real life. Um, awesome folks. That's it. And uh, another one in the books. Thanks Ash. Thanks.

Podcast

If You're NOT Split Testing You're LOSING Money

March 18, 2024

1:17:42

Hosted By

Rabah Rahil
CMO at Triple Whale

Guests

Ash Melwani
Co-Founder & CMO of MyObvi
Dylan Ander
CRO & SplitTesting Expert

Episode Description

In this episode, we go over the power of Split Testing and how to use it to your benefit as an ecommerce brand.#Adspend

Notes & Links

📧Subscribe to Whale Mail for exclusive industry insights and in-depth marketing breakdowns: https://www.getrevue.co/profile/tripl...

🐦 Follow us on Twitter for Industry insights https://twitter.com/triplewhale

Follow the people featured in this episode here:

- Rabah's Twitter: https://twitter.com/rabahrahil
- Ash's Twitter: https://twitter.com/ashvinmelwani
- Dylan's Twitter https://twitter.com/Splittestingcom

Transcription

Dylan Ander (00:00:00):

Let's say you have two ad sets that like, again, you're just changing one thing or you're doing a formal split test on Facebook. Mm-hmm <affirmative> you can't trust Facebook's, you know, numbers there. If you're doing it that way, you actually have to plug it into a statistical significance calculator. Mm-hmm <affirmative> so let's say your CPA is $32 on one ad set and 30 on another. What is the probability that it's due to random chance you don't know? So although you think, oh, 30 is better than 32, it might be due to random chance based on your sample size mm-hmm <affirmative>. So you actually have to plug that into a calculator to determine if it's statistically significant enough of a difference to deem that is not due to random chance.

Rabah Rahil (00:00:41):

We're back for our first live episode in person with my partner in crime as always Ashani and Dylan Ander split testing.com the best CRO agency on the planet. Um, we're gonna go through a bunch of cool stuff today, creative split testing. Um, you have some really cool brands that you work with that will also talk about as well. Um, but you're actually a bit of a dark horse on Twitter. So for people that don't know you, why don't you give us some of your background and then, um, we'll just jump in

Dylan Ander (00:01:09):

Yeah, a hundred percent. So, uh, been in econ my whole life in college, I started a print on demand store for sloths. Like the number one seller was like sloth prints on the butt. It was awesome. And it was like, I'm lazy. So we were drop shipping back in the good old, uh, Facebook is an ATM days. <laugh>, you know? Yeah, good old. But, um, yeah, we started there. Um, I felt rich, but the thing is the reason why I did super well versus other people, there was a tool called split testing.com back in the day was a free split testing tool. We were not the creators of it. We, their domain expired and we inherited split testing.com. Wow. Full disclosure. I don't wanna pretend like we were those guys <laugh> but when I was in college, I was literally 17, 18 years old and split testing my headlines back in the day, cuz it just like made sense in my head before interesting split testing was much of a thing.

Dylan Ander (00:01:55):

Like Jeff Bezos had a quote, you know, uh, I, he said that he attributes 90% of his success to experimentation, meaning split testing. Sure. Right. So if you're not supposed to have your store, obviously do it. It's super important. So that was our differentiator where I would like 20% boost on a, you know, Lander and boom scale. My ad spin I'd feel like a rich college kid and it was awesome. So I carried that out and eventually, um, I don't know if you know, Insta that follow on follow software back in the day helped work on the API there. Oh. And I white labeled like 50 different people for them to go sell Instagram automation services. Wow. So I got my sales in, I got my services in, I kind of learned the ropes there and then I just had an agency cause a friend like needed some marketing help.

Dylan Ander (00:02:36):

I was like $5,000 a month. And I was like, I don't know, 20, 21. I was like, it's a lot of money boys. Let's do it. Yo. Anyone know how to do SEO. And we, we just figured it out that way. I don't know if you guys have started agencies before, but when you first start, it's like, you know, you literally, I don't know, Hey, someone needs Facebook ads. Oh yeah. We've got a guy like we'll do it. Of course. And like you start and do it and do it. And there was so that's next core media, which most people know me from. Yeah. Um, and next core media grew and grew at Geico and planet fitness. I was doing a massive SEO contract with Geico. Um, I was consulting for planet fitness on some of their new ventures. Amazing. Um, and then eventually I kind of let all my employees go, all my clients go cause I was just so miserable.

Dylan Ander (00:03:14):

Like dark time agencies rough. But now, so at, at that point, what I did, this was about three, three and a half years ago I pivoted the agency and I was like, well, like, you know, media, buying's cool. Like a two to one, three to one, five to one on your money. Where could I get people like a hundred X ROI? Like literally if I'm gonna go in on the services and that's CRO, which is pretty solid because one split test can make a global increase. I mean, things like your header, your footer, your nav, like, you know, like tons of things are all on every page in the site. So a tweak in there I just recently did a split test, um, for a company called Cosk. Yeah. Literally 1.2 million in new revenue, cuz we changed one word in the header. Like it's crazy what it does. So I just wanted to create that service that like creates long lasting wealth and huge improvements to all these clients and you know yeah. Um, and, and, and as ad costs only get more, more expensive. You need to convert better to, you know, compete on those CPMs as Ash can talk about. Yeah. For days

Rabah Rahil (00:04:10):

<laugh> I love, I love that and Tom's gonna kill me cuz I always say I love that. Um, first off though, I do wanna give a shout out to you for scraping yourself together to come to this. You were just recently at, uh, probably one of the best festivals worldwide.

Dylan Ander (00:04:22):

Bonnaroo baby <laugh>. Yeah.

Rabah Rahil (00:04:23):

So definitely I appreciate what was the best at

Dylan Ander (00:04:26):

Um, do you know churches?

Rabah Rahil (00:04:28):

Yes,

Dylan Ander (00:04:29):

Dude. They're

Rabah Rahil (00:04:30):

With

Dylan Ander (00:04:30):

A V right? Yeah. With a V they're a metal band live. Like they literally like, like it's crazy, really fun fact too. I've actually written three billboard, hot hundred songs. I used to be a fulltime songwriter in Nashville while I was running my econ bro. Who's

Rabah Rahil (00:04:44):

Guy, what are you doing with that? Who's this dish, what are

Ash Melwani (00:04:46):

We doing?

Rabah Rahil (00:04:47):

Who is this guy? I that's about all the, all the I have <laugh> um, but yeah, bringing it back to the metal. One of the interesting things, no pun intended, um, was we have a forecasting calculator in triple well. And the thing that blows my mind is if you can just move a conversion rate by one or 2%, it's like the, the tide that lifts all boats where it's just such an interesting, like, I mean, it's just absolutely massive impact. I mean, you guys really, you have a really actually interesting testing infrastructure. So I want you kind of take us through that cuz you kind of have this low fidelity fast moving and then you move into kind of a high fidelity. Yeah. And I want you to tear it apart or tell him what he is doing well or not.

Dylan Ander (00:05:28):

Well, I'm a new Yorker baby. Let's

Rabah Rahil (00:05:29):

Go. <laugh> shout

Ash Melwani (00:05:31):

Out NYC. Absolutely. No, I mean, I've kind of told the story, you know, over and over again. But I think the biggest thing that kind of helped us get through the iOS updates, um, was split testing a lot of things. Right. Um, we used to do a lot of big changes first and then kind of dial each winner and then split test, you know, the smaller things. Right. So, you know, probably August September-ish, you know, is when we saw that impact of the iOS updates, you know, costs were going up, um, we weren't testing enough creatives and we really thought like, okay, the platform is just being the way that it is. And, and then that, and that's, that's all we can do. Right. We just have to wait. Yeah. But you don't have to wait, you shouldn't wait. Right. It's like, you can go and start making four or five different landing pages and see if you can even just double your conversion rate, right?

Ash Melwani (00:06:17):

Like you like all of, I think the last two years we were using a single landing page and we made no changes whatsoever. And it got us to where we were, which was fine. But now things are harder. You have to go and literally test, I think multiple landing pages, at least quarterly. Right. You know, at least have a cadence where you're setting up four different landing pages structurally this way you, you know that like these big changes could probably move the needle way more than just split testing headlines. But that comes next. Right. So what we ended up doing is like, we, you know, we tested four to five different styles of landing pages found our winner. You know, there was a, there was a, there was one specific winner that just blew past everything else. And then that is where we started dialing in.

Ash Melwani (00:07:02):

Right. That's when you start split testing your headlines, um, moving the placement of your CTA buttons, you know, do we put the buy boxes at the bottom? Do you put it at the top? Where do you put social proof? These like these things is where now if you have like a baseline of like 5% conversion rate, how do you get it to six to seven to eight? Yeah. Right. So that's like the biggest thing that we've kind of utilized and like pulled as like the main lever of these like rising ad costs to really like just convert more people. Yeah. Right. Cost is gonna go up. Like we, we saw it last year. Our, our CPC was like on average 1.9. So a dollar 90 cents today it's like $3 and 20. Right. But we're converting double. Yeah. So that's how I think we've kind of like survived, but now getting past that like 20 K a day and spend, we have to improve the, the conversion rate a little bit more and then also balancing increasing AOV.

Ash Melwani (00:07:58):

Right. So that's kind of where I wanted to kind of learn from you where like, okay, where's that balance of increasing conversion rate. Cause you can do that. You could be like, Hey, buy this $30 product. There's nothing else available straight to checkout, boom bang. Like that's it mm-hmm <affirmative> but then like your C is probably still 30, $40. Mm-hmm <affirmative> so now how do you get that $60 AOV? How do I get that? A hundred dollars AOV with maintaining conversion rate? So that's like the biggest thing I think brands should be focusing on and wondering if you have any tips for, for everyone out there.

Dylan Ander (00:08:27):

I mean, uh, got a couple, there

Ash Melwani (00:08:29):

We go. <laugh> passed it up.

Dylan Ander (00:08:31):

So, um, first thing that I always said, I made a, a tweet that not a lot of people liked about a month ago where I said any brand that is even remotely profitable that rebuilds their website, absolute idiots. If anyone is considering a full site redesign and a rebrands I've literally changed a font and conversions tanked or increased. Right. So like things as little as changing, like maybe your dark logo versus your light logo, like in your, your header, all of it matters. So would you ever run ads without attribution? Hell no. Right. So would you ever change a whole page on your site changing like 20 or 30 things and not knowing which made the change? Yeah. So the thing about Landers, which I'll get into is in a second is like if you're testing different Landers, you have a baseline. It's almost like putting a bandaid though.

Dylan Ander (00:09:13):

If I'm being honest where let's say you test one new Lander in like a new structure, I don't wanna say atory. Let's say it's a, you know, tid for tie, you wanted to move the buy box, you're doing a URL, redirect test, that type of stuff, the incremental improvement. It, you need to understand each level that's changing. So if you change like seven or eight things in the Landers, mm-hmm, <affirmative>, you're not gonna know which of those seven or eight things happen. So let's say you had a net 15% increase mm-hmm <affirmative> there might be an element that has a net like minus 4% increase. Right. But it, so it could have been a 19% if you incrementally did each of them. Right. Versus, you know, like having no attribution of which change right. Actually made the change. So that's the one gripe with testing new Lander.

Dylan Ander (00:09:51):

So first off never rebuild your whole website. That's just doesn't work. Yeah. Um, so for different Landers, the thing I'll say about focusing on Landers is, um, where a lot of people overlook CROs the offer, right? So one thing that you guys do a great job, I always preach like the power of free mm-hmm <affirmative>, which is what can you give away for free? You can give away info products, you can give away content, you can give access early access, you know, app access. Like there's so many things you can give away. Like you guys give away. I think the free, uh, scooper, sugar

Speaker 4 (00:10:17):

Cup scooper, all that, all of those,

Dylan Ander (00:10:20):

All the, I, I, I made a thread, like, I don't know, also like a month ago and shouted you guys out cuz you did a good job. So yeah, it's in the offer itself. So the way that we start every campaign for true CRO is go and ask your customers what they want. Mm-hmm <affirmative> cause you guys like for triple well, for Avi, like you guys may think, you know, what your customers want. Right. But only they know. Sure. And like, if anyone says I would click this or I would do that immediately fire that team member, get rid of them or slap yourself in the face. Cuz you as one person, you're not necessarily the demographic of user. Right. I don't know if you own a E eCom story that you're not the demographic of user. So you go and ask your customers, customer feedback survey.

Dylan Ander (00:10:55):

Why do you buy from us? You know, what makes us different? What's the best part of the experience? Get a little bit of the emotional side, like for econ brands, like, you know, like, um, what are you looking for when looking for collagen or supplements, right? Like what do you hate about supplements? You know, all that type of stuff and you get it and you shape your offer. Mm-hmm <affirmative> and with the Landers, making sure that your offer is well shaped, like that's so overlooked that I would rather you take four identical Landers and just kind of change up the actual offer itself. Yeah. And then from there start testing the rest of those Landers. Right? So it's like get the meat and potatoes, which you can double your conversion rate by figuring it out. So mm-hmm, <affirmative> on, I also own a couple of supplement brands.

Dylan Ander (00:11:31):

So on one of them we added plus like, you know, get a free ebook. Right. And it smashed conversion rate. It was awesome. So huge increase like where you offer that little like free, uh, the little free product try, like plus a free, you know, um, one of the things for like, like, like hair loss is probably one of your biggest pain points for collagen. Yeah. Yep. So how to maintain 10, you know, an ebook of 10 ways to maintain positive hair, right? Smart. Like that's the kind of thing that you can add as an offer to increase conversion rate, fundamentally, forget the tweaks, forget anything like you have to obviously have to AB test this and you know, make sure that one offer's converting better than others, but that's really the meat and potatoes of it. So it's like ask your customers and formulate an offer. So it's like before you touch your Lander, before you touch anything, just go on a walk like deep breath, put yourself one thing. Shout out Nick Harris. He's uh, our new chief strategy officer split, te.com. Nick it's, uh, at Nick D to C with a two D two C shout out Nick. We love you

Rabah Rahil (00:12:25):

Just recently on mentor pass as well. Shout out Kenny,

Dylan Ander (00:12:28):

Shout out Kenny. I just joined actually I think like yesterday, amazing killing it. If you guys, uh, need some support and help, I'm here to help. Um, so Nick, he actually taught me about like, you know, he worked at a lot, a lot of nine figure brands before empathy mapping, which is every step of you have an issue with hair loss. You therefore go to goo and start Googling. What is your experience and emotions you have at that point, when you go to the next stage and you're doing your research, how do you feel, what emotions are going on there and literally mapping the empathy of the customer all the way through. And this also goes to retention, email marketing, text messaging, like literally empathy mapping, the entire process of like from before they find you to, when they're becoming a raving fan, right. When is weakest, you know, it's kind of like also a self analysis of cool.

Dylan Ander (00:13:11):

They take it 20 days in. They don't, you know, or like for example, like on, on supplements, like with my brands three days in, Hey, my joint still hurt this thing doesn't work. And I'm like, Hey, you need to give it like two weeks to really work. Especially collagen as well. Yeah. So now start on the empathy mapping, right? Where right after the post purchase emails, like two or three days in like, Hey, I know you're waiting, um, think long term two to four weeks. Right? So you're able to map every single stage. Right. Um, and getting that empathy mapping all along. So Nick turned us on to empathy mapping, which absolutely changed the game for acquisition. Because by looking at how they're gonna feel over here, you can market it other than just saying like, Hey, let's solve the problem. No, no, no. Like let's actually like, Y the problem in like solution is really well defined and like there's a pathway and it makes it so much easier to tell the story. Yeah. Make good ads, good creatives that teach it, you know, that they can really harp on. Like maybe like for, for collagen, like hair loss. There's like four situations. I don't know. Like you've a receding hairline. It's thinning thin, like, you know, there's a couple different angles. Maybe you have to go and test those subtopics and have you do it customer feedback survey. Yep. Da da. So it's like just literally map the USPS, that's it? Yeah. That's

Rabah Rahil (00:14:21):

Literally what we do. <laugh> yeah. The empathy map is really interesting cause I, I enjoy something called jobs to be done. And there's something similar where there's two ways that you can kind of identify the job to be done. One, you take, you basically have this essentially like buyer journey, but it's a little more nuanced than that. Or the forces they're it's same, same, but different. And the forces are interesting because you don't only get a purchase when you have, you have these pushing and pulling forces essentially, and you only get the purchase or the behavior change when, um, you get, they call it a tip in there. So it's, it's really similar in terms of that empty mapping. And I really like that. Um, so we're not talking over people's heads. Can you explain like your five, like explain to a five year old kind of what AB testing is? So people that aren't kind of hip to it and kind of know what we're talking about. Give 'em a little foundational knowledge there.

Dylan Ander (00:15:06):

Yeah, exactly. So I'm gonna kind of take it through the lens of how I do the CRO campaign, which is CRO is just conversion rate optimization. If you get a hundred people to your website, how many people purchase okay. That is known as your conversion rate, which, you know, most people know what the conversion rate is. Um, it's just the art of increasing your conversion rate is what CRO has become. It's more of an umbrella term where boosting your average order value. Like, you know, it's really just revenue per user. So if someone clicks on your website, how much is that worth for you? So he said, it costs about $3 and 20 cents. If you're making $4 and 20 cents like four 20 blaze, then like, you know, you're <laugh> then you're, you're making, um, you know, you're definitely making profit there. So understanding the revenue per user or conversion rate, they're a little bit, you know, same, it makes sense.

Dylan Ander (00:15:52):

Right. Same but different. So, um, you know, you go to your customer feedback survey, then you go to heat mapping, which is, you know, seeing how people interact with your website. Yeah. There's, you know, we use Hotar at our agency. So, um, we use all the click maps, scroll maps, see how people interact with the website. So that's another thing that we do of like qualitative data that we put in, cuz people are clicking on a photo that doesn't go anywhere. There's clearly a problem with how it's like presented, you know? Right. If people like, if there's like a Nagar that a nav bar that has like six things across and people don't click on two of them remove 'em, it's probably gonna help your convers rate. So there's things like that that will help. And then ultimately after we get to that, we do this massive audit.

Dylan Ander (00:16:28):

So we become like professionals in your brands of whatever brand we're working on. And then it turns into split testing. Got it. So split testing is, or another known as AB testing, right? AB testing. Like I like using split testing cuz you can have many variants, right? Yeah. So it's not just an a and a, B it could be a, B, C, D E F G. And then there's a whole bunch of multivariate tests where you're changing a and B and then C and D and they go against each other. Yeah. So, but the whole point long story short is that you're testing different things on your website. Button color is the most simple to someone who's just starting an e-commerce, which is you have a green button. If we get a hundred people to the green button or a thousand people for the green button and a thousand people for a red button, we can see how much revenue comes in for the green button. Yeah. And the red button. Yeah. And that's just the most simple example of cool. If the red button has more revenue per user, like revenue per click, then awesome. I'm probably gonna go ahead with the red button and implement it on the pages that we do so you can change literally anything. My developers have shown me that there's nothing that can't be split, tested on a website and it's been awesome. So, uh, does that kind scratch that?

Rabah Rahil (00:17:33):

It's incredible helpful. The, the other question I had, so there's everybody throws around a term statistical significance. So I have a question about that. And then the other question too, that I had is, um, you were talking about earlier, uh, I think the fancy term is controls. If you have, have too many changes to the page, you don't have a really good controlled experiment because you don't know, like to your 0.1 change might have been a plus 20% push, but then you also had this other change that was a minus five. And so another change that was a minus two, then another change that was like a minus three. And so net you're seeing a plus 10 increase, but if you'd only done that plus 20 thing and you would've done it in a more, uh, controlled manner, if you will, where you're only testing one or two variables at a time. Yeah. You get a lot more insight from the experiment. So I guess first question, can you explain kind of statistical significance? What are you looking for there? Is it like an impression rate? Is it something or impression threshold or traffic or what have you, and then also, um, do you think it's good ever to test a lot of variables or should you always be doing like one or two kind of max mm-hmm

Dylan Ander (00:18:38):

<affirmative> totally. So, um, statistical significance, I'm just gonna go stats SI. That's what we call it. So, um, if something is stats or not every single piece of marketing, whether it's email marketing, Facebook ads, like the best media buyers, sit there, plug it into a calculator to see if something's stats sick. Right. So what statistical significance actually means is the percentage that it change is due to random chance, right? Because if I flip a coin, which is, you know, 50, 50 on heads and tails, if I flip it a thousand times in a row, it can land on heads a thousand times in a row. Right. So it's the, you know, the random chance that like your version is so the, you know, you can never have a hundred percent statistical significance. Cause that means something as fact, but it could be 99.9 9 9, 9, 9 99. So, you know, in science like, and like medical studies, there's like, you know, a version of someone without the medicine and someone with the medicine sure.

Dylan Ander (00:19:35):

In the medical communities, if you have like a 99.1, that's considered like dangerous to put in mankind. Yeah. With us, we can go as low as an 85, 90% confidence interval. So for whatever you're doing it, you need two things. Right? So like a thousand people visited here and 28 purchased thousand people here and you know, 40 purchased, right. You just plug that into a statistical significance calculator to not make it fancy. So let's say, you know, a big gripe I have on media buyers is let's say you have two ad sets that like, again, you're just changing one thing or you're doing a formal split test on Facebook. Mm-hmm you can't trust Facebook's, you know, numbers there. If you're doing it that way, you actually have to plug it into a statistical significance calculator. Mm-hmm <affirmative>. So let's say your CPA is $32 on one ad set and 30 on another.

Dylan Ander (00:20:20):

What is the probability that it's due to random chance you don't know? So although you think, oh, 30 is better than 32, it might be due to random chance based on your sample size mm-hmm <affirmative>. So you actually have to plug that into a calculator to determine if it's statistically significant enough of a difference to deem that is not due to random chance because 30 verse 32, fundamentally, if you assume that it's always gonna be 30, you don't know for sure as a predictor of the, the future, but you'll know if something's statistically significant between right. Between two different ones. So

Rabah Rahil (00:20:53):

Love that stat CIG fans I'm hip to it, baby. Yeah. <laugh>

Dylan Ander (00:20:58):

How you look in stats SIG today,

Rabah Rahil (00:21:01):

Stat C amazing.

Ash Melwani (00:21:03):

Well, what does that, what does that number look like for you guys when you're running your test? Right? Mm-hmm <affirmative> is there, is there a minimum, like sessions per experiment that you guys are looking for to get to that? You know,

Dylan Ander (00:21:14):

So it's a hot take cuz there's a lot of opinions on this. Yeah. Because if you're a small store, like for example, um, I come to you and there's a test and I say, Hey Ash, there's an 84% ch like chance, right? Like based on the statistical significance, mm-hmm <affirmative> 84% chance that this change that we have here will make you an additional $40,000 per month. There's still a chance that it could be negative. Right. Mm-hmm <affirmative> like, you know, there's uh, yeah. 16% that I gave. So like there's a 16% chance that this is completely random. Right. And there's an 84% chance that it's gonna be a $40,000 increase. Would you like me to implement that on your site? Yes or no? Yeah.

Rabah Rahil (00:21:53):

You'd take that bet. Right?

Dylan Ander (00:21:55):

Bet. Most people would take that bet. So it's like that type of thing where a lot of people say 95% or nothing, right? Like when you learn in high school, it's like a P value. So if you subtract the statistical significance, P value is 0.05. So 95% statistically significant. So for us, if it's below 85, we kind of say to the entrepreneur like that we're working with, Hey, like this is kind of your decision. Here's the facts. Here's exactly what it is. It's below our desired threshold. Right. But the way that we implement it is if we see all signs, right? So let's say your conversion rate increase, cuz you have a KPI like key performance indicator of what you're trying to increase some split tests. We're trying to increase average order value. Yeah. Some were trying to increase conversion rate. Some were trying to increase, add to cart rate and don't care about the purchases. There's so many different like test types that you can run.

Rabah Rahil (00:22:40):

What have

Dylan Ander (00:22:40):

You, yeah. Termin, exactly.

Rabah Rahil (00:22:41):

Terminating event can be whatever you want type of

Dylan Ander (00:22:44):

Thing. Just like ads, you know, like deeper in the funnel is typically better. So, um, yeah. I mean the minimum is really like what we evaluate. So if you go and Google optimize, which it's free, guys, I recommend you use Google optimize, um, the drag and drop feature. Isn't always the best. You kind of need to know how to code. If I'm being honest

Rabah Rahil (00:23:02):

Or I've broken multiple sides of it, honestly.

Dylan Ander (00:23:04):

Yeah. So we use Google optimize, but no joke if we're swapping a photo, my dev like one of my lead devs was really good. Can take two hours to make that test because what about all the different sizes of screens? It's gotta look perfect everywhere. Yeah. Right. So a lot of people just swap in a photo hope, you know, from Shopify and they think it's gonna work. And it tanks it's like, well, this objectively is way better. It's like, yeah, because it's not dynamic. There's different mobile types. There's like, there's a lot to go on there. So right. It takes a bunch. And then last part of that is there's also secondary metrics. Mm-hmm <affirmative> so let's say our main KPI is like purchases. Right? Right. So we, or sorry, conversion rate, we care more about lowering our, our CAC than actual like ROAS. Right? Mm-hmm <affirmative> so like there's a difference between the two, right? You just want the lowest cost per new customer. So like net CPA, of course. Um, if you're working on that front end conversion rate, mm-hmm, <affirmative> take a look at your add to carts. Every time you get an add to cart with your abandoned email sequence and assuming you have a text message sequence, you get a value per add to cart. Right. So if that has gone up, but your conversion rate stayed the same.

Dylan Ander (00:24:09):

True. Maybe that's good to implement. Yeah. Let's say your average order value has gone up, but your conversion rate's the same. That's great. Might wanna implement it. So there's secondary metrics and let's say you're focusing on SEO and things like that. Dwell time, time on site. Yeah. Number of pages you visited all inside of Google optimize it. It links in like directly to Google analytics. So there's not a single stat that you can't see. Well, I mean maybe like per channel, Google analytics gets a little wonky. That's the only thing I don't trust Google analytics for. But otherwise if it's hard coded and set up properly, that's where it is. So people saying there's a number of like sessions I need and stats CIG and all of that, where 85 is my general rule. If like you had to like, you know, gun to my head, I would say if it's below 85, kind of not the best, but it's really advanced where you look at all the secondary metrics, everything looks positive, but your first needle mover didn't happen. Yeah. You may still wanna actually implement it. So it's case by case. But yeah. Anyone feel free to DM me with a Google analytics or Google optimized screenshot, and I'll tell you what to do. Yeah.

Ash Melwani (00:25:06):

But like, okay. So, so for example, right, say let's take the red button, blue button, right. Are you looking at, if, if for example you have a hundred visitors and obviously blue is in the lead, right. Let's just say, is that enough for you to decide or pull the plug on that? Or like, I feel like you need a cert, at least for us when we're doing it, we're doing at least 5,000 sessions per variant. Mm-hmm <affirmative> just so that we know that. Okay. There's no, there's no chance. There's no randomness at all. Yeah. So like what does, is there a set number that you guys go with or is it more so like, I guess it's kind of maybe up to the, to the owner as well. Like they want to cut it at some point, you know?

Dylan Ander (00:25:42):

Exactly. So, um, if you ask the data purists, right? Like, like statisticians, not marketers, like they would say we would need a thousand conversions

Ash Melwani (00:25:52):

For sure. Right. Thousand conversions

Dylan Ander (00:25:53):

Conversions. Got it. Right. So purchases. Right. So that's not how we go at our firm. Right. Because not every brand's in, I mean, we've got plenty of eight, nine figure brands that, you know, we've got a couple thousand per variant, you know, even if it's multi variant, like of like, you know, seven or eight versions. Right. Um, that's where it starts transparently. We used to work, you know, when we started out, we worked with smaller brands. We were smaller agencies of course. And we had a minimum of a thousand sessions sessions yeah. That we would need per variant. And we would never call a test. I don't care if it has to run forever. Right. I don't care if it has to run for three months, can't have less than a thousand sessions. Cuz if you think about it, if you flip a coin a thousand times, do you trust where it's gonna land? Yeah. I kind of do. I think it's gonna be close to 50 50, so right. A thousand is kind of, if you just think about it, I mean it's completely arbitrary. Right. But a thousand per variant is my bare minimum. We never even still today. Yeah. No matter what it is, cannot do thousand per variant. So if you have a five way test, you need a thousand in each one. But also I'm not condoning once you're at a thousand, leave it the more you can afford to run. Right. That's the whole point.

Rabah Rahil (00:26:56):

Right? That's that's how I feel where ultimately one, I think it is a function of the business size mm-hmm <affirmative> because not only do you not have the data, but you don't have the time, like you can't wait for like you need to like, and so if you can at least derive directionality from it and saying like, Hey, you know, like if you're a small business, like it matters where like, if you're a small business and you move your conversion right up like one or 2%, it's not really that big of a deal. But when you're an eight, nine figure brand like a 1%, one point bump on your conversion rate is like almost business changing. So I think there's a certain aspect of understanding the context of the actual impact of the changes where it's just like a hundred sessions probably isn't enough, but maybe 500.

Rabah Rahil (00:27:37):

Isn't like, you can, for me, it's almost like the sniff test of like, if you showed this to 500 people, what do you think? Okay, cool. Mm-hmm <affirmative> are you fine to go ahead with that? Okay. Let's go. Cuz like to your point, Dylan, like a thousand PE a thousand conversions might be a three month, a quarter period for some brands where it's just like, you know, so you have to get into that. Um, that's incredible question. What one, how does Google optimize or how do all these people do the holdouts? Is it cookies? Or like how do they bifurcate kind of mm-hmm <affirmative> the audiences that are gonna see these different things. Cause ideally you don't want people polluting each, like if you're in cohort a or cohort B or cohort C mm-hmm <affirmative> you don't want any kind of cross pollination. Correct.

Dylan Ander (00:28:17):

Exactly. So Google optimize it's set and forget in that regard. So if you land on a session ID, even if you come back, right. So let's say I visit a product page green button instead of like a red button. Um, if I come back to that same page, like a day later, if the test is still running, I will see that green button again. Mm-hmm <affirmative> so it's kind of like a, not like you kind of don't have to think about it if you're using, you know, any like the paid ones that are really expensive, like VW O AB tasty Optimizly they're all the same, but so is Google optimized. Got it. If they visit once there's a session ID and if they come back, you don't have to worry about it. They're still gonna see the same one. Got it. Right. But then if you cut the test and you're at something new, you know, you're back to your code base. Yeah, exactly.

Rabah Rahil (00:28:56):

Makes sense. Awesome. Um, what's the craziest test you've seen, like in terms of performance where you, cause we were talking a little bit offline about like you were changing like X, Y, or Z, and you were like, oh my gosh, we made a bazillion dollars for our brand. Like what are some crazy kind of weird tests that you have seen before where you're like, oh, that's weird or that's interesting kind of stuff.

Dylan Ander (00:29:17):

Yeah. So, um, one of my senior CROs, uh, at split testin.com, he, so like one big thing that I always say is above the fold on mobile, get your damn ad to car button above the fold on your fucking mobile

Rabah Rahil (00:29:32):

Device.

Dylan Ander (00:29:33):

Yeah. So one thing about it is when you go into Google analytics go to device and I believe it's screen type, right? Or there's a secondary filter for screen type. You can take an average of the screen resolution of the mobile phones that visit your website. So when you make mockups, you should use that to default, right? Because there's people of smaller phones or much larger phones. So optimize for the ones that your customers are. So that's a really good, deep tip that most people don't know. But again, go to that average and make sure that average at all costs that adds a cart button is above the fold on mobile. So one time, one of my senior CROs was like, well, Dylan, like, there's the quantity selector. Do you want me to remove it? And I was like, fuck it. Yeah. Why not? Yeah.

Dylan Ander (00:30:17):

AOV. And they're like, well, the AOV is gonna get hurt a lot. Right. Cause like, you know, cool. You give them the option. We looked at the heat map and two people out of 20,000 visitors <laugh> clicked on the quantity selector. That's amazing. And then I was like, wait, hold on. And then I went to three other clients, I don't know, 10 in 10,000, like literally point something percent of customers are using that quantity selector. So hot take quantity, selectors suck. <laugh> you know, you need to make your buy box, actually be the ones that work. So we remove quantity selector. And now what we started doing is we put the quantity selector last. Yeah. And then there's a variant of just completely removing it nine times outta 10, which is probably stats Zig there. So we actually, we actually have a killer database of over 1500 split tests.

Dylan Ander (00:31:01):

We ran. So like internally we kind of get to see what's working. Not yeah. The benefit. So yeah, we get that ads card button above the fold, no matter what. So even if there's quantity or not quantity selectors, but variance. Um, like for example, like yeah. There's variants. So whether they're colors for fashion or there're flavors for supplements, we take that add to car button and we've been adding it above that. Remember you said that. So, so we literally added above there. So it's like, wait, so you're saying the ads, a car button goes above, like my t-shirt size and I'm saying, yeah. So first off do not get anyone at home. Don't go and do this. Don't just, you have to test it. Say like, that's the whole point. This is not financial advice. This is not financially not. Yeah. So on this unqualified opinion and not financial advice.

Dylan Ander (00:31:48):

Yeah. Like we found that to be like a hot take test that like get it above the fold so that buy button in like you need high contrast. Right. Green is something you usually test. It's known to be one of the most inviting colors that like obviously green button gang. I kind of wanna make a hashtag on Twitter, like green button gang. Just, I tossed it a couple times. See merch shit. Put that on the shirt. Damn. That's great merch. Yeah. So yeah, getting it above the fold is just like crucial. So, you know, forget the, the quantities and all that. But the only thing with that is it, none can be preselected because what will happen, we found out on customer service is if that out card button is above the fold. Right. And they like it and they just click on it. Let's say there's strawberry banana and chocolate.

Dylan Ander (00:32:29):

Mm-hmm <affirmative> the default will be the strawberry. Right. So, but that adds cart button. They just like click it and go and buy. Right. They may have wanted the banana or the chocolate. So you need to make sure that like when you, you click the ads to card button, the little error message says, you need to pick a selection select. So make sure there's no preselected ones on there. As long as there's no preselected put that ads card button, like literally almost interesting. Like, so like, you know, product it like, like product photo. Yeah. Title, the price button. <laugh> like literally nothing before any content before any, anything. And it also helps price anger. So price anchoring is another thing that's huge. Yeah. So a lot of people say the larger, the AOV, the smaller, the text should be the, you know, smaller, the AOV, the larger, the text should be like the size of the actual price.

Dylan Ander (00:33:16):

Right. We found that to be like, not necessarily the most true. I'd give it like a 70, 30 that like that's wrong based on our like database of tests. Yeah. We find almost always giving that price larger makes a huge difference. Yeah. So the three things that you want them to notice right away. Well, four things really, which is the product, what it looks like, the name of it, um, the price of it and where they can buy it, which is that adds to car button. So those are the four things. So it's called price anchoring where if you're buying sneakers, which I know you hate. Yeah. Uh

Ash Melwani (00:33:45):

<laugh>.

Dylan Ander (00:33:46):

It has to be, um, if you're looking at like $750 sneakers versus $75 sneakers, you shop differently. Yes. So you, as the e-com owner, it is your job to make sure they know how to scan the website, which is price anchoring. So just running the test where you have this big fat, bold price, even if it's an ugly $5,000 price tag, which is a lot of money for most consumers, um, still want it nice and big. Cause it's your responsibility to make sure. So right. Those are two hot takes. I've definitely got some others, one that we've been doing recently as well, which is hard to test and optimize. If you don't have sick developers, shout out Keith, we love you. Um, the second photo being UGC in your car cell, you guys do that really well. Don't you? It was one of my CRO hacks. I don't know if Ash has been, uh, scooping off me on Twitter. Amazing. Little

Ash Melwani (00:34:30):

Bit, a little bit. He's ever this little bit he's yeah.

Dylan Ander (00:34:34):

He's he's on, watch him, watch him. I'm giving too much. Damn. I'm getting out of a gig.

Ash Melwani (00:34:40):

Well, one thing that you, so just from listening to you now, right? There's so many things I wanna go back and test, right? So now for people who are like watching and, and like writing down these notes, like all I gotta, I gotta change the font size price. I gotta move the at to car button. I gotta do this. I gotta do that. How would you, what's the best way to structure all these tests and kind of carry out efficiently for people. Totally. You know, with let's let's say like you have, I, I don't know, maybe, maybe give the structure for, you know, small to mid-size businesses so that they can kind of yeah. Carry out these task.

Dylan Ander (00:35:11):

So it's the same for everyone. Yeah.

Dylan Ander (00:35:14):

Top down. That is it. So if you go on hot jar, which you can download for free. So no matter what size business you are going, download hot chart for free. It'll take a while for sessions to collect, but there's click maps, which is where you see what people are clicking on and scroll maps. So one of the hidden tricks is looking at the scroll map because it obviously goes down, you know, less and less, right. So if you think about it, if a hundred percent of people are looking at everything above the fold and only 12% are looking at the footer, it, the, you know, above the fold will have almost 10 times the impact of what's in the footer. Right. So it doesn't mean things on the bottom. Aren't good. Right? Cause sometimes like you want a long form page and just the buy box at the bottom, but we always start top down because above the fold is what has most of the like massive improvements cuz the most people see it. Right. So if you're testing something, that's two or three, like we call 'em buckets down the page, right. If you're testing the third bucket on your page, you may only get 40% of people seeing that. Right. So you definitely want things that a hundred percent of people see right.

Ash Melwani (00:36:18):

Goes. Okay. Um, and then, so for example, within these buckets, right? Say you wanna do, like, let's just talk about the product page, right? The buy box section, right? There's like about three or four things that you can test there, right? Yeah. Mm-hmm <affirmative> are, are you doing one test at a time? Are you doing like a multi variant test? Um, is that if, if you're doing a test at a time, is that over like four weeks mm-hmm <affirmative> or you do it in a week? Like what does that, what does that structure look

Dylan Ander (00:36:42):

Like? Yeah. So the same way that we had a minimum of that thousand per variant, yeah. Two weeks is the minimum for how long a test has to run. Right? Never run

Rabah Rahil (00:36:51):

Tech, agnostic,

Dylan Ander (00:36:52):

Traffic agnostic. Interesting. Even if you get to a certain level and the reason is weekday traffic reacts differently than weekend traffic. So if you do it for one week, you may have a really impressive weekend or a sale or something like that. Try normal. So if you have two of them, exactly it normalizes two weeks gives you enough that like it is. So for our agency, you know, we've biweekly calls and you know, biweekly tests, that's the whole point. So it's like, you need to let it run for two weeks minimum. So I don't care if it's and the other thing too is if you're like a really nine figure brands, you can just select and Google optimize like 20% of your traffic for testing mm-hmm <affirmative> so think about it in terms of ads, like what percent of your ad spend is testing

Ash Melwani (00:37:30):

10%.

Dylan Ander (00:37:31):

10%. Yeah. So cool. So if you want to use, be really conservative. Yeah. Use 10% of your traffic for testing. Yeah. Yeah. If you're a smaller brand, you love to use a hundred, but that's a little more risky. Right? Right. So it's kind of risk reward. Based on your side, the larger you are, the more flexibility you have, which, you know, those

Rabah Rahil (00:37:46):

Sensors when I was at, uh, my old agency and we would do like the brand lift tests and stuff like that, that was always a challenge because they, you could only do it with this audience and then you have this holdout and it would be like, usually you want to test against like your best audience and it would, there's this kind of again, to your point, like this risk award of like, yeah, we're gonna get learnings, but at the same time, like by definition, the goal isn't performance, it's the learnings. And so you, you would have to kind of, uh, weigh the risk and reward, like you said there, um, do you ever cut off a test early? Like when you see it shit to bed and you're like, oh my God, I can't like waste any more traffic

Dylan Ander (00:38:19):

If it's negatively statistically significant. What

Rabah Rahil (00:38:23):

Is that? Whoa. No, <laugh> hold on. Hold on.

Dylan Ander (00:38:26):

So just like it

Rabah Rahil (00:38:28):

Real mouth,

Dylan Ander (00:38:28):

Hold on. <laugh> no, Bon is a different real <laugh> so the same way an increase can be statistically significant. Yeah. A decrease can be statistically significant. So it might be negative 95%, negative 85%. Got it. So what you can do is you can cut that test variant. So more traffic goes to the rest. Got it. So there will be a point when it's like, damn, we're like a week in, so I will test for a week. So a week is my minimum. No matter what, if you press a button a week is the bare minimum that you can go right at a week, mark, then you can cuz there's like, we call 'em shark fins. Cause it can vary day to day, just like Google analytics or Facebook ads. If you put it on a graph. Of course. So your daily shark fins, you know, at least a week will like remove the shark fin.

Rabah Rahil (00:39:11):

Got it. Makes sense. That is interesting.

Dylan Ander (00:39:14):

So you can cut after one week if it's really, really bad, but you can't promote a winner in a week. Must be two weeks.

Rabah Rahil (00:39:19):

Yeah. No, that makes a ton of sense. I actually like that. Do you find reductive or additive testing is better? Like do you think when you take away stuff is better or do you like to add stuff?

Dylan Ander (00:39:32):

Totally depends on the brand. Okay. And we have data to back that up. Yeah. Fashion brands. When you look at it like sneakers, a lot of them like there's like your, you know, you're above the fold stuff. Like your ads card, button, description, warranty, shipping and testimonials. And that's it. So something like supplements, what you put in your body is sacred. Consumables are like sacred. There's a lot of infos are sacred. So, well, I mean, relax math. <laugh> easy, easy. So yeah, it really does depend on the industry. So, but I mean like as, as an agency, I would say a lot of the times we come in and people are like, man, we've added this thing. We've like worked harder. That's one thing that a lot of like people at home think that like the physically larger the test on the screen, the bigger the test, right.

Dylan Ander (00:40:20):

We've changed small little things that like are physically small on the screen, but make a massive difference. So understanding that difference too. So whether you remove or add understanding that first off, removing a small thing, it might be a bright green, like, you know, like the Klarna after pay all those things, we've moved those around or sometimes removed those and nothing got hit. But the thing is, all we did was remove that and like 40% conversion rate improvement on a PDP because really it's that bright color. That's distracting them from everything else. So just cuz it's physically like small doesn't mean that it's not a big test. Right. So that's number one. And number two, when we start with new clients, we find there's a lot of bullshit on the site. Yeah. Just remove it. Like yeah. Usually what, like one of our first tests that we run on homepages, undoubtedly is removing elements.

Dylan Ander (00:41:07):

That's like remove elements, like, like we have a nomenclature that we use like internally. So like homepage remove elements is pretty much the first test for every single person got. So it's like we test removing different options. So like, it'll be like an ABCD test. Like we think this one could definitely go this and this could go this and this could go and maybe move this guy up and we can structure that as one test to be pretty bold and give yourself, obviously, if they have enough traffic, like now, like we've got a higher threshold of clients we work with, but you know, making sure that we can. So removing things typically is like our first go to cuz we can like, I mean you can code it in two minutes. Yeah. Like you can on Google optimize, you can right. Click remove. Yeah. So like you can remove elements real fast.

Dylan Ander (00:41:45):

Yeah. So like, I mean I can sign a client, you know, two minutes later do that. So that's kind of like some quick ones we get. Yeah. So, and the other thing too, shout out, Nick, again, he told me that I was always talking about it wrong, which is like, we'll find some, like even in the sales process, I'd be like to my agency owners out there, we'll find some quick wins, we'll get some, you know, big, big, early victories. Like it's like, no, they're actually compounding data points. Yeah. Meaning if I improve your homepage, conversion rate 7%, which is only globally gonna be like 1%, you know, something like that. But then we improve the next test. Like another 4%. You now have a, you know, compounding improvement effect. And then now your emails, if they're going to the homepage and something else, now we do another test like three months later that does a 20% increase on the homepage. From where you first were assuming your conversion rate stayed the same, you're looking at like a 50% increase, not a four plus like 11 plus right. Or seven plus 11 plus you know, 20 it's exponential. So the compounding data Fs crucial.

Rabah Rahil (00:42:43):

Do you look at like new and returning cohorts as different or

Dylan Ander (00:42:49):

Absolutely.

Rabah Rahil (00:42:49):

Because going back to your point of like changing the website, I got a really amazing, uh, audit done. Shout out, uh, audit bunch of amazing

Dylan Ander (00:42:57):

Canadians Sean, you're the man, Sean. He's amazing Taylor. Yeah. They're they're our biggest partners, honestly.

Rabah Rahil (00:43:01):

Amazing people, amazing humans.

Dylan Ander (00:43:02):

By the way, if you look at audit, you'll see that it's backed by splittin.com database.

Rabah Rahil (00:43:07):

Amazing. We, we only hang with the homies. Um, and they did this amazing audit and we actually overhauled, uh, a site. This is when I was at my agency and it was beautiful and everything worked out well, but we saw a little bit of actual degradation in the conversion rate. And my thesis was that, I mean, when we looked at the actual data, it was almost all coming from returning customers. Mm-hmm <affirmative> and because you get a vibe and you get like, oh, and you move stuff around, it's kind of like, you know, you, you change your own grocery store layout or something that you're really accustomed to and people. And not that it wasn't a great thing. It was just like, people can get really used to inefficiencies <laugh> mm-hmm <affirmative> and we removed those inefficiencies and yeah, it took a little bit, it ended up normalized and netted out, excuse me, in the positive.

Rabah Rahil (00:43:51):

But it was just interesting to me because everything made sense and like it all worked out and the new customer conversion rate was fantastic, but we did take a little bit of hit in terms of those incumbent cohorts. Um, how do you deal with that? Or how do you think about that? Or like, uh, cause I I'm with you on that where there's just overhauling a website is just, I'd rather just play with Landers all day and just put paid traffic that way versus overhauling the website. Cuz that in, in retrospect, I think that's probably what we should have done is just kind of eased people into a new aesthetic versus like, oh, Hey look at this beautiful new site,

Dylan Ander (00:44:24):

Whole thing. Yeah. So I've got a fat piece of advice for everyone at home. So, um, there's four main ones. So when you go to Google analytics, you have those filters. There are those two little boxes that are at the top. You can post production, you can take my fingers <laugh> um, and you can filter by almost anything. You can make custom filters, but just go mobile and desktop. So when we're choosing a test as a winner, your mobile code base and your desktop code base are literally different websites. Like they're not even the same, like Shopify has two different code bases for your mobile and desktop website. So a test can win on mobile and lose on desktop. That's interesting. So that's number one. So if you're, if you're a CRO firm or anyone else, like if you guys are split testing, internal, whatever it is, if you just go based on total, uh, you just failed interest because like I said, you can have different changes.

Dylan Ander (00:45:16):

So think of four and then, so other than mobile and desktop, there's also new verse returning and that's also pre-done filters. You just hit the drop down and, you know mm. And, uh, yeah. And it hits, you know, new and returning. So those are four things that you need to look at. So I always make custom filters of desktop, new mobile, new desktop returning and desktop new. Yeah. So I don't know if I, I messed that up, but you, yeah. You guys get what I'm saying. So it's like, you know, you have four custom filters that now that's how we really judge mm-hmm <affirmative>. So everyone at the bare minimum look at mobile and desktop because we've had, we've found a lot where like, we're marketers, we're working on the computer. So not a lot of people have this. So I actually like pay for my team members.

Dylan Ander (00:45:58):

We have a little clip that you put your phone right next to your computer, which highly recommends you have your phone here should like put it on silent so you can work. But like, when you're looking at your website, pull it up right here and look at your phone. Cause people don't do that enough. Mobile is 90%. So yeah, this means 90%. This means 10%. Yeah. So going back on the mobile and desktop, what's super crucial is, you know, looking at each of those cohorts. So if it's winning on, let's say like returning users, tanks, but new users, it's great. That's called personalization tracking. So yeah. So personalization means you're feeding different experiences to different people based on what is on the website. So new verse returning is a behavior. Um, mobile verse desktop is a behavior. There's also location based, right? Like if you go to Neil patel.com, like, you know, the SEO legend, it says it literally it's my favorite.

Dylan Ander (00:46:46):

It says, I'm looking for a business in Brooklyn, New York to improve. <laugh> like, is it yours? So it's like personalization obviously to like location. So yeah, you can feed and Google optimize and you press it permanent. So it let's say like, you look like literally, if you don't do this already, Ash, go back to all your historical tests. Mm-hmm <affirmative> and filter every single result for new verse returning mm-hmm <affirmative> and you may be like, damn overall, it was losing, but it crushed it for new new ones. You can go on Google, optimize, set up an experiment for personalization and press like go and permanently. And now for the rest of eternity on your website, only new users see that experience versus old. So it takes no code, no anything mm-hmm <affirmative> you can personalize after that. So there's mobile desktop, new returning. And, and that's like, I don't know, 80, 90% of like the actual experimentation that we do the results.

Ash Melwani (00:47:35):

So you're probably deploying, like, even if it's just like, all right, mobile is crushing for new, you'll probably just deploy that and can the rest see that? I think that's where people kind of mess up. Yeah. And it's like, cause we redid our entire homepage. Right? Yeah. And what we saw was the same thing, same thing that you said, right. Returning was a lower conversion rate, but new, new customers were, were higher. Yeah. Um, I think we made that executive decision cuz overall it was more of a, it, it, it just depicted our brand a little bit better than what it was before, which was like just collection and buy boxes and stuff. But our customers were used to that. Right. Mm-hmm <affirmative> so that's probably why it decreased a little bit, but I think over time, right. People, customers get used to it and they're like, okay, I can find things again. But to your point, it's like, what if we just made that entire experience a little bit more personalized to new, to new visitors and then not have to worry about reteaching customers where they find their favorite products. Right? Yeah. It's a great, great

Dylan Ander (00:48:28):

Point. So on the homepage, the ones that we find are the best is like I said, above the fold. So to contextually now, like, you know, the listeners can like kind of follow along the logic, the hero image, like, you know yeah. Above the fold and the headline are two of the biggest that are ripe for personalization. Yeah. So let's say like for gummies, it's like the best collagen in the world. Right. So it's like that's for new visitors, right? Yeah. Cause they're now meeting the best collagen in the world. Exactly. What about the new returning, like returning users, like ready to re-up on your, you know, thing. So if people

Ash Melwani (00:49:01):

Deal something. Yeah.

Dylan Ander (00:49:02):

Well, no, not the deal. This is a headline where you're just changing the messaging to returning users, which is like, you know, um, improving hair loss. Right? Like now you can drive the value proposition instead of like introducing them to a brand. Cause they already know you they've been there before. Right. Right. Right. So it's like not necessarily a sale or a discount, you can go that route. Like I've done that before where like 20% returning discount. But like the thing is, you don't know if they purchased or not, you can target target it that way. True. But you can change your messaging just a little bit of like, you know, meet your new, you know, plug for health. Yeah. Versus like, you know, continue improving your health. Right. So those are two very different headlines, very different headlines fundamentally. Right. So that you can have for new verse returning

Rabah Rahil (00:49:43):

Or even, I guess you're right though. You'd have to have some sort of, uh, tracking in terms of purchase, but even being able to, and I think I, I get we're saying, cuz it does start to get into the offer, but um, a lot of people never subscribe on the first purchase. Right. Mm-hmm <affirmative> but at the same time, like almost everybody's subscription is like the best economics purchase. Like all things being equal. Like you would love people to hit the subscribe and say, versus mm-hmm <affirmative> actually just one off purchase. And so there's some things you can get into there. That's so fascinating. I never thought of it that way.

Ash Melwani (00:50:13):

Probably. So would you, so would you say that you would be pushing these changes live just through like Google optimize or is this something that you can also like how, how do people do this? Like is it only through that? Is there apps that you can do that?

Dylan Ander (00:50:24):

No. So there's, there's a lot of relatively too. Um, yes. You can build native, like I'm like, so you can build anything native. Yeah. Right. Like, like especially, I mean think like 90% of our clients are on Shopify. Right. Um, you can build anything native. That's like, you know, there's if your developer ever says, no, it means you need a better developer. That's

Rabah Rahil (00:50:41):

What the money's for. <laugh> that's

Dylan Ander (00:50:42):

What the money's for. Holy Lord. <laugh> my team's been blowing my mind lately. I'm like, whoa, you were able to do that. That's crazy. Yeah. So, um, you can make it custom. Um, Google optimiz has a really like convenient setting, which is how you go into the editor and however you changed it. Right. You can just go in and instead of like, you know how there's like the AB test multivariate test personalization it's right there. So you hit that personalization. You do the test experiment and you just click go and that's it. So that's the easiest way to do it. Like the least amount of code, the easiest way it's free. Yeah. That's why I recommend Google optimize for it. So there's like other personalization, like if you Google personalization, like that's the future of CRO. Yeah. Is how personalized can you make your website to every single different user? The message for people who are coming through Google ads, if you got down to it, like some of our biggest clients they have like, you know, I don't know, like collagen dash one collagen dash two collagen dash 5, 6, 7, 8, 9. And they all are identical except for like the

Ash Melwani (00:51:44):

Headline,

Dylan Ander (00:51:45):

The headline. Yeah, exactly. So it's like to match that. So getting into crazy personalization is how you ultimately scale, scale, scale, scale, like, yeah. Ads are great, but like each of these personalized like channels, like email and SMS, you know, like scaling ads, it has an ancillary benefit by getting more people in, but you need to make sure it's personalized. So yeah. Google optimizes the easiest way. But if you Google personalization, you'll see like Rebi, Rebi is like a really cool Shopify app. Do you guys use it?

Ash Melwani (00:52:12):

Uh, we just started looking into

Dylan Ander (00:52:14):

It. Yeah. Cool. So would recommend, um, for re and, but only for people that have like many skews, right. Because like it's really good at personalization and like AI learning of like

Rabah Rahil (00:52:22):

That's the Amazon, uh, kind of mimic, right?

Dylan Ander (00:52:25):

Exactly.

Rabah Rahil (00:52:26):

Because Amazon, like everybody's like Facebook news feed equivalent of Amazon, like everybody's, it's a bazillion different homepages. Like everybody it's that Amazon homepage is essentially unique to you. And so this is the Shopify

Dylan Ander (00:52:39):

Thing. So this is a shop by plugin. It's called Rebi Rey. Shout out if you wanna reach out to us for a good partnership pod that's yeah, we do. <laugh> sponsor to the podcast. <laugh> yeah, it's really good. But the only thing that I do use it for is, um, recommended purchases. That's what I found that Reba has nailed it. So that's a different area of personalization. Yeah. Than the actual stuff like personalization, if you Google it, like how do I personalize my, you know, E eCom store? It's like, oh, recommended purchases. Or you may also like, that's where Reba's really good for you put it in. And their AI ultimately like figures out what is actually the best. So right. For you guys. I do think Rebi on the bottom of your PDPs, right. Would probably be solid. Right, right. Just a guess. Just not financial advice. Yeah.

Ash Melwani (00:53:20):

No, no hundred percent. No, it makes sense. Cuz I think we're doing it in a way of where we're doing that research of like, okay, what are some similar or even through triple, it tells us frequently bought together. Right? Exactly. So it's on these products, like, all right, you may also want this, this and this. Right. Which is similar to the Amazon, you know, at Al to cart. So I think that might be good cuz then at least there's some type of intelligence that's like looking at all the factors and then showing that dynamically. I think Shopify had that. Right. I don't know if it's any good, to

Rabah Rahil (00:53:48):

Be honest. I knew I only knew of the plugin.

Ash Melwani (00:53:50):

Yeah. There's dynamic, uh, product recommendations. I don't know if like there works.

Dylan Ander (00:53:55):

Well, you do need to be on Shopify plus for that. Um, so that's like a minimum two to $4,000 per month going not everyone can yeah. I'll belly ache of the GA four problem too.

Rabah Rahil (00:54:08):

Get me start on GA four. Yeah. I can't believe Google's just gonna blow up your UA. It's nuts. There's like no roadmap. There's no succession plan. It's just like, oh F you guys, you gotta use this horrible, horrible thing called GA

Dylan Ander (00:54:19):

Fours horrendous. So horrendous. Um,

Rabah Rahil (00:54:22):

We're talking about Google analytics for, for anybody. And when we say UA, it's universal analytics, which is like, mm-hmm, <affirmative> the dummies version of, you can just put it in an absolute works versus yeah. Google analytics four, you actually have to understand data pipeline and you have to put a nice data pipeline together. It is. Yeah. Atrocious. So like you have to be really high or from my experience, it's been like really bad to implement it's you have to be really high level.

Dylan Ander (00:54:44):

It's fucking worse. It's horrible. Google. We love you. And we also hate you. Thanks a lot for nothing. No it's so yeah. So luckily Google optimize now integrates with GA four. I actually have like a little focus group of a couple of fellow nerds who are like trying to figure it out. Believe I cracked the code.

Rabah Rahil (00:55:00):

Oh, look at this, reach out to my man, if you need that G

Dylan Ander (00:55:02):

Shameless plug, which is not available to the public yet, but we made a software that goes straight into the Google analytics for API renames everything and talks and looks, we we're we're considering like I'm talking to a couple e-com owners, like what they want. Theoretically. My goal is to make a universal analytics for clone.

Rabah Rahil (00:55:23):

Amazing.

Dylan Ander (00:55:24):

So that's an upcoming thing. So like just sticking on the CRO side, like, you know, um, I've got a couple SAS products coming out that like literally will automate a lot of split testing, like full machine learning. I wrote a multi-arm bandit that like gets to statistical significance faster other than just 1, 2, 3, 4 1, 2, 3, 4. Cause you can start autopilot pushing traffic to the better performing ones, improve it faster. So I've got a couple innovations coming up that, uh, you guys will be hearing about

Rabah Rahil (00:55:50):

This one door closes another opens. I love that. <laugh> um, a couple more questions. One. When you're talking conversion rate, do you like to look at how many people actually come to the site or are you going straight from PDP to purchase? Cause I know people sometimes have a content play and I, I always recommend depending on the context and I want to answer for you, but just as my viewpoint was when people are coming to read content, I don't con I consider that like them coming to my library, versus when people get into like a slash product or something, I consider that them actually kind of coming into my shop. And I actually like to measure that. Cause if not, if you have a huge content database, like, and you're getting a ton of content traffic, you might have like a 0.5 conversion rate and it like people aren't coming there to shop. And so it's like, it kind of can pervert that conversion rate. How do you think of that?

Dylan Ander (00:56:40):

I want you to answer that first. Yeah.

Rabah Rahil (00:56:41):

How do you think of that? Cause you guys do have pretty strong content

Ash Melwani (00:56:44):

Too. Yeah. I think the biggest thing is we'll never like exactly what you said, right? It's I'm I'm sending you there just for the purpose of education. Yes. Right. So for example, if I think we send out weekly recipes, right? Um, yeah. It's a way to kind of cross sell. So like if you've, if you purchased a fruity cereal and we're showing you a recipe of, uh, cocoa cereal, waffles, right. It's like, okay, you could still use your fruity cereal collagen for that. But then like, it kind of shows you there's other flavors that you can utilize it for, but it's tough because you don't, there's no, there's no call to action on the recipe page. Right. I want you to be there to either continue looking at other recipes. So like the CTA is go to the next one, not go to the product. Right.

Dylan Ander (00:57:26):

May wanna test that homie.

Rabah Rahil (00:57:28):

Well, and it be fair though, to just push back is like, you might already have the

Ash Melwani (00:57:31):

Product. Yeah.

Rabah Rahil (00:57:32):

So you've already bought it's

Ash Melwani (00:57:33):

True.

Rabah Rahil (00:57:33):

It is true learning how to use it and consume it.

Ash Melwani (00:57:35):

And so a lot of those, a lot of those, like for example, a lot of the text and emails that go out because of that, a lot of the conversions that happen after are view through not click through mm-hmm <affirmative> right.

Rabah Rahil (00:57:49):

Z

Dylan Ander (00:57:50):

Manager command. Yeah. It's like, like, so I do agree with you. So I have like three or four clients that just smash SEO. Yeah. And they're looking at the blogs and they're like, how do we convert to customers? How do we convert to customers? And in Google analytics there's pathways. Yeah. That you can see entrance pages and exit pages. Yeah. Yep. So are they coming from like the main homepage clicking to a blog and then just bouncing? Yeah. Are they first on a blog then go to the homepage product page and then drop. Yeah. So like, you know, you kind of figure out where the leaky hole is in the bucket. Yep. So with the content, there's definitely different KPIs like for SEO. I mean, it, it sounds crazy, but like I'm actually trying to help the SEO team further as like a CRO agency.

Dylan Ander (00:58:32):

Like let's get your time on page time on higher. Like let's get exactly like all those metrics that you get to improve and get better usable content. Yeah. There was a client today where we looked at their heat map and they did a little summary of the top in this article. You'll learn this, this and this. And they didn't have hyperlinks going down. Yeah. So we saw in the heat map that it was, and I was like, let's test this and boom, oh my God. The time on site, like went through the roof, it was like, um, like a 60% increase in time on site, like of the page and the clickthrough rate to other pages was like a bumped 30%. Mm. So you're a hundred percent right. That there's different, like intentions and, and also go back to empathy mapping. Yeah. That's the other thing too, of like understanding where they are in their journey with you. That's what it ultimately comes back to as a north star.

Rabah Rahil (00:59:15):

Makes sense. Powerful anchor links. Um, yeah, that makes a ton of sense. Think I just lost my train of thought.

Ash Melwani (00:59:21):

I had a really big question, I think. Right. Um, the biggest thing that I think we were talking about before we started, right. Is CR uh, conversion rate versus like a, uh, AOV. Right. Um, right now I think, you know, towards the beginning of the year, it's like, how do I improve my conversion rate? I just wanna acquire customers at the lowest cost right now. Right. Mm-hmm <affirmative>, which is, I think we nailed that. And I think we took out all the, the friction of buying. It's like, all right, you come to the website, you go through this landing page, you understand what the product is? All right. You can either buy one bottle or two and that's it. Right. Um, to, to some people's point it's like remove the two and just go all in on the one. And your conversion rate will probably skyrocket, but my C is still gonna be 30, $40.

Ash Melwani (01:00:03):

I'll probably move a little bit more volume. But like, how do I, I think the main question here is how do I balance between conversion rate, but then AOV as well. Right. Mm-hmm <affirmative> so like, my CAC is like, I can right now, like 20 K a day, my pack is at 50. Right. My AOVs at 60. Yeah. I can't, I can't get anything lower as I scale. Right. Mm-hmm <affirmative> so the only thing I can do, like I can make better creatives. I don't think it's gonna move the needle as much as moving the needle on CRO. Right. Right. So either I get people to buy more or just, I think that's my, my thing is I need to get people to buy more. Right. Mm-hmm <affirmative> so how do I do that without impacting converge? Totally.

Dylan Ander (01:00:43):

Totally. So, um, comes down to that original part of asking your customers what they want and just give it to them. Like, what is their goal is their goal to change their lifestyle. If it is you need three products, you need three different products in a bundle together to change your actual lifestyle or health or hair or anything like that. Mm-hmm <affirmative>. So it comes back to that offer that you give, dig in more to the offer, right. Find offers that are more money, more value. Like you get what you pay for. Right. And you can sell what they're paying for too. So that's my answer. Number one. Mm-hmm <affirmative> answer number two. Um, the default, um, of like where you see, um, you know, of like which bundle you see first, right? Of like what they click on. That's another way that like easy way doesn't really hurt conversion rate. Like make sure your default is like, if you have one, three and five, sorry guys, if you, if you have one, three and five in your buy boxes, make sure three or make sure three or five is usually, I mean, there's something we test, but make sure three or five is like your default that people go to first. Yeah.

Ash Melwani (01:01:40):

And it doesn't hurt conversion rate or you've seen it not hurt conversion

Dylan Ander (01:01:44):

Rate. No, it doesn't hurt conversion rate and boost say V a bit. So again, not financial advice. Go test, try it. That's another one. And then obviously post-purchase upsells. Yeah. So that's another thing that you do. Prepurchase upsells is something also that actually can hurt conversion rates. Yeah. If you continue selling. Yeah. Yeah. So agree free. So in the card itself, if you're selling it doesn't mean don't do pre-purchase upsells. Right. But post-purchase upsells. Meaning like to, to those who don't know at home, one click up sell as your Firestone, you have the best mastermind. We love you. Um, the game. Oh. As, as there's the man, um, you know, Zipify one click up sell. So like after they purchase, it's like, would you like to buy another? Yeah. So that offer right there. You can go crazy with those. Yeah. I have a trick that I use for supplement brands, which can, you can work for works best for consumables, but here's a little dirty secret. Mm. What does your upsell look like after people buy? I know you probably have like a couple of conditionals. Yeah. But overall what's your most popular upsell?

Ash Melwani (01:02:41):

Um, so that we have two, right? So for one of our main funnels, which is our burn, um, supplement, we get another one for X amount off. Right. Mm-hmm <affirmative> that, that has like a 16% take rate. Mm-hmm <affirmative> I tried a new one for our collagen one. So on the landing page, I'm testing, showing, uh, four flavors. And then in the post purchase, I'm showing four new flavors of like, Hey, these are limited edition flavors. You can only get it here that had a 20% take rate.

Rabah Rahil (01:03:07):

That's cool. Which crazy

Dylan Ander (01:03:08):

On one bottle

Ash Melwani (01:03:09):

On one bottle. Yes.

Dylan Ander (01:03:10):

Cool. You wanna hear something crazy? <laugh> are you ready? Yeah. Hold your pants. Try.

Rabah Rahil (01:03:15):

Nice, please.

Dylan Ander (01:03:17):

Upsell eight bottles.

Ash Melwani (01:03:19):

Jesus.

Rabah Rahil (01:03:21):

If anybody does, it's your community. I've seen some of those pictures.

Dylan Ander (01:03:25):

Yeah. It's crazy. So like so much if you're it's it's insane. Right. So it's like, here's

Ash Melwani (01:03:29):

The thing. I probably won't see these people for another year though. <laugh>

Rabah Rahil (01:03:31):

That's

Dylan Ander (01:03:32):

The one. Would you rather the money now or now? Or never's

Rabah Rahil (01:03:35):

A fair point. That's a fair point for sure.

Dylan Ander (01:03:36):

So, and there's a discount, but you're getting it now and you're getting guaranteed sales. So the thing is, um, we have a conditional based on like, if it's a first time buyer, second time buyer again, personalization. Yeah. Right. Um, I upsell way more bottles than they need. So for my supplement brands, my smallest upsell is five bottles. Right. And the take rate is around like nine or 10%, which is not much less than yours, but mine's worth damn a lot more. And also when you push more units together, your cogs go down. Yeah. Cost of goods go down for sure. Which is again, more margins. So it's like less

Rabah Rahil (01:04:10):

Picking all these things. Yeah,

Dylan Ander (01:04:11):

Exactly. So interesting. The post purchases be an animal. Like some people go really aggressive on email marketing, you opt into their like pop up. Yeah. Um, they literally send you four emails a day for the next three days. Yeah. You either get hell of unsubscribes or hell of sales. You get both. Right. So it's like a big burn strategy. Like, I mean, PE already got the sale. Why not upsell like six or eight? You know? Cause if people really like the product, they're like, ah, I'm gonna use this anyways. Like I can just put it in my house. Now I get a 40% discount. And with that many units, you can give a 40% discount and your unit economics are killer.

Ash Melwani (01:04:46):

Yeah. Makes sense.

Rabah Rahil (01:04:48):

That's amazing. Wow. I will caution people though about hitting email too hard. I had a, uh, agency that took over when I left and they were sending like 78 million emails a month and like the first month and a half, like the numbers looked amazing. But the growth in, like it was a negative net growth in his list subscriber. And like people are getting pissed off and like you can't for sure. Email something where you need to make sure that it's tactful. Like it can't just be a money grab and SMS is even worse or, uh, more sensitive where it's just, just such an intimate medium. So definitely again, whatever the goal is, like, if it is a money grab, you know, smash, email do thing, but yeah, exactly. Email should bet full play in my opinion. And you need to make sure that you're generating value for the user, not just using them as a bank account. Um, one last question and we'll wrap up. Yep. Shipping. You guys are messing around with shipping. How like important is free shipping, no shipping. Cuz you guys did some really interesting stuff with shipping, right. That changed.

Ash Melwani (01:05:42):

We're about to bit. We about to. Yeah.

Dylan Ander (01:05:44):

Well, so you tell me what your plan is and I'll tell you if I, uh,

Ash Melwani (01:05:47):

So the plan, um, customers, if you're listening, sorry. <laugh>

Ash Melwani (01:05:53):

So we wanna best college <laugh> so we wanna, so right now we offer free shipping. Yeah, right. Um, right now what we do, no threshold. No, there's no T there's technically no threshold, but when you do add to cart, it does say you've unlocked it at like $25. Got it. Just so that it's like, oh, I've earned this right customers. Um, you're gonna get it regardless. Um, so now what we wanna do is we wanna get these people who are buying just a single bottle to get one more thing. Yeah. So we we're thinking of raising that threshold to actually put it at 50 and then charging flat fee of four, $5. Right. Mm-hmm <affirmative> um, and then if, if that first threshold is 50 and then the next threshold for free gift is 75 and then a 1 25 and then 1 75. Right. So it's like one more product just to get to the next tier. It's not like two or three. Right. It's like just add one more thing. Right. Mm-hmm <affirmative> so we're thinking of using free shipping as that initial, like gamification like, oh, I just unlocked this. Well, wait $25 more. I can get a shaker go. Yeah. $25 more. I can get this. Right. So that's the test we wanna do. Um, just have to get around to it.

Dylan Ander (01:07:00):

Truthfully. That sounds like several tests in one I would stick to, you know, like if you want to gamify it, that's like a whole plan. Right? Like you would need a CRO rollout

Ash Melwani (01:07:09):

To confirm, well, it's all there. It's basically all there. Yeah.

Dylan Ander (01:07:11):

So, so here's a dirty trick. Everyone has done a countdown timer before an e-commerce. Right, right. Is it a really countdown timer? Sorry. Customers have a, it's not a real countdown timer. You can still buy after 10 minutes or whatever. Yeah. Um, there's there's only three units left by now, before they're out of stock. God, that's kind cool. So, you know, those have gotten you, especially on the shoes. So like

Rabah Rahil (01:07:35):

The shoes might be real. That'll be fair. <laugh> yeah, they are. That's

Dylan Ander (01:07:38):

True. Well, you never know. That's why you put it there. I, so, um, so what we've done before is keep your full free shipping mm-hmm

Ash Melwani (01:07:44):

<affirmative>

Dylan Ander (01:07:45):

But say free shipping at 50, at 60, 75 and a hundred, but it's always free shipping.

Ash Melwani (01:07:53):

Interesting.

Dylan Ander (01:07:54):

So people think it, and then they get to the cart and they're like, wait, hold on. I thought it was $50 unlocked. It's 30 click boom.

Ash Melwani (01:08:02):

Choose.

Dylan Ander (01:08:04):

Yeah. So this is so it's like an intentional typo or error. Yeah. That's in the customer's favor.

Ash Melwani (01:08:09):

Right?

Rabah Rahil (01:08:10):

This is really interesting. So I don't know if you guys' logistics, uh, can do this, but somebody was actually doing something similar where, um, basically it was like a two day shipping and you would get like the two day shipping for free, but like not for free, like, everybody was pretty much like their logistics were capable of two day shipping mm-hmm <affirmative> but you, I don't remember if you would pay a little more or you would hit a threshold and then you'd get the two day shipping, so you'd get it faster or what have you. But it was in the same vein of expedited delivery. That's that's what it was. Yeah. Yeah. In the same vein where like I get my stuff faster because I bought more. So anyways, I'm kind of in the weeds there, but that's uh, that's in incredible. Um,

Ash Melwani (01:08:46):

I'm gonna try that immediately.

Rabah Rahil (01:08:47):

Immediately. That's about a being

Dylan Ander (01:08:49):

Ash's totally DM me a bunch of times. Like, Hey man, I'm gonna implement this right now and get back to you. And he never gets back to

Ash Melwani (01:08:55):

<laugh>. I dunno. Some of these tests. Yeah,

Dylan Ander (01:08:59):

Damn. I'm like, man, you're reciting some CRO hacks buddy.

Ash Melwani (01:09:02):

<laugh> yeah. This one I'm gonna have to definitely test. Cause I, I was thinking about it last night. I was talking to Ron about it and I was like, all well, we're gonna have to, you know, change the shipping. And like you said, it's a lot of steps. So makes sense to just up the tier and just,

Dylan Ander (01:09:17):

Yeah, that's it. One other thing on this too, in the announcement bar at the top, there's this thing called, I think it's called AOV countdown or like AOV countdown time or something like that. And it says free shipping at like $75 and they add $25 to cart and it says 50 more dollars to free shipping. Like it's a dynamic Shopify app. Like it's a plugin.

Rabah Rahil (01:09:36):

We had something similar for, uh, I used to work for, uh, my agency. We had a beard care client and we had a bundle builder page cuz one of the things that he was getting crushed with was AOV like his, his Mo AOV was

Dylan Ander (01:09:47):

Anthony make

Rabah Rahil (01:09:48):

Box. No, ah,

Dylan Ander (01:09:49):

No

Rabah Rahil (01:09:50):

<laugh>

Dylan Ander (01:09:50):

No took a swing.

Rabah Rahil (01:09:51):

Um, but it was this bundle builder where it was essentially like this little sticky side bar and you had this collection of products and like, oh, buy this, that and that. And we controlled those products cuz you would make a collection to make sure that they were high margin cuz you didn't want people to buy like, you know, three low margin products. And then all of a sudden you get 50% off kind of thing. Yeah. And so you have to be, make sure the economics net out, but it worked out really well. Wasn't great for new customers, which would make sense mm-hmm <affirmative> but um, for our returning customers, we, we really smashed that. It crushed in terms of boosting, um, the AOV in terms of our new cus or our, uh, returning customers to get 'em to spend more money cuz you already like the product.

Rabah Rahil (01:10:28):

And then we would have like a brush and then these, these things that kind of like built well and then you get like 30% off and we get this higher AOB. So the, the acquisition cost was there. So, um, alright, one last creepy question and then we'll wrap up mm-hmm <affirmative> so this is actually, uh, I didn't do my homework. I've been traveling. I'm sorry folks. And, and Dylan actually saved me, but, and this is why he's gonna have a better answer probably than me and Ash. But the question is, is it more efficacious to dry off in the shower, out of the shower? I'm gonna go first. So I'll give you some time to answer and you already have the answer, so we'll get, we'll have you wrap up cause you probably have the best answer, but my vote is out of the shower because, because sometimes I'm a little loosey goosey with the towel and it's the ban of my, my existence when like the little corner hits a puddle and then I like get like some of the towel wet. So I'm, I'm an outta the shower guy where

Ash Melwani (01:11:19):

Uh, um, very similar. Yeah, yeah. Out the shower. Uh, but also you have the, uh, bath mat outside in it kind of. Exactly. So I'll I'll quick pat and then hop out. Yeah. Yeah. That's my,

Rabah Rahil (01:11:31):

I'll see your quick pad in the shower.

Ash Melwani (01:11:32):

Yeah, no, I have my, I have my towel like, like hanging so I could just grab it quick pat and then I'm out. Not fully.

Dylan Ander (01:11:38):

So you're an in the shower guy.

Rabah Rahil (01:11:41):

Well, yeah, I guess what we have, we have to define dry and off then, right? Yeah. How

Ash Melwani (01:11:44):

Long?

Rabah Rahil (01:11:44):

Because I think I am. I have, cause I'm

Ash Melwani (01:11:46):

Technically dry when I can I'm

Rabah Rahil (01:11:48):

In the shower pattern outta the shower. The other thing too, Tim shower,

Ash Melwani (01:11:51):

Pattern out, shower

Rabah Rahil (01:11:53):

Dryer. Well, the other thing I just realized too is the shower configuration is matters too. Like if it's a bathtub shower, configuration, it's a totally different strategy than if it's a standup shower for me. Yeah.

Dylan Ander (01:12:02):

It's true. That's why you can't take CRO financial advice. Like there's so much that

Rabah Rahil (01:12:07):

Shower test.

Dylan Ander (01:12:07):

Well, here's the thing. So

Rabah Rahil (01:12:08):

What's the, what's the, what's the right answer

Dylan Ander (01:12:10):

Here. The reason I did this, the right answer in college, we had a, um, a public speaking class and they said you can debate any topic, more ridiculous, the better. And this was our, our, our debate. So the, in the structured debate we won for in the shower, in

Rabah Rahil (01:12:23):

The

Dylan Ander (01:12:24):

Wow, because

Rabah Rahil (01:12:25):

I'm 0% in on that.

Dylan Ander (01:12:26):

Well, here's the thing. You're in Texas. We are in New York when it's cause you winter, you're gonna be freezing when you get outta the shower. So you're not gonna want that. That's number one. Number two, you save yourself water on the floor where you can slip and potentially die. But

Rabah Rahil (01:12:39):

That's poor people. Stuff who got bath mats. <laugh> we're

Dylan Ander (01:12:42):

Rich. Whoa, come on. You're you're gonna say that no water gets beyond the bath mat or is your whole bathroom, a bath mat, but

Rabah Rahil (01:12:47):

It's tile. Right? And actually what's

Dylan Ander (01:12:50):

Tile on water

Rabah Rahil (01:12:51):

To be fair. I lived in Indiana and our house had a bathroom in the carpet. Which how, how or

Ash Melwani (01:12:57):

Carpet in the

Rabah Rahil (01:12:58):

Bathroom. How ridiculous is that? That

Ash Melwani (01:12:59):

Type it's

Rabah Rahil (01:13:00):

Okay. Oh, I will give you the temperature control. You, you definitely get cold and you slap that. You grab

Dylan Ander (01:13:05):

The towel,

Rabah Rahil (01:13:05):

You close the curtain.

Ash Melwani (01:13:06):

You're telling me your bathroom. Doesn't steam up completely.

Dylan Ander (01:13:09):

It does.

Rabah Rahil (01:13:11):

What's the pen sometimes too. If you have a sketchy water heater and stuff, but the

Dylan Ander (01:13:14):

Temperature, you said rich people, shit. I don't have a sketchy heater. Exactly.

Rabah Rahil (01:13:17):

The temperature, the temperature control is a good vector. I didn't think of

Dylan Ander (01:13:20):

That. Well, here's one downside of in the shower.

Rabah Rahil (01:13:22):

What about the danger of the, do you do the, the foot up on the side or something like that? Cause what if you're in a standup shower,

Dylan Ander (01:13:27):

The hard to reach

Rabah Rahil (01:13:29):

Did the towel dip is like the ban of my existence. I can't do it cuz like you,

Ash Melwani (01:13:32):

I don't want it to touch. I don't wanna, you

Rabah Rahil (01:13:33):

Render like a corner of the towel useless cuz it's wet now. Like you can't draw off with a wet towel. So how do you get around that problem?

Dylan Ander (01:13:38):

Well, that is the

Ash Melwani (01:13:40):

Accidentally drop

Rabah Rahil (01:13:41):

The towel done. Right? I drop it outside. I don't know. But

Dylan Ander (01:13:44):

If you're outside bend over and like you're, there's like water coming out now. Like everything's wet. Nobody's on the floor and then you can slip and die.

Rabah Rahil (01:13:52):

I don't know. I

Dylan Ander (01:13:53):

Think, I think the safety risk

Rabah Rahil (01:13:55):

Outside

Dylan Ander (01:13:55):

I'm team inside. You're

Rabah Rahil (01:13:57):

Inside from

Dylan Ander (01:13:57):

Team

Ash Melwani (01:13:57):

Inside. I mean, I've been doing this for 28

Rabah Rahil (01:14:00):

Years.

Ash Melwani (01:14:01):

I can't change now guys. <laugh> so team outside, I get the points. They're all valid team outside.

Rabah Rahil (01:14:07):

Amazing. All right. Tweet at us. Are you team inside or team outside? Shower Dylan. This is such a blast dude. Thank you so much for coming on. Split testing.com. Tell people how they can get more involved. Do you, where can they follow you? This time is yours. My friends

Dylan Ander (01:14:19):

Split testing.com. That's uh, long. The short of it. It's what we do. Um, find me on Twitter. DM me, ask me questions. I'm genuinely here to support. I have, there's so many answers that I have now that I didn't have when I was either starting or in the middle of my journey. It's about CRO I've. I've also earned, you know, dozens of millions in ads and all of that. So if you guys just have e-comm questions, come to me, DMS are open. It is me. Uh, I am there to help. So you know, if you guys are brand doing over, I don't know, you know, couple million a year, definitely top we've found money back guarantee. It'll blow your minds. But uh, other than that, yeah. Um, I'm here to help here to support, follow [email protected] or split testing com without the dot on Twitter. And um,

Rabah Rahil (01:15:01):

Amazing.

Dylan Ander (01:15:01):

Appreciate it. Gentlemen,

Rabah Rahil (01:15:02):

Tell us about the vitamin shop. Tell us about the mentor pass. You know what to do

Ash Melwani (01:15:07):

Actually tweeted me today. I'm I'm very grateful. Uh, picture of, uh, shop. Well,

Rabah Rahil (01:15:11):

You guys were just out. Was it

Ash Melwani (01:15:12):

In Jersey? Yeah, just did a little popup, uh, in Jersey just to get some, uh, some, uh, taste, the table

Rabah Rahil (01:15:18):

Wrap looked good. The whole setups clean

Dylan Ander (01:15:20):

You size.

Rabah Rahil (01:15:21):

We pops it slaps

Ash Melwani (01:15:22):

For sure

Dylan Ander (01:15:23):

Slaps,

Ash Melwani (01:15:23):

But yeah, if you're ever buy a vitamin shop, check it out. Avi, send me a picture on Twitter. Um, on mentor pass right now and uh, just follow me on, uh, Twitter at Ash Mulani.

Rabah Rahil (01:15:34):

You've been dropping some thread heat.

Ash Melwani (01:15:36):

Yeah, recently

Rabah Rahil (01:15:37):

I feel a little jelly. Good. Isn't it putting some numbers down, stepping up.

Ash Melwani (01:15:42):

We're keep ride the momentum.

Rabah Rahil (01:15:44):

Amazing. You're really, uh, settling into your guru status here. Right?

Ash Melwani (01:15:49):

GU GU showing what's working for us and amazing buddy.

Dylan Ander (01:15:53):

I wish I could be a guru like Ash

Rabah Rahil (01:15:57):

Capital.

Ash Melwani (01:15:57):

We gotta wrap this up.

Rabah Rahil (01:15:58):

Amazing. All right, folks, we're headed to the, uh, New York whale road show. We are gonna be in Toronto, London and LA. Uh, it's gonna be awesome sauce. We have a bunch of cool stuff coming out. So if you wanna follow more [email protected]. If you wanna get more involved, we're actually fingers crossed going to triple whale.com and then the apps moving to app. I know that's good. Remove the try. Love it's cleaner that way. Thank God. A little social network reference. So, uh, fingers cross. We're gonna be pushing that out. So we're really excited there. And then what else we got? Yeah. If you wanna support my obnoxious sneaker habit, I on the LA, what was that? Kenny, Kenny was telling me that Shaq actually donates all of his proceeds to like animal shelters. And I'm over here buying egregious actual sneakers. So if you wanna support my obnoxious sneaker habit, go book a mentor pass. Um, they're really fun. I love it. Kenny's built a really cool

Ash Melwani (01:16:49):

Platform's

Rabah Rahil (01:16:50):

A lot fun. Been a really big blast. Uh, wait,

Dylan Ander (01:16:51):

Show him your shoes, like lift your leg.

Rabah Rahil (01:16:53):

I don't know if you can see these. Yeah,

Dylan Ander (01:16:55):

I'm not, no I'm saying lift your leg, man.

Rabah Rahil (01:16:57):

I'm not as do it, but I'll take we'll we'll

Ash Melwani (01:17:00):

Postproduction

Rabah Rahil (01:17:01):

Postproduction but I, I got some heat on today. It's right. They're in like a top, top two, top three, uh, proper heat, but we're going the fancy Mito of course. And then the fancy dinner afterwards. So, you know, that's it. Yeah. And, and the boss is in town. You gotta look good. You gotta look good. Of course. Um, awesome folks, if you enjoyed this, share it. Subscribe, smash that subscribe button and, uh, Dylan, thank you so much again for not only setting this up. This is our first in person in person last. I love it in real life. Um, awesome folks. That's it. And uh, another one in the books. Thanks Ash. Thanks.

One Tab
To Replace Them All

Supercharge your growth with a purpose-built ecomOS for brands and agencies.

Before ...
Shopify
Ads Manager
Klaviyo
TikTok
Pinterest
Bliss
Triple Whale