Easy Split Testing For Small Business Owners

I was in Florida just about to indulge in my favorite food of all, the delectable stone crab claw. The ocean waters lapped up against the edge of the pier where I sat, and pelicans were busy soaring and swooping a few feet away.

Suddenly I noticed that the upright poles of the pier, which had pencil-pointed ends sticking up into the sun, were all tipped in white paint on the left side of the dock. But on the right side of the dock they were all painted black. What kind of secret signal to the boaters arriving by water for their stone crab is that, I wondered? Maybe sail boats on one side, motorized craft on the other? Maybe personal craft on one side and commercial fishing boats on the other?  I’m not a boater, so I couldn’t read the secret pier-pole language. I watched the pelicans strut up and down the docks and dive for fish while I thought about it and waited for my stone crab to land on my table.

Split testing is like those white versus black-tipped dock poles. Let’s say you write a sales letter inviting your customers to purchase a product from you. The elements of the sales letter include the title, the sales copy, the price point of the offer, and a picture of the product sitting on a shelf. But you wonder if the sales would be higher if you changed the sales letter around a little bit, so you develop a second version of it. In version B you keep all the elements the same except the picture – you change the picture to show someone actually using the product instead of it sitting on the shelf. You sit and wonder which version to use – A or B. Suddenly, you decide to divide the list of customers you’re going to send the sales letter to in half, and you send version A to half and version B to the other half. And you track the numbers – which one has better open rates and which one results in more sales? You’ve just done split testing.

Split testing is when you use two versions of an offer, split your customer list, and see which version does best for you. This is also sometimes called A/B testing. Version A of the offer versus version B. There’s a rule of thumb in split testing that you only change ONE element of the sales letter at a time. Why? Because, if version B does better than version A you want to know why. And if you change more than one element you don’t know why. Let’s say you changed the picture AND the title. Oh no! Was it the picture that caused the better response, or the different title? If you made the mistake of changing more than one element in version A versus version B, you have no way of knowing.

Lately, there is a new philosophy that is starting to crop up in marketing that says “let’s have two completely different versions and change several elements at a time. Who cares what the element was that performed better anyway? Whichever sales letter does better, why do we care why? We just drop the underperformer and go with the top performer.” This topic came up at a conference I was attending a few weeks ago – who cares which element made the difference? Strictly speaking, from a corporate marketer’s perspective, you are trying to “prove” and quantify what you do to put numbers on it – because corporations look all the time at their marketing costs and seek to contain them and scientifically get the most bang for their buck. From a small business owner’s standpoint, though, you do sales letters all the time, and you may just want to try two totally different ways of putting an offer out there. You are curious which does better for you but maybe you don’t care so much about exactly why.

But just to be clear, pure A/B testing (or split testing) means that you change only one element in your two different versions so that you can definitely say to yourself, “Ha! That picture that showed the person actually using the product sold twice as many as the one that showed the product sitting on the shelf!” You know exactly what made the difference so that you can replicate it and do it again with success.

So, you may be asking, what does all that have to do with my favorite stone crab claws and the black versus white paint on the tips of the pier poles? Well, you know I’m a curious person. I just about couldn’t stand not knowing what those black versus white tipped poles were secretly signalling to boaters. What if I arrived by boat next time I lusted for those delectable stone crab claws? So when my platter of claws arrived I sheepishly asked the server what those poles meant to the boaters. She laughed and said, “It’s not secret boating language. We were testing out if black tips on the poles would keep the pelicans from perching there any better than white paint on the poles. But it doesn’t matter. Pelicans are going to perch on pier poles no matter if the tips are black or white. Pelicans do what pelicans do – they perch on poles.”

There you have it – the stone crab place was doing a split test. They had version A and version B of pier poles – and that paint color was their only difference. All the poles where the same height. All the poles had pointed ends. The only difference was the paint on the pointed end. The pier poles were a perfect example of what you can do in your marketing. Split testing. A/B testing. To pelicans it doesn’t matter at all. But to your customers, it just might.