Classic scales

Measuring performance with A/B Testing

Last month I decided to take some of my own advice and understand how to improve my blog with some focused study and A/B testing (Science!). The objective was to understand how many of the Marketing Resource downloads were genuine, and whether the users would be prepared to share their details or promote the site in exchange for the download.

I decided to begin by measuring the performance of my most trafficked page (Marketing Resources), and then trying a few variations to see what worked best. To get started, I stopped publishing. I figured that by not publishing anything meant I’d get a clear view of traffic & behavior without any wobbles from posts I make.

Then I upgraded the download software being used for my Marketing Resources section, allowing me to track individual downloads by date and to force subscriptions or tweets as required. I also installed an optional social pop-up system (this does have a force share option too).

For the first two weeks I setup the Marketing Resources page so that every download required visitors to either send a supporting tweet, or enter their email addresses. For the remaining two weeks, the page prompted visitors for an optional social share.

The results were shocking! By forcing users to share their details or make a social post, the page generated fewer downloads, but had considerably more traffic, shares and user registrations. Optional shares results in absolutely no change from normal, meaning less traffic but twice as many downloads. Here’s the summary data:

Mandatory Subscription or Sharing Optional Sharing
Tweets +8 +0
Subscriptions +60 +0
Downloads -50% 0% change
Traffic 20% increase 0% change

Depending on what’s important to your site, you can interpret this accordingly. More consumption of your material, but with no idea who’s consuming it & no obvious promotion from it. Or less consumption, but with a better idea of who’s reading & some extra social media mileage. For me there’s no question that I’d prefer the latter.

I’m going to be doing some more experimentation with how my Marketing Resources page prompts for shares, and I’ll update you next month with my findings. Hope this was helpful.