This Email Experiment Went Horribly Wrong

54 amazing comments

email experimentHere at Blog Tyrant we place a lot of emphasis on getting email subscribers and building a mailing list that is engaged, happy and active.

We also like to share practical and tested strategies for bloggers to try out themselves, and sometimes that means sharing the things that don’t work so well.

Today we’ve got one such example.

A few weeks ago we changed the lead magnet from an eBook to a free training video and the results were way worse than expected.

Let’s take a look at what went wrong.

What did we change?

As you know, we advocate using a strategy whereby new visitors are encouraged to sign up to the mailing list by the promise of a free eBook.

We were using an eBook called My 5-Step Process to Profitable Blogging for a number of years and it was converting quite well in terms of people visiting the sign and signing up to the mailing list to get it.

As you can see below, sometimes the conversion rate was really excellent. I don’t share this to boast, but rather to help you see the mistake we made next in the hope that you can avoid it on your own sites.

conversion

But, it needed an update and so we decided to create a free training video that went over the whole process of how to find topics and write posts so that they rank well on Google and earn money through affiliates.

The graphic we used to promote this was an animated gif of one of the drawings that we did in the video.

And… it totally flopped.

The first day sign ups dropped by about 25%.

By the second day they had dropped by around 80% and never seemed to recover.

After two or three days of this I knew I had either made a technical error (like the sign up link was broken) or that the new offer just wasn’t hitting the mark.

What happened next?

After doing a big investigation I realized that it was not a technical error but that the new lead magnet was just not performing as well as the old one.

To my mind, there were a few possibilities as to why:

  • The promotion graphic was a gif
    We changed the promotional graphic from an eBook cover to a little animated gif. This might have been problematic as people seem to really like the feeling of a “physical” product and the book cover goes some way towards creating the vibe of something substantial.
  • The perceived value was less
    While the training video was quite useful in terms of the results you could get from watching it, perhaps a 15 minute video seems less valuable than a large eBook that people can keep. Especially as so many excellent videos are produced almost daily by many channels on YouTube.
  • The video target and topic was off
    Lastly, it was possible that the topic we talked about in the eBook was not the right one for the audience that was arriving on Blog Tyrant and looking to subscribe to the mailing list. Usually this is made up of beginners and, while the video was aimed at new bloggers, it might have been pitched a little too advanced.

After looking at all of this I decided to just change it to a new eBook that we were working on that was a 10,000-word guide on how to start a blog and build it so as to maybe craft a new career online.

I was really interested to see whether the simple act of putting an eBook back as the lead magnet would boost subscribers, even if it was different to the previously successful version.

It was.

After a few hours of replacing the training video with the free eBook we saw subscriber rates rocket back up to their previous levels. It was a complete recovery.

You can see the eBook we’re using now at the bottom of each page on Blog Tyrant, and click through to see the copy that the landing page is using to explain the sign up process.

What is the takeaway from this experiment?

One of the main lessons for me was how important it is to test and track changes instead of just making big edits in the hope that they will work.

A great advantage about running an internet business is that you can split test almost everything – there are hardly any equivalents of a big billboard or radio advert where you can’t actual track whether your spend is converting or not.

split test

For example, using a website like Visual Website Optimizer we can run a simple multivariate test. This involves creating two landing pages that are identical except for one element and then seeing which one coverts best.

The thing that always surprises me about these tests is how often the successful version goes against what you would expect to be the winner. Sometimes you will see really common bits of marketing wisdom completely turned on their head on your particular test case.

If you can’t afford a service like VWO you can always do your own mini-tests. It involves a bit more work and is a bit less accurate, but can be achieved with a few WordPress plugins that give you new functionality in conjunction with your mailing list provider.

For example, if you’re using AWeber for your mailing list you can just create two sign up forms for the same list, call them Test-1 and Test-2, and then create two WordPress pages with the same content but the different sign up forms.

You then simply use a promotional plugin like Boxzilla which allows you to create some cool pop ups or slide out boxes and run send traffic to each of the landing pages for a set period of time. You might do three days on one, three days on the next, and then see which gets the better results.

Here’s one I created with this plugin and a bit of Photoshop:

box

Of course this doesn’t give you the deeper data and the fast-paced feedback of a system where you are doing all of these tests simultaneously, but it can give you some good initial insights like which destination page and offer combination is more attractive to visitors.

Final thoughts

Our email subscriber list really is the lifeblood of a successful and sustainable blogging business and, as such, we should all try our best to build one that that is large, engaged and happy. This means doing a lot of testing and continually trying to improve the experience for readers.

Have you ever had any experiments that went really right or really wrong? I’d love to hear about them in the comments below.

SO, WHAT'S NEXT?

Hi, I'm Ramsay. If you enjoyed this post you might like to check out:

Finally, hit the button below to get a free report and email updates so you're never out of touch.

LEARN MORE

54 Comments. Join in. *Closed after 30 days*

  • Renard Moreau

    [ Smiles ] I would say that your email experiment was a success, because you know for a fact what works and what does not work.

    Fabulous blog post!


    1. Ramsay

      Thanks!


  • Adeyemi Adetilewa

    Learnt something new as usual.

    Thank you Ramsey for sharing your results. I love the title of your new lead magnet.

    Is there a way for your old subscribers to have a copy of this book without re-subscribing to your mailing list?


    1. Ramsay

      The same link for the old one works if you still have that email?


  • chris

    Typo in your first sentence. Duuuuude.


    1. Ramsay

      Yucky.


    2. Tyler

      haha, I love this guy.


      1. Ramsay

        Yeah, Chris is a fixture of this place. Says and does what he wants. 😉


  • Kirsten

    Fascinating! I found through SURVEY that my people prefer written vs video content. As a writer, I should be excited, but I now love creating video courses and content. Surveys are super helpful and I love this idea of the split test! I wonder also how something like hot jar would show your page with the book freebie vs the video freebie…


    1. Ramsay

      Man I haven’t done a survey in a long time. Good idea.


  • Mania Mavridou

    Just wanted to say that I HATE gifs!
    I find them irritating, distracting and cheap.
    I think it’s one of the silliest online inventions!

    Good experiment, however!


    1. Ramsay

      This one was just a screen capture of the actual video. Nothing too crass.


      1. Mania Mavridou

        Ok, to be honest, I haven’t seen yours.
        I talked generally about gifs.


  • Mike

    How interesting Ramsay, with all the talk about Video being the current medium of choice-and eBooks being ‘Dead’.
    Really goes to show you that some things are perennial, eBooks are still very popular.
    Email is still the dominant means of earning an income, after all this time.


    1. Ramsay

      Yep, it’s very interesting.


  • chris

    I used to offer a long-form ebook and I thought it would be a great offering. A few YEARS later, I changed it to a simple two-page checklist and it converts several times better. And to that, I’ll add that it helps to offer different lead magnets across a web site to capture people interested in topics within a niche


    1. Tyler

      @chris, would love to check that out. You’re clearly ahead of the pack. Thanks for the comments dude. Always learning from you 😉


    2. Ramsay

      Amazing. Love the simple outcomes like that.


  • Georgia

    Signed up to receive the guide but nothing has come through on my email to confirm. Does it take a while?


    1. Ramsay

      Should be instant. Check your junk folder. If it’s not there I’ll resend.


  • Tyler

    Hello Ramsay! I personally feel you’re crushing it, especially after that huge Facebook ad you just ran.

    I wanted to mention, gifs don’t work for some niches, and work great for other niches.

    Quick example, I recently discovered GIFs rotate into your LinkedIn Profile… Which is where most of my brand/following has been generated due to ‘high amount of service engagements’ — and I wanted to mention that it not only helped me with my click through… It also helped me pull out more clicks beyond the blog.

    Here’s the front end: https://www.linkedin.com/in/tableau-consulting/
    Here’s the content: https://www.linkedin.com/pulse/dev3lopcom-3-tableau-consulting-globally-quality-quantity-garrett/ (I like writing spicey content that drags people in – because my niche is always billing clients $200-300 per hour, and getting them in these blogs is always complicated).

    So, I went with the jurassic park ‘hold on to your butts’ gif. Because he was a computer dude, and it’s him. Audience + gif usage = success or it sounds like, in this case, failure.

    I’d love to see the data visualized – Google data studio embeds now, when will you transition to proving the values VS saying it? Just curious, I’ve been strategizing and whiteboarding ideas around explaining benefits of ‘work’ with data, to tell the story.

    I’m a 1 person army, ranking by massive businesses who have cheated to rank for nearly a decade. Which means I’m doing 1% of their efforts, thanks do badass help coming from you mate.

    Dude, I love you work, please keep it up. Never consider my comments backward, or negative to what you do. You’re a champion.

    My wife was just in my office, and I said out loud ‘see this guy is talking about me again, he’s just that good’ — you really have a way of touching something I’ve just done, or interested in learning.

    My family to yours, thanks for helping me.
    Keep crushing dude. Your writing style teaches me daily.

    Until next time, #tldr comments ftw.
    Cheers,
    Tyler


    1. Ramsay

      That’s very interesting. I never use LinkedIn because I can’t handle the format, but I know a lot of people who get a lot of leads from it.

      Thanks so much for the kind words. I really, really appreciate that.


      1. Tyler

        Don’t mention it, Ramsay. I agree with you (the usual).

        I don’t enjoy the format, nor do I appreciate the founders using it to tell people their opinions on politics.

        “Stick to the tech” – I always recommend. Or you look like a fool.


  • Shafi Khan

    Haha interesting case study Ramsey. Sometimes what we expect to be successful turn out to be total disappointment.

    I would agree that an ebook converts better because one can keep it on their device and go through it easily as and when required. Also, one can highlight important parts and create a summarize version.

    Thanks for sharing your case study.

    Regards,

    Shafi Khan


    1. Ramsay

      Thank you, Shafi.


  • Darius Gaynor

    Great case study! This is why split testing is very important. Don’t just change things and think it is better.


    1. Ramsay

      Thanks Darius. Nice to see you here again.


  • Robin Khokhar

    Hi Ramsay,
    I would say that Experiments can help us to get more traffic. If we are not going to do anything, then there will be no results.
    So, It was a good effort.
    Thanks for sharing.


    1. Ramsay

      Good point!


  • Maximus

    Thanks Ramsay. This is of course the beauty of running an online business, you can actually experiment. Something CocaCola tried with their formula and almost went under.

    Thanks for sharing also.


    1. Ramsay

      Oh I haven’t hear about that. I’ll look it up.


  • Benjamin Houy

    I’m glad you quickly identified the source of the problem. I regularly have problems like this and this taught me two things:

    1) Do not make change just for the fun of it. Only make changes if you are almost sure it will have a positive impact

    2) Document every single chance. It happens so many times that I changed a “detail”, destroyed my conversions and then didn’t remember what I did exactly.

    Also, Google Optimize is a pretty cool alternative to VWO.


    1. Ramsay

      I need to spend some time playing with GO. Have you had any luck with it?


      1. Benjamin Houy

        Yes, it replaced Optimizely for me. I love that it integrates with Google Analytics and find it very easy to use.

        The only drawback is that modifying styling requires CSS knowledge.


  • Max

    In my view, an ebook feels like more of a “tangible” product, whereas video content is something we only ever consume for free (YouTube, social media, etc). Great blog – definitely given me food for thought as I start preparing my lead generation resources.


    1. Ramsay

      Yep, I think the tangible feeling is key.


  • Vishal Ostwal

    Hi Ramsay,

    Seems like the type of context and content are both sensitively related.

    The concepts like A/B testing didn’t look important to me, and I used to be sort of ignorant to them. To my surprise, even the tiny changes did matter.

    For instance, I simply used a SumoMe welcome mat on my blog … and some people subscribed. I offered an eBook – one or two more subscribers (I’m at a rockbottom stage).

    Lesson learned – “Experiment. Don’t keep the blog stagnant.”

    PS. I never took the topic of starting an email newsletter seriously.

    Though, I should’ve.

    Doing so many things at once sort of overwhelms me. But now I’ve been changing one thing at a time. Perhaps, I take more time to learn compared to others.

    I still get those ‘a-ha’ moments regarding some simple blogging stuff.


    1. Ramsay

      How did the welcome mat convert for you?


      1. Vishal Ostwal

        It was good.

        In fact, it was the first time tried collecting emails, as an experiment.

        But I found it inconvenient, so I replace it with a simple slider that asked for emails and gave my eBook.

        PS. I gave all that I created for free. Never asked for emails (Didn’t want to bug people). I might want to change that. Should’ve learned this lesson earlier.


  • Rick Rouse

    I think every blogger on the planet who mails to a list has had similar experiences, Ramsay. Thanks for sharing your “lessons learned” with us. Your blog is a goldmine of great info!


    1. Ramsay

      Glad it’s helpful!


  • Steve

    It’s amazing how tired, irrelevant and boring so many lead magnets and optin offerings are on so many blogs and I often wonder if they actually check their stats or split test them at all.

    This is a really basic example of what most don’t do Ramsay and you’ve highlighted it in its simplest form what can change with just minor tweeks. Kaizan!

    Also worth noting that although you found a serious downturn in optins by changing your offer, this doesn’t necessarily mean that the gif doesn’t work! It appealed to some. Still money on the table?

    Email lists are also there to be segmented to catch people at different levels, different stages, different requirements, preferred learning experiences etc.

    So the gif may work for some people somewhere else along the funnel. Perhaps worth another round of testing but at a different level/entry point?


    1. Ramsay

      Really great point! I should explore that some more. Than you for the helpful comment, Steve.


  • Cindy Seaton

    Well what do you know, your predictions were right! I saw your blog in my inbox yesterday and didn’t open it, thinking I would open it later when I had time ( I love your posts by the way!) This morning I woke early and decided to do some research on what was the best time to release a blog post. I typed it in my search engine, and low and behold, your blog came up first. I’m so jealous 🙂 I’ve been releasing blogs posts on Mondays at 6:30 pm, and Thursdays at 7am. Maybe it’s time for a change. Thanks for the tips.


  • Sgarbi

    Learning with our own mistakes is pretty painful. But it usually works pretty well. It takes courage to expose ourselves. Chapeau!


  • Santo De Rose

    It looks like you accidentally hit a knowledge jackpot.
    App developing companies have found that no matter how many simulations they run with an app, their best results came from “just putting it out there”. They found that releasing the app in a “flawed” state resulted in honest results from users which then enabled them to address those problems more accurately.
    So I say…good accident tyrant!


    1. Ramsay

      ha ha!


  • Ahmad Imran

    Ramsay, hope you are well. Just a quick couple of lines to congratulate you on a brilliant post. It just shows that if the message is clear, we don’t need to fill our posts with 3000 words every time.

    Short, to-the-point and full of useful information. Keep this going and these short lessons and mistakes/successes are really worth knowing, sharing and implementing.

    Have a nice weekend ahead.

    Ahmad


    1. Ramsay

      Thanks mate. Appreciate that coming from you.


  • Rohan Singh

    Getting 48% conversion rate, really impressive. Last time, I learnt from your facebook experiment and this time from email experiment. Good work. Thanks, Ramsay.


    1. Ramsay

      It’s all luck. Glad it helped.


  • Susan Velez

    Hi Ramsay,

    I think every blogger should focus on growing their email subscribers. Since I’ve started my blog, that’s been my focus. Although it’s been a slow growing process, I finally reached my first 100 subscribers.

    To tell you the truth, I’ve never tried using a free ebook giveaway to see if I can improve the conversions.

    I am using a free 7-day course. I also have a free WordPress resource library.

    My conversion rates are nowhere near 48%. I definitely need to see if I can work on improving my conversion rates.

    Maybe it’s time that I sit down and create a free ebook to give away as a freebie.

    I personally know that I would prefer a video to an ebook. But that doesn’t necessarily mean that everyone else would.

    Thanks for sharing what you found out. Looks like I’ve got some work to do to today to see if I can improve my opt-in rates.

    Have a great day 🙂

    Susan


    1. Ramsay

      Let us know how you go!


  • Freddy G. Cabrera

    Hey Ramsay!

    This is a very interesting case study. I have been testing different lead magnets at my blog, as well. You have saved a bit of time here, but I’m still going to test things because different audiences behave differently.

    I think digital books (ebooks) are still the number one lead magnet for internet marketers. It must be because it is easy to download and read through.

    You can save the pdf file anywhere and access it almost on any smart device. You can’t really do that with videos, right!

    Thank you for sharing these helpful insights!

    Cheers! 😀


Leave a Reply

Your email address will not be published. Required fields are marked *

Scroll Up
Tweet
Share