auston

How effective is our App Download plugin? Up to 84% more effective

How do we prove it’s effective?


I believe the ultimate testament to the quality of your product is results. At 1Sale, we saw results when we put the idea behind our App Download plugin to work. Literally thousands more downloads. But that was a little bit different than UserPath’s download plugin, so in order to prove UserPath is effective, we ate our own dog food. However, we don’t have any of our own apps. So, we enlisted the help of a friend. This blog post goes over the results of our first A/B test, using Teepy – a Curated T-Shirt Shop for Amazon. It’s a way to collect, buy, and gift tees for any style or occasion.

We’ll be posting the results of more A/B tests for both App Download and Click-to-call as the results reach statistical significance, because what good are the results of an experiment if they are not reproducible?

The Experiment


The question for our experiment is pretty straight forward – Are people who visit a site for a mobile app on the desktop empowered to download the app? After a bit of looking around, it seems that most of the time, the answer is “No”. However, our previous experience and some anecdata lead us to a hypothesis:

A form that allows someone to send themselves a link to an app from the desktop will increase download intent.

So what exactly do we consider “download intent”? For the purposes of our experiment:

  1. If there is not a way to text a download link to yourself – then the click of an app store button is considered download intent.
  2. If there is a way to text a download link to yourself – then completion of a form that texts you a download link is considered download intent on desktop.

Now, let’s look at our A/B test. First, let’s examine our control, which we named “No UserPath”. It’s just the Teepy.co website, without the App Download plugin installed.

Teepy Site Before the UserPath App Download plugin was installed
Teepy Site Before the UserPath App Download plugin was installed

Next, let’s see what our independent variable looks like, we named it “With UserPath”. It’s a box that slides in from the right on a 1 second timer with a some text that tells the user they can download the Teepy app by filling out the form.

Teepy-with-userpath
Teepy site with the App Download plugin installed. Yielded 84.4% increase in Download Intent.

We ended up running the experiment for 2 weeks and 1 day. We waited that long because we wanted statistically significant results as determined by Optimizely, the A/B testing platform we used.

The Results


We believe the results speak for themselves. Teepy was able to achieve an 84.4% increase in download intent with UserPath’s App Download plugin. Additionally, engagement increased 28.4% with UserPath installed.

Optimizely Screenshot Shows 85% gains with a UserPath plugin.
Optimizely Screenshot Shows ~84% gains with the UserPath App Download plugin installed.

At the bottom of the funnel, they actually increased downloads 33% when compared to the 15 day time period before we installed UserPath. We asked the co-founder of Teepy, Alex Lopez why Teepy downloads increased so dramatically and this is what he told us: “It was definitely UserPath because I did not do anything else.”

With the App Download plugin installed, Teepy saw a 33% increase in downloads over the previous time period of the same length.
With the UserPath installed, Teepy saw a 33% increase in downloads compared to the previous 2 weeks.

I will be running more experiments in the upcoming weeks & publishing the results here, whether they bode well for UserPath or not. I believe that being honest will help us improve our product & give customers a clear look as to whether or not UserPath would be useful given their scenario.

Did you find this post valuable?

Sign up to get our high-quality content in your inbox – focused on improving customer acquisition, activation, retention, referrals & revenue.

* indicates required


UserPath

Ready to try UserPath for yourself?

Sign Up For Free!
auston

Auston

Co-founder at UserPath. Often blogging about the stuff we're doing to make your apps better and users satisfied. Sometimes writing about experiments or research. Learner. Thinker. Programmer.