Introduction

Once again, I decided to experiment with business solutions to increase profits and better understand my customers. Sometimes the client is motivated not by what you bet on and your actions arise the reaction opposite to what you think they do.

                                                                     Travis Smith

I found a way to learn about their preferences at no particular cost with an A/B test and I’ll talk about how I studied the influence of a Quantity Discount widget design on the store’s conversion. Well, let’s go!

Content:

A/B Testing as a Way to Increase Profits

Why do we need A/B tests? Think of a situation where the project is launched and gains traffic while users actively use the resource. And one day you decide to change something, for example, a CTA phrase or add a tiered pricing widget.

How will this affect sales? At first glance, the buyer comes to the store for the goods, that is, the widget’s design and font descriptions should be of little importance to customers. However, in practice, this is not the case. The visual appeal of your resource greatly affects sales, sometimes in a truly incomprehensible way. A/B testing was invented in order to experimentally determine the influence of these factors and provide a better design.

A/B Design Testing

I stumbled across the Internet to the description of this Quantity Discount application. The decision came suddenly. I have long wanted to try to sell goods in bulk, and at the same time decided to check out the ready-made design templates provided with the app. Frankly, the results puzzled me. Look at these samples to see why.

At the first glance, I found this design the most attractive and applied it first. By the way, I already had written the main indicators of success for the past month and wanted to compare them with those obtained during the test. And that’s what happened: with 40 orders, 6 people made a purchase of 3 units of goods at a time. The store’s conversion was 15%, which is 3% higher than in the period before the test. I thought, “Not bad,” but did not stop the experiment, as there is no limit to perfection.

Next, I decided to apply this design. Initially, I expected to see the conversion rate decrease, because this pink background seemed unpleasantly glaring and bright to me. But, as I already said, there will be surprises. Truly magically, the conversion rose to 19%, while out of 56 orders 12 customers took advantage of the offer to buy several products at once. For me, it was an unexpected success, as I was sure I would see a decline in activity, and would have been able to analyze how many conversions the widget brings with a knowingly nasty design.

Moreover, this application affects not only the conversion but also the average order value. In this case, the average order value rose by 11%. This splash was not caused by seasonal fluctuations, as I tested other designs too, then returned this one and the situation repeated. Wow, nothing else to say.
What other designs did I test:

White text on a blue background caused a slight conversion change up to 14% with the an increase of 8% in average order value.

Another green widget, now with more contrast. Conversion rate increased by 16%, the average order value by 10%. I concluded that the green color does not cause a sense of urgency among buyers and began to experiment with bright and red widgets.

The conversion rate remained the same in this case while the average order value increased to 11%. That’s when I thought I was on the right path, and I chose the next design:

This one doesn’t seem like anything special, as it doesn’t even have a red header. But this design finally forced me to stop guessing at the tastes of users and look at empirical indicators only: 21% conversion rate and 15% increase in the average order value. This one became a leader among different designs I tested, so I chose it to research the impact of the call-to-action phrase. Below you can see the parameters that I applied to this widget design.

Message Block
Background: 140,100,0
Border Color: 255,255,255
Border Width: 1px
Border Radius: 3px
Border Style: None
Inner Padding: 18px
Text Color: 255, 255, 255
Text Size: 22px
Text Style: None

Tiered Pricing Text Lines
Background: 80,80,80
Border Color: 255,255,255
Border Width: 1px
Border Style: Dotted
Top Margin: 16px
Top Padding: 16px
Line Height: 26px
Text Color: 255,255,255
Text Size: 15px
Text Style: None

CTA Testing

In this test, I used the most successful design in conjunction with various descriptions of the proposal. For example:

  • Get more discounts
  • Special offer for our customers
  • Buy cheaper

I will say right away that the influence of the text was smaller than that of the design, but not small enough to ignore. The first option was based on previous texts, so I could not calculate its impact on the conversion rate. The “Special offer for our customers” version showed a slight increase in user activity: the conversion approached 22% with an increase in average order value of 18%. The “Buy cheaper” phrase had the same results as “Get more discounts”.

Conclusions

The only lesson I learned from these trials and mistakes is: do not underestimate what you can not understand. Fortunately, there are tools that enable you to empirically determine the best parameters for the design of individual elements. As you can see, these experiments bring tangible results, so in case you’re still not sure whether they are really worthwhile, it’s time to get down to work. You can use my research and results as a guide. Even if you have other metrics, you will get valuable experience during the experiment and will know exactly what your shop lacks to achieve success.