eBay Promoted Listings have a Significant Effect on Sales – Experimental Results
Yesterday I described how we designed a comprehensive trial to test the effectiveness of the eBay Promoted Listings feature, and today I’m sharing the initial results.
The trial started with a 13-week period with no Promoted Listings campaigns to test our baseline sell-through rate and how it varied. This included all our Buy-It-Now/Best Offer listings – just over 15,000 listings each week across a wide range of eBay categories, including fashion, electronics, collectables and vehicle parts & accessories.
The sell-through rate is defined as the number of items where we agreed a sale on eBay, divided by the total number of listings live that week. We then calculated the percentage uplift or downlift each week from the average sell-through rate for the period. There was one clear outlier – the sales in week 10 – the August Bank Holiday weekend – which we discarded when calculating the average sell-through rate and the standard deviations.
The green and red lines represent the upper and lower sell-through rates which can be explained by random variation (for the statistically-minded, they are at two standard deviations). If we get results above the green line, then we can say that we’ve had a statistically significant impact: ie the “treatment” works.
Just to make sure that cancellations and unpaid items didn’t make a difference, we re-ran this analysis to compared orders which were actually dispatched and it did not show any significant difference.
We then compared the weeks where we ran a promoted listing campaign. If we put the promotion on just half of the inventory (evens or odds) then we compared the sell-through rate to the undosed half to remove any time/weather effects. If we dosed the entire inventory then we compared with the control period long-run sell-through rate.
There is a clear and statistically significant increase in sell-through rate when promoting listings in eight out of the nine weeks. The effect is large, increasing the sell-through rate by an average of 27%.
Finally, we looked at how the Promoted Listing percentage affected the sell-through. Would paying more get us an even better result?
The results broadly suggest that the more you pay the more you get, but the relationship is not as strong as the previous findings. (For the statistically minded, R^2 correlation is 43%). We suspect that this will vary by category – some categories will see much more competition for the top spots and require higher bidding – while other categories are easier so even a low bid will result in an uplift in sales. Stuff U Sell sells across a wide range of categories so our results are spread out.
Tomorrow we look at what these results might mean for your business this year.
Disclosure: As mentioned in the first article, eBay refunded the promoted listings fees to allow this study to be conducted, but had no say in the design or implementation of this research, nor editorial control over this write-up.