menu
April 13, 2017

Self-learning search improves revenue for e-commerce

We are happy to announce comparative results from using 904Labs self-learning search and a highly, but manually, tuned Apache Solr search engine on an e-commerce site. Over a period of one month, and with almost 55,000 unique visitors, 904Labs self-learning search engine helped increase revenue by 36% and purchases by 31%. This is an impressive achievement, and solid evidence that self-learning search can benefit revenue and user engagement for online retailers.

To make this comparison possible, we have partnered with Sooqr Search, a leading search technology company providing instant, relevant, and customizable site search for over 300 customers globally, and one of Sooqr Search's customers, Autostyle.nl, a leading Dutch online retailer for car accessories. Autostyle.nl is the prime go-to destination for individuals and car service businesses around the Netherlands who want to tune their cars. Autostyle.nl has presence in the Netherlands, Belgium, Germany, and the United Kingdom. Their Dutch website, on which we tested our system, attracts millions of pageviews and hundreds of thousands of searches per month.

Experimental setup

For our comparison, we performed an A/B test using Optimizely to ensure unbiased results. The traffic allocation was set to 50% for Sooqr Search and 50% for Sooqr + 904Labs self-learning search, and we aimed at a flight period of one month to account for seasonal variation in user behavior. During the test, we are tracking the following goals, with total revenue set to be the primary goal of the test:

  1. Total revenue (Primary goal)
  2. Total and unique number of purchases
  3. Total and unique number of add-to-basket actions
  4. Total and unique number of clicks on search results
  5. Total and unique number of newsletter subscriptions
  6. Total and unique number of clicks anywhere on the page (also referred to as "engagement")

IP addresses from the Autostyle backoffice and most of Autostyle's dealers, who use the site for customer service, are excluded from the test. Before starting the A/B test, the 904Labs self-learning search engine was trained on about 6,000 searches with clicks, extracted from a query log that was provided by Sooqr Search. During the A/B test, the 904Labs self-learning search was using clicks on search results to continuously learn from user behavior.

Results

The results of the A/B test are staggering:

  1. 36% improvement on total revenue
  2. 31% improvement on total purchases
  3. 7.5% improvement on total clicks on search results
Results of A/B test on Autostyle comparing 904Labs to Sooqr

What the results show is that having a manually optimized search engine can be of benefit, but having 904Labs' intelligent search layer on top of it can  boost user engagement and revenue even more. What we see here is a great collaboration of humans and machine. People have used Sooqr Search's search tools to optimize boosts for their Solr and add synonyms to the system, which were also used by 904Labs self-learning search. On top of that, 904Labs self-learning automatically extracts additional keywords, and optimizes search results for all queries by automatically deciding the best boosts from user behavior.

If we look closer at the results, and especially at the user segmentation between mobile and non-mobile users, we find that 904Labs self-learning search does even better for mobile. The results for mobile users are:

  1. 80% improvement on total revenue
  2. 70% improvement on total purchases
  3. 7.5% improvement on total clicks
Results of A/B test on Autostyle (mobile traffic) comparing 904Labs to Sooqr

With the number of mobile users increasing fast, and with the constraints of small screens on mobile devices with little room for search results, it is becoming more and more important to offer the best results at the top, saving users from multiple interactions with the search page. 904Labs self-learning search is ready to serve mobile customers and substantially increase their user experience.

What's next?

904Labs self-learning search dramatically improves user engagement and revenue for e-commerce sites. It is not only mobile ready, but it shows to benefit user engagement even more on mobile devices. 904Labs self-learning search adds an intelligent layer to the current search infrastructure and takes into account business rules, synonyms, or customized boosting (if present) and extends them to optimize search results across all queries. The results of a month of A/B testing provide solid proof for this claim: good search is important and it can add a lot to user engagement and revenue for online retailers.

Get in touch to learn how we can help your online business with better search.