• Expedia Group Technology - Data Quickly identifying winning ranking models before committing to A/B tests Authors: Adam Woznica, Benjamin Stieger, and Stefania Ebli Photo by Il Vagabiondo on Unsplash Expedia Group ™ covers a portfolio of brands such as Expedia.com, Hotels.com, and Vrbo, that power lodging searches for millions of travel shoppers every day. • In this competitive market matching users to hotel inventory is crucial , as users tend to quickly switch from website to website. • As a result, having the best lodging recommendation and ranking gives the best chance of winning a sale. • The lodging ranking system at Expedia Group consists of three main components: Selection [1] Relevance ranking [2] Re-ranking with business adjustments [3] These three steps are optimized to maximize conversion rate (CVR) and several business metrics . • Typically, all changes in the ranking stack - whether a new ranking algorithm, a new business adjustment, new hyperparameters, or an infrastructure improvement - are A/B tested to assess the impact on the business metrics. • The limitations of classic A/B tests A key limitation of A/B testing is its requirement for large sample sizes to maintain control over significance levels, test power, and the minimum relative uplift of the business metric.

Article Summaries:

  • Expedia Group Technology - Data Quickly identifying winning ranking models before committing to A/B tests Authors: Adam Woznica, Benjamin Stieger, and Stefania Ebli Photo by Il Vagabiondo on Unsplash Expedia Group ™ covers a portfolio of brands such as Expedia.com, Hotels.com, and Vrbo, that power lodging searches for millions of travel shoppers every day. In this competitive market matching users to hotel inventory is crucial , as users tend to quickly switch from website to website. As a result, having the best lodging recommendation and ranking gives the best chance of winning a sale. The l

Sources: