• Ever searched for something specific, only to be met with results that are close, but not quite ? • On Etsy’s Search Relevance team, that frustration is exactly what we are tackling. • Our goal is simple yet ambitious: to help buyers find exactly what they’re looking for, and to help sellers reach the people seeking their special products. • Search plays a central role in that mission. • Historically, Etsy’s search models have relied heavily on engagement signals - such as clicks, add-to-carts, and purchases - as proxies for relevance. • These signals are objective, but they can also be biased: popular listings get more clicks, even when they’re not the best match for a specific query.

Article Summaries:

  • Etsy’s Search Relevance team is deploying large language models (LLMs) to enhance the precision of its marketplace search. The new Semantic Relevance Evaluation and Enhancement Framework blends human‑curated “golden” relevance labels with LLM‑generated judgments across millions of query‑listing pairs. It defines three relevance tiers-Relevant, Partially relevant, and Irrelevant-to capture nuanced intent. The framework includes a family of ML models tuned for accuracy, latency, and cost, and integrates relevance signals into both offline evaluation and real‑time search. By adding semantic relevance to traditional engagement metrics, Etsy aims to deliver more intent‑aware results for buyers while helping sellers reach the right audience.
  • Etsy’s Search Relevance team has launched a Semantic Relevance Evaluation and Enhancement Framework that uses large language models (LLMs) to improve how listings match buyer intent. The system builds on human‑curated “golden” relevance labels and scales judgment with an LLM that evaluates millions of query‑listing pairs. Three relevance categories-Relevant, Partially Relevant, and Irrelevant-guide the model, which is tuned for accuracy, latency, and cost. The framework integrates semantic relevance signals into Etsy’s search pipeline, enabling both large‑scale offline evaluation and real‑time enhancement of search results to better serve buyers and sellers.
  • Etsy’s Search Relevance team is deploying large language models (LLMs) to add a semantic layer to its search ranking. The new framework combines human‑curated “golden” relevance labels with LLM‑generated judgments to scale evaluation across millions of query‑listing pairs. Three relevance categories-Relevant, Partially Relevant, and Irrelevant-guide model training. Etsy builds a family of semantic relevance models that balance accuracy, latency, and cost, and integrates them into both offline testing and real‑time search. By supplementing traditional engagement signals with intent‑aware semantic scores, Etsy aims to deliver more precise results for buyers while helping sellers reach the right audience.
  • Etsy’s Search Relevance team is deploying large language models (LLMs) to add a semantic layer to its existing engagement‑based search ranking. The new framework defines three relevance categories-Relevant, Partially relevant, and Irrelevant-based on user research. Human‑curated “golden” labels are combined with LLM‑generated judgments to scale training across millions of query‑listing pairs. Multiple semantic relevance models are tuned for accuracy, latency, and cost, and their signals are integrated into Etsy’s search pipeline for both offline evaluation and real‑time ranking. The goal is to deliver more intent‑aware results, improving buyer satisfaction while helping sellers reach the right audience.
  • Etsy’s Search Relevance team is deploying large language models (LLMs) to add a semantic layer to its search engine, complementing traditional engagement‑based signals such as clicks and purchases. The new “Semantic Relevance Evaluation and Enhancement Framework” combines human‑curated “golden” relevance labels with LLM‑generated judgments across millions of query‑listing pairs. It defines three relevance categories-Relevant, Partially Relevant, and Irrelevant-and trains models that balance accuracy, latency, and cost for both offline evaluation and real‑time search. By integrating these semantic relevance signals into production, Etsy aims to deliver more intent‑aware results that better serve buyers and help sellers reach the right audience.

Sources: