• The Algorithm That Saved Reddit 21% on BigQuery Slots Written by Michael Petro BigQuery serves as the central compute engine of Reddit’s data platform. • It powers ingestion, batch ETL, feature engineering, experimentation, analytics, and so much more. • While BigQuery is performant and extremely scalable, these qualities make it easy to spend enormous amounts on compute without the right guardrails. • In this blog post, we’ll walk through how we flattened Reddit’s BigQuery slot cost growth, and reduced our average slot hour cost by 21%. • Background Cloud infrastructure billing models typically fall into one of the two pricing paradigms: consumption pricing or capacity pricing. • In a consumption pricing model, you pay for resource consumption regardless of traffic shape; infrastructure scales to demand and you only pay for usage.

Article Summaries:

  • The Algorithm That Saved Reddit 21% on BigQuery Slots Written by Michael Petro BigQuery serves as the central compute engine of Reddit’s data platform. It powers ingestion, batch ETL, feature engineering, experimentation, analytics, and so much more. While BigQuery is performant and extremely scalable, these qualities make it easy to spend enormous amounts on compute without the right guardrails. In this blog post, we’ll walk through how we flattened Reddit’s BigQuery slot cost growth, and reduced our average slot hour cost by 21%. Background Cloud infrastructure billing models typically fall

Sources: