Cloud Masters Episode #116
BigQuery Editions & strategies for transitioning from expiring BigQuery flat-rate commitments
How to transition from flat-rate to BigQuery Editions and on-demand pricing without overspending.
Cloud Masters Episode #116

With DoiT Spot Scaling, automate your AWS Spot Instances to save up to 90% on compute spend without compromising reliability.

Cloud Masters
Cloud Masters
BigQuery Editions & strategies for transitioning from expiring BigQuery flat-rate commitments
Loading
/
Cloud Masters
Cloud Masters
BigQuery Editions & strategies for transitioning from expiring BigQuery flat-rate commitments
Loading
/

Episode notes

Key Moments

00:00: Introduction
00:26: How we got here
01:40: What happens if you do nothing?
02:55: Do jobs perform differently on BigQuery Editions?
03:49: Configuring baseline / max slots post-transition
06:23: When to use on-demand vs. Editions
07:35: How to treat spiky workloads
09:00: Handling compute-intensive, slot-light jobs
10:44: BigQuery’s minimum billing period
12:38: Mixing and matching Editions
15:20: Splitting your BigQuery workloads
17:15: Right pricing model for dev/test workloads?
18:18: Right pricing model for Spiky ETL workloads?
20:04: Slot wastage and how to avoid it
22:14: Scripts for analyzing slot utilization
25:57: Best practices for keeping BigQuery optimized
31:37: Moving jobs around to reduce slots needed
33:55: Should you commit to Editions? When?
40:28: Best Practices for dbt, Dataform, BI tools

Useful BigQuery scripts:

About the guests

Sayle Matthews
Sayle Matthews is a Google Cloud Data Practice Lead at DoiT International focusing on data solutions. Sayle has a solid track record of designing and implementing efficient solutions to complex problems. Among a myriad of diverse technical knowledge, Sayle is currently acting as one of the leaders in advising customers on optimizing BigQuery for the recently launched Editions pricing models.
Matthew Richardson
Matthew Richardson is a Google Cloud Data Practice Lead at DoiT International specializing in the Data & Analytics space, with certifications across all major cloud providers — Google Cloud, AWS, Azure — as well as Snowflake, DBT, Python, Tableau and SAS. He has deep experience in the use of a variety of Data Engineering, ETL, BI reporting & programming tool sets including the above products plus Matillion, Talend, Cognos, Teradata and Business Objects in addition. Matthew currently works with customers in optimizing their Data Modelling strategies particularly in BigQuery, Redshift or Snowflake, ensuring customers are getting the most out of their data on a consistent basis.
Sayle Matthews is a Google Cloud Data Practice Lead at DoiT International focusing on data solutions. Sayle has a solid track record of designing and implementing efficient solutions to complex problems. Among a myriad of diverse technical knowledge, Sayle is currently acting as one of the leaders in advising customers on optimizing BigQuery for the recently launched Editions pricing models.
Matthew Richardson is a Google Cloud Data Practice Lead at DoiT International specializing in the Data & Analytics space, with certifications across all major cloud providers — Google Cloud, AWS, Azure — as well as Snowflake, DBT, Python, Tableau and SAS. He has deep experience in the use of a variety of Data Engineering, ETL, BI reporting & programming tool sets including the above products plus Matillion, Talend, Cognos, Teradata and Business Objects in addition. Matthew currently works with customers in optimizing their Data Modelling strategies particularly in BigQuery, Redshift or Snowflake, ensuring customers are getting the most out of their data on a consistent basis.

Related content

BigQuery Editions and What You Need to Know
BigQuery Editions and What You Need to Know
Important things to know about BigQuery Editions, its autoscaler, and pitfalls to avoid.
Read more
Automate BigQuery reservations and assignments using Dataform
How to use Dataform to automate the process of adjusting BigQuery’s pricing plans and reservations based on specific periods, enabling you to optimize performance and cost.
Read more
BigQuery: Migration to Standard Edition from On-Demand in 5 Steps
A step-by-step guide for determining whether it’s cost-beneficial to switch from on-demand to Editions.
Read more
Let’s do it

From cost optimization to cloud migration, machine learning and CloudOps,
we’re here to make the public cloud easy.

From cost optimization to cloud migration, machine learning and CloudOps, we’re here to make the public cloud easy.

Ready to get started?

Get in touch

Company

Offering

Support

Never miss an update.

Subscribe to updates, news and more.

Schedule a call with our team

You will receive a calendar invite to the email address provided below for a 15-minute call with one of our team members to discuss your needs.

You will be presented with date and time options on the next step