BigQuery cost guardrails that won't break your teams

Start with blast radius, not the invoice

By the time a team is asking who ran the expensive query, the useful part of the conversation is usually already over. BigQuery bills don’t normally go sideways because one analyst suddenly turned reckless. They go sideways because the platform never made a serious distinction between exploration, scheduled transformations, and workloads that now behave like products. Once all of that shares the same billing path and the same failure model, the invoice is just the most annoying place where the architecture finally tells the truth.

That’s why we don’t think about guardrails as policing. Good guardrails are really about blast radius. They decide which mistakes are allowed to fail cheaply, which workloads deserve predictable completion, and which ones have earned their own lane entirely. Skip that part and jump straight to quotas, and people stop treating the controls as part of the system. They treat them as obstacles to work around.

That distinction matters because cost control goes bad fast once it turns personal. If the platform keeps letting incompatible workload shapes collide, somebody will eventually get blamed for the bill even though the design made the outcome inevitable. That’s lazy, and worse, it keeps the system exactly as fragile as it was before.

The warehouse is doing more than one job

Ad hoc exploration is one job. Scheduled transforms are another. Dashboard traffic, embedded reporting, and service-facing reads are a third again. They don’t want the same kind of protection, and pretending they do is how teams end up with controls that exist on paper and fail in practice.

Exploration needs a cheap way to be wrong. Somebody should be able to ask a messy question, miss a predicate, or poke at a table they don’t understand yet without accidentally torching the week’s spend. That’s where the early controls in on-demand versus slots make sense. Cheap failure is part of the point.

Scheduled transformations are different. If a model is part of the platform, randomly killing it with the same safety device you’d use for ad hoc SQL usually means the warehouse never got a real production lane. And once you have repeated BI traffic or service reads with latency expectations, you’ve got a workload that behaves much more like a small product than a casual query. That’s where reservation isolation starts earning its keep, especially when dashboard traffic is what’s driving the churn in the first place.

None of that is especially glamorous. It just means the warehouse has to admit that not all SQL is the same kind of work. Humans keep trying to flatten that distinction because one policy feels simpler. Then they act surprised when the “simple” setup turns into a weekly argument about exceptions.

maximum_bytes_billed is a seatbelt, not a constitution

maximum_bytes_billed is useful because it does one thing clearly. It lets BigQuery reject a query before execution if the estimated bytes processed cross the cap. For exploratory work, that’s great. The user is still in curiosity mode, and cheap rejection is often more valuable than letting the query run just to prove it was a bad idea.

Where teams get themselves into trouble is treating that control like a foundational rule for the whole warehouse. It isn’t. It’s a blunt pre-execution check based on estimated bytes processed. On clustered tables especially, that matters, because the estimate can be higher than what the final billed bytes would have been after execution behavior played out. That doesn’t make the feature broken. It means the feature is doing exactly what it was built to do, and that job is not “guarantee stable production behavior for every workload in the platform.”

So when a scheduled transform or recurring dashboard keeps hitting that limit, we don’t assume the answer is just a bigger cap. Sometimes the answer is better partitioning. Sometimes it’s a calmer serving model. Sometimes the workload simply shouldn’t still be sitting in the same lane as exploratory SQL. The limit is useful when it protects the right class of work. It’s a trap when teams keep widening it because they never separated the work in the first place.

interactive-dev:
  max_bytes_billed: 10GB
  billing: on-demand

interactive-prod:
  max_bytes_billed: 50GB
  billing: on-demand

scheduled-transforms:
  max_bytes_billed: null
  billing: reservation

dashboards:
  max_bytes_billed: null
  billing: reservation

Quotas matter, but boundaries do more of the work

BigQuery gives you default daily query quotas for on-demand usage, and you can tighten them with custom quotas or per-user limits. That’s useful. It just isn’t enough on its own. If ad hoc work shares a project with executive dashboards, the quota stops being a technical control and turns into a political argument. If batch transforms, backfills, and live reporting all share one billing lane, every exception starts looking like a reason to weaken the policy for everybody.

We’d rather keep the system boring. Let sandbox work stay on-demand with hard limits. Let production readers and scheduled transforms move behind reservations once they’ve earned it. Give legitimate exceptions an explicit path instead of quietly softening the default every time somebody important wants to run one very special expensive query that is apparently unlike all the others. It never is.

Storage classification matters too, but it’s solving a different problem. Physical versus logical storage should stay a separate decision. Teams love piling unrelated controls into one bucket and calling it governance. Usually it’s just confusion with a nicer label.

Weekly cost review should feel calm

The best cost review is boring enough that nobody tries to dodge it. Look at the top billed users and statement types over the last week. Check whether BI traffic is getting noisier. Check whether scheduled jobs are rewriting more than they should. Check whether a serving workload should move up the precompute ladder instead of staying live out of habit. Check whether the whole spike is really a table-shape problem, which is where warehouse cost spikes tend to start anyway.

If you only look when the bill hurts, cost control turns into blame. People get defensive, queries get discussed like moral failures, and the actual design issues stay right where they were. Weekly review changes the tone. It makes cost visible before it becomes dramatic, which means the conversation can stay architectural instead of personal. That’s a much better use of everyone’s time than holding a little trial every time someone forgets a filter.

select
  user_email,
  statement_type,
  count(*) as jobs,
  round(sum(total_bytes_billed) / 1e12, 2) as tb_billed,

from
  `region-eu`.INFORMATION_SCHEMA.JOBS_BY_PROJECT

where
  creation_time >= timestamp_sub(current_timestamp(), interval 7 day)
  and job_type = 'QUERY'

group by
  user_email,
  statement_type,

order by tb_billed desc;

The rule

BigQuery guardrails work when they match the work. Exploration should be able to fail cheaply. Production should be able to run predictably. BI workloads shouldn’t get infinite live compute just because they arrived through a dashboard instead of a script.

Once the controls line up with those realities, cost review gets less emotional and a lot more useful.