Load everything, then make the report
240illustrative read units
See the difference.
The same report. Less data to read.
Read only what the report needs
21illustrative read units
Change the report request
See which data gets read
01
240 unitsRead everything
Load all dates and fields, then keep the requested dates and selected fields for the report.
Day 0130 synthetic days →Day 30
02
21 unitsFocus on what matters
Day 0130 synthetic days →Day 30
ReadNot neededEach block represents an equal amount of data.
91.3% modeled read reduction; not a bill or time estimate.
The broad approach loads the full dataset before selecting the report’s dates and fields. Both paths finish with the same report. Equal-size units simplify this illustration.
How this relates to cost
BIGQUERY
How does this connect to a BigQuery bill?
Query processing and stored data are separate parts of the bill. Choose a compute billing model to see the difference.
Storage is separate.
Keeping data has its own charge over time. Reading fewer fields does not remove the data you retain.
These are alternative compute billing models, plus storage. This demo estimates neither a bill nor a monetary saving.
Same updated report. Two methods compared.
Start with six orders. Two change and two new sales arrive. The updated report has eight.
Rebuild the whole report
8orders written
A straightforward rebuild.
Repeats the work for unchanged orders.
Write only changed and new orders
4orders written
Leaves unchanged orders alone.
Needs reliable change tracking.
WrittenKept as-is
See the steps and the fictional orders
Replay the update. Each grid below shows the report at that step.
#1003$160 → $168
#1005Pending → Paid
#1007 + #1008Two new sales
Reduced motion: advance manually or play without animation.
A
Replace all
0
B
Apply changes
0
ChangedNewKept
See counts and assumptions
Different work, counted separately
| Operation | Full refresh | Incremental |
|---|---|---|
| Change markers checked | 0 | 0 |
| Source orders read | 0 | 0 |
| Report orders written | 0 | 0 |
Finding changes is part of the work.
The source provides reliable version markers separately from full order details. Both paths inspect eight markers in this example. Capturing those markers also takes work upstream. Reads and writes count logical orders, not all physical work inside a platform.
No deletions, retries, concurrent changes or derived totals are modeled. Finding destination records can add work. A full rebuild does not inherently need version comparisons. Animation speed is only for explanation.
The grids always show the report, in the same order on both sides. Reading highlights activity but does not update its values. Values change only when the report is written; markers and full record details are included in each card’s accessible description.
Which setup fits the work?
Independent tasks
Cloud Run Jobs
- Strong point
- Keep in mind
Coordinated processing
Dataproc / Serverless Spark
- Strong point
- Keep in mind
A shared environment
Dataproc / Spark cluster
- Strong point
- Keep in mind
Adjust assumptions & see technical details
Shape the scenario
For one CPU-only task, not the total across many tasks. Current Cloud Run Jobs limits and supported CPU/memory combinations need checking before deployment.
Cloud Run Jobs
Serverless Spark
Spark cluster
What to weigh up
A natural candidate means worth evaluating, not a proven fit. This is a scenario guide, not a benchmark or architecture approval. Validate limits, data movement, recovery, quotas, and real workload behavior before choosing.
Dataproc / managed Apache Spark includes cluster and serverless options. Here, Cloud Run means Cloud Run Jobs for batch processing.
Synthetic illustrations, not measured client results or time/cost forecasts.
Discuss your project