the dashboard was fast until the data got big, and then it wasn't, and the reason was that it was doing all of its work over again every time anyone looked at it.
the project was a live view over public conflict-event data, the kind of thing where new records arrive continuously and people expect the charts to reflect them now, not after a refresh they have to sit through. the simplest way to build that is to recompute the aggregates from scratch on each query. scan everything, group it, sum it, return it. it's correct, it's easy to reason about, and it falls apart the moment "everything" stops being small, because the cost of a full scan grows with the data while the user's patience does not.
the alternative is to stop recomputing and start updating. instead of rescanning the whole table, i streamed the changes as they happened, captured straight from the database, and kept the aggregates current by applying only what changed. a new event nudges the running totals it affects and nothing else. the dashboard reads numbers that are already up to date rather than asking the database to derive them on the spot. at ten million rows that was roughly seventy times faster than the full-scan version, and the gap widens as the data grows, which is the property you actually want.
the honest part is that incremental computation is not free. it's more moving parts, a stream to keep running, state to maintain, and a class of bugs the naive version simply doesn't have, where your running total drifts out of agreement with the source and you have to notice and reconcile. for a small dataset that's a bad trade. recompute is simpler and you should keep it.
so the decision isn't "incremental is better." it's knowing where the line is. below some size, rescanning everything is the right, boring choice. above it, the scan becomes the bottleneck and the extra machinery earns its keep. the skill is seeing which side of that line you're on before your users feel it, not after.