SECONDS_PER_DAY
SECONDS_PER_DAY = 86400
Seconds in a day, for turning a daily rate into a window's worth of writes.
How much a rollup window collapses, derived rather than assumed.
Compaction rolls several fine windows into one coarse window, and the saving comes from a subject
written more than once inside that window: only its last value needs to survive. So the collapse
factor is writes per DISTINCT subject, which is the coupon-collector expectation - throw w
writes at s subjects and the number of subjects hit at least once is
s * (1 - (1 - 1/s)^w).
This corrects an earlier flat assumption. A first draft of the cost model used 0.42 for every level, which was invented. The real factor is 1.00 at every level below a day, because a site with hundreds of thousands of subjects and tens of thousands of writes a day almost never writes the same subject twice in a minute, and it only becomes interesting at monthly rollups. That reverses what compaction is for at fine levels: the saving there is recompression and delta re-anchoring, not collapse.
The uniform assumption is the conservative one. Real writes are skewed - a few hot subjects take a large share - and skew means fewer distinct subjects for the same write count, so the real collapse is at least what this reports. An estimate that promises less saving than it delivers is the right direction to be wrong in.
factor(int $writes, int $subjects) : float
The collapse factor for one population over one window.
| int | $writes | Writes falling inside the window. |
| int | $subjects | Distinct subjects those writes could land on. |
Writes per distinct subject, never below 1.0. One means nothing collapses.
distinct(int $writes, int $subjects) : float
How many distinct subjects a number of writes touches.
| int | $writes | Writes falling inside the window. |
| int | $subjects | Distinct subjects those writes could land on. |
The expected count, which is fractional because it is an expectation.
forWindow(float $writesPerDay, int $subjects, int $windowSeconds) : float
The collapse factor for a daily write rate rolled up over a window.
| float | $writesPerDay | Writes a day across the population. |
| int | $subjects | Distinct subjects. |
| int | $windowSeconds | The rollup window. |
The factor.