\Drupal\strata\Estimate Estimator

Projects what a site's history will cost, from measured constants rather than guesses.

Every number this class multiplies was measured and is named, so a projection can be argued with rather than believed. The operation sizes come from mutating a real corpus and serializing only the changed field keys; the compression ratios come from the codec catalog's own reference measurements, taken frame by frame through the shipped codecs; the delta-coding reduction comes from re-encoding the corpus against each subject's previous version; the access-churn rate comes from core's own session write interval.

Three findings this model makes, which matter more than the totals.

Two thirds of a Drupal site's write volume is user access-timestamp churn, so how that one field is treated moves the bill more than anything else about the site. Recording it as a compact login event costs 24% less than a full field delta and less than dropping it, because dropping the operation leaves the row dirty for the reconciler to pick up later at full price.

Files are most of the stored bytes and get almost none of the compression benefit, which is why they have their own ladder, their own budget line and a different default: store each unique content once, forever, by digest, rather than versioning it.

The tree is what a large quiet site pays for. Base manifests scale with how many subjects a site has and how many of them change, not with how busy any one of them is, so a site with a million mostly-idle users pays more than a small site under constant load.

What the base interval does, stated carefully. Lengthening it divides the number of bases, but each base then covers a longer window and records more changed subjects, so the two effects largely cancel. The interval is a real dial on restore latency - a deeper replay walks more segments - and only a weak one on stored bytes. A projection that claimed otherwise would send an operator to tune the wrong thing.

Summary

Methods
Properties
Constants
__construct
project
populations
operationsPerDay
rawBytesPerDay
treeBytes
codeBytes
writesPerMonth
ratio
No public properties found
SECONDS_PER_DAY
SECONDS_PER_MONTH
SECONDS_PER_YEAR
DAYS_PER_YEAR
ACCESS_WRITES_PER_ACTIVE_USER
USER_EDITS_PER_USER
NODE_EDITS_PER_NODE
ROW_WRITES_PER_USER
STATE_WRITES_PER_DAY
CONFIG_WRITES_PER_DAY
OP_BYTES
DELTA_REDUCTION
TREE_BYTES_PER_CHANGED_SUBJECT
CODE_BYTES_PER_DEPLOY
FLUSH_RATIO
COMPACTION_RATIO
WRITES_PER_FLUSH
COMPACTION_REWRITE_SHARE
No protected methods found
No protected properties found
No protected constants found
accessBytes
opSize
byte
opBytes
compactionRatio
No private constants found

Constant

SECONDS_PER_DAY

SECONDS_PER_DAY = 86400

Seconds in a day.

SECONDS_PER_MONTH

SECONDS_PER_MONTH = 2629800

Seconds in a month, taken as a thirtieth of a year so twelve months make a year exactly.

SECONDS_PER_YEAR

SECONDS_PER_YEAR = 31557600

Seconds in a year.

DAYS_PER_YEAR

DAYS_PER_YEAR = 365.25

Days in a year, matching the seconds above.

ACCESS_WRITES_PER_ACTIVE_USER

ACCESS_WRITES_PER_ACTIVE_USER = 12.0

Access writes a day per daily-active user.

Core writes the access field only when requestTime - lastAccessedTime exceeds session_write_interval, which defaults to 180 seconds. At three sessions a day of about twelve minutes each that is twelve writes.

USER_EDITS_PER_USER

USER_EDITS_PER_USER = 0.12

Profile edits a day per registered user.

NODE_EDITS_PER_NODE

NODE_EDITS_PER_NODE = 0.019

Content edits a day per node.

ROW_WRITES_PER_USER

ROW_WRITES_PER_USER = 0.25

Custom table row writes a day per registered user.

Attributed to users rather than to rows because these are the tables a site's own modules write on behalf of a person doing something.

STATE_WRITES_PER_DAY

STATE_WRITES_PER_DAY = 580.0

State and key-value writes a day, site-wide.

Bookkeeping by cron and by modules, which does not scale with the size of the site.

CONFIG_WRITES_PER_DAY

CONFIG_WRITES_PER_DAY = 12.0

Configuration writes a day, site-wide.

OP_BYTES

OP_BYTES = ['access_event' => 48, 'access_delta' => 184, 'user_edit' => 593, 'node_edit' => 870, 'row_write' => 217, 'state_write' => 217, 'config_write' => 2000]

Operation record sizes in bytes, including the 120-byte header.

Measured by mutating a real corpus and serializing only the changed top-level field keys. Where a realm had several measured cases the mean of the day-to-day ones is used and the first-capture case is excluded, since a first capture happens once and a daily edit happens daily. The table and configuration realms had no measured case of their own; the small-field figure stands in for a row write, and a whole configuration object is taken at two kilobytes.

DELTA_REDUCTION

DELTA_REDUCTION = 0.257

Share of journal bytes delta coding removes before compression.

25.7% in aggregate. On the rewrite class alone it is 93%: a 5,967-byte blob with one flag flipped encodes to 94 bytes against its previous version. Small operations dominate by count, which is why the aggregate figure is so much lower than the headline one.

TREE_BYTES_PER_CHANGED_SUBJECT

TREE_BYTES_PER_CHANGED_SUBJECT = 240.0

Bytes a base manifest spends per subject that changed inside its window.

Calibrated against one measurement: 7.23 GB a year of manifests for a site of 262,000 subjects writing 82,892 operations a day with hourly bases. One measured point fixes one constant, and strata:calibrate replaces it with the site's own.

CODE_BYTES_PER_DEPLOY

CODE_BYTES_PER_DEPLOY = 62772.0

Stored bytes the code realm costs per deploy.

A real module measured 762,344 bytes compressed on a first capture; a deploy changes a fraction of it and delta codes against the previous version, which came to 8.98 MiB a year over 150 deploys.

FLUSH_RATIO

FLUSH_RATIO = 4.28

Compression ratio on the flush path.

Level 1 with no dictionary, which is what a flush inside a web request can afford: 422.9 MB/s against 105.1 MB/s once a dictionary is loaded per frame.

COMPACTION_RATIO

COMPACTION_RATIO = 5.86

Compression ratio after compaction.

Level 19 with a per-realm dictionary, at 2.4 MB/s. Slow enough that it only belongs in compaction, where nothing is waiting on it.

WRITES_PER_FLUSH

WRITES_PER_FLUSH = 4

Write requests one flush performs.

A pack of frames, a segment manifest, a commit and the ref that names it. Frames below a megabyte are batched into one pack object, which is the difference between a few million requests a month and tens of millions, and the index of subjects belongs to a base anchor on its own interval rather than to every flush.

COMPACTION_REWRITE_SHARE

COMPACTION_REWRITE_SHARE = 0.05

Share of stored objects compaction rewrites each month.

Recompression rewrites whole packs rather than single frames, so the request cost is per pack and not per frame.

Properties

$opBytes

$opBytes : array

Type

array<string|int, mixed>

$compactionRatio

$compactionRatio : ?float

Type

float|null

Methods

__construct()

__construct(array  $opBytes = [], float|null  $compactionRatio = null) : mixed

Constructs an estimator.

Parameters

array $opBytes

Operation sizes overriding the measured ones, keyed as OP_BYTES is. A calibration pass supplies what this host actually produced.

float|null $compactionRatio

The compression ratio to apply, or NULL to read it from the codec catalog.

Returns

mixed —

project()

project(\Drupal\strata\Estimate\Measurement  $measurement) : \Drupal\strata\Estimate\Projection

Projects a measurement.

Parameters

\Drupal\strata\Estimate\Measurement $measurement

The site and the policy.

Returns

\Drupal\strata\Estimate\Projection —

What it is projected to cost.

populations()

populations(\Drupal\strata\Estimate\Measurement  $measurement) : list<array{name: string, realm: string, ops: float, bytes: int}>

Every write population a site has, each with its own rate and its own operation size.

Populations rather than realms, because the entity realm is three different things: access churn at 48 bytes and two thirds of the volume, profile edits at 593, and content edits at 870. One mean over that mix would be wrong in both directions at once, and it is exactly the mix the access-churn decision turns on.

Parameters

\Drupal\strata\Estimate\Measurement $measurement

The site.

Returns

list

The populations.

operationsPerDay()

operationsPerDay(\Drupal\strata\Estimate\Measurement  $measurement) : array<string,float>

Operations a day, by realm.

Parameters

\Drupal\strata\Estimate\Measurement $measurement

The site.

Returns

array

Realm value keyed to operations a day.

rawBytesPerDay()

rawBytesPerDay(\Drupal\strata\Estimate\Measurement  $measurement) : array<string,float>

Uncompressed operation bytes a day, by realm.

Parameters

\Drupal\strata\Estimate\Measurement $measurement

The site.

Returns

array

Realm value keyed to bytes a day.

treeBytes()

treeBytes(\Drupal\strata\Estimate\Measurement  $measurement, float  $opsPerDay) : int

Stored bytes of base manifests over the retention period.

Parameters

\Drupal\strata\Estimate\Measurement $measurement

The site.

float $opsPerDay

Operations a day across every realm.

Returns

int —

Bytes.

codeBytes()

codeBytes(\Drupal\strata\Estimate\Measurement  $measurement) : int

Stored bytes of the code realm over the retention period.

Parameters

\Drupal\strata\Estimate\Measurement $measurement

The site.

Returns

int —

Bytes.

writesPerMonth()

writesPerMonth(\Drupal\strata\Estimate\Measurement  $measurement) : int

Write requests a month.

Parameters

\Drupal\strata\Estimate\Measurement $measurement

The site.

Returns

int —

Requests.

ratio()

ratio() : float

The compression ratio in use.

Returns

float —

The ratio.

accessBytes()

accessBytes(\Drupal\strata\Estimate\Measurement  $measurement) : int

The size of one access-churn record under the configured mode.

Parameters

\Drupal\strata\Estimate\Measurement $measurement

The site.

Returns

int —

Bytes.

opSize()

opSize(string  $realm, \Drupal\strata\Estimate\Measurement  $measurement) : int

The operation size to use for one realm.

Parameters

string $realm

The realm value.

\Drupal\strata\Estimate\Measurement $measurement

The site, which decides how access churn is recorded.

Returns

int —

Bytes.

byte()

byte(string  $key) : int

One operation size, overridden or measured.

Parameters

string $key

A key of OP_BYTES.

Returns

int —

Bytes.