MIN_SAMPLES
MIN_SAMPLES = 32
Fewest samples worth building a dictionary from.
Below this the dictionary is longer than what it would save and describes one document rather than a realm.
Builds a dictionary for one realm out of what that realm actually writes.
A dictionary is what makes small frames compress at all. A 16 KiB frame has almost no history to match against, so the same field names and the same JSON scaffolding are re-encoded in every frame; a dictionary supplies that history up front. Measured through the shipped codec: 4.87x without and 5.86x with, on the reference corpus at 16 KiB frames.
Two ways to build one, and which wins depends on the corpus, so both are built and scored.
Concatenating samples produces a raw content dictionary, which zstd matches against verbatim; the
zstd binary's trainer produces a compact entropy-oriented one. On 200 near-identical entity
payloads the raw dictionary measured 11.64x against the trainer's 8.42x; on 400 deliberately mixed
payloads the trainer measured 5.42x against 4.96x. Neither is generally better, so a run compresses
a held-out slice with each candidate and keeps whichever actually won. Nothing here assumes.
No PHP extension exposes zstd's training API - ext-zstd ships the dictionary compress and
decompress calls and nothing else - so the trained candidate needs the binary and is simply absent
when it is not installed.
$codec : \Drupal\strata\Codec\CompressionCodecInterface
__construct(\Drupal\strata\Codec\CompressionCodecInterface $codec, string|null $binary = null) : mixed
Constructs a trainer.
| \Drupal\strata\Codec\CompressionCodecInterface | $codec | The codec candidates are scored through, so the score is what this host will actually get. |
| string|null | $binary | Path to the |
train(list$samples) : array{bytes: string, source: string, ratio: float, samples: int}|null
Builds the best dictionary the samples support.
| list |
$samples | Representative payloads from one realm. |
The winning candidate with the ratio it measured, or NULL when there is too little data or no candidate beat compressing without a dictionary at all.
baseline(list$samples) : float
What a corpus compresses to without any dictionary.
The number a candidate has to beat, and the reason a realm whose payloads have nothing in common ends up with no dictionary rather than a useless one.
| list |
$samples | Payloads to score against. |
The ratio.
concatenate(list$build) : string
The raw content candidate: samples end to end, newest first, up to the ceiling.
Newest first because a dictionary's tail is what zstd matches most cheaply, and the samples a realm is writing now are the ones the next frames will look like.
| list |
$build | The samples. |
The dictionary bytes, empty when there is nothing to concatenate.
measure(list$samples, string|null $dictionary) : float
The ratio a corpus compresses at, with a dictionary or without one.
| list |
$samples | Payloads. |
| string|null | $dictionary | Dictionary bytes, or NULL. |
Raw bytes over compressed bytes, 1.0 when nothing could be measured.