\Drupal\strata\Compaction Compactor

Makes stored history cheaper without making it less restorable.

Two jobs, deliberately separated. Recompression rewrites objects at a setting the flush path cannot afford, changes no content address, and destroys nothing - it is safe to run on a schedule and safe to interrupt. Pruning removes restore targets, so it runs only when asked, refuses on any doubt, and always produces a receipt.

The refusal rules are the load-bearing part:

A prune refuses outright when the reachability walk could not read something. An incomplete walk under-reports what is live, and under-reporting is the direction that deletes what a restore needs. Refusing costs storage; guessing costs the backup.
A candidate is dropped only when all three reachability classes agree - no reachable commit needs it, no live frame was compressed against its dictionary, and nothing decodes against it as a delta parent. Every candidate held back is named in the receipt, so the refusals are visible rather than implied.
A pack is deleted only when every frame in it is collectable. One live frame keeps the whole object, because a pack is a flat concatenation and there is no way to remove part of one.
A pack the index no longer points at is deleted only when every frame in it is provably filed somewhere else. A recompressed pack leaves its predecessor with no index row naming it, which is indistinguishable from a stale index unless each frame is found at its new home first.
A prune whose deletion the store refuses part way returns a refusal rather than trimming the index anyway. On a store spread across buckets that is what a bucket being down looks like, and the index rows are the only thing that still names the objects left behind.

Summary

Methods

Properties

Constants

__construct
compact
prune
levels

No public properties found

No public constants found

No protected methods found

No protected properties found

No protected constants found

collect
remove
liveFrameIn
supersededPacks
nearPackKeys
packKeys

provider
index
recompressor
reachability
levels
lease
logger
reanchorer
rollup
migrator
placement
notifier

No private constants found

final

File: src/Compaction/Compactor.php
Package: Application
Class hierarchy: \Drupal\strata\Compaction\Compactor
See also

Properties

$provider

$provider : \Drupal\strata\Storage\StorageProviderInterface

Type

StorageProviderInterface

$index

$index : \Drupal\strata\Cas\FrameIndexInterface

Type

FrameIndexInterface

$recompressor

$recompressor : \Drupal\strata\Compaction\Recompressor

Type

Recompressor

$reachability

$reachability : \Drupal\strata\Compaction\Reachability

Type

Reachability

$levels

$levels : \Drupal\strata\Compaction\LevelPolicy

Type

LevelPolicy

$lease

$lease : \Drupal\strata\Flush\Lease

Type

Lease

$logger

$logger : \Psr\Log\LoggerInterface

Type

LoggerInterface

$reanchorer

$reanchorer : ?\Drupal\strata\Delta\Reanchorer

Type

Reanchorer|null

$rollup

$rollup : ?\Drupal\strata\Compaction\Rollup

Type

Rollup|null

$migrator

$migrator : ?\Drupal\strata\Tier\TierMigrator

Type

TierMigrator|null

$placement

$placement : ?\Drupal\strata\Tier\PlacementIndexInterface

Type

PlacementIndexInterface|null

$notifier

$notifier : ?\Drupal\strata\Event\Notifier

Type

Notifier|null

Methods

__construct()

__construct(\Drupal\strata\Storage\StorageProviderInterface  $provider, \Drupal\strata\Cas\FrameIndexInterface  $index, \Drupal\strata\Compaction\Recompressor  $recompressor, \Drupal\strata\Compaction\Reachability  $reachability, \Drupal\strata\Compaction\LevelPolicy  $levels, \Drupal\strata\Flush\Lease  $lease, \Psr\Log\LoggerInterface  $logger, \Drupal\strata\Delta\Reanchorer|null  $reanchorer = null, \Drupal\strata\Compaction\Rollup|null  $rollup = null, \Drupal\strata\Tier\TierMigrator|null  $migrator = null, \Drupal\strata\Tier\PlacementIndexInterface|null  $placement = null, \Drupal\strata\Event\Notifier|null  $notifier = null) : mixed

Constructs a compactor.

Parameters

\Drupal\strata\Storage\StorageProviderInterface	$provider	The store to rewrite and prune.
\Drupal\strata\Cas\FrameIndexInterface	$index	Repointed by a rewrite and trimmed by a prune.
\Drupal\strata\Compaction\Recompressor	$recompressor	Does the rewriting.
\Drupal\strata\Compaction\Reachability	$reachability	Decides what may be removed.
\Drupal\strata\Compaction\LevelPolicy	$levels	The retention ladder, which decides what is old enough to consider.
\Drupal\strata\Flush\Lease	$lease	Keeps two compactions from overlapping, and keeps one from overlapping a flush.
\Psr\Log\LoggerInterface	$logger	Records what a pass did.
\Drupal\strata\Delta\Reanchorer\|null	$reanchorer	Breaks delta chains that have grown past the cap, or NULL to leave them alone. Runs before the recompression, because re-anchoring rewrites the frames a recompression would otherwise have to rewrite a second time.
\Drupal\strata\Compaction\Rollup\|null	$rollup	Folds each level's segments into the level above, or NULL to leave the ladder alone. Runs before the recompression so the coarse segments it writes are the ones densified.
\Drupal\strata\Tier\TierMigrator\|null	$migrator	Moves aged objects into the tier they belong in, or NULL on a store with one destination. Runs after the recompression, because a pack is densified once and the dense form is the one worth copying into a bucket nobody is going to rewrite again.
\Drupal\strata\Tier\PlacementIndexInterface\|null	$placement	Where each object lives, or NULL when there is only one place it could be. Used to keep the recompression on the nearest tier: a cold pack was densified on its way out, and reading it again every pass would spend class-B requests in the bucket the tiering exists to keep quiet.
\Drupal\strata\Event\Notifier\|null	$notifier	Announces a prune that removed something or refused, or NULL to announce nothing.

Returns

mixed —

compact()

compact(int  $budget = 0, bool  $prune = false, bool  $apply = false) : \Drupal\strata\Compaction\CompactionReport

Runs a compaction pass.

Parameters

int	$budget	Most stored bytes to read while recompressing; zero for no limit.
bool	$prune	TRUE to follow the rewrite with a prune. FALSE is the cron default, because removing a restore target is not something a scheduled job should do without having been asked.
bool	$apply	Passed through to the prune: FALSE produces the same receipt and deletes nothing.

Returns

\Drupal\strata\Compaction\CompactionReport —

What the pass did.

prune()

prune(bool  $apply = false, int  $limit = 1000) : \Drupal\strata\Compaction\PruneReceipt

Removes what no reachable history needs, or explains why it would not.

Parameters

bool	$apply	FALSE to produce the receipt without deleting anything. The numbers are the same either way, which is what makes the dry run worth reading.
int	$limit	Most frames to consider in one pass.

Returns

\Drupal\strata\Compaction\PruneReceipt —

What was removed, what was held back, and why.

levels()

levels() : \Drupal\strata\Compaction\LevelPolicy

The retention ladder this compactor works to.

Returns

\Drupal\strata\Compaction\LevelPolicy —

The policy.

collect()

collect(bool  $apply, int  $limit) : \Drupal\strata\Compaction\PruneReceipt

The prune itself, before anything is announced.

Parameters

bool	$apply	FALSE to report what would be removed without removing it.
int	$limit	Most orphan candidates to consider.

Returns

\Drupal\strata\Compaction\PruneReceipt —

The receipt.

remove()

remove(array  $collectable, array  $superseded, list  $kept, bool  $apply, float  $started) : \Drupal\strata\Compaction\PruneReceipt

Deletes the objects holding a set of collectable frames.

A pack survives while it holds one frame anything needs. That means a pass can free nothing even with collectable frames in hand, which is correct and is reported rather than worked around: the alternative is rewriting the pack without the dead frames, which is a compaction job and not a deletion.

Parameters

array	$collectable	Collectable records keyed by content address.
array	$superseded	Object key keyed to its size, for packs the index has moved every frame out of.
list	$kept	Lines already collected for the receipt.
bool	$apply	FALSE to report without deleting.
float	$started	When the pass began.

Returns

\Drupal\strata\Compaction\PruneReceipt —

The receipt.

liveFrameIn()

liveFrameIn(string  $pack, list  $collectable) : string|null

A frame in a pack that is not in the collectable set.

Parameters

string	$pack	The pack id.
list	$collectable	Frames in this pack that may go.

Returns

string|null —

The address of a frame that must stay, or NULL when the whole pack may go.

supersededPacks()

supersededPacks(list  $kept) : array<string,int>

Packs whose every frame the index now files somewhere else.

Recompression writes a denser pack and repoints the index at it, leaving the predecessor in place: a row pointing at an object that was never written is data loss, while an object no row points at is garbage, and the safe order produces garbage. This is what collects it.

A pack is superseded only when every frame in its directory is found in the index under a DIFFERENT object. A frame the index does not know at all makes the pack un-sweepable, because an absent row means either that the frame was collected or that the index is stale, and the two are indistinguishable from here. Refusing costs one stale object until the next reindex.

Parameters

list

$kept

Collects a line for each pack that was examined and held.

Returns

array —

Object key keyed to its size in bytes.

nearPackKeys()

nearPackKeys() : list<string>

Pack keys the recompressor should look at.

Everything, on a store with one destination. On a tiered store, only the packs the placement index files in the nearest tier: a pack that has already been promoted was densified before it went, and reading it back every pass to find out that it will not shrink further is a class-B request in exactly the bucket the tiering exists to keep quiet. A pack with no placement row is included, because unknown must not read as cold.

Returns

list —

Object keys.

packKeys()

packKeys() : list<string>

Every pack key in the store, oldest level first.

Returns

list —

Object keys.

see
package	Application
uses
method
property
property-read
property-write