\Drupal\strata\Cas ObjectStore

Puts a value into the store and gets it back.

The one path every captured byte travels: split into frames, deduplicate against the index, compress, seal, batch into a pack, upload. Reading reverses it, fetching only the ranges a value actually needs.

Order matters and is fixed. Compression runs before sealing, since ciphertext does not compress. The content address is taken from the DECODED frame, so the address is stable across a change of codec, cipher, level or dictionary; two sites with different settings storing the same content agree on what to call it. And the address is passed to the cipher as associated data, so a frame relocated to another key fails to open rather than opening as the wrong content.

Every frame carries the codec, cipher and dictionary that wrote it in its index record rather than inheriting the store's current settings, so changing a setting never orphans what is already written. The same fields are written into the OBJECT as well - a standalone frame carries them in a plaintext header, a packed frame in the pack's own directory - so the whole index is reconstructible from the bucket after the local tables are dropped.

Summary

Methods
Properties
Constants
__construct
write
commit
written
read
frame
dictionaryId
index
No public properties found
FRAME_PREFIX
PACK_PREFIX
No protected methods found
No protected properties found
No protected constants found
deltaSource
store
drain
unwrap
fetchPacked
parentBytes
dictionaryFor
written
provider
index
codecs
cipher
framer
packer
level
dictionary
dictionaryId
dictionaries
chainPolicy
No private constants found

Constant

FRAME_PREFIX

FRAME_PREFIX = 'frames'

Key prefix standalone frames are written under.

PACK_PREFIX

PACK_PREFIX = 'packs'

Key prefix packs are written under.

Properties

$written

$written : int

Encoded bytes this store has written since it was constructed.

Monotonic rather than resettable, so a caller measuring one flush takes the difference across it and two callers sharing a store cannot clear each other's reading.

Type

int

$codecs

$codecs : \Drupal\strata\Codec\CodecRegistry

Type

CodecRegistry

$cipher

$cipher : \Drupal\strata\Crypto\CipherInterface

Type

CipherInterface

$framer

$framer : \Drupal\strata\Cas\Framer

Type

Framer

$packer

$packer : \Drupal\strata\Cas\Packer

Type

Packer

$level

$level : ?int

Type

int|null

$dictionary

$dictionary : ?string

Type

string|null

$dictionaryId

$dictionaryId : ?string

Type

string|null

$dictionaries

$dictionaries : ?\Drupal\strata\Codec\Dictionary\DictionaryStore

Type

DictionaryStore|null

$chainPolicy

$chainPolicy : ?\Drupal\strata\Delta\ChainDepthPolicy

Type

ChainDepthPolicy|null

Methods

__construct()

__construct(\Drupal\strata\Storage\StorageProviderInterface  $provider, \Drupal\strata\Cas\FrameIndexInterface  $index, \Drupal\strata\Codec\CodecRegistry  $codecs, \Drupal\strata\Crypto\CipherInterface  $cipher, \Drupal\strata\Cas\Framer  $framer = new \Drupal\strata\Cas\Framer(), \Drupal\strata\Cas\Packer  $packer = new \Drupal\strata\Cas\Packer(), int|null  $level = null, string|null  $dictionary = null, string|null  $dictionaryId = null, \Drupal\strata\Codec\Dictionary\DictionaryStore|null  $dictionaries = null, \Drupal\strata\Delta\ChainDepthPolicy|null  $chainPolicy = null) : mixed

Constructs a store.

Parameters

\Drupal\strata\Storage\StorageProviderInterface $provider

Where objects are written.

\Drupal\strata\Cas\FrameIndexInterface $index

The deduplication and reference index.

\Drupal\strata\Codec\CodecRegistry $codecs

Used to pick a writer and to find a reader for whatever wrote an existing frame.

\Drupal\strata\Crypto\CipherInterface $cipher

Seals frames on the way out.

\Drupal\strata\Cas\Framer $framer

Splits values into frames.

\Drupal\strata\Cas\Packer $packer

Batches small frames.

int|null $level

Compression level, or NULL for the codec's default.

string|null $dictionary

Dictionary bytes to compress with, or NULL.

string|null $dictionaryId

Identifier recorded in each frame record so the dictionary can be found again. Required whenever $dictionary is given, since a frame that cannot name its dictionary cannot be decoded.

\Drupal\strata\Codec\Dictionary\DictionaryStore|null $dictionaries

Where a dictionary a frame names but this store is not holding can be fetched from. Without it, retraining would strand every frame written against the previous version.

\Drupal\strata\Delta\ChainDepthPolicy|null $chainPolicy

Bounds how long a delta chain may grow, or NULL to store every frame standalone. A chain is a dependency chain - frame N needs frame N-1 - so without a cap a hot subject would build a chain thousands of links long and every read of it would walk the lot.

Throws

\InvalidArgumentException

When a dictionary is supplied without an id.

Returns

mixed —

write()

write(string  $value, list  $previous = []) : list<string>

Stores a value and returns the frame map that reassembles it.

Frames the index already holds are skipped entirely: no compression, no sealing, no upload. The map still lists them, so two values sharing content share frames.

Parameters

string $value

The bytes to store.

list $previous

The frame map of the subject's previous version, when there is one. A rewritten value compresses against its own previous version far better than against anything else - measured at 63.70x on the rewrite class, where a 5,967-byte blob with one flag flipped encodes to 94 bytes - and this is where that gain comes from. Empty when there is no previous version or when the caller does not have it, which stores standalone.

Throws

\RuntimeException

When a codec, the cipher or the provider fails.

Returns

list

Frame content addresses, in order. An empty value produces an empty map.

commit()

commit() : int

Flushes any partially filled pack.

Called at the end of a segment so nothing is left buffered in memory across a request.

Throws

\RuntimeException

When the provider fails.

Returns

int —

How many packs were written.

written()

written() : int

Encoded bytes written since this store was constructed.

Counts what actually reached the provider, so a frame the index already held contributes nothing. Deduplication is most of what this module does, and a figure that counted skipped frames would report a flush as costing what it avoided.

Returns

int —

The running total.

read()

read(list  $map) : string

Rebuilds a value from its frame map.

Parameters

list $map

Frame content addresses, in order, as returned by ObjectStore::write().

Throws

\RuntimeException

When a frame is unknown, absent, undecodable, or does not match its address.

Returns

string —

The value.

frame()

frame(string  $hash) : string|null

Fetches and decodes one frame.

A packed frame is fetched with a ranged read, so reading one frame out of a 1 MiB pack costs that frame's bytes rather than the whole pack. An endpoint without range support falls back to the whole object.

Parameters

string $hash

Content address of the decoded frame.

Throws

\RuntimeException

When the frame is known but absent, its codec cannot be read here, its dictionary is missing, or it fails to open.

Returns

string|null —

The decoded frame, or NULL when the index does not know it.

dictionaryId()

dictionaryId() : string|null

The dictionary this store can decode against, if any.

A frame naming a different dictionary cannot be read here, so a verify pass asks for this rather than assuming every dictionary a frame might name is available.

Returns

string|null —

The dictionary id, or NULL when the store holds none.

index()

index() : \Drupal\strata\Cas\FrameIndexInterface

The index this store writes to.

Returns

\Drupal\strata\Cas\FrameIndexInterface —

The index.

deltaSource()

deltaSource(string  $value, list  $previous) : array{hash: string, bytes: string, depth: int}|null

The previous version a value can be delta coded against, if any.

Only a single-frame value against a single-frame previous version. A value spanning several frames is already large enough that the framer's own deduplication covers the unchanged parts, and coding frame two of the new value against frame one of the old one would produce a chain whose links do not correspond to anything.

Parameters

string $value

The value being stored.

list $previous

The previous version's frame map.

Returns

array{hash: string, bytes: string, depth: int}|null —

The parent frame's address, its decoded bytes and its chain depth, or NULL when this value should be stored standalone.

store()

store(string  $hash, string  $bytes, array{hash: string, bytes: string, depth: int}|null  $against = null) : void

Encodes one frame and either buffers or uploads it.

Parameters

string $hash

Content address of the decoded frame.

string $bytes

The decoded frame.

array{hash: string, bytes: string, depth: int}|null $against

The previous version to code against, or NULL to store standalone.

Throws

\RuntimeException

When a codec, the cipher or the provider fails.

Returns

void —

drain()

drain(bool  $force = false) : int

Uploads full packs, and a partial one when forced.

Parameters

bool $force

TRUE to upload a pack that has not reached its target.

Throws

\RuntimeException

When the provider fails.

Returns

int —

How many packs were written.

unwrap()

unwrap(string  $hash, \Drupal\strata\Cas\FrameRecord  $record) : string

Fetches a standalone frame and strips its header.

The header's own account of the codec and cipher is checked against the index rather than trusted over it. They disagree only when the index describes a different object than the one at that key, and decoding either way would produce garbage that looks like data.

Parameters

string $hash

Content address of the decoded frame.

\Drupal\strata\Cas\FrameRecord $record

The frame's record.

Throws

\RuntimeException

When the object is absent, unreadable, or describes itself differently than the index does.

Returns

string —

The sealed body.

fetchPacked()

fetchPacked(\Drupal\strata\Cas\FrameRecord  $record) : string

Fetches a frame that lives inside a pack.

Parameters

\Drupal\strata\Cas\FrameRecord $record

The frame's record.

Throws

\RuntimeException

When the pack is absent or shorter than the record says.

Returns

string —

The encoded frame.

parentBytes()

parentBytes(string  $hash, string  $parent) : string

The decoded parent of a delta frame.

Reading a delta frame reads its parent, and that parent may itself be a delta, so a read walks the chain. ChainDepthPolicy is what keeps that walk short; this only refuses when the parent is not there at all, because a delta frame without its parent decodes to nothing usable and returning it would be worse than failing.

Parameters

string $hash

The frame being read, for the error.

string $parent

Its delta parent's address.

Throws

\RuntimeException

When the parent is unknown or unreadable.

Returns

string —

The parent's decoded bytes.

dictionaryFor()

dictionaryFor(string  $hash, string|null  $id) : string|null

The dictionary bytes one frame has to be decoded against.

Usually the one this store is compressing with, since most frames were written by a store configured the same way. After a retrain they are not the same, and a frame written against an older version has to be given that version - which is why an older version is never pruned while a frame names it.

Parameters

string $hash

The frame's content address, for the error.

string|null $id

The dictionary id the frame recorded, or NULL when it was written without one.

Throws

\RuntimeException

When the frame names a dictionary this store cannot produce. Decoding against the wrong dictionary yields plausible garbage, so this refuses instead.

Returns

string|null —

The dictionary bytes, or NULL when the frame needs none.