\Drupal\strata\Cas Packer

Batches small frames into one object.

At 16 KiB a frame is far too small to be its own object. R2 charges $4.50 per million Class A operations and a 15-second flush interval already costs about 202,000 of them per month before any frames are counted; storing each frame separately would add one per frame and put the request bill an order of magnitude above the storage bill.

A pack is a flat concatenation with the directory at the end. The frame bytes carry no framing of their own, so a reader that knows a frame's offset and length fetches exactly that range with one ranged GET and never downloads the rest. The local frame index holds the same offsets, so an ordinary read never touches the directory and costs no extra request; the directory is what lets the local index be rebuilt from the bucket after it is dropped.

Summary

Methods
Properties
Constants
__construct
target
buffered
pending
isFull
shouldStoreAlone
add
flush
discard
extract
No public properties found
DEFAULT_TARGET
MIN_TARGET
MAX_TARGET
No protected methods found
No protected properties found
No protected constants found
No private methods found
target
pending
buffered
No private constants found

Constant

DEFAULT_TARGET

DEFAULT_TARGET = 1048576

Target pack size in bytes.

MIN_TARGET

MIN_TARGET = 65536

Smallest useful target. Below this the request saving stops covering the read amplification.

MAX_TARGET

MAX_TARGET = 134217728

Largest target. A pack this size makes a ranged read the only sane way to touch it.

Properties

$target

$target : int

Target size in bytes.

Type

int

$pending

$pending : list}>

Frames waiting to be flushed, in insertion order.

Type

array<int, array{hash: string, bytes: string, meta: array}> —

$buffered

$buffered : int

Bytes currently buffered.

Type

int

Methods

__construct()

__construct(int  $target = \self::DEFAULT_TARGET) : mixed

Constructs a packer.

Parameters

int $target

Target pack size in bytes, between Packer::MIN_TARGET and Packer::MAX_TARGET.

Throws

\InvalidArgumentException

When the target is outside the supported range.

Returns

mixed —

target()

target() : int

The configured target size.

Returns

int —

Target pack size in bytes.

buffered()

buffered() : int

Bytes currently buffered.

Returns

int —

The buffered total.

pending()

pending() : int

How many frames are waiting.

Returns

int —

The pending count.

isFull()

isFull() : bool

Whether the buffer has reached its target.

Returns

bool —

TRUE when Packer::flush() should be called.

shouldStoreAlone()

shouldStoreAlone(int  $size) : bool

Whether a frame should bypass packing entirely.

A frame at or above the target gains nothing from being batched and would push every frame behind it into a second pack, so it is stored as its own object.

Parameters

int $size

Encoded frame size in bytes.

Returns

bool —

TRUE when the frame should be stored standalone.

add()

add(string  $hash, string  $bytes, array  $meta = []) : void

Adds a frame to the buffer.

Parameters

string $hash

Content address of the decoded frame.

string $bytes

The encoded frame, after compression and sealing.

array $meta

What the frame's index record needs and the object bytes cannot supply: raw, codec, cipher and optionally dictionary, parent and depth. Written into the pack's directory verbatim.

Throws

\InvalidArgumentException

When the hash is not a valid digest, the frame is empty, or the frame belongs in its own object.

Returns

void —

flush()

flush() : array<string,mixed>|null

Seals the buffer into a pack and clears it.

The pack id is the content address of the whole object, directory included. The directory is derived from the frames and their order, so two flushes producing the same content produce the same object and the store deduplicates whole packs as well as frames.

Returns

array|null —

Keys id (the object's content address), bytes (the frames followed by the directory) and entries (hash, offset, length and the metadata the frame was added with), or NULL when nothing was buffered.

discard()

discard() : void

Discards the buffer without producing a pack.

Used when a flush fails and the frames will be rebuilt from the journal rather than retried from memory.

Returns

void —

extract()

extract(string  $pack, int  $offset, int  $length) : string

Extracts one frame from a pack's bytes.

Parameters

string $pack

The pack bytes.

int $offset

Byte offset of the frame.

int $length

Frame length in bytes.

Throws

\RuntimeException

When the range falls outside the pack, which means the index and the object disagree.

Returns

string —

The encoded frame.