DEFAULT_SIZE
DEFAULT_SIZE = 65536
Block size in bytes.
64 KiB. See the class docblock for the measurements this comes from.
Splits a file into fixed blocks so a change costs the blocks it touched.
Whole-object storage would re-upload a 256 MiB video because two bytes of metadata changed. Blocks make the cost proportional to the edit: hash each one, store the ones that are new, and keep an ordered list of digests as the file's version.
64 KiB, and the size was measured rather than picked. On a 256 MiB file, a 2 MiB in-place edit costs 2.1 MiB at 64 KiB blocks against 3.0 MiB at 1 MiB blocks and 8.0 MiB at 4 MiB, and hashing the whole file takes 0.40 s either way - 645 MB/s. Smaller blocks would keep shrinking the patch and start costing more in per-block overhead than they save.
What fixed blocks cannot do, stated plainly. They cannot follow an insertion. Inserting 2 MiB
at the front shifts every block after it, so 60.8% of the file looks changed at every block size.
That is what ShiftDetector is for: a high changed ratio is the signature of a shift rather than an
edit, and it is reported rather than silently paid for. Content-defined chunking is the answer to
that case and costs 4.29 MB/s, which is 150x slower, so it is never a default.
split(resource $stream) : \Generator<int,array{index: int, offset: int, hash: string, bytes: string}>
Splits an open stream, yielding each block as it is read.
A generator, so splitting a 256 MiB file holds one block in memory rather than the file.
| resource | $stream | An open, readable stream positioned where splitting should start. |
When the argument is not a stream.
When a read fails.
Each block's position, content address and content, in order.
secondsFor(int $bytes) : float
How long splitting a file of a given size takes on the reference host.
Printed next to the file-capture switch, because a first capture of an 80 GB media tree is a number an operator should see before turning it on rather than after.
| int | $bytes | File size. |
Seconds.