FrameSize

interface FrameSize

import type { FrameSize } from '@driftengine/capture';

In depth

The engine never opens a file. A host hands it a FrameSource — a video element in a browser, a decoder on a native host, a folder of stills in a test — and the engine reads it one frame at a time, in order, into a buffer of its own. A minute of 1080p is eight gigabytes of pixels, so a clip is walked rather than held, and what a capture keeps is chosen as it goes past.

A frame may arrive later than it is asked for, since every browser decoder is asynchronous, so frameAt may answer a promise and the two readers here are async. A host with its frames already in hand answers without one and pays nothing: an awaited value that is not a promise costs a microtask, and this is a clip's pace rather than a frame's.

Frames are chosen by motion, not by number. What reconstruction needs is baseline: two views far enough apart to triangulate, and no more of them than the budget allows. So the frames kept are spaced by equal motion — a slow pan gives up most of its frames, a fast one keeps them — and a clip that never moves is one frame, which is what it holds. every is the other way to choose, for a caller that knows its clip: every Nth frame, under the same budget.

Motion is the mean absolute difference between one frame and the last, over the colours and not the alpha, which is a measure of how much the picture changed rather than of how far anything travelled. What it gives up is telling a pan from a light being switched on; what would make it wrong is a clip whose brightness swings without the camera moving, and the answer then is the feature matching of Task 9 rather than a cleverer average.

Properties