activationBound

function activationBound(shape: NetworkShape, weights: Float32Array, low: Float32Array, high: Float32Array): number

import { activationBound } from '@driftengine/texture';

Parameters

In depth

What a consumer asks before it chooses half precision. Past 65,504 the format has no finite value, and a partial sum can pass that on its way to a total that does not — so this bounds each neuron by its bias plus the sum of every weight times the largest magnitude its input can take, which covers every prefix of the sum at once. Intervals are carried through the layers, and a rectifier clips them, which is what keeps the bound from growing without cause.