HMath.DeepLearning.Precision¶

Description

Explicit INT8 quantization and software FP16/BF16 matrix multiplication. Integer GEMM accumulates exact integers, then applies scales; it rejects signed-32-bit accumulator overflow. Half inputs round ties to even, with FP32 accumulation and one final conversion to Double. Mixed precision retains Double master weights; unscale/overflow decisions happen before any update.

On this page¶

API	Purpose
Quantization	Integer quantization policy: Symmetric around zero or Asymmetric with a learned zero point.
Symmetric	Declaration and type contract below.
Asymmetric	Declaration and type contract below.
QuantizedTensor	Opaque Int8 tensor with shape,optional channel axis,scales,and zero points.
quantize	Calibrate finite nonempty values per tensor or selected axis.
dequantize	Decode owned INT8 values using scale*(q-zeroPoint), rejecting floating arithmetic overflow.
quantizedValues	Return signed INT8 storage in logical row-major order.
quantizationScales	Return one positive scale per calibrated channel, or a single per-tensor scale.
quantizationZeroPoints	Return integer zero offsets under the selected symmetric/asymmetric calibration.
int8Gemm	Multiply two INT8 matrices with exact integer accumulation; supports per-row A/per-column B or per-tensor scales.
Precision	Mixed-precision storage/rounding selector FP16 or BF16; operations report finite/overflow conditions through the checked precision API.
FP16	Declaration and type contract below.
BF16	Declaration and type contract below.
reducedGemm	Round operands to the chosen 16-bit format, accumulate in Float32, and return checked Double values.
LossScaler	Dynamic mixed-precision loss-scale state with finite-step growth tracking.
LossScaler	Declaration and type contract below.
lossScale	Positive scaling factor applied to loss/gradients in mixed-precision training.
scaleGrowthInterval	Number of consecutive finite updates before loss-scale growth.
finiteSteps	Current count of consecutive finite updates since the last reset/growth.
initialLossScaler	Create a positive finite loss scale and positive growth interval; zero accumulated finite steps.
scaleLoss	Multiply a finite loss by the checked current scale; overflow is a structured failure.
unscaleGradients	False means overflow: every caller must skip the entire optimizer update.
updateLossScaler	Halve scale after overflow; double after growthInterval finite updates, with a bounded maximum.
gradientNorm	Compute a checked scaled Euclidean norm; unrepresentable final norms fail explicitly.
clipGradients	Clip norm through the shared overflow-safe automatic-differentiation gradient utility.

Documentation

Quantization

data Quantization Source #

Integer quantization policy: Symmetric around zero or Asymmetric with a learned zero point.

Constructors

Symmetric
Asymmetric

Instances

Instances details

Show Quantization Source #
Instance details Defined in HMath.DeepLearning.Precision Methods showsPrec :: Int -> Quantization -> ShowS # show :: Quantization -> String # showList :: [Quantization] -> ShowS #
Eq Quantization Source #
Instance details Defined in HMath.DeepLearning.Precision Methods (==) :: Quantization -> Quantization -> Bool # (/=) :: Quantization -> Quantization -> Bool #

QuantizedTensor

data QuantizedTensor Source #

Opaque Int8 tensor with shape,optional channel axis,scales,and zero points. Construct through the checked quantization API.

Instances

Instances details

Show QuantizedTensor Source #
Instance details Defined in HMath.DeepLearning.Precision Methods showsPrec :: Int -> QuantizedTensor -> ShowS # show :: QuantizedTensor -> String # showList :: [QuantizedTensor] -> ShowS #
Eq QuantizedTensor Source #
Instance details Defined in HMath.DeepLearning.Precision Methods (==) :: QuantizedTensor -> QuantizedTensor -> Bool # (/=) :: QuantizedTensor -> QuantizedTensor -> Bool #

quantize

quantize :: Quantization -> Maybe Int -> Tensor Double -> Either MathError QuantizedTensor Source #

Calibrate finite nonempty values per tensor or selected axis. Symmetric uses [-127,127], asymmetric [-128,127], with ties-to-even rounding.

dequantize

dequantize :: QuantizedTensor -> Either MathError (Tensor Double) Source #

Decode owned INT8 values using scale*(q-zeroPoint), rejecting floating arithmetic overflow.

quantizedValues

quantizedValues :: QuantizedTensor -> [Int8] Source #

Return signed INT8 storage in logical row-major order.

quantizationScales

quantizationScales :: QuantizedTensor -> [Double] Source #

Return one positive scale per calibrated channel, or a single per-tensor scale.

quantizationZeroPoints

quantizationZeroPoints :: QuantizedTensor -> [Int] Source #

Return integer zero offsets under the selected symmetric/asymmetric calibration.

int8Gemm

int8Gemm :: QuantizedTensor -> QuantizedTensor -> Either MathError (Tensor Double) Source #

Multiply two INT8 matrices with exact integer accumulation; supports per-row A/per-column B or per-tensor scales.

Precision

data Precision Source #

Mixed-precision storage/rounding selector FP16 or BF16; operations report finite/overflow conditions through the checked precision API.

Constructors

FP16
BF16

Instances

Instances details

Show Precision Source #
Instance details Defined in HMath.DeepLearning.Precision Methods showsPrec :: Int -> Precision -> ShowS # show :: Precision -> String # showList :: [Precision] -> ShowS #
Eq Precision Source #
Instance details Defined in HMath.DeepLearning.Precision Methods (==) :: Precision -> Precision -> Bool # (/=) :: Precision -> Precision -> Bool #

reducedGemm

reducedGemm :: Precision -> Tensor Double -> Tensor Double -> Either MathError (Tensor Double) Source #

Round operands to the chosen 16-bit format, accumulate in Float32, and return checked Double values.

LossScaler

data LossScaler Source #

Dynamic mixed-precision loss-scale state with finite-step growth tracking.

Constructors

LossScaler
Fields lossScale :: Double Positive scaling factor applied to loss/gradients in mixed-precision training. scaleGrowthInterval :: Int Number of consecutive finite updates before loss-scale growth. finiteSteps :: Int Current count of consecutive finite updates since the last reset/growth.

Instances

Instances details

Show LossScaler Source #
Instance details Defined in HMath.DeepLearning.Precision Methods showsPrec :: Int -> LossScaler -> ShowS # show :: LossScaler -> String # showList :: [LossScaler] -> ShowS #
Eq LossScaler Source #
Instance details Defined in HMath.DeepLearning.Precision Methods (==) :: LossScaler -> LossScaler -> Bool # (/=) :: LossScaler -> LossScaler -> Bool #

initialLossScaler

initialLossScaler :: Double -> Int -> Either MathError LossScaler Source #

Create a positive finite loss scale and positive growth interval; zero accumulated finite steps.

scaleLoss

scaleLoss :: LossScaler -> Double -> Either MathError Double Source #

Multiply a finite loss by the checked current scale; overflow is a structured failure.

unscaleGradients

unscaleGradients :: LossScaler -> [Double] -> Either MathError (Bool, [Double]) Source #

False means overflow: every caller must skip the entire optimizer update. Gradients are returned only when every scaled gradient is finite.

updateLossScaler

updateLossScaler :: Bool -> LossScaler -> Either MathError LossScaler Source #

Halve scale after overflow; double after growthInterval finite updates, with a bounded maximum.

gradientNorm

gradientNorm :: [Double] -> Either MathError Double Source #

Compute a checked scaled Euclidean norm; unrepresentable final norms fail explicitly.

clipGradients

clipGradients :: Double -> [Double] -> Either MathError [Double] Source #

Clip norm through the shared overflow-safe automatic-differentiation gradient utility.