HMath.Autodiff

Module index · Numerical conventions

Description

Pure CPU automatic differentiation. Dual nests for higher orders. Expr compiles to an immutable, reusable reverse tape; dynamic graphs are ordinary Haskell functions building expressions at run time. Tapes own their saved primals, so no ambient mutable graph or lifetime is involved. Accumulation is explicit in GradientState; reverse evaluation itself never accumulates. Custom rules return a value and local partial derivatives (first order only). Checkpoints store the subgraph recipe and recompute its tape on backprop. Complex gradients use real coordinates: dLdz = (dLdx+i*dLdy)2 (Wirtinger).

On this page

API	Purpose
Dual	A tangent may itself be a dual number.
Dual	Declaration and type contract below.
primal	Extract the primal value from Dual value tangent without evaluating a derivative.
tangent	Extract the directional derivative seed/result from a Dual value.
forward	Evaluate a scalar dual function with unit tangent; returns primal and first derivative.
forwardJVP	Evaluate a vector dual function along a checked same-length direction; every output must be finite.
Expr	Opaque reverse-mode differentiable scalar expression,composed from explicit inputs and operations.
variable	Declare a zero-based graph input. compileGraph validates the declared input count.
constant	Embed a literal; nonfinite literals are rejected when a graph is compiled.
customOperation	Supply one local partial derivative per argument.
customUnary	First-order scalar custom rule; the second callback is the analytic local derivative.
detach	Preserve the primal value while removing every reverse edge through this expression.
stopGradient	Preserve the primal value while removing every reverse edge through this expression.
checkpoint	Store a graph-building recipe instead of the internal tape; replay it during reverse propagation.
CompiledGraph	Opaque reusable automatic-differentiation computation graph with explicit input/output structure.
compileGraph	Compile an immutable reusable graph with n inputs and a hard limit of one million scalar nodes.
evaluateGraph	Re-evaluate a compiled graph at exactly n finite inputs, without requesting gradients.
gradient	Return a fresh reverse gradient.
valueAndGradient	Build a dynamic graph and return its checked value and reverse gradient; repeated input leaves accumulate.
differentiate	Construct a symbolic derivative with respect to one graph input.
hessian	Return input-by-input symbolic second derivatives; all evaluated derivatives must be finite.
jacobian	Return output-by-input Jacobian rows, including shape-preserving empty output handling.
jvp	Multiply the automatic Jacobian by a finite input direction; returns one value per output.
vjp	Multiply an output cotangent by the automatic Jacobian; returns one value per input.
hvp	Multiply the symbolic Hessian by a checked finite input direction.
GradientState	Explicit gradient accumulation owner; evaluation does not mutate it implicitly.
GradientState	Declaration and type contract below.
accumulatedGradient	Gradient components currently retained by explicit GradientState accumulation; reverse evaluation alone does not mutate it.
zeroGradients	Create an explicit zero accumulation buffer for a nonnegative input count.
accumulateGradients	Add one same-length finite gradient without modifying the previous state.
resetGradients	Return a zero buffer of the original size; existing state remains reusable.
gradientHook	Apply an explicit checked transformation; changing gradient length is rejected.
clipGradient	Clip Euclidean norm while preserving direction, including vectors whose unscaled norm exceeds Double range.
gradientCheck	Returns maximum absolute discrepancy.
complexGradient	Differentiate a real objective of [real,imaginary] and return its Wirtinger derivative with respect to conjugate z.
differentiableStep	Take a symbolic gradient-descent step at the supplied expressions; local substitution preserves outer derivatives across repeated steps.
ADTensor	Row-major symbolic tensors.
adTensor	Construct a checked row-major symbolic tensor.
adValues	Return logical row-major graph expressions, preserving shared input identifiers.
adShape	Return tensor dimensions, with [] denoting a scalar.
adBroadcast	Broadcast through explicit repeated scalar graph leaves; reverse gradients sum over repetitions.
adAdd	Broadcast and combine symbolic tensors using the existing checked tensor shape contract.
adMultiply	Broadcast and combine symbolic tensors using the existing checked tensor shape contract.
adMatmul	Symbolic batched matrix multiplication with checked contraction dimensions.
adContract	Contract selected axes with matching sizes; uncontracted dimensions retain tensor order.
adSum	Reduce every tensor element to one symbolic scalar.

Documentation

Dual

data Dual a Source #

A tangent may itself be a dual number. Branches use the primal ordering.

Constructors

Dual a a

Instances

Instances details

(Floating a, Eq a) => Floating (Dual a) Source #
Instance details Defined in HMath.Autodiff Methods pi :: Dual a # exp :: Dual a -> Dual a # log :: Dual a -> Dual a # sqrt :: Dual a -> Dual a # (**) :: Dual a -> Dual a -> Dual a # logBase :: Dual a -> Dual a -> Dual a # sin :: Dual a -> Dual a # cos :: Dual a -> Dual a # tan :: Dual a -> Dual a # asin :: Dual a -> Dual a # acos :: Dual a -> Dual a # atan :: Dual a -> Dual a # sinh :: Dual a -> Dual a # cosh :: Dual a -> Dual a # tanh :: Dual a -> Dual a # asinh :: Dual a -> Dual a # acosh :: Dual a -> Dual a # atanh :: Dual a -> Dual a # log1p :: Dual a -> Dual a # expm1 :: Dual a -> Dual a # log1pexp :: Dual a -> Dual a # log1mexp :: Dual a -> Dual a #
Num a => Num (Dual a) Source #
Instance details Defined in HMath.Autodiff Methods (+) :: Dual a -> Dual a -> Dual a # (-) :: Dual a -> Dual a -> Dual a # (*) :: Dual a -> Dual a -> Dual a # negate :: Dual a -> Dual a # abs :: Dual a -> Dual a # signum :: Dual a -> Dual a # fromInteger :: Integer -> Dual a #
Fractional a => Fractional (Dual a) Source #
Instance details Defined in HMath.Autodiff Methods (/) :: Dual a -> Dual a -> Dual a # recip :: Dual a -> Dual a # fromRational :: Rational -> Dual a #
Show a => Show (Dual a) Source #
Instance details Defined in HMath.Autodiff Methods showsPrec :: Int -> Dual a -> ShowS # show :: Dual a -> String # showList :: [Dual a] -> ShowS #
Eq a => Eq (Dual a) Source #
Instance details Defined in HMath.Autodiff Methods (==) :: Dual a -> Dual a -> Bool # (/=) :: Dual a -> Dual a -> Bool #
Ord a => Ord (Dual a) Source #
Instance details Defined in HMath.Autodiff Methods compare :: Dual a -> Dual a -> Ordering # (<) :: Dual a -> Dual a -> Bool # (<=) :: Dual a -> Dual a -> Bool # (>) :: Dual a -> Dual a -> Bool # (>=) :: Dual a -> Dual a -> Bool # max :: Dual a -> Dual a -> Dual a # min :: Dual a -> Dual a -> Dual a #
Differentiable (Dual a) Source #
Instance details Defined in HMath.API.Algebra Associated Types type Differential (Dual a)   Instance details Defined in HMath.API.Algebra type Differential (Dual a) = a Methods differential :: Dual a -> Differential (Dual a) Source #
type Differential (Dual a) Source #
Instance details Defined in HMath.API.Algebra type Differential (Dual a) = a

primal

primal :: Dual a -> a Source #

Extract the primal value from Dual value tangent without evaluating a derivative. Example: primal (Dual 3 1) == 3.

tangent

tangent :: Dual a -> a Source #

Extract the directional derivative seed/result from a Dual value. Example: tangent (Dual 3 1) == 1.

forward

forward :: Num a => (Dual a -> Dual a) -> a -> (a, a) Source #

Evaluate a scalar dual function with unit tangent; returns primal and first derivative.

forwardJVP

forwardJVP :: ([Dual Double] -> [Dual Double]) -> [Double] -> [Double] -> Either MathError ([Double], [Double]) Source #

Evaluate a vector dual function along a checked same-length direction; every output must be finite.

Expr

data Expr Source #

Opaque reverse-mode differentiable scalar expression,composed from explicit inputs and operations. Compile/evaluate through the checked autodiff graph API.

Instances

Instances details

Floating Expr Source #
Instance details Defined in HMath.Autodiff Methods pi :: Expr # exp :: Expr -> Expr # log :: Expr -> Expr # sqrt :: Expr -> Expr # (**) :: Expr -> Expr -> Expr # logBase :: Expr -> Expr -> Expr # sin :: Expr -> Expr # cos :: Expr -> Expr # tan :: Expr -> Expr # asin :: Expr -> Expr # acos :: Expr -> Expr # atan :: Expr -> Expr # sinh :: Expr -> Expr # cosh :: Expr -> Expr # tanh :: Expr -> Expr # asinh :: Expr -> Expr # acosh :: Expr -> Expr # atanh :: Expr -> Expr # log1p :: Expr -> Expr # expm1 :: Expr -> Expr # log1pexp :: Expr -> Expr # log1mexp :: Expr -> Expr #
Num Expr Source #
Instance details Defined in HMath.Autodiff Methods (+) :: Expr -> Expr -> Expr # (-) :: Expr -> Expr -> Expr # (*) :: Expr -> Expr -> Expr # negate :: Expr -> Expr # abs :: Expr -> Expr # signum :: Expr -> Expr # fromInteger :: Integer -> Expr #
Fractional Expr Source #
Instance details Defined in HMath.Autodiff Methods (/) :: Expr -> Expr -> Expr # recip :: Expr -> Expr # fromRational :: Rational -> Expr #
Show Expr Source #
Instance details Defined in HMath.Autodiff Methods showsPrec :: Int -> Expr -> ShowS # show :: Expr -> String # showList :: [Expr] -> ShowS #

variable

variable :: Int -> Expr Source #

Declare a zero-based graph input. compileGraph validates the declared input count.

constant

constant :: Double -> Expr Source #

Embed a literal; nonfinite literals are rejected when a graph is compiled.

customOperation

customOperation :: ([Double] -> Either MathError (Double, [Double])) -> [Expr] -> Expr Source #

Supply one local partial derivative per argument. Rules must be deterministic and total; outputs and arity are checked.

customUnary

customUnary :: (Double -> Double) -> (Double -> Double) -> Expr -> Expr Source #

First-order scalar custom rule; the second callback is the analytic local derivative.

detach

detach :: Expr -> Expr Source #

Preserve the primal value while removing every reverse edge through this expression.

stopGradient

stopGradient :: Expr -> Expr Source #

Preserve the primal value while removing every reverse edge through this expression.

checkpoint

checkpoint :: ([Expr] -> Expr) -> [Expr] -> Expr Source #

Store a graph-building recipe instead of the internal tape; replay it during reverse propagation.

CompiledGraph

data CompiledGraph Source #

Opaque reusable automatic-differentiation computation graph with explicit input/output structure.

compileGraph

compileGraph :: Int -> Expr -> Either MathError CompiledGraph Source #

Compile an immutable reusable graph with n inputs and a hard limit of one million scalar nodes.

evaluateGraph

evaluateGraph :: CompiledGraph -> [Double] -> Either MathError Double Source #

Re-evaluate a compiled graph at exactly n finite inputs, without requesting gradients.

gradient

gradient :: ([Expr] -> Expr) -> [Double] -> Either MathError [Double] Source #

Return a fresh reverse gradient. Previous calls and explicit GradientState values are unaffected.

valueAndGradient

valueAndGradient :: ([Expr] -> Expr) -> [Double] -> Either MathError (Double, [Double]) Source #

Build a dynamic graph and return its checked value and reverse gradient; repeated input leaves accumulate.

differentiate

differentiate :: Int -> Expr -> Either MathError Expr Source #

Construct a symbolic derivative with respect to one graph input. Opaque custom first-order rules reject higher orders.

hessian

hessian :: ([Expr] -> Expr) -> [Double] -> Either MathError [[Double]] Source #

Return input-by-input symbolic second derivatives; all evaluated derivatives must be finite.

jacobian

jacobian :: ([Expr] -> [Expr]) -> [Double] -> Either MathError [[Double]] Source #

Return output-by-input Jacobian rows, including shape-preserving empty output handling.

jvp

jvp :: ([Expr] -> [Expr]) -> [Double] -> [Double] -> Either MathError [Double] Source #

Multiply the automatic Jacobian by a finite input direction; returns one value per output.

vjp

vjp :: ([Expr] -> [Expr]) -> [Double] -> [Double] -> Either MathError [Double] Source #

Multiply an output cotangent by the automatic Jacobian; returns one value per input.

hvp

hvp :: ([Expr] -> Expr) -> [Double] -> [Double] -> Either MathError [Double] Source #

Multiply the symbolic Hessian by a checked finite input direction.

GradientState

newtype GradientState Source #

Explicit gradient accumulation owner; evaluation does not mutate it implicitly. Use the accumulation API to retain/reset gradients.

Constructors

GradientState
Fields accumulatedGradient :: [Double] Gradient components currently retained by explicit GradientState accumulation; reverse evaluation alone does not mutate it.

Instances

Instances details

Show GradientState Source #
Instance details Defined in HMath.Autodiff Methods showsPrec :: Int -> GradientState -> ShowS # show :: GradientState -> String # showList :: [GradientState] -> ShowS #
Eq GradientState Source #
Instance details Defined in HMath.Autodiff Methods (==) :: GradientState -> GradientState -> Bool # (/=) :: GradientState -> GradientState -> Bool #

zeroGradients

zeroGradients :: Int -> Either MathError GradientState Source #

Create an explicit zero accumulation buffer for a nonnegative input count.

accumulateGradients

accumulateGradients :: GradientState -> [Double] -> Either MathError GradientState Source #

Add one same-length finite gradient without modifying the previous state.

resetGradients

resetGradients :: GradientState -> GradientState Source #

Return a zero buffer of the original size; existing state remains reusable.

gradientHook

gradientHook :: ([Double] -> Either MathError [Double]) -> GradientState -> Either MathError GradientState Source #

Apply an explicit checked transformation; changing gradient length is rejected.

clipGradient

clipGradient :: Double -> [Double] -> Either MathError [Double] Source #

Clip Euclidean norm while preserving direction, including vectors whose unscaled norm exceeds Double range.

gradientCheck

gradientCheck :: Double -> ([Expr] -> Expr) -> [Double] -> Either MathError Double Source #

Returns maximum absolute discrepancy. The numerical check is deliberately separate from automatic derivatives; h must be finite and strictly positive.

complexGradient

complexGradient :: ([Expr] -> Expr) -> Complex Double -> Either MathError (Complex Double) Source #

Differentiate a real objective of [real,imaginary] and return its Wirtinger derivative with respect to conjugate z.

differentiableStep

differentiableStep :: Double -> ([Expr] -> Expr) -> [Expr] -> Either MathError [Expr] Source #

Take a symbolic gradient-descent step at the supplied expressions; local substitution preserves outer derivatives across repeated steps.

ADTensor

data ADTensor Source #

Row-major symbolic tensors. Broadcasting is delegated to the checked tensor shape engine. Matrix/contraction rules are ordinary scalar graph operations, so reverse accumulation includes all repeated/broadcast input contributions.

Instances

Instances details

Show ADTensor Source #
Instance details Defined in HMath.Autodiff Methods showsPrec :: Int -> ADTensor -> ShowS # show :: ADTensor -> String # showList :: [ADTensor] -> ShowS #

adTensor

adTensor :: [Int] -> [Expr] -> Either MathError ADTensor Source #

Construct a checked row-major symbolic tensor. Shape product must match expression count.

adValues

adValues :: ADTensor -> [Expr] Source #

Return logical row-major graph expressions, preserving shared input identifiers.

adShape

adShape :: ADTensor -> [Int] Source #

Return tensor dimensions, with [] denoting a scalar.

adBroadcast

adBroadcast :: [Int] -> ADTensor -> Either MathError ADTensor Source #

Broadcast through explicit repeated scalar graph leaves; reverse gradients sum over repetitions.

adAdd

adAdd :: ADTensor -> ADTensor -> Either MathError ADTensor Source #

Broadcast and combine symbolic tensors using the existing checked tensor shape contract.

adMultiply

adMultiply :: ADTensor -> ADTensor -> Either MathError ADTensor Source #

Broadcast and combine symbolic tensors using the existing checked tensor shape contract.

adMatmul

adMatmul :: ADTensor -> ADTensor -> Either MathError ADTensor Source #

Symbolic batched matrix multiplication with checked contraction dimensions.

adContract

adContract :: [Int] -> [Int] -> ADTensor -> ADTensor -> Either MathError ADTensor Source #

Contract selected axes with matching sizes; uncontracted dimensions retain tensor order.

adSum

adSum :: ADTensor -> Expr Source #

Reduce every tensor element to one symbolic scalar.