Optimization and differentiation: API index

Find an operation by its purpose. Entries link to the complete signature and contract.

HMath.Autodiff

API	Purpose
Dual	A tangent may itself be a dual number.
Dual	Declaration and type contract below.
primal	Extract the primal value from Dual value tangent without evaluating a derivative.
tangent	Extract the directional derivative seed/result from a Dual value.
forward	Evaluate a scalar dual function with unit tangent; returns primal and first derivative.
forwardJVP	Evaluate a vector dual function along a checked same-length direction; every output must be finite.
Expr	Opaque reverse-mode differentiable scalar expression,composed from explicit inputs and operations.
variable	Declare a zero-based graph input. compileGraph validates the declared input count.
constant	Embed a literal; nonfinite literals are rejected when a graph is compiled.
customOperation	Supply one local partial derivative per argument.
customUnary	First-order scalar custom rule; the second callback is the analytic local derivative.
detach	Preserve the primal value while removing every reverse edge through this expression.
stopGradient	Preserve the primal value while removing every reverse edge through this expression.
checkpoint	Store a graph-building recipe instead of the internal tape; replay it during reverse propagation.
CompiledGraph	Opaque reusable automatic-differentiation computation graph with explicit input/output structure.
compileGraph	Compile an immutable reusable graph with n inputs and a hard limit of one million scalar nodes.
evaluateGraph	Re-evaluate a compiled graph at exactly n finite inputs, without requesting gradients.
gradient	Return a fresh reverse gradient.
valueAndGradient	Build a dynamic graph and return its checked value and reverse gradient; repeated input leaves accumulate.
differentiate	Construct a symbolic derivative with respect to one graph input.
hessian	Return input-by-input symbolic second derivatives; all evaluated derivatives must be finite.
jacobian	Return output-by-input Jacobian rows, including shape-preserving empty output handling.
jvp	Multiply the automatic Jacobian by a finite input direction; returns one value per output.
vjp	Multiply an output cotangent by the automatic Jacobian; returns one value per input.
hvp	Multiply the symbolic Hessian by a checked finite input direction.
GradientState	Explicit gradient accumulation owner; evaluation does not mutate it implicitly.
GradientState	Declaration and type contract below.
accumulatedGradient	Gradient components currently retained by explicit GradientState accumulation; reverse evaluation alone does not mutate it.
zeroGradients	Create an explicit zero accumulation buffer for a nonnegative input count.
accumulateGradients	Add one same-length finite gradient without modifying the previous state.
resetGradients	Return a zero buffer of the original size; existing state remains reusable.
gradientHook	Apply an explicit checked transformation; changing gradient length is rejected.
clipGradient	Clip Euclidean norm while preserving direction, including vectors whose unscaled norm exceeds Double range.
gradientCheck	Returns maximum absolute discrepancy.
complexGradient	Differentiate a real objective of [real,imaginary] and return its Wirtinger derivative with respect to conjugate z.
differentiableStep	Take a symbolic gradient-descent step at the supplied expressions; local substitution preserves outer derivatives across repeated steps.
ADTensor	Row-major symbolic tensors.
adTensor	Construct a checked row-major symbolic tensor.
adValues	Return logical row-major graph expressions, preserving shared input identifiers.
adShape	Return tensor dimensions, with [] denoting a scalar.
adBroadcast	Broadcast through explicit repeated scalar graph leaves; reverse gradients sum over repetitions.
adAdd	Broadcast and combine symbolic tensors using the existing checked tensor shape contract.
adMultiply	Broadcast and combine symbolic tensors using the existing checked tensor shape contract.
adMatmul	Symbolic batched matrix multiplication with checked contraction dimensions.
adContract	Contract selected axes with matching sizes; uncontracted dimensions retain tensor order.
adSum	Reduce every tensor element to one symbolic scalar.

HMath.Optimization.Common

API	Purpose
Objective	Scalar objective function over a parameter vector, supplied by the optimization caller.
Gradient	Caller-supplied gradient components in parameter order; dimension/finite checks occur in the consuming optimizer.
Hessian	Caller-supplied second-derivative matrix in parameter order, consumed by checked Newton methods.
OptimizeOptions	Finite optimization work budgets,tolerance,and method step rate.
OptimizeOptions	Declaration and type contract below.
optMaxIterations	Maximum optimization iterations.
optMaxEvaluations	Maximum objective/evaluation work budget.
optTolerance	Positive finite method-specific stopping tolerance.
optLearningRate	Positive finite step-size/rate where used by the method.
defaultOptimizeOptions	Default optimization options: 1000 iterations, 20000 evaluations, tolerance 1e-8 and learning rate 0.01.
OptimizeResult	Accepted parameters and diagnostics. optGradientNorm is the Euclidean gradient norm for smooth methods, the projected/proximal mapping for proximal methods, and a method-specific geometric stopping residual for derivative-free methods.
OptimizeResult	Declaration and type contract below.
optPoint	Returned parameter vector at the accepted estimate.
optValue	Objective value at optPoint.
optGradientNorm	Gradient,projected/proximal,or geometric stopping residual as defined by the selected method.
optIterations	Completed optimization iterations.
optEvaluations	Recorded objective/evaluation count.
optConverged	Whether the selected method met its stopping criterion; this does not certify a global optimum.
checkedOptions	Require finite positive tolerancerate and positive iterationevaluation budgets.
checkedPoint	Reject NaN/infinity, including finite-input arithmetic overflow caught at solver boundaries.
checkedValue	Evaluate a deterministic objective at finite coordinates and reject a nonfinite value.
checkedGradient	Evaluate a finite gradient of exactly the input dimension.
dot	Internal same-length dot kernel; public algorithms validate shapes and finite arithmetic at their boundaries.
norm	Scaled Euclidean norm.
add	Internal same-length vector kernels; shape validation belongs to the public checked caller.
sub	Internal same-length vector kernels; shape validation belongs to the public checked caller.
scale	Multiply each vector component by a scalar.
identity	Dense identity matrix for a checked nonnegative dimension.
matVec	Dense row-major matrix-vector kernel for shapes already validated by the caller.
transpose	The transpose function transposes the rows and columns of its argument.
matMul	Dense row-major matrix multiplication for compatible checked inner dimensions.
outer	Outer product preserving input coordinate order.
solveDense	Checked square dense LU solve; singular and nonfinite systems return MathError.
result	Build diagnostics from a final objective and gradient, charging both callbacks.
checkBudget	Reject a proposed evaluation total that exceeds the supplied limit; equality is permitted.
LearningRate	Selected constant or scheduled learning-rate policy, with method-specific parameters validated by the schedule function.
ConstantRate	Declaration and type contract below.
StepDecay	Declaration and type contract below.
ExponentialDecay	Declaration and type contract below.
CosineAnnealing	Declaration and type contract below.
Warmup	Declaration and type contract below.
learningRate	Evaluate a nonnegative explicit schedule at a zero-based step; warmup scales the initial steps linearly.
EarlyStopping	Explicit monitoring state retaining the best score,nonimproving count,and stop flag; it does not mutate an optimizer automatically.
EarlyStopping	Declaration and type contract below.
bestScore	Best monitored score so far, or Nothing before the first observation.
staleSteps	Consecutive monitoring updates without sufficient improvement.
stopped	Whether early-stopping patience has been exhausted.
earlyStopping	Track strict improvement beyond minDelta; stop after patience stale observations.
LineSearch	Step selection policy: Armijo backtracking or strong-Wolfe conditions.
Armijo	Declaration and type contract below.
StrongWolfe	Declaration and type contract below.
lineSearch	Backtracking Armijo or bracket-and-zoom strong Wolfe (c1=1e-4,c2=.9).

HMath.Optimization.Constrained

API	Purpose
Constraint	Scalar Equality or LessEqual constraint over the parameter vector; inequality functions use the solver's <=0 convention.
Equality	Declaration and type contract below.
LessEqual	Declaration and type contract below.
feasibility	Return maximum absolute equality or positive inequality violation, with checked values and gradients.
PenaltyMethod	Constrained-optimization strategy selecting quadratic penalty,log barrier,augmented Lagrangian,or interior-point treatment.
QuadraticPenalty	Declaration and type contract below.
LogBarrier	Declaration and type contract below.
InteriorPoint	Declaration and type contract below.
AugmentedLagrangian	Declaration and type contract below.
constrainedMinimize	Outer continuation or multiplier updates around the named constrained objective.
LPResult	Linear-program primal solution,objective value,retained basis indices,and iteration count.
LPResult	Declaration and type contract below.
lpPoint	Returned primal variable vector of the linear program.
lpValue	Objective value at the returned LP point.
lpBasis	Zero-based basis indices retained by the simplex solution.
lpIterations	Number of simplex iterations performed.
linearProgramming	Primal two-phase simplex with Bland entering/leaving rules.
revisedSimplex	Revised primal simplex, A*x=b,x>=0.
quadraticProgramming	Strictly convex QP: min .5x'Hx+c'x, Ax<=b.
slsqp	Sequential least-squares QP with an analytic Hessian and exact linearized equality/active inequality constraints.
cobyla	Derivative-free constrained linear-approximation trust region.
weightedObjective	Compose nonnegative scalarization weights and checked objective values; at least one weight must be positive.
paretoFront	Return original indices of nondominated objective rows under componentwise minimization; duplicates remain nondominated.

HMath.Optimization.FirstOrder

API	Purpose
GradientMethod	First-order optimizer selector, including momentum and adaptive-moment variants.
GradientDescent	Declaration and type contract below.
Momentum	Declaration and type contract below.
Nesterov	Declaration and type contract below.
AdaGrad	Declaration and type contract below.
RMSProp	Declaration and type contract below.
AdaDelta	Declaration and type contract below.
Adam	Declaration and type contract below.
AdamW	Declaration and type contract below.
AMSGrad	Declaration and type contract below.
AdaMax	Declaration and type contract below.
Nadam	Declaration and type contract below.
RAdam	Declaration and type contract below.
OptimizerState	Opaque accumulated first-order optimizer moments/history.
initializeOptimizer	Allocate zero momentum/moment buffers for a nonnegative parameter count.
optimizerStep	Apply one checked stateful update using the supplied gradient and positive rate.
minimizeGradient	Minimize with explicit gradient/state and fixed learning rate.
minimizeGradientWithCallback	Observe each finite iterate after its objective/gradient evaluation.
stochasticGradient	One pass over explicit sample/batch gradients; ordering and shuffling are caller-owned, allowing exact replay.
miniBatchGradient	Average a nonempty explicit set of same-length checked sample gradients at the current parameters.
projectedGradient	Project every update and stop on the projected-gradient mapping, including constrained boundary optima.
proximalGradient	Apply prox(rate,point) after each gradient step; residual is the proximal-gradient mapping norm.
fista	Accelerated proximal gradient using the Beck-Teboulle t recurrence and a fresh mapping residual at accepted points.
coordinateDescent	Cyclic coordinates or blocks; every coordinate must occur exactly once.

HMath.Optimization.Global

API	Purpose
GlobalMethod	Seeded bounded stochastic search strategy.
SimulatedAnnealing	Declaration and type contract below.
DifferentialEvolution	Declaration and type contract below.
GeneticAlgorithm	Declaration and type contract below.
ParticleSwarm	Declaration and type contract below.
CMAES	Declaration and type contract below.
globalMinimize	Run a reproducible seeded stochastic algorithm in a finite box. size>=4; budget exhaustion returns the best finite incumbent with convergence=False.
basinHopping	Basin hopping uses genuine local Nelder–Mead descents followed by seeded Metropolis transitions.

HMath.Optimization.Local

API	Purpose
ScalarMethod	Bounded one-dimensional minimization strategy (golden section,Brent,or ternary search).
GoldenSection	Declaration and type contract below.
TernarySearch	Declaration and type contract below.
BrentSearch	Declaration and type contract below.
minimizeScalar	Minimize on a finite bracket a<b.
QuasiNewton	Quasi-Newton update selector such as BFGS,DFP,SR1,or limited-memory BFGS.
BFGS	Declaration and type contract below.
LBFGS	Declaration and type contract below.
DFP	Declaration and type contract below.
SR1	Declaration and type contract below.
minimizeQuasiNewton	Run the named inverse-Hessian secant update with safeguarded curvature and a strong-Wolfe descent search.
NewtonMethod	Newton/trust-region strategy selected for checked local minimization.
Newton	Declaration and type contract below.
NewtonCG	Declaration and type contract below.
TrustRegion	Declaration and type contract below.
Dogleg	Declaration and type contract below.
minimizeNewton	Use an analytic square Hessian with Newton/CG or trust-region acceptance; singular systems and collapsed regions fail.
nonlinearCG	Polak–Ribiere+ nonlinear CG with descent restart and strong Wolfe search.
ResidualFunction	Caller-supplied residual vector for nonlinear least-squares fitting.
JacobianFunction	Caller-supplied residual Jacobian with rows corresponding to residual components and columns to parameters.
LeastSquaresLoss	Robust residual penalty selector.
SquaredLoss	Declaration and type contract below.
HuberLoss	Declaration and type contract below.
CauchyLoss	Declaration and type contract below.
SoftL1	Declaration and type contract below.
leastSquares	lambda=0 selects Gauss–Newton, positive lambda selects LM.
nelderMead	Optimize an axis-initialized simplex with reflection, expansion, contraction, and shrink steps; report simplex spread.
powell	Powell direction-set method with bounded line minimization on [-radius,radius].