HMath.Optimization.Common

Module index · Numerical conventions

Description

Shared checked Double objective contracts. Finite deterministic callbacks return structured failures. Budgets count objectivegradientHessian calls, including line-search trials, rather than only accepted steps.

On this page

API	Purpose
Objective	Scalar objective function over a parameter vector, supplied by the optimization caller.
Gradient	Caller-supplied gradient components in parameter order; dimension/finite checks occur in the consuming optimizer.
Hessian	Caller-supplied second-derivative matrix in parameter order, consumed by checked Newton methods.
OptimizeOptions	Finite optimization work budgets,tolerance,and method step rate.
OptimizeOptions	Declaration and type contract below.
optMaxIterations	Maximum optimization iterations.
optMaxEvaluations	Maximum objective/evaluation work budget.
optTolerance	Positive finite method-specific stopping tolerance.
optLearningRate	Positive finite step-size/rate where used by the method.
defaultOptimizeOptions	Default optimization options: 1000 iterations, 20000 evaluations, tolerance 1e-8 and learning rate 0.01.
OptimizeResult	Accepted parameters and diagnostics. optGradientNorm is the Euclidean gradient norm for smooth methods, the projected/proximal mapping for proximal methods, and a method-specific geometric stopping residual for derivative-free methods.
OptimizeResult	Declaration and type contract below.
optPoint	Returned parameter vector at the accepted estimate.
optValue	Objective value at optPoint.
optGradientNorm	Gradient,projected/proximal,or geometric stopping residual as defined by the selected method.
optIterations	Completed optimization iterations.
optEvaluations	Recorded objective/evaluation count.
optConverged	Whether the selected method met its stopping criterion; this does not certify a global optimum.
checkedOptions	Require finite positive tolerancerate and positive iterationevaluation budgets.
checkedPoint	Reject NaN/infinity, including finite-input arithmetic overflow caught at solver boundaries.
checkedValue	Evaluate a deterministic objective at finite coordinates and reject a nonfinite value.
checkedGradient	Evaluate a finite gradient of exactly the input dimension.
dot	Internal same-length dot kernel; public algorithms validate shapes and finite arithmetic at their boundaries.
norm	Scaled Euclidean norm.
add	Internal same-length vector kernels; shape validation belongs to the public checked caller.
sub	Internal same-length vector kernels; shape validation belongs to the public checked caller.
scale	Multiply each vector component by a scalar.
identity	Dense identity matrix for a checked nonnegative dimension.
matVec	Dense row-major matrix-vector kernel for shapes already validated by the caller.
transpose	The transpose function transposes the rows and columns of its argument.
matMul	Dense row-major matrix multiplication for compatible checked inner dimensions.
outer	Outer product preserving input coordinate order.
solveDense	Checked square dense LU solve; singular and nonfinite systems return MathError.
result	Build diagnostics from a final objective and gradient, charging both callbacks.
checkBudget	Reject a proposed evaluation total that exceeds the supplied limit; equality is permitted.
LearningRate	Selected constant or scheduled learning-rate policy, with method-specific parameters validated by the schedule function.
ConstantRate	Declaration and type contract below.
StepDecay	Declaration and type contract below.
ExponentialDecay	Declaration and type contract below.
CosineAnnealing	Declaration and type contract below.
Warmup	Declaration and type contract below.
learningRate	Evaluate a nonnegative explicit schedule at a zero-based step; warmup scales the initial steps linearly.
EarlyStopping	Explicit monitoring state retaining the best score,nonimproving count,and stop flag; it does not mutate an optimizer automatically.
EarlyStopping	Declaration and type contract below.
bestScore	Best monitored score so far, or Nothing before the first observation.
staleSteps	Consecutive monitoring updates without sufficient improvement.
stopped	Whether early-stopping patience has been exhausted.
earlyStopping	Track strict improvement beyond minDelta; stop after patience stale observations.
LineSearch	Step selection policy: Armijo backtracking or strong-Wolfe conditions.
Armijo	Declaration and type contract below.
StrongWolfe	Declaration and type contract below.
lineSearch	Backtracking Armijo or bracket-and-zoom strong Wolfe (c1=1e-4,c2=.9).

Documentation

Objective

type Objective = [Double] -> Either MathError Double Source #

Scalar objective function over a parameter vector, supplied by the optimization caller.

Gradient

type Gradient = [Double] -> Either MathError [Double] Source #

Caller-supplied gradient components in parameter order; dimension/finite checks occur in the consuming optimizer.

Hessian

type Hessian = [Double] -> Either MathError [[Double]] Source #

Caller-supplied second-derivative matrix in parameter order, consumed by checked Newton methods.

OptimizeOptions

data OptimizeOptions Source #

Finite optimization work budgets,tolerance,and method step rate. Start from defaultOptimizeOptions.

Constructors

OptimizeOptions
Fields optMaxIterations :: Int Maximum optimization iterations. optMaxEvaluations :: Int Maximum objective/evaluation work budget. optTolerance :: Double Positive finite method-specific stopping tolerance. optLearningRate :: Double Positive finite step-size/rate where used by the method.

Instances

Instances details

Show OptimizeOptions Source #
Instance details Defined in HMath.Optimization.Common Methods showsPrec :: Int -> OptimizeOptions -> ShowS # show :: OptimizeOptions -> String # showList :: [OptimizeOptions] -> ShowS #
Eq OptimizeOptions Source #
Instance details Defined in HMath.Optimization.Common Methods (==) :: OptimizeOptions -> OptimizeOptions -> Bool # (/=) :: OptimizeOptions -> OptimizeOptions -> Bool #

defaultOptimizeOptions

defaultOptimizeOptions :: OptimizeOptions Source #

Default optimization options: 1000 iterations, 20000 evaluations, tolerance 1e-8 and learning rate 0.01. Customize with record updates; algorithms validate the selected settings.

OptimizeResult

data OptimizeResult Source #

Accepted parameters and diagnostics. optGradientNorm is the Euclidean gradient norm for smooth methods, the projected/proximal mapping for proximal methods, and a method-specific geometric stopping residual for derivative-free methods. Stochastic searches cannot certify global optimality.

Constructors

OptimizeResult

Fields optPoint :: [Double] Returned parameter vector at the accepted estimate. optValue :: Double Objective value at optPoint. optGradientNorm :: Double Gradient,projected/proximal,or geometric stopping residual as defined by the selected method. optIterations :: Int Completed optimization iterations. optEvaluations :: Int Recorded objective/evaluation count. optConverged :: Bool Whether the selected method met its stopping criterion; this does not certify a global optimum.

Instances

Instances details

Show OptimizeResult Source #
Instance details Defined in HMath.Optimization.Common Methods showsPrec :: Int -> OptimizeResult -> ShowS # show :: OptimizeResult -> String # showList :: [OptimizeResult] -> ShowS #
Eq OptimizeResult Source #
Instance details Defined in HMath.Optimization.Common Methods (==) :: OptimizeResult -> OptimizeResult -> Bool # (/=) :: OptimizeResult -> OptimizeResult -> Bool #

checkedOptions

checkedOptions :: OptimizeOptions -> Either MathError () Source #

Require finite positive tolerancerate and positive iterationevaluation budgets.

checkedPoint

checkedPoint :: [Double] -> Either MathError () Source #

Reject NaN/infinity, including finite-input arithmetic overflow caught at solver boundaries.

checkedValue

checkedValue :: Objective -> [Double] -> Either MathError Double Source #

Evaluate a deterministic objective at finite coordinates and reject a nonfinite value.

checkedGradient

checkedGradient :: Gradient -> [Double] -> Either MathError [Double] Source #

Evaluate a finite gradient of exactly the input dimension.

dot

dot :: [Double] -> [Double] -> Double Source #

Internal same-length dot kernel; public algorithms validate shapes and finite arithmetic at their boundaries.

norm

norm :: [Double] -> Double Source #

Scaled Euclidean norm. A mathematical norm beyond Double range remains positive infinity.

add

add :: [Double] -> [Double] -> [Double] Source #

Internal same-length vector kernels; shape validation belongs to the public checked caller.

sub

sub :: [Double] -> [Double] -> [Double] Source #

Internal same-length vector kernels; shape validation belongs to the public checked caller.

scale

scale :: Double -> [Double] -> [Double] Source #

Multiply each vector component by a scalar.

identity

identity :: Int -> [[Double]] Source #

Dense identity matrix for a checked nonnegative dimension.

matVec

matVec :: [[Double]] -> [Double] -> [Double] Source #

Dense row-major matrix-vector kernel for shapes already validated by the caller.

transpose

transpose :: [[a]] -> [[a]] #

The transpose function transposes the rows and columns of its argument.

Laziness

Expand

transpose is lazy in its elements

>>> take 1 (transpose ['a' : undefined, 'b' : undefined])
["ab"]

Examples

Expand

>>> transpose [[1,2,3],[4,5,6]]
[[1,4],[2,5],[3,6]]

If some of the rows are shorter than the following rows, their elements are skipped:

>>> transpose [[10,11],[20],[],[30,31,32]]
[[10,20,30],[11,31],[32]]

For this reason the outer list must be finite; otherwise transpose hangs:

>>> transpose (repeat [])
* Hangs forever *

matMul

matMul :: [[Double]] -> [[Double]] -> [[Double]] Source #

Dense row-major matrix multiplication for compatible checked inner dimensions.

outer

outer :: [Double] -> [Double] -> [[Double]] Source #

Outer product preserving input coordinate order.

solveDense

solveDense :: [[Double]] -> [Double] -> Either MathError [Double] Source #

Checked square dense LU solve; singular and nonfinite systems return MathError.

result

result :: Int -> Int -> Bool -> Objective -> Gradient -> [Double] -> Either MathError OptimizeResult Source #

Build diagnostics from a final objective and gradient, charging both callbacks.

checkBudget

checkBudget :: OptimizeOptions -> Int -> Either MathError () Source #

Reject a proposed evaluation total that exceeds the supplied limit; equality is permitted.

LearningRate

data LearningRate Source #

Selected constant or scheduled learning-rate policy, with method-specific parameters validated by the schedule function.

Constructors

ConstantRate Double
StepDecay Double Int Double
ExponentialDecay Double Double
CosineAnnealing Double Double Int
Warmup Int LearningRate

Instances

Instances details

Show LearningRate Source #
Instance details Defined in HMath.Optimization.Common Methods showsPrec :: Int -> LearningRate -> ShowS # show :: LearningRate -> String # showList :: [LearningRate] -> ShowS #
Eq LearningRate Source #
Instance details Defined in HMath.Optimization.Common Methods (==) :: LearningRate -> LearningRate -> Bool # (/=) :: LearningRate -> LearningRate -> Bool #

learningRate

learningRate :: LearningRate -> Int -> Either MathError Double Source #

Evaluate a nonnegative explicit schedule at a zero-based step; warmup scales the initial steps linearly.

EarlyStopping

data EarlyStopping Source #

Explicit monitoring state retaining the best score,nonimproving count,and stop flag; it does not mutate an optimizer automatically.

Constructors

EarlyStopping
Fields bestScore :: Maybe Double Best monitored score so far, or Nothing before the first observation. staleSteps :: Int Consecutive monitoring updates without sufficient improvement. stopped :: Bool Whether early-stopping patience has been exhausted.

Instances

Instances details

Show EarlyStopping Source #
Instance details Defined in HMath.Optimization.Common Methods showsPrec :: Int -> EarlyStopping -> ShowS # show :: EarlyStopping -> String # showList :: [EarlyStopping] -> ShowS #
Eq EarlyStopping Source #
Instance details Defined in HMath.Optimization.Common Methods (==) :: EarlyStopping -> EarlyStopping -> Bool # (/=) :: EarlyStopping -> EarlyStopping -> Bool #

earlyStopping

earlyStopping :: Int -> Double -> Double -> EarlyStopping -> Either MathError EarlyStopping Source #

Track strict improvement beyond minDelta; stop after patience stale observations.

LineSearch

data LineSearch Source #

Step selection policy: Armijo backtracking or strong-Wolfe conditions.

Constructors

Armijo
StrongWolfe

Instances

Instances details

Show LineSearch Source #
Instance details Defined in HMath.Optimization.Common Methods showsPrec :: Int -> LineSearch -> ShowS # show :: LineSearch -> String # showList :: [LineSearch] -> ShowS #
Eq LineSearch Source #
Instance details Defined in HMath.Optimization.Common Methods (==) :: LineSearch -> LineSearch -> Bool # (/=) :: LineSearch -> LineSearch -> Bool #

lineSearch

lineSearch :: LineSearch -> Int -> Objective -> Gradient -> [Double] -> [Double] -> Double -> [Double] -> Either MathError (Double, Double, [Double], Int) Source #

Backtracking Armijo or bracket-and-zoom strong Wolfe (c1=1e-4,c2=.9). The return includes all objective/gradient trials; descent is required.