HMath.MachineLearning.Regression¶

Description

Dense reference estimators. Every design includes an unpenalized intercept; rows are observations. Fit diagnostics distinguish budget exhaustion from a coefficient/residual convergence test. Gaussian-process classification uses the logistic Laplace approximation, rather than exact Bayesian integration.

On this page¶

API	Purpose
Penalty	Minimize mean squared error divided by two plus L1 and/or squared L2 coefficient penalties.
Ordinary	Declaration and type contract below.
Ridge	Declaration and type contract below.
Lasso	Declaration and type contract below.
ElasticNet	Declaration and type contract below.
LinearModel	Affine predictor; coefficient zero is the intercept and remaining coefficients follow feature order.
coefficients	Intercept followed by slopes in input feature order.
fitLinear	Fit affine least squares or coordinate-descent elastic net.
predictLinear	Predict one finite response per query row using the fitted intercept and slopes.
polynomialFeatures	Build univariate columns x, x^2, ..., x^degree; the regression intercept supplies the constant.
fitWeighted	Fit affine least squares with nonnegative observation weights.
fitRobust	Huber IRLS with fixed positive residual threshold, including intercept.
BayesianLinear	Posterior-mean linear model together with coefficient covariance, returned by Bayesian regression fitting.
fitBayesian	Isotropic zero-mean Gaussian prior, observation precision beta.
bayesianMean	Posterior-mean fitted linear model.
bayesianCovariance	Posterior covariance matrix of linear coefficients.
RecursiveLS	Owned online regression coefficients and inverse information, with fixed forgetting factor.
recursiveLS	Initial inverse information delta I; forgetting factor in (0,1].
updateRecursiveLS	Assimilate one feature row/response using the matrix inversion lemma.
recursiveCoefficients	Current intercept and slopes; updating creates a new state.
GLMFamily	Bernoulli mean with logistic link, or nonnegative response mean with log link.
BinomialLogit	Declaration and type contract below.
PoissonLog	Declaration and type contract below.
GLM	Fitted generalized linear model with intercept and family-specific inverse link.
fitGLM	Damped Newton/IRLS with nonnegative slope ridge.
predictGLM	Return conditional means: probabilities for BinomialLogit, positive rates for PoissonLog.
MultinomialModel	Softmax class weights and sorted original integer labels.
fitMultinomial	Fit all class logits by full-batch gradient descent with slope ridge.
predictMultinomial	Return sorted (original label, probability) pairs for each observation.
Kernel	Positive-semidefinite linear, polynomial, or Gaussian radial basis kernel parameters.
LinearKernel	Declaration and type contract below.
PolynomialKernel	Declaration and type contract below.
RBFKernel	Declaration and type contract below.
kernel	Evaluate a checked PSD kernel on equal-length finite feature vectors.
SVM	Binary kernel decision state, retaining original training observations and signed dual weights.
fitSVM	Deterministic two-coordinate SMO for binary labels -1/+1.
decisionSVM	Return signed kernel margins; zero is the fitted decision boundary.
predictSVM	Classify margins as -1 or +1, with zero assigned +1.
GaussianProcess	Zero-mean Gaussian-process regression posterior with retained training covariance solve.
fitGaussianProcess	Fit a GP with strictly positive observation-noise variance and a checked PSD kernel.
predictGaussianProcess	Return posterior latent-function (mean, variance), excluding observation noise.
GPClassifier	Logistic Gaussian-process classifier fitted with a Laplace Gaussian posterior approximation.
fitGPClassifier	Newton mode fitting for 0/1 labels.
predictGPClassifier	Approximate class-one probabilities using the logistic Gaussian variance correction.
BayesFamily	Gaussian features, nonnegative multinomial counts, or binary Bernoulli features.
GaussianBayes	Declaration and type contract below.
MultinomialBayes	Declaration and type contract below.
BernoulliBayes	Declaration and type contract below.
NaiveBayes	Fitted class priors and family-specific independent feature likelihoods.
fitNaiveBayes	Fit class likelihoods.
predictNaiveBayes	Select the class with greatest log posterior; preserve the original integer labels.
Discriminant	Class priors, means, and regularized inverse covariance/log-determinant state.
fitDiscriminant	False = pooled covariance LDA; True = class covariance QDA.
predictDiscriminant	Classify with Gaussian discriminant log likelihoods using training covariance state.

Documentation

Penalty

data Penalty Source #

Minimize mean squared error divided by two plus L1 and/or squared L2 coefficient penalties.

Constructors

Ordinary
Ridge Double
Lasso Double
ElasticNet Double Double

Instances

Instances details

Show Penalty Source #
Instance details Defined in HMath.MachineLearning.Regression Methods showsPrec :: Int -> Penalty -> ShowS # show :: Penalty -> String # showList :: [Penalty] -> ShowS #
Eq Penalty Source #
Instance details Defined in HMath.MachineLearning.Regression Methods (==) :: Penalty -> Penalty -> Bool # (/=) :: Penalty -> Penalty -> Bool #

LinearModel

data LinearModel Source #

Affine predictor; coefficient zero is the intercept and remaining coefficients follow feature order.

Instances

Instances details

Show LinearModel Source #
Instance details Defined in HMath.MachineLearning.Regression Methods showsPrec :: Int -> LinearModel -> ShowS # show :: LinearModel -> String # showList :: [LinearModel] -> ShowS #
Eq LinearModel Source #
Instance details Defined in HMath.MachineLearning.Regression Methods (==) :: LinearModel -> LinearModel -> Bool # (/=) :: LinearModel -> LinearModel -> Bool #

coefficients

coefficients :: LinearModel -> [Double] Source #

Intercept followed by slopes in input feature order.

fitLinear

fitLinear :: MLConfig -> Penalty -> Dataset -> [Double] -> Either MathError (Fit LinearModel) Source #

Fit affine least squares or coordinate-descent elastic net. Ordinary/ridge use an SVD minimum-norm solve; iterative residual is coefficient displacement.

predictLinear

predictLinear :: LinearModel -> Dataset -> Either MathError [Double] Source #

Predict one finite response per query row using the fitted intercept and slopes.

polynomialFeatures

polynomialFeatures :: Int -> [Double] -> Either MathError Dataset Source #

Build univariate columns x, x^2, ..., x^degree; the regression intercept supplies the constant.

fitWeighted

fitWeighted :: Dataset -> [Double] -> [Double] -> Either MathError LinearModel Source #

Fit affine least squares with nonnegative observation weights. Zero-weight rows contribute no error.

fitRobust

fitRobust :: MLConfig -> Double -> Dataset -> [Double] -> Either MathError (Fit LinearModel) Source #

Huber IRLS with fixed positive residual threshold, including intercept.

BayesianLinear

data BayesianLinear Source #

Posterior-mean linear model together with coefficient covariance, returned by Bayesian regression fitting.

Instances

Instances details

Show BayesianLinear Source #
Instance details Defined in HMath.MachineLearning.Regression Methods showsPrec :: Int -> BayesianLinear -> ShowS # show :: BayesianLinear -> String # showList :: [BayesianLinear] -> ShowS #
Eq BayesianLinear Source #
Instance details Defined in HMath.MachineLearning.Regression Methods (==) :: BayesianLinear -> BayesianLinear -> Bool # (/=) :: BayesianLinear -> BayesianLinear -> Bool #

fitBayesian

fitBayesian :: Double -> Double -> Dataset -> [Double] -> Either MathError BayesianLinear Source #

Isotropic zero-mean Gaussian prior, observation precision beta. Unlike frequentist ridge the prior also regularizes the intercept.

bayesianMean

bayesianMean :: BayesianLinear -> LinearModel Source #

Posterior-mean fitted linear model.

bayesianCovariance

bayesianCovariance :: BayesianLinear -> Dataset Source #

Posterior covariance matrix of linear coefficients.

RecursiveLS

data RecursiveLS Source #

Owned online regression coefficients and inverse information, with fixed forgetting factor.

Instances

Instances details

Show RecursiveLS Source #
Instance details Defined in HMath.MachineLearning.Regression Methods showsPrec :: Int -> RecursiveLS -> ShowS # show :: RecursiveLS -> String # showList :: [RecursiveLS] -> ShowS #
Eq RecursiveLS Source #
Instance details Defined in HMath.MachineLearning.Regression Methods (==) :: RecursiveLS -> RecursiveLS -> Bool # (/=) :: RecursiveLS -> RecursiveLS -> Bool #

recursiveLS

recursiveLS :: Int -> Double -> Double -> Either MathError RecursiveLS Source #

Initial inverse information delta I; forgetting factor in (0,1]. Input dimension excludes the automatically supplied intercept.

updateRecursiveLS

updateRecursiveLS :: RecursiveLS -> [Double] -> Double -> Either MathError RecursiveLS Source #

Assimilate one feature row/response using the matrix inversion lemma. Reject lost definiteness.

recursiveCoefficients

recursiveCoefficients :: RecursiveLS -> [Double] Source #

Current intercept and slopes; updating creates a new state.

GLMFamily

data GLMFamily Source #

Bernoulli mean with logistic link, or nonnegative response mean with log link.

Constructors

BinomialLogit
PoissonLog

Instances

Instances details

Show GLMFamily Source #
Instance details Defined in HMath.MachineLearning.Regression Methods showsPrec :: Int -> GLMFamily -> ShowS # show :: GLMFamily -> String # showList :: [GLMFamily] -> ShowS #
Eq GLMFamily Source #
Instance details Defined in HMath.MachineLearning.Regression Methods (==) :: GLMFamily -> GLMFamily -> Bool # (/=) :: GLMFamily -> GLMFamily -> Bool #

GLM

data GLM Source #

Fitted generalized linear model with intercept and family-specific inverse link.

Instances

Instances details

Show GLM Source #
Instance details Defined in HMath.MachineLearning.Regression Methods showsPrec :: Int -> GLM -> ShowS # show :: GLM -> String # showList :: [GLM] -> ShowS #
Eq GLM Source #
Instance details Defined in HMath.MachineLearning.Regression Methods (==) :: GLM -> GLM -> Bool # (/=) :: GLM -> GLM -> Bool #

fitGLM

fitGLM :: MLConfig -> GLMFamily -> Double -> Dataset -> [Double] -> Either MathError (Fit GLM) Source #

Damped Newton/IRLS with nonnegative slope ridge. Residual is the penalized likelihood gradient norm.

predictGLM

predictGLM :: GLM -> Dataset -> Either MathError [Double] Source #

Return conditional means: probabilities for BinomialLogit, positive rates for PoissonLog.

MultinomialModel

data MultinomialModel Source #

Softmax class weights and sorted original integer labels.

Instances

Instances details

Show MultinomialModel Source #
Instance details Defined in HMath.MachineLearning.Regression Methods showsPrec :: Int -> MultinomialModel -> ShowS # show :: MultinomialModel -> String # showList :: [MultinomialModel] -> ShowS #
Eq MultinomialModel Source #
Instance details Defined in HMath.MachineLearning.Regression Methods (==) :: MultinomialModel -> MultinomialModel -> Bool # (/=) :: MultinomialModel -> MultinomialModel -> Bool #

fitMultinomial

fitMultinomial :: MLConfig -> Double -> Dataset -> [Int] -> Either MathError (Fit MultinomialModel) Source #

Fit all class logits by full-batch gradient descent with slope ridge. Residual is weight displacement.

predictMultinomial

predictMultinomial :: MultinomialModel -> Dataset -> Either MathError [[(Int, Double)]] Source #

Return sorted (original label, probability) pairs for each observation.

Kernel

data Kernel Source #

Positive-semidefinite linear, polynomial, or Gaussian radial basis kernel parameters.

Constructors

LinearKernel
PolynomialKernel Int Double Double
RBFKernel Double

Instances

Instances details

Show Kernel Source #
Instance details Defined in HMath.MachineLearning.Regression Methods showsPrec :: Int -> Kernel -> ShowS # show :: Kernel -> String # showList :: [Kernel] -> ShowS #
Eq Kernel Source #
Instance details Defined in HMath.MachineLearning.Regression Methods (==) :: Kernel -> Kernel -> Bool # (/=) :: Kernel -> Kernel -> Bool #

kernel

kernel :: Kernel -> [Double] -> [Double] -> Either MathError Double Source #

Evaluate a checked PSD kernel on equal-length finite feature vectors.

SVM

data SVM Source #

Binary kernel decision state, retaining original training observations and signed dual weights.

Instances

Instances details

Show SVM Source #
Instance details Defined in HMath.MachineLearning.Regression Methods showsPrec :: Int -> SVM -> ShowS # show :: SVM -> String # showList :: [SVM] -> ShowS #
Eq SVM Source #
Instance details Defined in HMath.MachineLearning.Regression Methods (==) :: SVM -> SVM -> Bool # (/=) :: SVM -> SVM -> Bool #

fitSVM

fitSVM :: MLConfig -> Kernel -> Double -> Dataset -> [Double] -> Either MathError (Fit SVM) Source #

Deterministic two-coordinate SMO for binary labels -1/+1. The reported residual is the largest box-constrained dual KKT violation.

decisionSVM

decisionSVM :: SVM -> Dataset -> Either MathError [Double] Source #

Return signed kernel margins; zero is the fitted decision boundary.

predictSVM

predictSVM :: SVM -> Dataset -> Either MathError [Double] Source #

Classify margins as -1 or +1, with zero assigned +1.

GaussianProcess

data GaussianProcess Source #

Zero-mean Gaussian-process regression posterior with retained training covariance solve.

Instances

Instances details

Show GaussianProcess Source #
Instance details Defined in HMath.MachineLearning.Regression Methods showsPrec :: Int -> GaussianProcess -> ShowS # show :: GaussianProcess -> String # showList :: [GaussianProcess] -> ShowS #
Eq GaussianProcess Source #
Instance details Defined in HMath.MachineLearning.Regression Methods (==) :: GaussianProcess -> GaussianProcess -> Bool # (/=) :: GaussianProcess -> GaussianProcess -> Bool #

fitGaussianProcess

fitGaussianProcess :: Kernel -> Double -> Dataset -> [Double] -> Either MathError GaussianProcess Source #

Fit a GP with strictly positive observation-noise variance and a checked PSD kernel.

predictGaussianProcess

predictGaussianProcess :: GaussianProcess -> Dataset -> Either MathError [(Double, Double)] Source #

Return posterior latent-function (mean, variance), excluding observation noise.

GPClassifier

data GPClassifier Source #

Logistic Gaussian-process classifier fitted with a Laplace Gaussian posterior approximation.

Instances

Instances details

Show GPClassifier Source #
Instance details Defined in HMath.MachineLearning.Regression Methods showsPrec :: Int -> GPClassifier -> ShowS # show :: GPClassifier -> String # showList :: [GPClassifier] -> ShowS #
Eq GPClassifier Source #
Instance details Defined in HMath.MachineLearning.Regression Methods (==) :: GPClassifier -> GPClassifier -> Bool # (/=) :: GPClassifier -> GPClassifier -> Bool #

fitGPClassifier

fitGPClassifier :: MLConfig -> Kernel -> Dataset -> [Double] -> Either MathError (Fit GPClassifier) Source #

Newton mode fitting for 0/1 labels. A 1e-10 diagonal jitter stabilizes the prior covariance.

predictGPClassifier

predictGPClassifier :: GPClassifier -> Dataset -> Either MathError [Double] Source #

Approximate class-one probabilities using the logistic Gaussian variance correction.

BayesFamily

data BayesFamily Source #

Gaussian features, nonnegative multinomial counts, or binary Bernoulli features.

Constructors

GaussianBayes
MultinomialBayes
BernoulliBayes

Instances

Instances details

Show BayesFamily Source #
Instance details Defined in HMath.MachineLearning.Regression Methods showsPrec :: Int -> BayesFamily -> ShowS # show :: BayesFamily -> String # showList :: [BayesFamily] -> ShowS #
Eq BayesFamily Source #
Instance details Defined in HMath.MachineLearning.Regression Methods (==) :: BayesFamily -> BayesFamily -> Bool # (/=) :: BayesFamily -> BayesFamily -> Bool #

NaiveBayes

data NaiveBayes Source #

Fitted class priors and family-specific independent feature likelihoods.

Instances

Instances details

Show NaiveBayes Source #
Instance details Defined in HMath.MachineLearning.Regression Methods showsPrec :: Int -> NaiveBayes -> ShowS # show :: NaiveBayes -> String # showList :: [NaiveBayes] -> ShowS #
Eq NaiveBayes Source #
Instance details Defined in HMath.MachineLearning.Regression Methods (==) :: NaiveBayes -> NaiveBayes -> Bool # (/=) :: NaiveBayes -> NaiveBayes -> Bool #

fitNaiveBayes

fitNaiveBayes :: BayesFamily -> Double -> Dataset -> [Int] -> Either MathError NaiveBayes Source #

Fit class likelihoods. Smoothing is a variance floor for Gaussian and a pseudocount otherwise.

predictNaiveBayes

predictNaiveBayes :: NaiveBayes -> Dataset -> Either MathError [Int] Source #

Select the class with greatest log posterior; preserve the original integer labels.

Discriminant

data Discriminant Source #

Class priors, means, and regularized inverse covariance/log-determinant state.

Instances

Instances details

Show Discriminant Source #
Instance details Defined in HMath.MachineLearning.Regression Methods showsPrec :: Int -> Discriminant -> ShowS # show :: Discriminant -> String # showList :: [Discriminant] -> ShowS #
Eq Discriminant Source #
Instance details Defined in HMath.MachineLearning.Regression Methods (==) :: Discriminant -> Discriminant -> Bool # (/=) :: Discriminant -> Discriminant -> Bool #

fitDiscriminant

fitDiscriminant :: Bool -> Double -> Dataset -> [Int] -> Either MathError Discriminant Source #

False = pooled covariance LDA; True = class covariance QDA. A positive diagonal ridge makes singular/constant-feature data explicitly regularized.

predictDiscriminant

predictDiscriminant :: Discriminant -> Dataset -> Either MathError [Int] Source #

Classify with Gaussian discriminant log likelihoods using training covariance state.