HMath.Interchange¶

Description

Checked fixed-width numerical serialization and array exchange. The binarytextNumPy formats are independently versioned from HMath's package version. Binary/raw formats preserve IEEE bit patterns; text preserves numerical values and signed zero, but not NaN payloads. Foreign objects and arbitrary code/pickles are never deserialized. DLPack uses the native C API declared in hmath_dlpack.h; optional HDF5ArrowONNX conversion is provided by the explicitly selected tools/interchange_adapters.py adapters.

On this page¶

API	Purpose
array	Construct a bounded view.
Array	Opaque validated bytes and descriptive UTF-8 metadata (key/value strings).
Endian	Explicit byte order.
LittleEndian	Declaration and type contract below.
BigEndian	Declaration and type contract below.
DType	Numeric wire types; no platform-dependent Int, object or record dtype.
Bool8	Declaration and type contract below.
I8	Declaration and type contract below.
U8	Declaration and type contract below.
I16	Declaration and type contract below.
U16	Declaration and type contract below.
I32	Declaration and type contract below.
U32	Declaration and type contract below.
I64	Declaration and type contract below.
U64	Declaration and type contract below.
F32	Declaration and type contract below.
F64	Declaration and type contract below.
C64	Declaration and type contract below.
C128	Declaration and type contract below.
Limits	Caller-configurable finite resource budgets, checked before allocations.
Limits	Declaration and type contract below.
maxBytes	Maximum aggregate encodedinputbacking bytes, including relevant format overhead.
maxElements	Maximum logical scalar element count.
maxRank	Maximum array dimension count (implementation cap 255).
maxMetadata	Maximum descriptive metadata/header byte budget.
maxArchiveEntries	Maximum archive member count; aggregate uncompressed bytes are also bounded.
defaultLimits	128 MiB of bytes, 8 million samples, rank 16, 64 KiB metadata, 256 members.
Order	Physical order of a packed output; logical values are always row major.
COrder	Declaration and type contract below.
FOrder	Declaration and type contract below.
InterchangeError	Decoders never throw on malformed bytes; IO failures have their own case.
InvalidData	Declaration and type contract below.
Unsupported	Declaration and type contract below.
LimitExceeded	Declaration and type contract below.
InterchangeIOError	Declaration and type contract below.
Element	Fixed-width element encoding.
elementType	Given a Haskell Element type proxy, return its exact fixed-width wire DType.
putElement	Given a Haskell Element type proxy, return its exact fixed-width wire DType.
getElement	Given a Haskell Element type proxy, return its exact fixed-width wire DType.
contiguousArray	Given limits,dtype,endian,physical order,shape,then exact packed bytes, construct a canonical byte-strided array.
validateShape	Given limits then dimensions, validate rank, nonnegative axes and element product, returning Int element count.
arrayDType	Return the fixed-width wire dtype of an Array; this is independent of Haskell host Int/Double defaults.
arrayEndian	Return explicit payload byte order; single-byte dtypes are endian-independent.
arrayShape	Return logical dimensions in axis order; [] is scalar and zero axes are valid.
arrayStrides	Return signed byte strides for each axis.
arrayOffset	Return byte offset of logical origin within arrayBytes backing storage.
arrayBytes	Return immutable backing bytes, which may include padding or data not reached by the logical strided view.
arrayMetadata	Return descriptive UTF-8-compatible key/value metadata strings in stored order; metadata is not executable.
withMetadata	Metadata is UTF-8 in binary files; keys must be unique and nonempty.
elementBytes	Return fixed byte width of a DType, including complex pairs (C64=8,C128=16) and Bool8=1.
elementCount	Return logical shape product; scalar arrays have one element and empty axes have zero.
denseStrides	Dense strides use Integer arithmetic before conversion.
fromValues	Given limits,endian,shape,then typed Element list, encode to a contiguous array with that Element dtype.
toValues	Decode logical row-major values as the requested Element type, requiring an exact dtype match.
fromTensor	Given limits,endian,then typed Haskell Tensor, encode logical row-major values and preserve shape.
toTensor	Decode an Array of matching Element dtype to a contiguous Haskell Tensor, preserving logical shape/order.
fromMatrix	Given limits,endian,then typed Matrix, encode its row-major values as a rank-two Array.
toMatrix	Decode a matching typed rank-two Array to Matrix.
repack	Pack arbitrary views, preserving dtype and exact component bits.
rawEncode	Given limits,destination endian,physical order,then Array, repack its logical payload and return raw bytes without shapeheader metadata.
rawDecode	Given limits,dtype,endian,order,shape,then bytes, construct a packed Array.
encodeBinary	Given limits then Array, encode HMath wire-format v1 including shapestridesoffsetmetadatabacking storage and CRC-32.
decodeBinary	Given limits then bytes, validate HMath v1 magicheaderflags/CRC and exact framing, returning an Array.
crc32	Standard CRC-32, compatible with ZIP and zlib, with a fixed 256-entry table.

Documentation

array

array :: Limits -> DType -> Endian -> [Int] -> [Int] -> Int -> ByteString -> Either InterchangeError Array Source #

Construct a bounded view. Bool8 bytes reachable through the view must be 0 or 1. Floating payloads preserve IEEE NaNs, signed zero and infinities.

Array

data Array Source #

Opaque validated bytes and descriptive UTF-8 metadata (key/value strings).

Instances

Instances details

Show Array Source #
Instance details Defined in HMath.Interchange.Array Methods showsPrec :: Int -> Array -> ShowS # show :: Array -> String # showList :: [Array] -> ShowS #
Eq Array Source #
Instance details Defined in HMath.Interchange.Array Methods (==) :: Array -> Array -> Bool # (/=) :: Array -> Array -> Bool #

Endian

data Endian Source #

Explicit byte order. Single-byte types ignore it.

Constructors

LittleEndian
BigEndian

Instances

Instances details

Show Endian Source #
Instance details Defined in HMath.Interchange.Array Methods showsPrec :: Int -> Endian -> ShowS # show :: Endian -> String # showList :: [Endian] -> ShowS #
Eq Endian Source #
Instance details Defined in HMath.Interchange.Array Methods (==) :: Endian -> Endian -> Bool # (/=) :: Endian -> Endian -> Bool #

DType

data DType Source #

Numeric wire types; no platform-dependent Int, object or record dtype.

Constructors

Bool8
I8
U8
I16
U16
I32
U32
I64
U64
F32
F64
C64
C128

Instances

Instances details

Bounded DType Source #
Instance details Defined in HMath.Interchange.Array Methods minBound :: DType # maxBound :: DType #
Enum DType Source #
Instance details Defined in HMath.Interchange.Array Methods succ :: DType -> DType # pred :: DType -> DType # toEnum :: Int -> DType # fromEnum :: DType -> Int # enumFrom :: DType -> [DType] # enumFromThen :: DType -> DType -> [DType] # enumFromTo :: DType -> DType -> [DType] # enumFromThenTo :: DType -> DType -> DType -> [DType] #
Show DType Source #
Instance details Defined in HMath.Interchange.Array Methods showsPrec :: Int -> DType -> ShowS # show :: DType -> String # showList :: [DType] -> ShowS #
Eq DType Source #
Instance details Defined in HMath.Interchange.Array Methods (==) :: DType -> DType -> Bool # (/=) :: DType -> DType -> Bool #
Ord DType Source #
Instance details Defined in HMath.Interchange.Array Methods compare :: DType -> DType -> Ordering # (<) :: DType -> DType -> Bool # (<=) :: DType -> DType -> Bool # (>) :: DType -> DType -> Bool # (>=) :: DType -> DType -> Bool # max :: DType -> DType -> DType # min :: DType -> DType -> DType #

Limits

data Limits Source #

Caller-configurable finite resource budgets, checked before allocations. Byte budgets include metadata/file overhead as well as raw payload. The archive budget applies to the SUM of uncompressed members, preventing bombs.

Constructors

Limits

Fields

maxBytes :: Int
Maximum aggregate encodedinputbacking bytes, including relevant format overhead.
maxElements :: Int
Maximum logical scalar element count.
maxRank :: Int
Maximum array dimension count (implementation cap 255).
maxMetadata :: Int
Maximum descriptive metadata/header byte budget.
maxArchiveEntries :: Int
Maximum archive member count; aggregate uncompressed bytes are also bounded.

Instances

Instances details

Show Limits Source #
Instance details Defined in HMath.Interchange.Array Methods showsPrec :: Int -> Limits -> ShowS # show :: Limits -> String # showList :: [Limits] -> ShowS #
Eq Limits Source #
Instance details Defined in HMath.Interchange.Array Methods (==) :: Limits -> Limits -> Bool # (/=) :: Limits -> Limits -> Bool #

defaultLimits

defaultLimits :: Limits Source #

128 MiB of bytes, 8 million samples, rank 16, 64 KiB metadata, 256 members.

Order

data Order Source #

Physical order of a packed output; logical values are always row major.

Constructors

COrder
FOrder

Instances

Instances details

Show Order Source #
Instance details Defined in HMath.Interchange.Array Methods showsPrec :: Int -> Order -> ShowS # show :: Order -> String # showList :: [Order] -> ShowS #
Eq Order Source #
Instance details Defined in HMath.Interchange.Array Methods (==) :: Order -> Order -> Bool # (/=) :: Order -> Order -> Bool #

InterchangeError

data InterchangeError Source #

Decoders never throw on malformed bytes; IO failures have their own case.

Constructors

InvalidData String
Unsupported String
LimitExceeded String
InterchangeIOError String

Instances

Instances details

Show InterchangeError Source #
Instance details Defined in HMath.Interchange.Array Methods showsPrec :: Int -> InterchangeError -> ShowS # show :: InterchangeError -> String # showList :: [InterchangeError] -> ShowS #
Eq InterchangeError Source #
Instance details Defined in HMath.Interchange.Array Methods (==) :: InterchangeError -> InterchangeError -> Bool # (/=) :: InterchangeError -> InterchangeError -> Bool #

Element

class Element a where Source #

Fixed-width element encoding. Instances deliberately exclude machine Int. Custom instances must write exactly elementBytes bytes and use a bounded decoder. Built-in instances preserve floating-point bit representations.

Methods

elementType :: proxy a -> DType Source #

Given a Haskell Element type proxy, return its exact fixed-width wire DType.

putElement :: Endian -> a -> Put Source #

Given byte order then one typed Element, emit its binary representation with Data.Binary.Put.

getElement :: Endian -> Get a Source #

Given byte order, decode one typed Element with Data.Binary.Get; surrounding array decoders enforce byte/storage budgets.

Instances

Instances details

Element Int16 Source #
Instance details Defined in HMath.Interchange.Array Methods elementType :: proxy Int16 -> DType Source # putElement :: Endian -> Int16 -> Put Source # getElement :: Endian -> Get Int16 Source #
Element Int32 Source #
Instance details Defined in HMath.Interchange.Array Methods elementType :: proxy Int32 -> DType Source # putElement :: Endian -> Int32 -> Put Source # getElement :: Endian -> Get Int32 Source #
Element Int64 Source #
Instance details Defined in HMath.Interchange.Array Methods elementType :: proxy Int64 -> DType Source # putElement :: Endian -> Int64 -> Put Source # getElement :: Endian -> Get Int64 Source #
Element Int8 Source #
Instance details Defined in HMath.Interchange.Array Methods elementType :: proxy Int8 -> DType Source # putElement :: Endian -> Int8 -> Put Source # getElement :: Endian -> Get Int8 Source #
Element Word16 Source #
Instance details Defined in HMath.Interchange.Array Methods elementType :: proxy Word16 -> DType Source # putElement :: Endian -> Word16 -> Put Source # getElement :: Endian -> Get Word16 Source #
Element Word32 Source #
Instance details Defined in HMath.Interchange.Array Methods elementType :: proxy Word32 -> DType Source # putElement :: Endian -> Word32 -> Put Source # getElement :: Endian -> Get Word32 Source #
Element Word64 Source #
Instance details Defined in HMath.Interchange.Array Methods elementType :: proxy Word64 -> DType Source # putElement :: Endian -> Word64 -> Put Source # getElement :: Endian -> Get Word64 Source #
Element Word8 Source #
Instance details Defined in HMath.Interchange.Array Methods elementType :: proxy Word8 -> DType Source # putElement :: Endian -> Word8 -> Put Source # getElement :: Endian -> Get Word8 Source #
Element Bool Source #
Instance details Defined in HMath.Interchange.Array Methods elementType :: proxy Bool -> DType Source # putElement :: Endian -> Bool -> Put Source # getElement :: Endian -> Get Bool Source #
Element Double Source #
Instance details Defined in HMath.Interchange.Array Methods elementType :: proxy Double -> DType Source # putElement :: Endian -> Double -> Put Source # getElement :: Endian -> Get Double Source #
Element Float Source #
Instance details Defined in HMath.Interchange.Array Methods elementType :: proxy Float -> DType Source # putElement :: Endian -> Float -> Put Source # getElement :: Endian -> Get Float Source #
Element (Complex Double) Source #
Instance details Defined in HMath.Interchange.Array Methods elementType :: proxy (Complex Double) -> DType Source # putElement :: Endian -> Complex Double -> Put Source # getElement :: Endian -> Get (Complex Double) Source #
Element (Complex Float) Source #
Instance details Defined in HMath.Interchange.Array Methods elementType :: proxy (Complex Float) -> DType Source # putElement :: Endian -> Complex Float -> Put Source # getElement :: Endian -> Get (Complex Float) Source #

contiguousArray

contiguousArray :: Limits -> DType -> Endian -> Order -> [Int] -> ByteString -> Either InterchangeError Array Source #

Given limits,dtype,endian,physical order,shape,then exact packed bytes, construct a canonical byte-strided array. Require exact byte count; invalid shapedatalimits fail.

validateShape

validateShape :: Limits -> [Int] -> Either InterchangeError Int Source #

Given limits then dimensions, validate rank, nonnegative axes and element product, returning Int element count. Invalid data or exceeded limits return InterchangeError.

arrayDType

arrayDType :: Array -> DType Source #

Return the fixed-width wire dtype of an Array; this is independent of Haskell host Int/Double defaults.

arrayEndian

arrayEndian :: Array -> Endian Source #

Return explicit payload byte order; single-byte dtypes are endian-independent.

arrayShape

arrayShape :: Array -> [Int] Source #

Return logical dimensions in axis order; [] is scalar and zero axes are valid.

arrayStrides

arrayStrides :: Array -> [Int] Source #

Return signed byte strides for each axis. Unlike HMath.Tensor strides, these are bytes, not elements.

arrayOffset

arrayOffset :: Array -> Int Source #

Return byte offset of logical origin within arrayBytes backing storage.

arrayBytes

arrayBytes :: Array -> ByteString Source #

Return immutable backing bytes, which may include padding or data not reached by the logical strided view. Use toValues or repack for logical contents.

arrayMetadata

arrayMetadata :: Array -> [(String, String)] Source #

Return descriptive UTF-8-compatible key/value metadata strings in stored order; metadata is not executable.

withMetadata

withMetadata :: Limits -> [(String, String)] -> Array -> Either InterchangeError Array Source #

Metadata is UTF-8 in binary files; keys must be unique and nonempty. The four-byte-per-codepoint bound is intentionally conservative before encoding.

elementBytes

elementBytes :: DType -> Int Source #

Return fixed byte width of a DType, including complex pairs (C64=8,C128=16) and Bool8=1.

elementCount

elementCount :: Array -> Int Source #

Return logical shape product; scalar arrays have one element and empty axes have zero.

denseStrides

denseStrides :: Order -> Int -> [Int] -> [Int] Source #

Dense strides use Integer arithmetic before conversion. Call only on validated shape/byte sizes; zero axes use zero strides to avoid overflow.

fromValues

fromValues :: Element a => Limits -> Endian -> [Int] -> [a] -> Either InterchangeError Array Source #

Given limits,endian,shape,then typed Element list, encode to a contiguous array with that Element dtype. Require exact shape/count match and resource bounds; failures return InterchangeError.

toValues

toValues :: Element a => Array -> Either InterchangeError [a] Source #

Decode logical row-major values as the requested Element type, requiring an exact dtype match. No silent numeric cast occurs; mismatch returns Unsupported/InvalidData.

fromTensor

fromTensor :: Element a => Limits -> Endian -> Tensor a -> Either InterchangeError Array Source #

Given limits,endian,then typed Haskell Tensor, encode logical row-major values and preserve shape. The Element instance determines wire dtype; resource bounds can fail.

toTensor

toTensor :: Element a => Array -> Either InterchangeError (Tensor a) Source #

Decode an Array of matching Element dtype to a contiguous Haskell Tensor, preserving logical shape/order. Dtype or checked tensor-construction failure returns InterchangeError.

fromMatrix

fromMatrix :: Element a => Limits -> Endian -> Matrix a -> Either InterchangeError Array Source #

Given limits,endian,then typed Matrix, encode its row-major values as a rank-two Array. Dtype comes from Element; resource limits can fail.

toMatrix

toMatrix :: Element a => Array -> Either InterchangeError (Matrix a) Source #

Decode a matching typed rank-two Array to Matrix. Rank/dtype or checked shape conversion failures return InterchangeError.

repack

repack :: Limits -> Endian -> Order -> Array -> Either InterchangeError Array Source #

Pack arbitrary views, preserving dtype and exact component bits. Complex byte swapping reverses each real/imaginary component separately.

rawEncode

rawEncode :: Limits -> Endian -> Order -> Array -> Either InterchangeError ByteString Source #

Given limits,destination endian,physical order,then Array, repack its logical payload and return raw bytes without shapeheader metadata. Checked resourcelayout failures propagate.

rawDecode

rawDecode :: Limits -> DType -> Endian -> Order -> [Int] -> ByteString -> Either InterchangeError Array Source #

Given limits,dtype,endian,order,shape,then bytes, construct a packed Array. Caller supplies all metadata; exact byte-count and shape/resource validation is enforced.

encodeBinary

encodeBinary :: Limits -> Array -> Either InterchangeError ByteString Source #

Given limits then Array, encode HMath wire-format v1 including shapestridesoffsetmetadatabacking storage and CRC-32. Invalid metadata/layout or exceeded aggregate byte budget fails.

decodeBinary

decodeBinary :: Limits -> ByteString -> Either InterchangeError Array Source #

Given limits then bytes, validate HMath v1 magicheaderflags/CRC and exact framing, returning an Array. Truncation, corruption, unsupported metadata or exceeded budgets return InterchangeError.

crc32

crc32 :: ByteString -> Word32 Source #

Standard CRC-32, compatible with ZIP and zlib, with a fixed 256-entry table.

module HMath.Interchange.Text

module HMath.Interchange.Numpy

module HMath.Interchange.Compression

module HMath.Interchange.IO