HMath.Interchange.IO

Module index · Numerical conventions

Description

Bounded strict file codecs, fixed-chunk raw streaming, and read-only mapped arrays. Mappings are bracketed and protected by a lock; even an IO closure escaping the callback reports a closed mapping rather than touching freed memory. Callers must keep mapped files immutable (POSIX truncation can otherwise cause SIGBUS). Windows mappings use the wide-character API.

On this page

API	Purpose
readArrayFile	File reading checks the declared size before allocating, then caps the actual read too, handling concurrent growth.
writeArrayFile	Writes to the supplied path after encoding succeeds.
RawSpec	Raw physical storage: fixed dtypeendianorder and shape, no header.
RawSpec	Declaration and type contract below.
readRawChunks	Read physical-order chunks (one-dimensional arrays); the caller retains the original RawSpec for reconstructing dimensions.
writeRawChunks	Write fixed-size raw chunks.
MappedArray	Scope-bound mapping; no pointer or borrowed ByteString is exposed.
withMappedArray	Given limits,RawSpec,path,then rank-2 callback, map read-only raw data for that callback's lifetime and unmap on exit.
mappedValue	Read one typed scalar using checked multidimensional indexing.
mappedShape	Return dimensions of a callback-scoped read-only mapped array.
mappedReadBytes	Copy a bounded byte range.

Documentation

readArrayFile

readArrayFile :: Limits -> (ByteString -> Either InterchangeError Array) -> FilePath -> IO (Either InterchangeError Array) Source #

File reading checks the declared size before allocating, then caps the actual read too, handling concurrent growth. Files are closed before return.

writeArrayFile

writeArrayFile :: Limits -> (Limits -> Array -> Either InterchangeError ByteString) -> FilePath -> Array -> IO (Either InterchangeError ()) Source #

Writes to the supplied path after encoding succeeds. Applications needing transactional replacement can write a sibling temporary and rename it.

RawSpec

data RawSpec Source #

Raw physical storage: fixed dtypeendianorder and shape, no header. The stream has exactly the product(shape)*itemsize bytes, including empty arrays.

Constructors

RawSpec DType Endian Order [Int]

Instances

Instances details

Show RawSpec Source #
Instance details Defined in HMath.Interchange.IO Methods showsPrec :: Int -> RawSpec -> ShowS # show :: RawSpec -> String # showList :: [RawSpec] -> ShowS #
Eq RawSpec Source #
Instance details Defined in HMath.Interchange.IO Methods (==) :: RawSpec -> RawSpec -> Bool # (/=) :: RawSpec -> RawSpec -> Bool #

readRawChunks

readRawChunks :: Limits -> RawSpec -> Int -> Handle -> (Array -> IO (Either InterchangeError Bool)) -> IO (Either InterchangeError ()) Source #

Read physical-order chunks (one-dimensional arrays); the caller retains the original RawSpec for reconstructing dimensions. Memory is O(chunk), excluding any data retained by the callback. False cancels successfully without consuming the rest. Full completion requires EOF, detecting tails.

writeRawChunks

writeRawChunks :: Limits -> RawSpec -> Int -> Handle -> (Int -> IO (Either InterchangeError (Maybe ByteString))) -> IO (Either InterchangeError ()) Source #

Write fixed-size raw chunks. The producer receives a maximum byte count, must return whole elements, and signals EOF with Nothing. Short streams, overlong streams, noncanonical bools and empty nonfinal chunks fail. Errors may leave a partial output; the handle remains owned by the caller.

MappedArray

data MappedArray s Source #

Scope-bound mapping; no pointer or borrowed ByteString is exposed.

withMappedArray

withMappedArray :: Limits -> RawSpec -> FilePath -> (forall s. MappedArray s -> IO a) -> IO (Either InterchangeError a) Source #

Given limits,RawSpec,path,then rank-2 callback, map read-only raw data for that callback's lifetime and unmap on exit. Validate exact file sizespecbudgets; pathIOformat failures return InterchangeError. Do not retain borrowed pointers beyond the callback.

mappedValue

mappedValue :: Element a => MappedArray s -> [Int] -> IO (Either InterchangeError a) Source #

Read one typed scalar using checked multidimensional indexing. Physical endian conversion happens through the same Element decoder as normal arrays.

mappedShape

mappedShape :: MappedArray s -> [Int] Source #

Return dimensions of a callback-scoped read-only mapped array. The region token prevents its raw view escaping withMappedArray.

mappedReadBytes

mappedReadBytes :: MappedArray s -> Int -> Int -> IO (Either InterchangeError ByteString) Source #

Copy a bounded byte range. Copies may safely survive mapping closure.