Processing#

The preferred API either indexes one frame with index_frame or reuses an Indexer across frames that share configuration.

One-Off Indexing#

lauelab.indexing.index_frame(frame, *, geometry, crystal=None, peak_params=None, index_params=None, detector_index=0, detector_id=None, cosmic_filter=False, start=(0, 0), group=(1, 1), depth=None, mask=None, metadata=None, keep_image=True)[source]#

Index one frame with a temporary Indexer.

Parameters:

  • frame (np.ndarray | str | Path) – Two-dimensional NumPy array with a dtype in SUPPORTED_FRAME_DTYPES, or path to a supported HDF5 frame.

  • geometry (str | Path | Geometry) – Parsed detector geometry or path to a geometry XML file.

  • crystal (str | Path | Crystal | None) – Crystal description or crystal XML path. If None, peak search and pixel-to-q conversion run without orientation indexing.

  • peak_params (PeakParams | None) – Peak-search configuration. Defaults to PeakParams.

  • index_params (IndexParams | None) – Orientation-indexing configuration. Defaults to IndexParams.

  • detector_index (int) – Physical detector slot in the geometry. This is not the ordinal position among active detectors.

  • detector_id (str | None) – Detector identifier to select instead of detector_index.

  • cosmic_filter (bool) – Must be False. Cosmic-ray filtering is unsupported for indexing; True raises InputError.

  • start (tuple[int, int]) – Full-detector ROI origin and pixel grouping in (x, y) order.

  • group (tuple[int, int]) – Full-detector ROI origin and pixel grouping in (x, y) order.

  • depth (float | None) – Physical sample depth in micrometres. None uses the HDF5 frame’s entry1/depth when present and finite, otherwise no depth.

  • mask (np.ndarray | None) – Optional mask matching the frame shape. Nonzero pixels are excluded.

  • metadata (FrameMetadata | Mapping[str, object] | None) – Optional frame metadata object or mapping.

  • keep_image (bool) – Retain the contiguous input image in the returned result.

Returns:

FrameResult – A self-contained frame result. The input image is retained by default.

Raises:

  • InputError – If configuration, detector selection, or frame input is invalid.

  • MemoryError – If a native stage cannot allocate required memory.

  • IndexingError – If a native numerical or internal indexing stage fails.

Return type:

FrameResult

Notes

Construct Indexer directly when processing multiple frames so parsed geometry and crystal state can be reused.

Reusable Indexer#

class lauelab.indexing.Indexer(geometry, crystal=None, *, peak_params=None, index_params=None, detector_index=0, detector_id=None, cosmic_filter=False)[source]#

Reusable in-process Laue frame indexer.

Peak search, pixel-to-q conversion, and crystal indexing run through the native library without subprocesses or per-frame intermediate text files.

Parameters:

  • geometry (str | Path | Geometry) – Parsed detector geometry or path to a geometry XML file.

  • crystal (str | Path | Crystal | None) – Crystal description or crystal XML path. If None, frames are peak searched and converted to q space without orientation indexing.

  • peak_params (PeakParams | None) – Peak-search configuration. Defaults to PeakParams.

  • index_params (IndexParams | None) – Orientation-indexing configuration. Defaults to IndexParams.

  • detector_index (int) – Physical detector slot in the geometry. This is not the ordinal position among active detectors.

  • detector_id (str | None) – Detector identifier to select instead of detector_index.

  • cosmic_filter (bool) – Must be False. Cosmic-ray filtering is unsupported for indexing; True raises InputError.

Raises:

  • InputError – If parameters or detector selection are invalid.

  • ValueError – If a geometry or crystal description is invalid.

  • ImportError – If the native indexing library is unavailable.

Notes

Reuse one instance for frames that share geometry, crystal, detector, and processing parameters. Concurrent calls to index() from several Python threads are unsupported. For parallel work, give each worker process its own indexer; see Batch indexing.

Construction normalizes whole-number floats in the parameter objects to int while preserving their values. The parameter types in indexer.peak_params can therefore differ from those supplied.

Use replace() to create a separately validated instance with changed configuration.

index(frame, *, start=(0, 0), group=(1, 1), depth=None, mask=None, metadata=None, keep_image=True)[source]#

Process one NumPy frame, HDF5 file, or scan frame without subprocesses.

Parameters:

  • frame (np.ndarray | str | Path | ScanFrame) – Two-dimensional NumPy array with a dtype in SUPPORTED_FRAME_DTYPES, path to a supported HDF5 frame whose image has one of those dtypes, or a ScanFrame naming one stored frame of a reconstructed point file. Pixel values enter peak search as their exact double value; no other dtype is converted.

  • start (tuple[int, int]) – Zero-based detector (x, y) origin for an in-memory frame.

  • group (tuple[int, int]) – Positive detector-pixel grouping factors as (x, y).

  • depth (float | None) – Physical sample depth in micrometres passed to pixel-to-q conversion. None uses the HDF5 frame’s entry1/depth when that dataset is present and finite, the stored depth of a ScanFrame, and no depth otherwise. An explicit finite value, including 0.0, overrides the file.

  • mask (np.ndarray | None) – Array with the same shape as frame. Values are converted to a boolean uint8 mask and passed to native peak search, where nonzero pixels are masked.

  • metadata (FrameMetadata | Mapping[str, object] | None) – Experiment metadata for XML output. Explicit values override metadata loaded from an HDF5 file.

  • keep_image (bool) – Retain the contiguous input image in the returned result’s image attribute.

Returns:

FrameResult – A self-contained result whose native backing allocations have already been released.

Raises:

  • InputError – If the frame, region, mask, metadata, or selected detector is invalid, or an HDF5 input is missing, unreadable, or malformed. File-reading errors are preserved as the exception’s cause.

  • MemoryError – If a native stage cannot allocate required memory.

  • IndexingError – If a native numerical or internal stage fails.

Return type:

FrameResult

Notes

For HDF5 and scan-frame input, detector start and group values from the file take precedence over method arguments when present; depth works the other way, the argument overriding the file. A scan frame is read exactly: one frame, its point’s detector metadata, and its physical depth. The rest of the stack is not loaded. A retained image can alias a contiguous array supplied by the caller. Native peak search uses a separate working copy, so smoothing does not modify that array.

A frame stored with a non-native byte order is byte-swapped to a native-order copy of the same dtype before processing. Floating-point frames must be finite.

Frame statistics exclude masked pixels and always describe the raw input image. Smoothing applies only to peak detection and fitting. Under automatic thresholding, a frame with no unmasked nonzero pixels returns a valid empty result with threshold_used set to NaN.

Native code can write diagnostics to stdout or stderr before Python raises an exception.

index_many(frames, *, keep_images=False)[source]#

Index frames in order while reusing parsed configuration.

Parameters:

  • frames (Iterable[np.ndarray | str | Path]) – Iterable of two-dimensional arrays with supported dtypes or supported HDF5 paths.

  • keep_images (bool) – Retain each input image in its result. Defaults to False to limit batch memory use.

Returns:

list[FrameResult] – Results in the same order as the input iterable.

Raises:

  • InputError – If a frame or its metadata is invalid.

  • MemoryError – If a native stage cannot allocate required memory.

  • IndexingError – If a native numerical or internal stage fails.

Return type:

list[FrameResult]

Notes

Frames are processed sequentially. Processing stops on the first exception.

iter_index(inputs, *, mask=None, workers=1, max_in_flight=None, should_stop=None, keep_images=False, poll_seconds=0.25)[source]#

Index frames in worker processes, yielding one outcome per input in order.

Parameters:

  • inputs (Iterable) – Iterable of FrameInput objects, or of frames accepted by index() (arrays or HDF5 paths), which are wrapped with default processing values and no input_id. The iterable is consumed lazily as the in-flight window admits work.

  • mask (np.ndarray | None) – Optional mask shared by every frame, with nonzero pixels excluded. It is sent to each worker once, not with every task.

  • workers (int) – Positive number of worker processes. Each builds its own Indexer from this indexer’s geometry path, crystal, parameters, detector selection, and cosmic_filter.

  • max_in_flight (int | None) – Bound on inputs submitted but not yet consumed, which also bounds the results held for in-order delivery. Must be at least workers. Defaults to 2 * workers.

  • should_stop – Optional callable returning True to request a cooperative stop. It is polled before every submission and at least every poll_seconds while waiting. On True, no further inputs are admitted, unstarted inputs are withdrawn, running inputs finish and are yielded, then iteration ends with stopped set.

  • keep_images (bool) – Retain each input image in its result. Defaults to False; a retained image adds one detector-sized array to every result sent between processes.

  • poll_seconds (float) – Longest interval between should_stop checks while waiting.

Returns:

FrameOutcomes – Iterate inside a with block. Each FrameOutcome holds either a FrameResult, including results with no peaks or patterns, or an expected input error from EXPECTED_INPUT_ERRORS, and the iteration continues after such errors.

Raises:

  • InputError – If workers, max_in_flight, should_stop, poll_seconds, or mask is invalid.

  • WorkerError – During iteration, if a worker cannot initialize, raises an unexpected exception, or the pool breaks. Iteration cannot continue.

Notes

Workers use the spawn start method; guard module-level calls with if __name__ == "__main__":. This method schedules computation only. Writing results, recording failures, and deciding what to do after a stop belong to the caller.

replace(**changes)[source]#

Create an indexer with selected configuration values replaced.

Parameters:

**changes – Constructor arguments to replace. Supported names are geometry, crystal, peak_params, index_params, detector_index, detector_id, and cosmic_filter.

Returns:

Indexer – A new, independently validated indexer.

Raises:

  • TypeError – If an unknown constructor argument is supplied.

  • InputError – If replacement parameters or detector selection are invalid.

Return type:

Indexer

Notes

Native geometry and crystal state is constructed for the new instance; it is not shared with the original indexer.

results_writer(path, **kwargs)[source]#

Create a streaming HDF5 writer bound to this indexer’s configuration.

write_results(results, path, *, frame_ids=None, overwrite=False)[source]#

Write an iterable of results to one indexing-results HDF5 file.

write_many_xml(results, path)[source]#

Write multiple results to one LaueGo XML document.

Parameters:

  • results (Iterable[FrameResult]) – Frame results written in iteration order.

  • path (str | Path) – Destination XML file. An existing file is replaced.

Raises:

  • RuntimeError – If any result has no XML snapshot.

  • OSError – If the destination cannot be written.

Notes

Results are serialized one at a time through XmlResultsWriter, so memory use does not grow with the number of results. The output is identical to writing the collected steps in one call. The document is written under the destination’s .partial name and renamed into place on success, so a failure leaves an existing destination as it was.

Parallel Indexing#

class lauelab.indexing.FrameInput(frame, input_id=None, start=(0, 0), group=(1, 1), depth=None, metadata=None)[source]#

One frame to index, with its identity and per-frame processing values.

Parameters:

  • frame (np.ndarray | str | Path | ScanFrame) – Two-dimensional array with a supported dtype, path to a supported HDF5 frame, or ScanFrame. An array is sent to the worker with the task; a path or scan frame is opened inside the worker.

  • input_id (Hashable | None) – Caller-defined stable identity carried unchanged into the outcome. None when the caller has no identity beyond the input position.

  • start (tuple[int, int]) – Full-detector ROI origin and pixel grouping in (x, y) order, as for Indexer.index().

  • group (tuple[int, int]) – Full-detector ROI origin and pixel grouping in (x, y) order, as for Indexer.index().

  • depth (float | None) – Physical sample depth in micrometres, or None to use the HDF5 frame’s entry1/depth when present.

  • metadata (object | Mapping[str, object] | None) – Optional frame metadata object or mapping.

class lauelab.indexing.FrameOutcome(input_index, input_id, result=None, error=None, seconds=0.0)[source]#

The result of attempting one input.

Parameters:

  • input_index (int) – Zero-based position of the input in the iterable passed to Indexer.iter_index().

  • input_id (Hashable | None) – The input_id of the corresponding FrameInput, or None.

  • result (object | None) – The FrameResult when the frame was processed, including a processed frame with no peaks or no patterns.

  • error (Exception | None) – The expected input error when the frame could not be processed, one of EXPECTED_INPUT_ERRORS. Exactly one of result and error is set.

  • seconds (float) – Wall time of the indexing call inside the worker.

property ok: bool#

Whether the frame was processed.

class lauelab.indexing.FrameOutcomes(indexer, inputs, *, mask, workers, max_in_flight, should_stop, keep_images, poll_seconds)[source]#

Ordered outcomes of a bounded parallel indexing run.

Instances are created by Indexer.iter_index(). Use one as a context manager and iterate over it; leaving the with block shuts the worker processes down whether the iteration finished, stopped early, or raised.

stopped#

True once should_stop returned True and admission ended.

Type:

bool

n_submitted#

Inputs handed to the worker pool.

Type:

int

n_yielded#

Outcomes returned so far.

Type:

int

n_cancelled#

Submitted inputs withdrawn before a worker started them, after a stop.

Type:

int

peak_in_flight#

Highest number of submitted-but-unconsumed inputs observed. It never exceeds max_in_flight.

Type:

int

Notes

Outcomes come back in input order. An input whose worker call is slow holds back later outcomes, which wait in the bounded in-flight window; no further inputs are admitted until the head is consumed. Inputs that were never started produce no outcome; compare n_submitted - n_cancelled with the number of inputs to find them.

Workers use the spawn start method, so a script calling Indexer.iter_index() at module level must guard that call with if __name__ == "__main__":.

close()[source]#

Withdraw unstarted work, wait for running work, and stop the workers.

Safe to call more than once. After close() the iterator is finished.

lauelab.indexing.EXPECTED_INPUT_ERRORS#

Built-in immutable sequence.

If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable’s items.

If the argument is a tuple, the return value is the same object.

EXPECTED_INPUT_ERRORS are the exception types that describe one input and are returned in FrameOutcome.error rather than raised; see Batch indexing.

Frame Inputs#

lauelab.indexing.load_mask(path)[source]#

Load a peak-search mask stored as a 34-ID-E HDF5 image.

Parameters:

path (str | Path) – HDF5 file whose entry1/data/data dataset holds the mask image. Any numeric dtype is accepted.

Returns:

numpy.ndarray – Two-dimensional boolean array. True marks a pixel excluded from peak search. This follows the LaueGo peaksearch -K convention: nonzero mask pixels are excluded and zero pixels remain available.

Raises:

  • OSError – If the file cannot be opened.

  • KeyError – If the image dataset is missing.

  • ValueError – If the dataset is not a two-dimensional numeric array.

Return type:

np.ndarray

Notes

The mask shape is checked against the frame when it is passed to Indexer.index(), not here.

class lauelab.indexing.ScanFrame(path, point_id, depth_index)[source]#

Reference to a reconstructed depth image for indexing.

Supply the point file path, point ID, and depth index to select a stored frame. The reference can be pickled, sent to a worker, and saved with indexing results. Each process opens the point file when it reads the frame; a scan catalog is unnecessary.

Parameters:

  • path (pathlib.Path or str) – A point file, such as points/<input stem>.h5 of a scan directory written by reconstruct_scan(), or a standalone file written by reconstruct_point(). ScanReader.point_path resolves a catalog point ID to this path.

  • point_id (str) – Expected point ID. Reading fails if the ID in the file differs.

  • depth_index (int) – Zero-based position of the frame in the point’s depth stack. Its physical depth in µm is read from the file.

lauelab.indexing.indexer.SUPPORTED_FRAME_DTYPES#

Built-in immutable sequence.

If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable’s items.

If the argument is a tuple, the return value is the same object.

SUPPORTED_FRAME_DTYPES lists the NumPy dtypes that Indexer.index accepts for a frame; see Frame input.