Processing#

The preferred API either indexes one frame with index_frame or reuses an Indexer across frames that share configuration.

One-Off Indexing#

lauelab.indexing.index_frame(frame, *, geometry, crystal=None, peak_params=None, index_params=None, detector_index=0, detector_id=None, cosmic_filter=False, start=(0, 0), group=(1, 1), depth=None, mask=None, metadata=None, keep_image=True)[source]#

Index one frame with a temporary Indexer.

Parameters:

  • frame (np.ndarray | str | Path) – Two-dimensional NumPy array with a dtype in SUPPORTED_FRAME_DTYPES, or path to a supported HDF5 frame.

  • geometry (str | Path | Geometry) – Parsed detector geometry or path to a geometry XML file.

  • crystal (str | Path | Crystal | None) – Crystal description or crystal XML path. If None, peak search and pixel-to-q conversion run without orientation indexing.

  • peak_params (PeakParams | None) – Peak-search configuration. Defaults to PeakParams.

  • index_params (IndexParams | None) – Orientation-indexing configuration. Defaults to IndexParams.

  • detector_index (int) – Physical detector slot in the geometry. This is not the ordinal position among active detectors.

  • detector_id (str | None) – Detector identifier to select instead of detector_index.

  • cosmic_filter (bool) – Must be False. Cosmic-ray filtering is unsupported for indexing; True raises InputError.

  • start (tuple[int, int]) – Full-detector ROI origin and pixel grouping in (x, y) order.

  • group (tuple[int, int]) – Full-detector ROI origin and pixel grouping in (x, y) order.

  • depth (float | None) – Physical sample depth in micrometres. None uses the HDF5 frame’s entry1/depth when present and finite, otherwise no depth.

  • mask (np.ndarray | None) – Optional mask matching the frame shape. Nonzero pixels are excluded.

  • metadata (FrameMetadata | Mapping[str, object] | None) – Optional frame metadata object or mapping.

  • keep_image (bool) – Retain the contiguous input image in the returned result.

Returns:

FrameResult – A self-contained frame result. The input image is retained by default.

Raises:

  • InputError – If configuration, detector selection, or frame input is invalid.

  • MemoryError – If a native stage cannot allocate required memory.

  • IndexingError – If a native numerical or internal indexing stage fails.

Return type:

FrameResult

Notes

Construct Indexer directly when processing multiple frames so parsed geometry and crystal state can be reused.

Reusable Indexer#

class lauelab.indexing.Indexer(geometry, crystal=None, *, peak_params=None, index_params=None, detector_index=0, detector_id=None, cosmic_filter=False)[source]#

Reusable in-process Laue frame indexer.

Peak search, pixel-to-q conversion, and crystal indexing run through the native library without subprocesses or per-frame intermediate text files.

Parameters:

  • geometry (str | Path | Geometry) – Parsed detector geometry or path to a geometry XML file.

  • crystal (str | Path | Crystal | None) – Crystal description or crystal XML path. If None, frames are peak searched and converted to q space without orientation indexing.

  • peak_params (PeakParams | None) – Peak-search configuration. Defaults to PeakParams.

  • index_params (IndexParams | None) – Orientation-indexing configuration. Defaults to IndexParams.

  • detector_index (int) – Physical detector slot in the geometry. This is not the ordinal position among active detectors.

  • detector_id (str | None) – Detector identifier to select instead of detector_index.

  • cosmic_filter (bool) – Must be False. Cosmic-ray filtering is unsupported for indexing; True raises InputError.

Raises:

  • InputError – If parameters or detector selection are invalid.

  • ValueError – If a geometry or crystal description is invalid.

  • ImportError – If the native indexing library is unavailable.

Notes

Reuse one instance for frames that share geometry, crystal, detector, and processing parameters. Concurrent calls to index() from several Python threads are unsupported. For parallel work, give each worker process its own indexer; see Batch indexing.

Construction normalizes whole-number floats in the parameter objects to int while preserving their values. The parameter types in indexer.peak_params can therefore differ from those supplied.

Use replace() to create a separately validated instance with changed configuration.

index(frame, *, start=(0, 0), group=(1, 1), depth=None, mask=None, metadata=None, keep_image=True)[source]#

Process one NumPy frame, HDF5 file, or scan frame without subprocesses.

Parameters:

  • frame (np.ndarray | str | Path | ScanFrame) – Two-dimensional NumPy array with a dtype in SUPPORTED_FRAME_DTYPES, path to a supported HDF5 frame whose image has one of those dtypes, or a ScanFrame naming one stored frame of a reconstructed point file. Pixel values enter peak search as their exact double value; no other dtype is converted.

  • start (tuple[int, int]) – Zero-based detector (x, y) origin for an in-memory frame.

  • group (tuple[int, int]) – Positive detector-pixel grouping factors as (x, y).

  • depth (float | None) – Physical sample depth in micrometres passed to pixel-to-q conversion. None uses the HDF5 frame’s entry1/depth when that dataset is present and finite, the stored depth of a ScanFrame, and no depth otherwise. An explicit finite value, including 0.0, overrides the file.

  • mask (np.ndarray | None) – Array with the same shape as frame. Values are converted to a boolean uint8 mask and passed to native peak search, where nonzero pixels are masked.

  • metadata (FrameMetadata | Mapping[str, object] | None) – Experiment metadata for XML output. Explicit values override metadata loaded from an HDF5 file.

  • keep_image (bool) – Retain the contiguous input image in the returned result’s image attribute.

Returns:

FrameResult – A self-contained result whose native backing allocations have already been released.

Raises:

  • InputError – If the frame, region, mask, metadata, or selected detector is invalid, or an HDF5 input is missing, unreadable, or malformed. File-reading errors are preserved as the exception’s cause.

  • MemoryError – If a native stage cannot allocate required memory.

  • IndexingError – If a native numerical or internal stage fails.

Return type:

FrameResult

Notes

For HDF5 and scan-frame input, detector start and group values from the file take precedence over method arguments when present; depth works the other way, the argument overriding the file. A scan frame is read exactly: one frame, its point’s detector metadata, and its physical depth. The rest of the stack is not loaded. A retained image can alias a contiguous array supplied by the caller. Native peak search uses a separate working copy, so smoothing does not modify that array.

A frame stored with a non-native byte order is byte-swapped to a native-order copy of the same dtype before processing. Floating-point frames must be finite.

Frame statistics exclude masked pixels and always describe the raw input image. Smoothing applies only to peak detection and fitting. Under automatic thresholding, a frame with no unmasked nonzero pixels returns a valid empty result with threshold_used set to NaN.

Native code can write diagnostics to stdout or stderr before Python raises an exception.

index_many(frames, *, keep_images=False)[source]#

Index frames in order while reusing parsed configuration.

Parameters:

  • frames (Iterable[np.ndarray | str | Path]) – Iterable of two-dimensional arrays with supported dtypes or supported HDF5 paths.

  • keep_images (bool) – Retain each input image in its result. Defaults to False to limit batch memory use.

Returns:

list[FrameResult] – Results in the same order as the input iterable.

Raises:

  • InputError – If a frame or its metadata is invalid.

  • MemoryError – If a native stage cannot allocate required memory.

  • IndexingError – If a native numerical or internal stage fails.

Return type:

list[FrameResult]

Notes

Frames are processed sequentially. Processing stops on the first exception.

iter_index(inputs, *, mask=None, workers=1, max_in_flight=None, should_stop=None, keep_images=False, poll_seconds=0.25)[source]#

Index frames in worker processes, yielding one outcome per input in order.

Parameters:

  • inputs (Iterable) – Iterable of FrameInput objects, or of frames accepted by index() (arrays or HDF5 paths), which are wrapped with default processing values and no input_id. The iterable is consumed lazily as the in-flight window admits work.

  • mask (np.ndarray | None) – Optional mask shared by every frame, with nonzero pixels excluded. It is sent to each worker once, not with every task.

  • workers (int) – Positive number of worker processes. Each builds its own Indexer from this indexer’s geometry path, crystal, parameters, detector selection, and cosmic_filter.

  • max_in_flight (int | None) – Bound on inputs submitted but not yet consumed, which also bounds the results held for in-order delivery. Must be at least workers. Defaults to 2 * workers.

  • should_stop – Optional callable returning True to request a cooperative stop. It is polled before every submission and at least every poll_seconds while waiting. On True, no further inputs are admitted, unstarted inputs are withdrawn, running inputs finish and are yielded, then iteration ends with stopped set.

  • keep_images (bool) – Retain each input image in its result. Defaults to False; a retained image adds one detector-sized array to every result sent between processes.

  • poll_seconds (float) – Longest interval between should_stop checks while waiting.

Returns:

FrameOutcomes – Iterate inside a with block. Each FrameOutcome holds either a FrameResult, including results with no peaks or patterns, or an expected input error from EXPECTED_INPUT_ERRORS, and the iteration continues after such errors.

Raises:

  • InputError – If workers, max_in_flight, should_stop, poll_seconds, or mask is invalid.

  • WorkerError – During iteration, if a worker cannot initialize, raises an unexpected exception, or the pool breaks. Iteration cannot continue.

Notes

Workers use the spawn start method; guard module-level calls with if __name__ == "__main__":. This method schedules computation only. Writing results, recording failures, and deciding what to do after a stop belong to the caller.

replace(**changes)[source]#

Create an indexer with selected configuration values replaced.

Parameters:

**changes – Constructor arguments to replace. Supported names are geometry, crystal, peak_params, index_params, detector_index, detector_id, and cosmic_filter.

Returns:

Indexer – A new, independently validated indexer.

Raises:

  • TypeError – If an unknown constructor argument is supplied.

  • InputError – If replacement parameters or detector selection are invalid.

Return type:

Indexer

Notes

Native geometry and crystal state is constructed for the new instance; it is not shared with the original indexer.

results_writer(path, **kwargs)[source]#

Create a streaming HDF5 writer bound to this indexer’s configuration.

write_results(results, path, *, frame_ids=None, overwrite=False)[source]#

Write an iterable of results to one indexing-results HDF5 file.

write_many_xml(results, path)[source]#

Write multiple results to one LaueGo XML document.

Parameters:

  • results (Iterable[FrameResult]) – Frame results written in iteration order.

  • path (str | Path) – Destination XML file. An existing file is replaced.

Raises:

  • RuntimeError – If any result has no XML snapshot.

  • OSError – If the destination cannot be written.

Notes

Results are serialized one at a time through XmlResultsWriter, so memory use does not grow with the number of results. The output is identical to writing the collected steps in one call. The document is written under the destination’s .partial name and renamed into place on success, so a failure leaves an existing destination as it was.

Individual stages#

Find and fit peaks in one image, the first stage of indexing.

This function uses the same native code as Indexer.index() and returns the same peaks for the same image, mask, and settings. No geometry or crystal description is needed, so you can compare peak-search settings before running the later stages.

Parameters:

  • image (np.ndarray) – Two-dimensional NumPy array with a dtype in SUPPORTED_FRAME_DTYPES. Peak search converts each pixel value exactly to float64. To search a frame that is stored in a file, use the image of a FrameResult or read the array with h5py.

  • boxsize (int) – Peak-search settings with the names, defaults, units, and validation of the PeakParams fields.

  • max_rfactor (float) – Peak-search settings with the names, defaults, units, and validation of the PeakParams fields.

  • min_size (float) – Peak-search settings with the names, defaults, units, and validation of the PeakParams fields.

  • min_separation (int) – Peak-search settings with the names, defaults, units, and validation of the PeakParams fields.

  • threshold (float | None) – Peak-search settings with the names, defaults, units, and validation of the PeakParams fields.

  • threshold_ratio (float | None) – Peak-search settings with the names, defaults, units, and validation of the PeakParams fields.

  • peak_shape (Literal['Lorentzian', 'Gaussian']) – Peak-search settings with the names, defaults, units, and validation of the PeakParams fields.

  • max_peaks (int | None) – Peak-search settings with the names, defaults, units, and validation of the PeakParams fields.

  • smooth (bool) – Peak-search settings with the names, defaults, units, and validation of the PeakParams fields.

  • mask (np.ndarray | None) – Array with the same shape as image. Peak search excludes the pixels where the mask is nonzero.

Returns:

PeakSearch – The fitted peaks in frame pixel coordinates, the frame statistics, and the validated settings in params.

Raises:

  • InputError – If a setting, the image, or the mask is invalid.

  • MemoryError – If native peak search cannot allocate required memory.

  • IndexingError – If native peak search fails numerically or internally.

Return type:

PeakSearch

Notes

Convert the peaks to scattering vectors with Geometry.pixels_to_q(), which accepts PeakSearch.peaks directly, then pass the vectors to index_orientations().

class lauelab.indexing.PeakSearch(peaks, params, threshold_used, threshold_ratio, total_sum, sum_above_threshold, num_above_threshold, peak_minwidth, peak_maxwidth, peak_max_cent_to_fit, peak_boxsize, seconds)[source]#

Peaks and frame statistics from peak_search().

Parameters:

  • peaks (numpy.ndarray) – Structured peak array with shape (n,). The fields are those of FrameResult.peaks except qhat: fit_x, fit_y, intens, integral, hwhm_x, hwhm_y, tilt, chisq, and background. fit_x and fit_y are zero-based frame pixel coordinates of the searched image. The peaks are sorted by the maximum pixel value in the blob used for each fit, largest first.

  • params (lauelab.indexing.indexer.PeakParams) – The validated peak-search configuration, including defaults. Pass it as peak_params to Indexer to index a scan with the same settings.

  • threshold_used (float) – Intensity threshold used by peak search. This is NaN when the threshold is automatic and the image has no unmasked nonzero pixels.

  • threshold_ratio (float) – Ratio used to calculate the automatic threshold. This is NaN when an absolute threshold is set.

  • total_sum (float) – Sum of unmasked raw image pixel values.

  • sum_above_threshold (float) – Sum of pixel values above threshold_used.

  • num_above_threshold (int) – Number of pixels above threshold_used.

  • peak_minwidth (float) – Effective fitting parameters configured by the native peak search.

  • peak_maxwidth (float) – Effective fitting parameters configured by the native peak search.

  • peak_max_cent_to_fit (float) – Effective fitting parameters configured by the native peak search.

  • peak_boxsize (int) – Effective fitting parameters configured by the native peak search.

  • seconds (float) – Elapsed native peak-search time in seconds.

Notes

Fields shared with FrameResult have the same meaning.

property n_peaks: int#

Number of detected peaks.

lauelab.indexing.index_orientations(qhat, crystal, *, kev_max_calc=30.0, kev_max_test=35.0, angle_tolerance_deg=0.12, cone_deg=72.0, hkl_prefer=(0, 0, 1), max_data=250)[source]#

Find crystal orientations that fit scattering vectors, the last stage of indexing.

This function uses the same native code as Indexer.index() and returns the same patterns for the same scattering vectors, crystal, and settings. You can compare indexing settings or select a subset of peaks without repeating peak search or pixel-to-q conversion.

Parameters:

  • qhat (np.ndarray) – Scattering vectors with shape (n, 3) in the 34-ID-E laboratory frame, such as the result of Geometry.pixels_to_q() or the qhat field of FrameResult.peaks. Each row is normalized before use.

  • crystal (str | Path | Crystal) – Crystal description or crystal XML path.

  • kev_max_calc (float) – Indexing settings with the names, defaults, units, and validation of the IndexParams fields.

  • kev_max_test (float) – Indexing settings with the names, defaults, units, and validation of the IndexParams fields.

  • angle_tolerance_deg (float) – Indexing settings with the names, defaults, units, and validation of the IndexParams fields.

  • cone_deg (float) – Indexing settings with the names, defaults, units, and validation of the IndexParams fields.

  • hkl_prefer (tuple[int, int, int]) – Indexing settings with the names, defaults, units, and validation of the IndexParams fields.

  • max_data (int) – Indexing settings with the names, defaults, units, and validation of the IndexParams fields.

Returns:

tuple[Pattern, …] – A tuple of the orientations found, or an empty tuple if none are found. Pattern.pk_index contains zero-based row indices into qhat.

Raises:

  • InputError – If a setting or the crystal is invalid, or qhat does not have shape (n, 3), contains a non-finite value, or contains a zero vector among the rows used.

  • MemoryError – If native crystal indexing cannot allocate required memory.

  • IndexingError – If native crystal indexing fails numerically or internally.

Return type:

tuple[Pattern, …]

Notes

The row order is significant. This function uses only the first max_data rows. peak_search() sorts its peaks by the maximum pixel value of each blob, largest first, so if qhat is computed from all of those peaks in order, the peaks with the brightest blobs are indexed. If qhat has fewer than two rows, this function returns an empty tuple, as Indexer.index() does.

Use Geometry.pixels_to_q for pixel-to-q conversion; see the Geometry reference for details. For a worked example of all three stages, see Run the indexing stages separately.

Parallel Indexing#

class lauelab.indexing.FrameInput(frame, input_id=None, start=(0, 0), group=(1, 1), depth=None, metadata=None)[source]#

One frame to index, with its identity and per-frame processing values.

Parameters:

  • frame (np.ndarray | str | Path | ScanFrame) – Two-dimensional array with a supported dtype, path to a supported HDF5 frame, or ScanFrame. An array is sent to the worker with the task; a path or scan frame is opened inside the worker.

  • input_id (Hashable | None) – Caller-defined stable identity carried unchanged into the outcome. None when the caller has no identity beyond the input position.

  • start (tuple[int, int]) – Full-detector ROI origin and pixel grouping in (x, y) order, as for Indexer.index().

  • group (tuple[int, int]) – Full-detector ROI origin and pixel grouping in (x, y) order, as for Indexer.index().

  • depth (float | None) – Physical sample depth in micrometres, or None to use the HDF5 frame’s entry1/depth when present.

  • metadata (object | Mapping[str, object] | None) – Optional frame metadata object or mapping.

class lauelab.indexing.FrameOutcome(input_index, input_id, result=None, error=None, seconds=0.0)[source]#

The result of attempting one input.

Parameters:

  • input_index (int) – Zero-based position of the input in the iterable passed to Indexer.iter_index().

  • input_id (Hashable | None) – The input_id of the corresponding FrameInput, or None.

  • result (object | None) – The FrameResult when the frame was processed, including a processed frame with no peaks or no patterns.

  • error (Exception | None) – The expected input error when the frame could not be processed, one of EXPECTED_INPUT_ERRORS. Exactly one of result and error is set.

  • seconds (float) – Wall time of the indexing call inside the worker.

property ok: bool#

Whether the frame was processed.

class lauelab.indexing.FrameOutcomes(indexer, inputs, *, mask, workers, max_in_flight, should_stop, keep_images, poll_seconds)[source]#

Ordered outcomes of a bounded parallel indexing run.

Instances are created by Indexer.iter_index(). Use one as a context manager and iterate over it; leaving the with block shuts the worker processes down whether the iteration finished, stopped early, or raised.

stopped#

True once should_stop returned True and admission ended.

Type:

bool

n_submitted#

Inputs handed to the worker pool.

Type:

int

n_yielded#

Outcomes returned so far.

Type:

int

n_cancelled#

Submitted inputs withdrawn before a worker started them, after a stop.

Type:

int

peak_in_flight#

Highest number of submitted-but-unconsumed inputs observed. It never exceeds max_in_flight.

Type:

int

Notes

Outcomes come back in input order. An input whose worker call is slow holds back later outcomes, which wait in the bounded in-flight window; no further inputs are admitted until the head is consumed. Inputs that were never started produce no outcome; compare n_submitted - n_cancelled with the number of inputs to find them.

Workers use the spawn start method, so a script calling Indexer.iter_index() at module level must guard that call with if __name__ == "__main__":.

close()[source]#

Withdraw unstarted work, wait for running work, and stop the workers.

Safe to call more than once. After close() the iterator is finished.

lauelab.indexing.EXPECTED_INPUT_ERRORS#

Built-in immutable sequence.

If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable’s items.

If the argument is a tuple, the return value is the same object.

EXPECTED_INPUT_ERRORS are the exception types that describe one input and are returned in FrameOutcome.error rather than raised; see Batch indexing.

Frame Inputs#

lauelab.indexing.load_mask(path)[source]#

Load a peak-search mask stored as a 34-ID-E HDF5 image.

Parameters:

path (str | Path) – HDF5 file whose entry1/data/data dataset holds the mask image. Any numeric dtype is accepted.

Returns:

numpy.ndarray – Two-dimensional boolean array. True marks a pixel excluded from peak search. This follows the LaueGo peaksearch -K convention: nonzero mask pixels are excluded and zero pixels remain available.

Raises:

  • OSError – If the file cannot be opened.

  • KeyError – If the image dataset is missing.

  • ValueError – If the dataset is not a two-dimensional numeric array.

Return type:

np.ndarray

Notes

The mask shape is checked against the frame when it is passed to Indexer.index(), not here.

lauelab.indexing.read_frame(frame)[source]#

Read one stored frame without processing it.

Parameters:

frame (str | PathLike[str] | ScanFrame) – Path to a 34-ID-E HDF5 frame, or a ScanFrame selecting a reconstructed depth image.

Returns:

tuple – (image, metadata, processing). image is the two-dimensional pixel array indexed [y, x], in the type stored in the file. metadata is a dict of the FrameMetadata field names found in the file. processing holds any of start and group, each an (x, y) pair giving the full-detector ROI origin and pixel grouping, and depth in µm. A key is absent when the file does not record it.

Raises:

InputError – If the file cannot be read or is not a supported frame.

class lauelab.indexing.ScanFrame(path, point_id, depth_index)[source]#

Reference to one reconstructed depth image for processing.

Supply the point file path, point ID, and depth index to select a stored frame. The reference can be pickled, sent to a worker, and saved with indexing results. Each process opens the point file when it reads the frame; a scan catalog is unnecessary.

Parameters:

  • path (pathlib.Path or str) – A point file, such as points/<input stem>.h5 of a scan directory written by reconstruct_scan(), or a standalone file written by reconstruct_point(). ScanReader.point_path resolves a catalog point ID to this path.

  • point_id (str) – Expected point ID. Reading fails if the ID in the file differs.

  • depth_index (int) – Zero-based position of the frame in the point’s depth stack. Its physical depth in µm is read from the file.

lauelab.indexing.indexer.SUPPORTED_FRAME_DTYPES#

Built-in immutable sequence.

If no argument is given, the constructor returns an empty tuple. If iterable is specified the tuple is initialized from iterable’s items.

If the argument is a tuple, the return value is the same object.

SUPPORTED_FRAME_DTYPES lists the NumPy dtypes that Indexer.index accepts for a frame; see Frame input.