Tutorial 5: Analysis from memory

Instead of loading images and peaks from a file and converting them to polar coordinates at each step, we developed an approach that works directly from a pygid.Conversion instance. After the analysis, all results can be saved to a NeXus file for further use.

---

1. pygid.Conversion initialization

Create a pygid.Conversion class instance based on the experimental parameters and load metadata using pygid. The detailed description can be found in pygid documentation.

import pygid

# Experimental metadata
exp_metadata = pygid.ExpMetadata(
    start_time = r"2025-09-09T20:36:23.076828",
    end_time    = r"2025-09-09T20:37:24.076828",
    source_type = "synchrotron",
    source_name = "ESRF ID10",
    detector = "eiger4m",
    monitor = 294302)

# Sample metadata from YAML file
smpl_metadata = pygid.SampleMetadata(path_to_load="../../example/sample.yaml")

poni_path = '../../example/laB6_2025_09_05.poni'  # path to the PONI file
mask_path = '../../example/mask.npy'              # path to the mask file
filename = '../../example/eiger4m_0000.h5'        # path to the raw data
dataset = '/entry/data0/image'                    # image root in the raw data file
frame_num = None                                  # frame number to process

# Set the experimental parameters
params = pygid.ExpParams(
	poni_path = poni_path,
    mask_path = mask_path,
    fliplr = True,
    flipud = True,
    # transp = False,
    ai = 0.075
)

# Set the range, resolution and intensity corrections
matrix = pygid.CoordMaps(
    params,
    vert_positive = True, hor_positive = True,
    q_xy_range = (0, 3.5), q_z_range = (0, 3.5), dq = 0.002,
)

# Load the data
conversion = pygid.Conversion(
    matrix = matrix,
    path = filename,
    dataset = dataset,
    frame_num = frame_num
)

NOTE: the choice of the frame number will be ignored during the analysis. All images loaded at this step (pygid.Conversion) will be processed.

---

2. mlgidBASE initialization

The initialization of mlgidBASE should be performed before the conversion.

from mlgidbase import mlgidBASE
analysis = mlgidBASE(pygid_conversion=conversion)

2026-05-26 13:56:48.193521376 [W:onnxruntime:Default, device_discovery.cc:283 GetGpuDevices] Failed to detect devices under "/sys/class/drm/card0": device_discovery.cc:93 ReadFileContents Failed to open file: "/sys/class/drm/card0/device/vendor"

---

3. Run analysis methods

Run analysis functions as described in previous tutorials:

analysis.run_detection()
analysis.run_fitting()
analysis.run_matching(
    cif_prepr = r'../../example/prepr_cifs.pickle',
    peaks_type='segments')

INFO - Loading model

---

4. Save result

analysis.save_result(
    path_to_save='../../example/BA2PbI4.h5',
    smpl_metadata=smpl_metadata,
    exp_metadata=exp_metadata,
)

INFO - Saved in /home/docs/checkouts/readthedocs.org/user_builds/mlgidbase/checkouts/latest/example/BA2PbI4.h5 in group entry_0000

NOTE: The converted image is not available in memory after saving

---

Parameters

• path_to_save (str) – output path for the HDF5 result file.

• h5_group (str) – name of the NXentry dataset within the HDF5 file.

• overwrite_file (bool) – if True, overwrites existing HDF5 file. Default is True.

• overwrite_group (bool) – if True, overwrites existing NXentry group. Default is True.

• exp_metadata (ExpMetadata )– instance containing experimental information

• smpl_metadata (SampleMetadata ) – instance containing sample information

---

Batch processing

The analysis is not implemented for batch processing. For usage with huge amount of images use an external loop for data conversion and analysis:

for frame_num in range(100):
    # load a single image
    conversion = pygid.Conversion(
        matrix = matrix,
        path = filename,
        dataset = dataset,
        frame_num = frame_num)

    # run analysis
    analysis = mlgidBASE(pygid_conversion=conversion)
    analysis.run_detection()
    analysis.run_fitting()
    analysis.run_matching(
        cif_prepr = r'../../example/prepr_cifs.pickle',
        peaks_type='segments')

    # save result
    analysis.save_result(
        path_to_save='result_from_conversion.h5',
        smpl_metadata = smpl_metadata,
        exp_metadata=exp_metadata,
        overwrite_file=False,
        overwrite_group=False)