qsirecon.config module

A Python module to maintain unique, run-wide QSIRecon settings.

This module implements the memory structures to keep a consistent, singleton config. Settings are passed across processes via filesystem, and a copy of the settings for each run and subject is left under <output_dir>/sub-<participant_id>/log/<run_unique_id>/qsirecon.toml. Settings are stored using ToML. The module has a to_filename() function to allow writing out the settings to hard disk in ToML format, which looks like:

This config file is used to pass the settings across processes, using the load() function.

Configuration sections

class qsirecon.config.environment

Read-only options regarding the platform and environment.

Crawls runtime descriptive settings (e.g., default FreeSurfer license, execution environment, nipype and QSIRecon versions, etc.). The environment section is not loaded in from file, only written out when settings are exported. This config section is useful when reporting issues, and these variables are tracked whenever the user does not opt-out using the --notrack argument.

cpu_count = 2

Number of available CPUs.

exec_docker_version = None

Version of Docker Engine.

exec_env = 'posix'

A string representing the execution platform.

free_mem = 6.4

Free memory at start.

nipype_version = '1.8.6'

Nipype’s current version.

overcommit_limit = '50%'

Linux’s kernel virtual memory overcommit limits.

overcommit_policy = 'heuristic'

Linux’s kernel virtual memory overcommit policy.

templateflow_version = '24.2.2'

The TemplateFlow client version installed.

version = '0.23.1'

QSPrep’s version.

class qsirecon.config.execution

Configure run-level settings.

aggr_ses_reports = 4

Maximum number of sessions aggregated in one subject’s visual report.

bids_database_dir = None

Path to the directory containing SQLite database indices for the input BIDS dataset.

bids_description_hash = None

Checksum (SHA256) of the dataset_description.json of the BIDS dataset.

bids_dir = None

An existing path to the dataset, which must be BIDS-compliant.

bids_filters = None

A dictionary of BIDS selection filters.

boilerplate_only = False

Only generate a boilerplate.

dataset_links = {}

A dictionary of dataset links to be used to track Sources in sidecars.

debug = []

Debug mode(s).

derivatives = {}

Path(s) to search for pre-computed derivatives

freesurfer_input = None

Directory containing FreeSurfer directories to use for recon workflows.

fs_license_file = None

An existing file containing a FreeSurfer license.

fs_subjects_dir = None

FreeSurfer’s subjects directory.

classmethod init()

Create a new BIDS Layout accessible with layout.

input_dir = None

An existing path to the input data, which may not be BIDS-compliant (in which case a BIDS-compliant version will be created and stored as bids_dir).

interactive_reports_only = False

Only build the interactive reports, based on the output directory.

layout = None

A BIDSLayout object, see init().

log_dir = None

The path to a directory that contains execution logs.

log_level = 25

Output verbosity.

low_mem = None

Utilize uncompressed NIfTIs and other tricks to minimize memory allocation.

notrack = False

Do not collect telemetry information for QSIRecon.

output_dir = None

Folder where derivatives will be stored.

participant_label = None

List of participant identifiers that are to be preprocessed.

reports_only = False

Only build the reports, based on the reportlets found in a cached working directory.

run_uuid = '20241007-155022_1bd431ef-740b-492b-8d05-c699575da9f5'

Unique identifier of this particular run.

skip_odf_reports = False

Disable ODF recon reports.

sloppy = False

Run in sloppy mode (meaning, suboptimal parameters that minimize run-time).

templateflow_home = PosixPath('/home/docs/.cache/templateflow')

The root folder of the TemplateFlow client.

work_dir = PosixPath('/home/docs/checkouts/readthedocs.org/user_builds/qsirecon/checkouts/0.23.1/docs/work')

Path to a working directory where intermediate results will be available.

write_graph = False

Write out the computational graph corresponding to the planned postprocessing.

class qsirecon.config.workflow

Configure the particular execution graph of this workflow.

b0_threshold = None

Any value in the .bval file less than this will be considered a b=0 image.

infant = False

Configure pipelines specifically for infant brains

input_type = None

Specifies which pipeline was used to preprocess data in bids_dir.

output_resolution = None

Isotropic voxel size for outputs.

qsirecon_suffixes = []

List of reconstruction workflow names, derived from the recon_spec.

recon_spec = None

Recon workflow specification.

class qsirecon.config.nipype

Nipype settings.

crashfile_format = 'txt'

The file format for crashfiles, either text (txt) or pickle (pklz).

get_linked_libs = False

Run NiPype’s tool to enlist linked libraries for every interface.

classmethod get_plugin()

Format a dictionary for Nipype consumption.

classmethod init()

Set NiPype configurations.

memory_gb = None

Estimation in GB of the RAM this workflow can allocate at any given time.

nprocs = 2

Number of processes (compute tasks) that can be run in parallel (multiprocessing only).

omp_nthreads = None

Number of CPUs a single process can access for multithreaded execution.

plugin = 'MultiProc'

NiPype’s execution plugin.

plugin_args = {'maxtasksperchild': 1, 'raise_insufficient': False}

Settings for NiPype’s execution plugin.

remove_unnecessary_outputs = True

Clean up unused outputs after running

resource_monitor = False

Enable resource monitor.

stop_on_first_crash = True

Whether the workflow should stop or continue after the first error.

Usage

A config file is used to pass settings and collect information as the execution graph is built across processes.

from qsirecon import config
config_file = config.execution.work_dir / '.qsirecon.toml'
config.to_filename(config_file)
# Call build_workflow(config_file, retval) in a subprocess
with Manager() as mgr:
    from .workflow import build_workflow
    retval = mgr.dict()
    p = Process(target=build_workflow, args=(str(config_file), retval))
    p.start()
    p.join()
config.load(config_file)
# Access configs from any code section as:
value = config.section.setting

Logging

class qsirecon.config.loggers

Keep loggers easily accessible (see init()).

cli = <Logger cli (WARNING)>

Command-line interface logging.

default = <RootLogger root (WARNING)>

The root logger.

classmethod init()

Set the log level, initialize all loggers into loggers.

• Add new logger levels (25: IMPORTANT, and 15: VERBOSE).

• Add a new sub-logger (cli).

• Logger configuration.

interface = <Logger nipype.interface (INFO)>

NiPype’s interface logger.

utils = <Logger nipype.utils (INFO)>

NiPype’s utils logger.

workflow = <Logger nipype.workflow (INFO)>

NiPype’s workflow logger.

Other responsibilities

The config is responsible for other conveniency actions.

• Switching Python’s multiprocessing to forkserver mode.

• Set up a filter for warnings as early as possible.

• Automated I/O magic operations. Some conversions need to happen in the store/load processes (e.g., from/to Path <-> str, BIDSLayout, etc.)

qsirecon.config.dumps()

Format config into toml.

class qsirecon.config.environment

Bases: _Config

Read-only options regarding the platform and environment.

Crawls runtime descriptive settings (e.g., default FreeSurfer license, execution environment, nipype and QSIRecon versions, etc.). The environment section is not loaded in from file, only written out when settings are exported. This config section is useful when reporting issues, and these variables are tracked whenever the user does not opt-out using the --notrack argument.

cpu_count = 2

Number of available CPUs.

exec_docker_version = None

Version of Docker Engine.

exec_env = 'posix'

A string representing the execution platform.

free_mem = 6.4

Free memory at start.

nipype_version = '1.8.6'

Nipype’s current version.

overcommit_limit = '50%'

Linux’s kernel virtual memory overcommit limits.

overcommit_policy = 'heuristic'

Linux’s kernel virtual memory overcommit policy.

templateflow_version = '24.2.2'

The TemplateFlow client version installed.

version = '0.23.1'

QSPrep’s version.

class qsirecon.config.execution

Bases: _Config

Configure run-level settings.

aggr_ses_reports = 4

Maximum number of sessions aggregated in one subject’s visual report.

bids_database_dir = None

Path to the directory containing SQLite database indices for the input BIDS dataset.

bids_description_hash = None

Checksum (SHA256) of the dataset_description.json of the BIDS dataset.

bids_dir = None

An existing path to the dataset, which must be BIDS-compliant.

bids_filters = None

A dictionary of BIDS selection filters.

boilerplate_only = False

Only generate a boilerplate.

dataset_links = {}

A dictionary of dataset links to be used to track Sources in sidecars.

debug = []

Debug mode(s).

derivatives = {}

Path(s) to search for pre-computed derivatives

freesurfer_input = None

Directory containing FreeSurfer directories to use for recon workflows.

fs_license_file = None

An existing file containing a FreeSurfer license.

fs_subjects_dir = None

FreeSurfer’s subjects directory.

classmethod init()

Create a new BIDS Layout accessible with layout.

input_dir = None

An existing path to the input data, which may not be BIDS-compliant (in which case a BIDS-compliant version will be created and stored as bids_dir).

interactive_reports_only = False

Only build the interactive reports, based on the output directory.

layout = None

A BIDSLayout object, see init().

log_dir = None

The path to a directory that contains execution logs.

log_level = 25

Output verbosity.

low_mem = None

Utilize uncompressed NIfTIs and other tricks to minimize memory allocation.

notrack = False

Do not collect telemetry information for QSIRecon.

output_dir = None

Folder where derivatives will be stored.

participant_label = None

List of participant identifiers that are to be preprocessed.

reports_only = False

Only build the reports, based on the reportlets found in a cached working directory.

run_uuid = '20241007-155022_1bd431ef-740b-492b-8d05-c699575da9f5'

Unique identifier of this particular run.

skip_odf_reports = False

Disable ODF recon reports.

sloppy = False

Run in sloppy mode (meaning, suboptimal parameters that minimize run-time).

templateflow_home = PosixPath('/home/docs/.cache/templateflow')

The root folder of the TemplateFlow client.

work_dir = PosixPath('/home/docs/checkouts/readthedocs.org/user_builds/qsirecon/checkouts/0.23.1/docs/work')

Path to a working directory where intermediate results will be available.

write_graph = False

Write out the computational graph corresponding to the planned postprocessing.

qsirecon.config.from_dict(settings, init=True, ignore=None)

Read settings from a flat dictionary.

Parameters:

• setting (dict) – Settings to apply to any configuration

• init (bool or Container) – Initialize all, none, or a subset of configurations.

• ignore (Container) – Collection of keys in setting to ignore

qsirecon.config.get(flat=False)

Get config as a dict.

qsirecon.config.init_spaces(checkpoint=True)

Initialize the spaces setting.

qsirecon.config.load(filename, skip=None, init=True)

Load settings from file.

Parameters:

• filename (os.PathLike) – TOML file containing QSIRecon configuration.

• skip (dict or None) – Sets of values to ignore during load, keyed by section name

• init (bool or Container) – Initialize all, none, or a subset of configurations.

class qsirecon.config.loggers

Bases: object

Keep loggers easily accessible (see init()).

cli = <Logger cli (WARNING)>

Command-line interface logging.

default = <RootLogger root (WARNING)>

The root logger.

classmethod init()

Set the log level, initialize all loggers into loggers.

• Add new logger levels (25: IMPORTANT, and 15: VERBOSE).

• Add a new sub-logger (cli).

• Logger configuration.

interface = <Logger nipype.interface (INFO)>

NiPype’s interface logger.

utils = <Logger nipype.utils (INFO)>

NiPype’s utils logger.

workflow = <Logger nipype.workflow (INFO)>

NiPype’s workflow logger.

class qsirecon.config.nipype

Bases: _Config

Nipype settings.

crashfile_format = 'txt'

The file format for crashfiles, either text (txt) or pickle (pklz).

get_linked_libs = False

Run NiPype’s tool to enlist linked libraries for every interface.

classmethod get_plugin()

Format a dictionary for Nipype consumption.

classmethod init()

Set NiPype configurations.

memory_gb = None

Estimation in GB of the RAM this workflow can allocate at any given time.

nprocs = 2

Number of processes (compute tasks) that can be run in parallel (multiprocessing only).

omp_nthreads = None

Number of CPUs a single process can access for multithreaded execution.

plugin = 'MultiProc'

NiPype’s execution plugin.

plugin_args = {'maxtasksperchild': 1, 'raise_insufficient': False}

Settings for NiPype’s execution plugin.

remove_unnecessary_outputs = True

Clean up unused outputs after running

resource_monitor = False

Enable resource monitor.

stop_on_first_crash = True

Whether the workflow should stop or continue after the first error.

class qsirecon.config.seeds

Bases: _Config

Initialize the PRNG and track random seed assignments

ants = None

Seed used for antsRegistration, antsAI, antsMotionCorr

classmethod init()

master = None

Master random seed to initialize the Pseudorandom Number Generator (PRNG)

numpy = None

Seed used by NumPy

qsirecon.config.to_filename(filename)

Write settings to file.

class qsirecon.config.workflow

Bases: _Config

Configure the particular execution graph of this workflow.

b0_threshold = None

Any value in the .bval file less than this will be considered a b=0 image.

infant = False

Configure pipelines specifically for infant brains

classmethod init()

input_type = None

Specifies which pipeline was used to preprocess data in bids_dir.

output_resolution = None

Isotropic voxel size for outputs.

qsirecon_suffixes = []

List of reconstruction workflow names, derived from the recon_spec.

recon_spec = None

Recon workflow specification.