Skip to content

deeplearning-prepare-data - Prepare input data for deep learning with PyTorch

This pipeline prepares images generated by Clinica to be used with the PyTorch deep learning library [Paszke et al., 2019]. Four types of tensors are proposed: 3D images, 3D patches, 2D slices or 3D region of interest (ROI).

Outputs from the t1-linear pipeline, t1-extensive pipeline and pet-linear pipeline can be processed (for the moment, t1-extensive is available from ClinicaDL). These pipelines are designed as a prerequisite for the deep learning classification algorithms presented in [Wen et al., 2020] and showcased in the ClinicaDL framework.

Prerequisites

You need to have performed the t1-linear pipeline or t1-extensive pipeline on your T1-weighted MRI or pet-linear pipeline on your PET images. There exists an option to convert custom NIfTI images into tensors. This option is chosen by adding the parameter custom to the command line instruction and a known suffix, as explained below.

Dependencies

If you installed the core of Clinica, this pipeline needs no further dependencies.

Running the pipeline

The pipeline can be run with the following command line:

clinica run deeplearning-prepare-data [OPTIONS] CAPS_DIRECTORY [[t1-linear|t1-extensive|pet-linear|custom]]
                                      [[image|slice|patch]]

where:

  • CAPS_DIRECTORY is the folder containing the results of the pipeline used as input and the output of the present command, both in a CAPS hierarchy.
  • The second positional argument is the name of the image modality whose outputs are used as inputs It can be t1-linear, t1-extensive or pet-linear. You can choose custom if you want to get a tensor from a custom filename.
  • The third positional argument is the format of the extracted tensors. You can choose between image to convert to PyTorch tensor the whole 3D image, patch to extract 3D patches, slice to extract 2D slices from the image and roi to extract 3D region of interest depending on the ROI mask.

Warning

The roi option will only be available in release 0.5.0 and will then migrate to ClinicaDL: https://github.com/aramis-lab/clinicadl/

By default, the features are extracted from the cropped image (see the documentation of the t1-linear pipeline). You can deactivate this behaviour with the --use_uncropped_image flag. The same behaviour is applied to pet-linear pipeline.

Pipeline options if you use patch extraction:

  • --patch_size: patch size. Default value: 50.
  • --stride_size: stride size. Default value: 50.

Pipeline options if you use slice extraction:

  • --slice_direction: slice direction. You can choose between 0 (sagittal plane), 1(coronal plane) or 2 (axial plane). Default value: 0.
  • --slice_mode: slice mode. You can choose between rgb (will save the slice in three identical channels) or single (will save the slice in a single channel). Default value: rgb.

Pipeline options if you use roi extraction:

  • --roi_list: list of ROI to be extracted. The masks corresponding to these ROIs should be written in <caps_directory>/masks/tpl-<tpl_name>.
  • --roi_uncrop_output: disables cropping option, so the output tensors have the same size as the whole image instead of the ROI size.
  • --custom_template (mandatory for custom): only used when modality is set to custom. Sets the value of <tpl_name>.
  • --custom_mask_pattern (optional): only used when modality is set to custom. Allows to choose a particular mask with a name following the given pattern.

ROI masks

ROI masks are compressed NIfTI files (.nii.gz) containing a binary mask of the same size as the input data it corresponds to. All masks must follow the pattern tpl-<tpl_name>_*_roi-<roi_name>_mask.nii.gz.

If the defined region is not cubic, deeplearning-prepare-data will automatically extract the smallest bounding box around the region and fill the remaining values with 0 (unless --roi_uncrop_output is specified).

Masks must correspond to the template used in the pipeline for registration. For t1-linear and pet-linear it is automatically set to MNI152NLin2009cSym. For a custom modality this value must be set using custom_template.

The chosen mask will correspond to the mask with the shortest name following the wanted pattern.

Example of a valid CAPS hierarchy:

CAPS_DIRECTORY
├── masks
│       ├── tpl-<tpl_name>
│       │       ├── tpl-<tpl_name>[_custom_pattern]_roi-<roi_1>_mask.nii.gz
│       │       ├── ...
│       │       └── tpl-<tpl_name>[_custom_pattern]_roi-<roi_N>_mask.nii.gz
│       └── tpl-MNI152NLin2009cSym
│               ├── tpl-MNI152NLin2009cSym_desc-Crop_res-1x1x1_roi-<roi_1>_mask.nii.gz
│               ├── tpl-MNI152NLin2009cSym_desc-Crop_res-1x1x1_roi-<roi_2>_mask.nii.gz
│               ├── tpl-MNI152NLin2009cSym_res-1x1x1_roi-<roi_1>_mask.nii.gz
│               └── tpl-MNI152NLin2009cSym_res-1x1x1_roi-<roi_2>_mask.nii.gz
└── subjects
        └── ...

The first two masks in tpl-MNI152NLin2009cSym/ contain desc-Crop, hence they can only be applied to cropped input images, and their size will be (169x208x179). On the contrary the last two masks in the same folder do not contain desc-Crop hence they can only be applied to uncropped input images and their size will be (193x229x193).

Pipeline options if you use pet-linear modality:

  • --acq_label: the label given to the PET acquisition, specifying the tracer used (acq-<acq_label>). It can be for instance 'fdg' for 18F-fluorodeoxyglucose or 'av45' for 18F-florbetapir;
  • --suvr_reference_region: the reference region used to perform intensity normalization (i.e. dividing each voxel of the image by the average uptake in this region) resulting in a standardized uptake value ratio (SUVR) map. It can be cerebellumPons or cerebellumPons2 (used for amyloid tracers) and pons or pons2 (used for FDG). See PET introduction for more details about masks versions.

Note

Same values used for the pet-linear pipeline are needed for both options.

Pipeline options if you use custom modality:

  • --custom_suffix: suffix of the filename that should be converted to the tensor format. The output will be saved into a folder named custom but the processed files will keep their original name. For instance, you can convert the images from the segmentation of the grey matter registered on the Ixi549Space. These images are obtained by running t1-volume pipeline (and SPM underhood). The suffix for these images is graymatter_space-Ixi549Space_modulated-off_probability.nii.gz.

Regarding the default values

When using patch or slice extraction, default values were set according to [Wen et al., 2020].

Note

The arguments common to all Clinica pipelines are described in Interacting with clinica.

Tip

Do not hesitate to type clinica run deeplearning-prepare-data --help to see the full list of parameters.

Outputs

In the following subsections, files with the .pt extension denote tensors in PyTorch format.

The full list of output files can be found in the ClinicA Processed Structure (CAPS) Specification.

Image-based outputs

Results are stored in the following folder of the CAPS hierarchy: subjects/<participant_id>/<session_id>/deeplearning_prepare_data/image_based/t1_linear.

For the t1-linear modality, the output file is <source_file>_space-MNI152NLin2009cSym[_desc-Crop]_res-1x1x1_T1w.pt: a tensor version of the 3D T1w image registered to the MNI152NLin2009cSym template and optionally cropped.

Corresponding folder and file names are obtained for the files processed with the t1-extensive and pet-linear modality.

For the case of files processed with the custom modality, files are stored in the following folder: subjects/<participant_id>/<session_id>/deeplearning_prepare_data/image/custom.

Patch-based outputs

Results are stored in the following folder of the CAPS hierarchy: subjects/<participant_id>/<session_id>/deeplearning_prepare_data/patch_based/t1_linear.

The output files are <source_file>[_<input_pattern>]_patchsize-<N>_stride-<M>_patch-<i>_T1w.pt: tensor version of the <i>-th 3D isotropic patch of size <N> with a stride of <M>.

Slice-based outputs

Results are stored in the following folder of the CAPS hierarchy: subjects/<participant_id>/<session_id>/deeplearning_prepare_data/slice_based/t1_linear.

The output files are <source_file>[_<input_pattern>]_axis-{sag|cor|axi}_channel-{single|rgb}_T1w.pt: tensor version of the <i>-th 2D slice in sagittal, coronal or axial plane using three identical channels (rgb) or one channel (single).

ROI based outputs

Results are stored in the following folder of the CAPS hierarchy: subjects/<participant_id>/<session_id>/deeplearning_prepare_data/roi_based/t1_linear.

The output files are <source_file>_space-<tpl_name>[_desc-{CropRoi|CropImage|Crop}]_[mask_pattern]_roi-<roi_name>_T1w.pt: tensor version of the selected 3D region of interest.

The key value following desc depends on the input and output image:

  • desc-CropROI: the input image contains desc-Crop and ROI cropping is enabled,
  • desc-CropImage: the input image contains desc-Crop and ROI cropping is disabled,
  • desc-Crop: the input image do not contain desc-Crop and ROI cropping is enabled,
  • <no_descriptor>: the input image do not contain desc-Crop and ROI cropping is disabled.

Going further

Describing this pipeline in your paper

Example of paragraph

These results have been obtained using the deeplearning-prepare-data pipeline of Clinica [Routier et al., 2021; Wen et al., 2020]. More precisely,

  • 3D images

  • 3D patches with patch size of <patch_size> and stride size of <stride_size>

  • 2D slices in {sagittal | coronal | axial} plane and saved in {three identical channels | a single channel}

  • 3D regions of interest depending on a mask given in input

were extracted and converted to PyTorch tensors [Paszke et al., 2019].

Tip

Easily access the papers cited on this page on Zotero.