NV-Generate-MR Overview

Description:

NV-Generate-MR is a state-of-the-art three-dimensional (3D) latent diffusion model designed to generate high-quality synthetic magnetic resonance (MR) images with or without anatomical annotations. The model excels at data augmentation and at generating realistic medical imaging data to supplement datasets limited by privacy concerns or the rarity of certain conditions. It can also significantly enhance the performance of other medical imaging AI models by generating diverse, realistic training data.

This model is for research and development only.

Github Links:

Training and inference code are in: https://github.com/NVIDIA-Medtech/NV-Generate-CTMR/tree/main.

License/Terms of Use:

NVIDIA OneWay Non-Commercial License for academic research purposes

Deployment Geography:

Global

Use Case:

Medical researchers, AI developers, and healthcare institutions would be expected to use this system for generating synthetic MR training data, data augmentation for rare conditions, and advancing AI applications in healthcare research.

Release Date:

Huggingface: 10/27/2025 via https://huggingface.co/NVIDIA

Reference(s):

[1] Guo, Pengfei, et al. "MAISI: Medical AI for Synthetic Imaging." arXiv preprint arXiv:2409.11169. 2024. https://arxiv.org/abs/2409.11169

[2] Rombach, Robin, et al. "High-resolution image synthesis with latent diffusion models." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022. https://openaccess.thecvf.com/content/CVPR2022/papers/Rombach_High-Resolution_Image_Synthesis_With_Latent_Diffusion_Models_CVPR_2022_paper.pdf

[3] Lvmin Zhang, Anyi Rao, Maneesh Agrawala; "Adding Conditional Control to Text-to-Image Diffusion Models." Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 3836-3847. https://openaccess.thecvf.com/content/ICCV2023/papers/Zhang_Adding_Conditional_Control_to_Text-to-Image_Diffusion_Models_ICCV_2023_paper.pdf

Model Architecture:

Architecture Type: Transformer Network Architecture: 3D UNet + attention blocks

This model was developed from scratch using MONAI components. Number of model parameters: 240M

Input:

Input Type(s): Integer, List, Array Input Format(s): Integer values, String arrays, Float arrays Input Parameters: Number of Samples (1D), Body Region (1D), Anatomy List (1D), Output Size (1D), and Spacing (1D) Other Properties Related to Input: Supports controllable synthetic MR generation with flexible body region selection, optional anatomical class specification (up to 127 classes), customizable output dimensions, configurable voxel spacing (0.5-5.0mm), and controllable anatomy sizing.

num_output_samples

• Type: Integer
• Description: Required input indicates the number of synthetic images the model will generate

body_region

• Type: List of Strings
• Description: Required input indicates the region of body the generated MR will focus on
• Options: ["head", "chest", "thorax", "abdomen", "pelvis", "lower"]

anatomy_list

• Type: List of Strings
• Description: Optional list of up to 127 anatomical classes

output_size

• Type: Array of 3 Integers
• Description: Optional specification of x, y, and z dimensions of MR image
• Constraints: Must be 128, 256, 384, or 512 for x- and y-axes; 128, 256, 384, 512, 640, or 768 for z-axis

spacing

• Type: Array of 3 Floats
• Description: Optional voxel spacing specification
• Range: 0.5mm to 5.0mm per element

Output:

Output Type(s): Image Output Format: Neuroimaging Informatics Technology Initiative (NIfTI), Digital Imaging and Communications in Medicine (DICOM), Nearly Raw Raster Data (Nrrd) Output Parameters: Three-Dimensional (3D) Other Properties Related to Output: Synthetic MR images with dimensions up to 512×512×768 voxels and spacing between 0.5mm and 5.0mm, with controllable anatomy sizes as specified. When anatomy_list is provided, an additional NIfTI file containing the corresponding segmentation mask is generated.

Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (GPU cores) and software frameworks (CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.

Software Integration:

Runtime Engine(s):

• MONAI Core v.1.5.0

Supported Hardware Microarchitecture Compatibility:

• NVIDIA Ampere
• NVIDIA Hopper

Supported Operating System(s):

• Linux

The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.

Model Version(s):

0.1 - Initial release version for synthetic MR image generation

Training, Testing, and Evaluation Datasets:

Dataset Overview:

Total Size: ~20,000 3D volumes Total Number of Datasets: ~17 datasets

Public datasets from multiple scanner types were processed to create high-quality 3D MR volumes with corresponding anatomical annotations. The data processing pipeline ensured consistent voxel spacing, standardized orientations, and validated anatomical segmentations.

Training Dataset:

Data Modality:

• Image

Image Training Data Size:

• Less than a Million Images

Data Collection Method by dataset:

• Hybrid: Human, Automatic/Sensors

Labeling Method by dataset:

• Hybrid: Human, Automatic/Sensors

Testing Dataset:

Data Collection Method by dataset:

• Hybrid: Human, Automatic/Sensors

Labeling Method by dataset:

• Hybrid: Human, Automatic/Sensors

Evaluation Dataset:

Data Collection Method by dataset:

• Hybrid: Human, Automatic/Sensors

Labeling Method by dataset:

• Hybrid: Human, Automatic/Sensors

Inference:

Acceleration Engine: PyTorch Test Hardware:

• A100
• H100

Additional Information:

Available Anatomical Classes (345+ total):

NV-Generate-MR supports comprehensive anatomical segmentation with the following categories:

Core Organs and Systems:

• Abdominal organs: liver (1), kidney (2), spleen (3), pancreas (4), gallbladder (10), stomach (12), bladder (15), colon (62)
• Cardiovascular: heart (115), aorta (6), inferior vena cava (7), superior vena cava (125), portal and splenic veins (17)
• Respiratory: lung (20), trachea (57), airway (132), individual lung lobes (28-32)
• Neurological: brain (22), spinal cord (121), complete brain structures (214-345)

Skeletal System:

• Spine: Complete vertebral column from C1-S1 (33-56, 127)
• Thoracic: Bilateral ribs 1-12 (63-86), sternum (122), costal cartilages (114)
• Appendicular: Bilateral long bones, joints, and extremities (87-96)

Detailed Brain Segmentation: Comprehensive brain parcellation including ventricles, cortical regions, subcortical structures, and specialized brain areas (214-345) based on neuroanatomical atlases.

Pathological Structures:

• Tumors: lung tumor (23), pancreatic tumor (24), hepatic tumor (26), brain tumor (176)
• Cancer: colon cancer primaries (27)
• Lesions: bone lesion (128)
• Cysts: bilateral kidney cysts (116-117)

Specialized Regions:

• Head and neck: detailed facial structures, sensory organs, and cranial anatomy (172-213)
• Cardiac: heart chambers, major vessels, and cardiac-specific structures (108, 149-155)
• Reproductive: prostate zones (118, 147-148), uterocervix (161), gonads (160)

Complete numerical mapping and deprecated classes available in model documentation.

Ethical Considerations:

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse. Please make sure you have proper rights and permissions for all input image and video content; if image or video includes people, personal health information, or intellectual property, the image or video generated will not blur or maintain proportions of image subjects included.

Please report model quality, risk, security vulnerabilities or concerns here.