You need to agree to share your contact information to access this model
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
You agree to not use the model to conduct experiments that cause harm to human subjects.
Log in or Sign Up to review the conditions and access this model content.
MOrphing HistoPAthology DIffusion (MoPaDi)
Abstract
Deep learning can extract predictive and prognostic biomarkers from histopathology whole slide images, but its interpretability remains elusive. We develop and validate MoPaDi (Morphing histoPathology Diffusion), which generates counterfactual mechanistic explanations. MoPaDi uses diffusion autoencoders to manipulate pathology image patches and flip their biomarker status by changing the morphology. Importantly, MoPaDi includes multiple instance learning for weakly supervised problems. We validate our method on four datasets classifying tissue types, cancer types within different organs, center of slide origin, and a biomarker – microsatellite instability. Counterfactual transitions were evaluated through pathologists’ user studies and quantitative cell analysis. MoPaDi achieves excellent image reconstruction quality (multiscale structural similarity index measure 0.966–0.992) and good classification performance (AUCs 0.76–0.98). In a blinded user study for tissue-type counterfactuals, counterfactual images were realistic (63.3–73.3% of original images identified correctly). For other tasks, pathologists identified meaningful morphological features from counterfactual images. Overall, MoPaDi generates realistic counterfactual explanations that reveal key morphological features driving deep learning model predictions in histopathology, improving interpretability.
Installation
Clone MoPaDi repository, create a virtual environment, e.g. with conda or mamba, and install required packages:
git clone https://github.com/KatherLab/mopadi.git && cd mopadi
mamba create -n mopadi python=3.8 -c conda-forge
pip install -r requirements.txt
Example jupyter notebooks, showing how to use these pretrained models to generate counterfactual images are provided on GitHub.
Models
In this repository you can find the following models:
- Tissue classes autoencoding diffusion model (trained on 224 x 224 px tiles from NCT-CRC-HE-100K dataset (Kather et al., 2018)) + linear 9 classes classifier (Adipose (ADI), background (BACK), debris (DEB), lymphocytes (LYM), mucus (MUC), smooth muscle (MUS), normal colon mucosa (NORM), cancer-associated stroma (STR), colorectal adenocarcinoma epithelium (TUM));
- Colorectal (CRC) cancer autoencoding diffusion model (trained on 512 x 512 px tiles (0.5 microns per px, MPP) from tumor regions from TCGA CRC cohort) + microsatellite instability (MSI) status MIL classifier (MSI high [MSIH] vs. nonMSIH);
- Breast cancer (BRCA) autoencoding diffusion model (trained on 512 x 512 px tiles (0.5 MPP) from tumor regions from TCGA BRCA cohort) + breast cancer type (invasive lobular carcinoma [ILC] vs. invasive ductal carcinoma [IDC]) and E2 center MIL classifiers;
- Pancancer autoencoding diffusion model (trained on 256 x 256 px tiles (varying MPP) from histology images from uniform tumor regions in TCGA WSI (Komura & Ishikawa, 2021)) + liver cancer types (hepatocellular carcinoma [HCC] vs. cholangiocarcinoma [CCA]) MIL & linear classifiers and lung cancer types (lung adenocarcinoma [LUAD] vs. lung squamous cell carcinoma [LUSC]) MIL & linear classifiers.
Examples of counterfactual images generated with corresponding models (please refer to the preprint for more examples):
Citation
If you find our work useful in your research or if you use parts of this code please consider citing our preprint:
@misc
{zigutyte2024mopadi,
title={ounterfactual Diffusion Models for Mechanistic Explainability of Artificial Intelligence Models in Pathology},
author={Laura Žigutytė and Tim Lenz and Tianyu Han and Katherine Jane Hewitt and Nic Gabriel Reitsam and Sebastian Foersch and Zunamys I Carrero and Michaela Unger and Alexander T Pearson and Daniel Truhn and Jakob Nikolas Kather},
year={2024},
eprint={2024.10.29.620913},
archivePrefix={bioRxiv},
url={https://www.biorxiv.org/content/10.1101/2024.10.29.620913v1},
}