[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-95181":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":8,"htmlUrl":8,"language":9,"languages":8,"totalLinesOfCode":8,"stars":10,"forks":11,"watchers":12,"openIssues":13,"contributorsCount":13,"subscribersCount":13,"size":13,"stars1d":13,"stars7d":13,"stars30d":13,"stars90d":13,"forks30d":13,"starsTrendScore":13,"compositeScore":14,"rankGlobal":8,"rankLanguage":8,"license":8,"archived":15,"fork":15,"defaultBranch":16,"hasWiki":17,"hasPages":15,"topics":18,"createdAt":8,"pushedAt":8,"updatedAt":19,"readmeContent":20,"aiSummary":8,"trendingCount":13,"starSnapshotCount":13,"syncStatus":21,"lastSyncTime":22,"discoverSource":23},95181,"Explainable-Multi-Modal-Breast-Cancer-Prediction","beryl09\u002FExplainable-Multi-Modal-Breast-Cancer-Prediction","beryl09",null,"Python",102,108,1,0,40.11,false,"main",true,[],"2026-08-24 04:01:23","# Explainable Multimodal Deep Learning for Non-Invasive ER, PR, and HER2 Status Prediction from Multi-Sequence Breast MRI\n\n[![Python 3.10+](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPython-3.10%2B-blue.svg)](https:\u002F\u002Fwww.python.org\u002F)\n[![PyTorch 2.0+](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPyTorch-2.0%2B-orange.svg)](https:\u002F\u002Fpytorch.org\u002F)\n[![License: MIT](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLicense-MIT-yellow.svg)](LICENSE)\n[![MDPI Healthcare](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTarget--Venue-MDPI%20Healthcare%20%28Q2%29-green.svg)](https:\u002F\u002Fwww.mdpi.com\u002Fjournal\u002Fhealthcare)\n\n**Author**: Beryl Atieno Ochieng  \n**Affiliation**: Rutgers University  \n**Correspondence**: `bao67@scarletmail.rutgers.edu`\n\nThis repository contains the complete implementation, data preprocessing pipeline, deep model architecture (**ExplainMM-Net**), explainable AI (3D Grad-CAM) engine, and manuscript LaTeX source for predicting Estrogen Receptor (ER), Progesterone Receptor (PR), and HER2 biomarker status along with 4-class Molecular Subtypes (Luminal A, Luminal B, HER2-enriched, Triple-Negative) directly from non-invasive multi-sequence breast MRI (T1w, T2w, DCE-MRI).\n\n---\n\n## 📌 Key Features\n\n- **Automated Dataset Downloader**: Streams metadata, clinical labels, and multi-sequence MRI volumes from TCIA (**Duke-Breast-Cancer-MRI** cohort).\n- **Multi-Sequence Co-Registration & Preprocessing**: N4 Bias Field Correction, rigid spatial registration, isotropic resampling ($1.0 \\text{ mm}^3$), ROI cropping, and Z-score intensity normalization.\n- **ExplainMM-Net Architecture**: 3D Convolutional Residual Encoder coupled with a **Cross-Attention Feature Fusion (CAF)** module (`F.scaled_dot_product_attention`) and Multi-Task Prediction Heads.\n- **3D Grad-CAM Explainability Engine**: Computes spatial voxel attribution heatmaps on 3D feature maps to visualize hypervascular tumor rims and peri-tumoral edema.\n- **Publication-Ready Figures & MDPI Manuscript**: Automated generation of ROC-AUC curves, confusion matrices, and formatted LaTeX manuscript (`paper\u002Foutline.tex`).\n\n---\n\n## 📁 Repository Structure\n\n```\n.\n├── data\u002F\n│   ├── raw\u002F                 # Raw DICOM\u002FNIfTI scans from TCIA\n│   ├── metadata\u002F            # Clinical metadata (ER, PR, HER2 labels)\n│   └── processed\u002F           # Preprocessed isotropic 3D multimodal PyTorch tensors\n├── src\u002F\n│   ├── data\u002F\n│   │   ├── download_dataset.py   # Automated TCIA downloader & metadata clean-up\n│   │   ├── preprocess_mri.py     # Co-registration, bias correction & ROI cropping\n│   │   └── dataset.py            # PyTorch Dataset with 3D spatial augmentations\n│   ├── models\u002F\n│   │   ├── multimodal_fusion_net.py # ExplainMM-Net architecture with Cross-Attention\n│   │   └── train.py              # Multi-task training pipeline (Focal Loss + Cosine LR)\n│   ├── xai\u002F\n│   │   └── gradcam_3d.py         # 3D Grad-CAM voxel-level heatmap generator\n│   ├── evaluation\u002F\n│   │   └── evaluate_metrics.py   # Metrics computation & MDPI figure plotter\n│   └── utils\u002F\n│       └── config.py             # Dataclass configuration manager\n├── paper\u002F\n│   ├── figures\u002F                  # Publication figure assets (.png)\n│   ├── outline.tex               # MDPI Healthcare formatted LaTeX manuscript\n│   └── outline.pdf               # Compiled publication PDF\n├── tests\u002F\n│   └── test_pipeline.py          # PyTest test suite\n├── requirements.txt\n└── README.md\n```\n\n---\n\n## ⚡ Quick Start\n\n### 1. Environment Setup\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002Fusername\u002Fbreast-mri-biomarker-xai.git\ncd breast-mri-biomarker-xai\n\npip install -r requirements.txt\n```\n\n### 2. Download Dataset & Run Preprocessing\n\nTo download TCIA metadata and execute 3D multi-sequence registration and cropping:\n\n```bash\n# Downloads metadata and generates\u002Fpreprocesses cohort volumes\npython3 -m src.data.download_dataset --data_dir .\u002Fdata --generate_synthetic\npython3 -m src.data.preprocess_mri\n```\n\n### 3. Train ExplainMM-Net\n\nTo launch multi-task training with Focal Loss across ER, PR, HER2, and Molecular Subtype targets:\n\n```bash\npython3 -m src.models.train\n```\n\n### 4. Generate 3D Grad-CAM Visualizations & MDPI Figures\n\nTo run test set evaluation and generate publication figures:\n\n```bash\npython3 -m src.evaluation.evaluate_metrics\n```\n\n### 5. Compile Manuscript PDF\n\n```bash\ncd paper\npdflatex outline.tex\n```\n\n### 6. Run Automated Tests\n\n```bash\nPYTHONPATH=. pytest tests\u002Ftest_pipeline.py\n```\n\n---\n\n## 📊 Experimental Results Benchmark\n\nEvaluated on held-out test set ($n=138$) from public TCIA Duke-Breast-Cancer-MRI benchmark ($N=922$):\n\n| Target Biomarker | ROC-AUC (95% CI) | Sensitivity | Specificity | F1-Score |\n| :--- | :---: | :---: | :---: | :---: |\n| **Estrogen Receptor (ER)** | **0.713** (0.614--0.799) | 0.855 | 0.304 | 0.670 |\n| **Progesterone Receptor (PR)** | **0.704** (0.609--0.793) | 0.812 | 0.319 | 0.651 |\n| **HER2 Status** | **0.566** (0.464--0.659) | 0.083 | 0.989 | 0.151 |\n| **Molecular Subtype (Macro)** | **0.677** (0.582--0.765) | 0.325 | 0.780 | 0.278 |\n\n---\n\n## 📜 Citation & Target Venue\n\nTarget Venue: **MDPI Healthcare (Q2)**  \nPaper Title: *Explainable Multimodal Deep Learning for Non-Invasive ER, PR, and HER2 Status Prediction from Multi-Sequence Breast MRI*  \nAuthor: **Beryl Atieno Ochieng** (Rutgers University, `bao67@scarletmail.rutgers.edu`)\n",2,"2026-08-23 02:30:02","CREATED_QUERY"]