amonfortc/FibroPred · Hugging Face

README: FibroPred Predictive System

Description

FibroPred is a predictive system designed to analyze clinical data and provide predictions for specific medical outcomes related to fibrosis. The tool employs machine learning models, primarily Random Forest classifiers, to forecast the likelihood of various events or conditions, such as mortality, necessity for transplantation, and progressive disease.

The system provides an easy-to-use interface built with Gradio, allowing users to input feature values and obtain predictions. Models and their configurations are stored and loaded dynamically, ensuring modularity and adaptability.

Features

Prediction Targets:
- Death: Likelihood of patient mortality based on clinical and diagnostic features.
- Binary Diagnosis: Classification of patients into specific diagnostic categories.
- Necessity of Transplantation: Assessment of whether a patient is likely to need a transplant.
- Progressive Disease: Prediction of disease progression based on longitudinal data.
Dynamic Model Loading: Models and feature sets are loaded dynamically from a pre-configured directory (models).
Gradio Interface: A tabbed interface for each prediction target, where users can input values interactively and receive predictions in real-time.

Predictive Features

Below are the features used for prediction across all targets:

Pedigree (0 - 67): Represents the familial history related to fibrotic conditions.
Age at diagnosis (36.0 - 92.0): Age of the patient at the time of diagnosis. A critical factor as progression and treatment response vary with age.
FVC (L) at diagnosis (0.0 - 5.0): Forced vital capacity in liters at the time of diagnosis, reflecting lung function.
FVC (%) at diagnosis (0.0 - 200.0): Forced vital capacity as a percentage of the expected value for the patient’s age and sex.
DLCO (%) at diagnosis (0.0 - 200.0): Diffusion capacity for carbon monoxide as a percentage, measuring gas exchange efficiency in the lungs.
RadioWorsening2y (0 - 3): Radiological assessment of lung deterioration over two years. Higher values indicate significant progression.
Severity of telomere shortening - Transform 4 (1 - 6): Indicates the degree of telomere shortening.
Progressive disease (0 - 1): Binary variable indicating whether the disease is progressive (1) or stable (0).
Antifibrotic Drug (0 - 1): Binary variable representing the use of antifibrotic drugs. 1 indicates use; 0 indicates none.
Prednisone (0 - 1): Binary variable reflecting prednisone usage. 1 indicates use; 0 indicates none.
Mycophenolate (0 - 1): Binary variable indicating mycophenolate usage. 1 indicates use; 0 indicates none.
FVC (L) 1 year after diagnosis (0.0 - 5.0): Forced vital capacity in liters one year after diagnosis, used to evaluate changes in lung function.
FVC (%) 1 year after diagnosis (0.0 - 200.0): Forced vital capacity as a percentage one year after diagnosis.
DLCO (%) 1 year after diagnosis (0.0 - 200.0): Diffusion capacity for carbon monoxide as a percentage one year after diagnosis.
Genetic mutation studied in patient (0 - 1): Binary variable indicating the presence of specific genetic mutations. 1 indicates mutation found; 0 indicates none.
Comorbidities (0 - 1): Binary variable representing the presence of relevant comorbidities. 1 indicates presence; 0 indicates absence.

Setup Instructions

Clone or download the repository.
Ensure Python 3.8+ is installed.
Install dependencies using the command:
```
pip install -r requirements.txt
```
Place trained models in the models directory. Models should be .pkl files containing:
- model: Trained Random Forest model.
- features: Feature list used during model training.

Usage

Run the application:
```
python app.py
```
Access the Gradio interface through the displayed local or public URL.
Select a prediction tab, input feature values, and click "Submit" to get predictions.

Key Scripts

app.py:
- Implements the Gradio interface.
- Maps user-friendly model names to actual models and their features.
fibropred_model.py:
- Contains the preprocessing pipeline, including imputation and feature selection.
- Includes functions for training, evaluation, and visualization.
model_utils.py:
- Functions to load models and their features dynamically.
- Handles predictions using preloaded models.
requirements.txt:
- Lists the Python dependencies required for the system.

Technical Highlights

Machine Learning Framework: Models are built using scikit-learn, leveraging Random Forest classifiers for robust predictions.
Visualization: The script includes utilities for plotting feature importance, ROC-AUC curves, and overfitting diagnostics.
Dynamic Handling: Feature lists and models are dynamically linked, ensuring flexibility when adding new prediction targets.

Results

The performance and feature importance for the prediction targets are detailed below:

Prediction Target: Death

Cross-validation Accuracy: The cross-validation results for "Death" show some variability across folds, but overall, the model achieves consistently high accuracy, indicating good generalization ability across subsets of the data.
Train vs Test Accuracy: The train-test accuracy comparison shows minimal overfitting, as the performance on both sets is closely aligned.

Feature Importance: Features such as "Progressive disease" and "DLCO (%) at diagnosis" are the most influential, highlighting their significant role in predicting mortality.

ROC-AUC Curve: The ROC-AUC curve illustrates strong model performance, with an area under the curve (AUC) of 0.92, confirming the model's ability to distinguish between positive and negative cases effectively.

Prediction Target: Binary Diagnosis

Cross-validation Accuracy: Variability in cross-validation accuracy is observed, but the model maintains high performance across most folds.

Feature Importance: Key predictors include "Prednisone" and "Antifibrotic Drug", reflecting the importance of treatment factors in classification.

ROC-AUC Curve: The high AUC value of 0.95 indicates excellent discrimination ability for the binary classification task.

Prediction Target: Progressive Disease

Cross-validation Accuracy: Accuracy scores across folds highlight variability, but peaks show strong model performance, suggesting robustness in certain data subsets.
Train vs Test Accuracy: The parity between training and testing accuracy confirms minimal overfitting and reliable generalization.

Feature Importance: "RadioWorsening2y" emerges as the dominant predictor, supported by secondary factors like "FVC (%) 1 year after diagnosis".

ROC-AUC Curve: With an AUC of 0.98, the model demonstrates exceptional predictive power for disease progression.

Prediction Target: Necessity of Transplantation

Cross-validation Accuracy: Cross-validation reveals excellent model accuracy, with minimal performance dips across folds.

Feature Importance: "Age at diagnosis" and "FVC (%) 1 year after diagnosis" are the most critical variables, underscoring their importance in assessing transplantation need.

ROC-AUC Curve: The model achieves an AUC of 1.00, reflecting perfect discrimination between cases where transplantation is needed and those where it is not.

Future Improvements

Optimizing Variable Names: Review and refine the naming conventions for variables to improve clarity and consistency, facilitating better understanding for medical practitioners and data scientists.
Improving Model Precision: Retrain the model with a larger and more diverse dataset, incorporating data from additional patients to enhance accuracy and generalization.
Identifying Optimal Medical Variables: Conduct a detailed analysis to identify which medical variables contribute most significantly to prediction accuracy and consider eliminating less relevant ones to simplify the model.
Testing Model Performance with Reduced Variables: Assess whether the model maintains strong predictive performance with a reduced set of optimized medical variables, which could enhance interpretability and efficiency.
Expanding Dataset Diversity: Incorporate data from different demographics, regions, and clinical conditions to ensure the model performs well across diverse patient groups.
Adding Longitudinal Data Analysis: Integrate longitudinal data to capture temporal patterns in disease progression, which could significantly enhance prediction capabilities.
Real-time Model Retraining: Develop an interface or mechanism for users to upload new patient data and retrain the model seamlessly, keeping it up-to-date with the latest insights.

Associated Space

Check out the interactive demo of this model on Hugging Face Spaces:

This README provides a comprehensive guide to understanding and using the FibroPred predictive system effectively.