pipeML

A flexible and modular machine learning framework designed to support leakage-free model training through custom cross-validation fold construction

Installation

You can install the development version of pipeML from GitHub with:

# install.packages("pak")
pak::pkg_install("VeraPancaldiLab/pipeML")

Description

pipeML is a flexible and leakage-aware machine learning framework for R designed for predictive modeling in high-dimensional biological data. The package integrates all key steps of the machine learning workflow — feature filtering, model training, validation, prediction, and interpretation — into a single reproducible pipeline.

A key design goal of pipeML is to support fold-aware feature construction, allowing features that depend on the dataset (e.g. enrichment scores, correlation-based features, or network-derived features) to be recomputed within each cross-validation fold. This prevents information leakage and ensures reliable performance estimation.

The framework is designed to integrate naturally with R/Bioconductor workflows, making it particularly suitable for omics and biomedical machine learning applications.

Figure 1. General structure of the pipeML machine learning pipeline.

Key Features

End-to-end ML workflow

• Integrated pipeline for feature filtering, model training, validation, prediction, and interpretation

Leakage-aware validation

• Custom cross-validation fold construction
• Support for fold-aware feature recomputation
• Prevents information leakage when using dataset-dependent features

Flexible model evaluation

• Repeated and stratified k-fold cross-validation
• Leave-one-dataset-out (LODO) evaluation for cross-cohort generalization

Feature filtering

• Near-constant and highly correlated features are removed from the training features (preprocess = TRUE, the default): once before the cross-validation, or inside each fold for features built by custom fold functions

Hyperparameter tuning

• Automatic optimization based on:

  • AUROC
  • AUPRC
  • C-index (survival)

Model interpretation

• SHAP values of the selected model, per sample and as global feature importance
• Performance visualization (ROC and PR curves with bootstrap confidence bands, Kaplan-Meier curves by predicted risk group)

Parallel computing

• Multi-core support for faster model training and cross-validation

Custom workflows

• Users can define custom fold construction functions
• The parameters of the feature construction can be tuned inside the cross-validation, like model hyperparameters
• These functions receive a bestune argument after tuning, to rebuild the features on the full training dataset for the final model

Supported Machine Learning Methods

Classification algorithms:

For classification tasks, we implemented a diverse set of classification algorithms that are benchmarked on the fly making extensive use of the R package caret.

• Bagged classification trees
• Random forests
• C5.0 decision trees
• Regularized logistic regression (elastic net)
• k-nearest neighbors (KNN)
• Classification and regression trees (CART)
• Lasso regression
• Ridge regression
• Support vector machines with linear and radial kernels
• Extreme Gradient Boosting (XGBoost)

Survival algorithms:

For time-to-event outcomes, pipeML implements a unified survival modeling framework based on the parsnip and workflows ecosystems, enabling consistent training, hyperparameter tuning, and evaluation across multiple survival model families.

• Cox proportional hazards model
• Elastic net–regularized Cox regression
• Parametric accelerated failure time (AFT) models
• Conditional inference survival trees
• Bagged CART survival models
• Oblique random survival forests

General usage

Below are basic examples showing how to use pipeML. For detailed tutorials, see Get started and the Articles.

Results (plots, fold files of custom workflows) are written to a Results/ folder in the working directory.

library(pipeML)

data <- data_example_classification
X <- data[, setdiff(colnames(data), "target")]
y <- data$target

set.seed(123)
train_idx <- caret::createDataPartition(y, p = 0.7, list = FALSE)
X_train <- X[train_idx, ]
X_test <- X[-train_idx, ]
y_train <- y[train_idx]
y_test <- y[-train_idx]

Training models

res <- compute_features.training.ML(features_train = X_train, 
                                    target_var = y_train,
                                    task_type = "classification",
                                    trait.positive = "1",
                                    metric = "AUROC",
                                    k_folds = 5,
                                    n_rep = 10,
                                    ncores = 2)

Predicting on new data

pred = compute_prediction(model = res$Model, 
                          test_data = X_test, 
                          target_var = y_test, 
                          task_type = "classification",
                          trait.positive = "1")
pred$AUC

Explaining the model

shap <- compute_shap_values(model_trained = res$Model, task_type = "classification")

Training and Testing Workflow

res <- compute_features.ML(features_train = X_train, 
                           features_test = X_test, 
                           coldata = data,
                           task_type = "classification",
                           trait = "target",
                           trait.positive = "1",
                           metric = "AUROC",
                           k_folds = 5,
                           n_rep = 10,
                           ncores = 2)

Issues

If you encounter any problems or have questions about the package, we encourage you to open an issue here. We’ll do our best to assist you!

Authors

pipeML was developed by Marcelo Hurtado in supervision of Vera Pancaldi and is part of the Pancaldi team. Currently, Marcelo is the primary maintainer of this package.

Citing pipeML

If you use pipeML in a scientific publication, please cite:

Hurtado, M., & Pancaldi, V. (2026). A new pipeline for cross-validation fold-aware machine learning prediction of clinical outcomes addresses hidden data-leakage in omics based ‘predictors’. bioRxiv. https://doi.org/10.64898/2026.03.12.711429