← All projectsR03

DATA SYSTEMS · ML EVALUATION · REPRODUCIBILITY

Reproducible evaluation for tabular classification

I built R03 around a question that is easy to miss when a model score looks good: did the evaluation procedure give the model information it should not have had, and could somebody else reproduce the number later?

The Python code, pinned dependencies, run records, charts, and verification script live in the R03 folder.

Open the R03 source folder ↗Open the verification record ↗

Stack Python / scikit-learn / pandas / NumPy

The setup

There are two deliberately different datasets here. The first is a bundled tabular reference dataset, used to exercise a normal scaling and logistic regression pipeline. The second has random features and random labels. It has no real signal, which makes it useful for exposing a flawed evaluation path.

For every run, I save the configuration, split seeds, dependency versions, dataset fingerprint, individual metrics, and the summary. The project then re-runs those records and compares the whole generated object rather than only checking one final score.

The leakage comparison

I selected 25 features before splitting the random-label data, then trained and evaluated a classifier. That path reported mean held-out accuracy of 0.840. The labels were random, so the score was not evidence of a useful classifier. It came from letting feature selection inspect all rows before the eventual test partition was protected.

I then put the same selector inside a scikit-learn pipeline, so it was fitted only on training rows. Mean accuracy fell to 0.544, close to chance for the same random-label task.

Chart comparing mean accuracy 0.840 for feature selection before the split against 0.544 for selection inside the safe training pipeline.
The high leaked score is a property of the flawed procedure, not a model discovery.

One score is not the whole result

The safe reference pipeline was also run across 30 predetermined train/test splits. Accuracy ranged from 0.958 to 0.993, with a mean of 0.978. The best individual split was 0.015 above that mean. I kept that result in the record to show why it would be misleading to report only the best run.

Chart showing a safe-pipeline accuracy range from 0.958 to 0.993 across 30 fixed splits, with mean 0.978.
The same pipeline does not have one permanent score; the split is part of the result.

Experiments

E01

Reference pipeline

A correctly split scaling and logistic-regression pipeline was run across five fixed seeds.

E02

Feature-selection leakage

Random-label data scored 0.840 after leaky selection and 0.544 with selection inside the training pipeline.

E04

Recorded-run verification

The saved E01–E03 objects were regenerated exactly from their recorded configuration and environment.

What I take from it

A pipeline is useful here because it puts feature selection and model fitting inside the training step. The more important habit is to protect the test data before making choices, record more than one split, and leave enough detail for the run to be checked again.

Limits

The random-label experiment is intentionally controlled; it does not say how large leakage effects will be in every real dataset. The reference dataset is not a medical model or a deployment claim. Exact regeneration currently depends on the recorded Python environment and versions, so it should be treated as a local reproducibility check, not a guarantee for every platform.

If you want to inspect it