Research · 2025–26
A tool that checks whether a dataset’s labels can be rebuilt from its own features, which I built after finding exactly that problem in my own model.
- 2,300+
- audio and text samples
- 450+
- hours of talks
- 10
- models compared
01How it started
I was building EngagePredict, a model that predicts how engaging a talk is from its audio and transcript. It looked at semantic, sentiment and prosodic features across 2,300+ samples and 450+ hours of TED talks.
The first version scored suspiciously well. When I dug in, it turned out the label could be reconstructed from the features themselves, so the model was partly just reading the answer back. I withdrew that version and rebuilt it honestly.
02What I built
LeakGuard is a screen you can run on a dataset you did not build. It decides whether the label can be reconstructed from the dataset’s own features, points to which features are responsible, and removes them.
The core idea is comparing how well simple linear models do against more flexible ones. A label computed straight from the columns gets recovered almost perfectly by a linear model, so a big gap is a red flag. That score is calibrated against a few deliberately leaky "anchor" labels built on the same data.
If the data cannot support a clear answer, LeakGuard says it is inconclusive instead of guessing.
03Decisions and trade-offs
04How it went
It turned into a paper, "Detecting and Localising Label Leakage in Benchmark Datasets", which is under review right now.
