Is your test set actually held out?
Paste a dataset of small molecules or peptides. This runs the same leakage checks ToxBench ran on Tox21, ClinTox and SIDER: duplicates, contradictory labels, shared scaffolds or sequence families either side of your split, and close analogues that make a test case easy for the wrong reason. Then it proposes a split that does not leak, and tells you what your test fold can and cannot support.
A header row is required, with a smiles column for small molecules or a
sequence column of one-letter amino acids for peptides. A
split column is optional: include it and your split is scored as well.
Peptides are grouped by sequence identity rather than scaffold, because a linear
peptide has no ring system and a scaffold split would silently become a random one.
Nothing is stored. Up to 4,000 molecules or 2,000 peptides.
Either one file holding the whole dataset, or your folds as separate files. Upload
the folds and the split column is written for you, so your own split
gets scored against the proposal.
What a toxicity benchmark looks like when you split it honestly
Four models, three datasets, five seeds, scored under random and scaffold splits. Every number here is read from the published results, including the ones that disagree with the headline.
Random versus scaffold
macro-AUROC, mean over 5 seedsLook closer
pick a datasetPredict, and see how far to trust it
Random forests on ECFP4, trained on the scaffold split. Every probability arrives with its nearest-neighbour similarity to the training set and the AUROC this benchmark actually measured for molecules that far away.
One molecule per line. A line may be a bare SMILES, or NAME SMILES.
One-letter amino acids, one peptide per line, up to 60 residues. These models were trained on small molecules, so a peptide falls outside what they can speak to. The app runs it anyway and shows you why the usual applicability-domain check does not apply here.