Ideas from other fields

CAPRI and CASP assessors learned that placing the binding site is all-or-nothing, so consensus over many models beat trusting one confidence score. Frequency analysis in codebreaking does the same with letter counts.

Public benchmarks support the direction: Protenix-v1 improves from 36.0% to 47.7% correct complexes going from 1 to 80 seeds, and contact frequency over 100 AF3 seeds reached PR-AUC 0.58 versus 0.36 for an AF2 ensemble on antibodies. Nobody has run it on HER2 nanobodies yet.

Run several seeds and engines on the two clean proxy complexes (9MTE, 9MTX) and score per-residue contact frequency.

If multi-seed consensus does not beat the contact map of the single best-ipTM model on PR-AUC on 9MTE/9MTX, abandon the ensemble. A second test: if split-half-shrunk consensus does not beat unshrunk mean contact, drop the shrinkage.

If this test fails, the idea is dropped, not argued for.

Turning a pile of disagreeing guesses into one honest probability map.

Wartime codebreakers summed small independent pieces of evidence until a hypothesis crossed a threshold.

The recipe is valid only if clues are independent, and clonal families of nanobodies break that. It was proposed, not run on the 1.8-million-row table.

Sum standardised log-odds from three or four sources on the proxy set, after clustering correlated sources.

If the log-odds sum does not beat the best single source by at least 0.03 macro PR-AUC on the proxies, revert to the single best source.

If this test fails, the idea is dropped, not argued for.

A calibrated 0-to-1 output built the way Bletchley built confidence.

Contact precision rose from 47% (CASP12, best L/5) to above 70% (CASP13); AlphaFold2 reached a median GDT_TS of 92.4 at CASP14.

A pseudo-alignment from real binder sequences marks the nanobody loops only, and gives identical profiles for ONU11 and ONU13. There is one antigen and no partner co-variation, so the idea dies for the antigen side and survives as a loop-variability map.

Use the binder-derived variability map as a feature for the nanobody chain only.

If the map does not raise CDR-residue PR-AUC over a plain position prior, drop it.

If this test fails, the idea is dropped, not argued for.

Testing the user's own hypothesis in the field where it was born, and writing down honestly where it stops.

MaxEnt, Poisson point-process models and PU learning are the same family; prevalence cannot be identified from presence-only data, so rankings and a prior are used.

The statistics map cleanly and clarify that prevalence does not matter for a rank-based per-complex score. But the presence-only prior itself scored only 0.04-0.05 average precision, and PU variants lowered PR-AUC in the NGS tests.

Already done in simulation. Keep the null models (exposure-only, domain-only) as yardsticks.

PU/MaxEnt-style reweighting must beat plain logistic regression on grouped cross-validation. It did not, so it is parked.

If this test fails, the idea is dropped, not argued for.

Borrowing a toolbox from people who count birds, then using it to find out which of our assumptions were wrong.

Ecologists separate the fundamental niche (where it could live) from the realised niche (where it actually can).

None of the four public HER2 nanobody epitopes overlap the roughly 51-residue dimer patch (mostly residues 221-351), which suggests it should help. No held-out run exists.

Down-weight the dimer patch in the antigen prior and score on the four public nanobody complexes.

If it does not lower mean average precision over exposed residues, or hurts held-out AP, remove it.

If this test fails, the idea is dropped, not argued for.

Reading a protein surface like a landscape with territories.

Conservators use a hand-set map of likely damage rather than inspecting every brushstroke.

A position-based prior was reported at +0.17 to +0.33 PR-AUC above the base rate on HER2 nanobodies, and a simulated CDR prior reaches 0.644 against 0.204 random. Numbering used a home-made script that may be off by one, and the germline-versus-mutation question is untested.

Redo numbering with ANARCI and score the position prior on the five HER2 nanobodies.

It must beat prevalence by at least +0.17 absolute PR-AUC; if not, fall back to a uniform, CDR3-weighted guess.

If this test fails, the idea is dropped, not argued for.

The easy half of the problem, solved by respecting where nanobodies usually grip.

Clinicians convert a test result into a post-test probability using the base rate.

The hive estimates about 27% positive predictive value at a 3% prior. The NGS data also carry no per-HER2-residue contrast, so this can only reweight the nanobody side.

Use a simulated clone-enrichment proxy as a weighted signal on the proxy complexes.

If a PU-weighted use of the signal does not beat a uniform plus exposure prior by 0.02 macro PR-AUC under leave-one-complex-out, drop it.

If this test fails, the idea is dropped, not argued for.

Explaining to a machine what every doctor already knows about false positives.

Clinical papers report relative risk and number needed to treat, not raw accuracy.

It already paid off: the organisers' 0.176 is only about 1.2 times the expected random score on its own proxy, and random reaches it 16% of the time.

Report lift and a shuffle null next to every number.

If an arm's macro lift over the per-complex shuffle null is below about 1.5x on both clean proxies, do not submit it as the main candidate.

If this test fails, the idea is dropped, not argued for.

Deflating a headline number with one division.

Courts weigh evidence against thresholds rather than a single number.

The area-weighted metric is cutoff-proof in principle, so the fraction of cutoffs or seeds passed can serve as a probability. Not yet compared with raw single-cutoff labels.

Compute the fraction of cutoffs passed on the proxies and use it as the target.

If the consensus-of-cutoffs score does not give higher weighted PR-AUC than single-cutoff labels, discard it.

If this test fails, the idea is dropped, not argued for.

Putting residues on trial.

Sports models are scored forward in time, never on the past they trained on.

9MTE and 9MTX (released 2025-11-19) are leakage-free proxies; older proxies may be memorised.

Compare each arm's rank on 9MTE/9MTX with its rank on the older 5MY6/7QVK.

If the ranks disagree by more than a pre-set margin, treat the pre-cutoff gain as memorisation and discard it.

If this test fails, the idea is dropped, not argued for.

A time machine for honest scoring.

Statisticians use bootstrap confidence intervals and hedge across entries.

Simulation gives a macro PR-AUC standard deviation of roughly 0.024-0.06 at n=3; unrelated submissions need a gap of about 0.07-0.16 to separate. This is a property of the metric, not a method that can fail.

Use the five-submission ladder as hedges and bootstrap locally.

If the local bootstrap intervals of two candidates overlap, submit the simpler one.

If this test fails, the idea is dropped, not argued for.

Playing the odds on purpose.

Forensic labs track every transfer with a register.

Already paid off: the audit caught a stale home page. Not a model idea, so it does not 'fail' in the scoring sense.

A local validator that checks the file against the printed construct.

A submission file that mismatches the template sequences must fail the local check.

If this test fails, the idea is dropped, not argued for.

Being the detective of your own pipeline.

Peaks that repeat across noisy copies are kept; the rest is dropped.

Peaks did replicate (Jaccard 0.48-0.61, 75% in CDR columns), but the first test showed denoised peaks did not beat the plain variability prior.

Already tried once; no further effort is planned.

Denoised peaks must raise CDR-residue PR-AUC over the variability prior. The first try said no.

If this test fails, the idea is dropped, not argued for.

Humming a protein into a phone.

Unsupervised alignment works when each language is large and the two have similar structure, using thousands of paired examples or a seed dictionary.

There is one HER2 sequence against 1e5-1e6 nanobodies, and only a few hundred binder labels. The premise fails; the practical lesson, a nanobody language-model paratope head, survives (literature PR-AUC about 0.65-0.70, unverified).

Train a paratope head on language-model embeddings with clustered splits.

If it does not beat the CDR position prior on held-out clusters, use the prior.

If this test fails, the idea is dropped, not argued for.

Failing in an instructive way and keeping the useful leftover.

Netflix was won by a blend plus new temporal features, with a margin of 0.00001 RMSE and a tie broken by 20 minutes. DREAM synergy was won with prior-knowledge features. Roelofs 2019 found little overfitting across 120 Kaggle competitions, with tiny test sets as outliers.

'Representation not model size' survives only in restricted form. AlphaFold2 is the counterexample: an end-to-end model won CASP14 by a wide margin.

Blend a structure-based prior with the nanobody-side prior, calibrated per complex.

If the blend does not beat the best single arm on the clean proxies, ship the single arm.

If this test fails, the idea is dropped, not argued for.

Learning from people who won by 20 minutes.