How this was made
A small organisation of AI agents did the research: an emperor that sets the shape, kings that steer, queens that own a subject, and workers that read and compute. Click any box to read what it concluded.
The org chart
1 emperor, 2 kings, 9 queens, 45 planned worker topics. Click any box.
Emperor
See hive/EMPEROR.md tick 0. Two kings split the problem at its natural seam: WORLD (what , VHHs and the phage--cryoEM assay physically are, and what the organisers actually measure) vs INFER (how to turn sequence + public structure + NGS into per- probabilities, how to validate with only 3 test complexes, and what other fields know). Nine queens x five workers keeps every king under ~5 queen digests. Deadline is 2026-10-31 so rounds are short and modeling runs in parallel with research.
The Emperor does no research. It only chooses how many kings, queens and workers there are, which of them may talk, and how much each may read. Its decision log is below.
source
hive/structure.jsonhive/EMPEROR.md
Who may talk to whom: the channels5 open message channels between queens
| Channel | Queens | Purpose |
|---|---|---|
label_truth | q3, q4, q6, q8 | What exactly is the label and how is it scored; resolve /detail vs /guideline contradictions |
structures_and_proxies | q1, q2, q5, q8 | Public / complexes, their epitopes and cutoffs, usable as proxy test sets |
data_representation | q3, q6, q7 | What the data can and cannot say; representation ideas to test |
methods_to_execution | q5, q6, q7, q8 | Which engine/feature is worth GPU-hours; budgets per arm |
serendipity | everyone | Broadcast of cross-domain ideas from q9 and every king lens |
source
hive/structure.json
The Emperor's decision log
The Emperor does not research. It writes down, at each "tick", what it observed and how it changed the hive.
Standing rules (inherited from the CASMI26 and OpenBind hives, adapted)
- Fan-in discipline. Worker writes one topic file. Queen reads only TL;DR + frontmatter + Implications of her workers and writes
research/<queen_id>/_QUEEN_DIGEST.md(<= 1,500 words). King reads only queen digests, writesresearch/_meta/<king_id>_KINGLOG.md. Emperor reads only king logs and coverage JSON. - Serendipity is mandatory. Every worker has a
serendipity:line; every queen digest a "Forced analogies" section; every king log a "Cross-domain transfers" section with at least one idea from outside biotech/ML. Kings rotate the lens list. - Stagnation rule. If a queen's second round yields < 2 new concrete sourced facts, the king re-assigns her to a different subdomain or a random field (art, law, philosophy, healthcare, ecology, astronomy, cryptography, music, sports analytics) with the instruction "find what this field knows about inferring where two things touch from weak noisy signals".
- Channels are explicit (structure.json). Emperor opens/closes by observed traffic.
- Restructure triggers. Split a king > 5 queens or log > 3,000 words; merge queens overlapping > 50% in tags; add a queen when a king log names an uncovered subdomain twice; kill a worker topic when two rounds yield nothing new.
- Every tick ends with artifacts on disk. No silent ticks.
- Resource rule. Check CPU/RAM/disk on C: and D: before any fan-out; <= ~8 concurrent agents; big downloads on the drive that has space; Explorer public partitions only, <= 2 GPUs (CLAUDE.md fair-use guardrails).
- Template-noun rule (new). The user's kickoff text is a reusable template ("protease", "mass spec"). Verify the real challenge first; never import the template's nouns. (Learned on CASMI26, repeated here: SEPIQ has no protease and no mass spectrometer.)
Tick 0 — 2026-10-17 (initial structure)
Observation. Fresh folder. Real challenge fetched from sepiq.uk: HER2 VHH epitope/paratope prediction, 3 held-out cryo-EM complexes (ONU11, ONU13, ONU23), Track A per-residue PR-AUC macro-averaged over 3 complexes, 5 submissions per team, window closes 2026-10-31 (14 days), results 2026-12-15. Training signal = AVIDa-hHER2 (1.8M NGS reads, a few hundred labels), SAbDab-nano, public PDB. Biggest structural facts: (i) n = 3 test targets means the metric is dominated by variance and by whether the epitope region is right (HER2 has four domains and known epitope hot spots); (ii) co-folding engines are explicitly allowed and public HER2 complexes are explicitly allowed even if they leak ("as long as the answer is correct"); (iii) the baseline scores 0.176 on its own proxy, so the bar is low and a good structure-based prior can win; (iv) the user's thesis (data representation of the NGS data is the breakthrough) is testable only through q7 and needs real execution, not just reading.
Decision. Two kings:
king_world- biology, molecules, assay and institution. 4 queens x 5 workers (HER2 biology, VHH immunology/structure, assay/NGS/ground truth, challenge intelligence).king_infer- sequence-to-interface methods. 5 queens x 5 workers (structure prediction + docking, epitope ML, data representation and integration, compute/validation/execution, cross-domain serendipity). Why two, not one: nine queen digests (~13k words) would exceed one king's budget. Why not three: no third natural seam; q4 stays small. Serendipity: a dedicated queen (q9) AND a lens on every queen.
Time policy (new, because of the 14-day window). Research waves are capped at ~48 h. The dashboard is built from whatever has landed by then and keeps filling in. Modeling starts in parallel, not after: an engineering lane (outside the hive) builds the proxy test set, runs co-folding on the three targets as soon as sequences are captured, and produces a safe submission-1 early; the ladder decisions come from q8. The user asked for dashboard first; this is a deliberate, flagged deviation forced by the deadline.
Concurrency policy. At most 8 workers in flight at a time (fan-out of ~45 workers happens in waves, queens ordered by dependency: Wave 1 = q4, q3, q1, q2 (what is the label and what is HER2/VHH); Wave 2 = q5, q6, q7, q8, q9).
Context budget per level: worker <= 1 topic; queen <= 5 TL;DRs; king <= 5 digests; emperor <= 2 king logs.
Next Emperor tick after the Wave-1 queen digests exist: reassign any stagnant queen, open the proxy-test channel with real PDB ids, decide whether q7 gets an execution worker pool.
Tick 1 — 2026-10-17 (round 1 closed: 46 workers, 9 digests, 2 king logs)
What happened. All nine queens delivered; no stagnation. q3/q4 captured verbatim pages (research/_meta/sepiq_verbatim.md); q7 ran
real tables (1,843,875 VHH, 19,367 labelled: 7,316 Yes / 12,051 Not) and killed the user thesis four ways (DCA/Potts, clonal DMS,
mimotope complementarity, PU/denoising): the NGS data carry no per-HER2-residue contrast; they inform only the VHH side. Antigen side must
come from structure (co-fold consensus, exposure, public footprints). HER2 interface base rate ~3-5% (not 12.6%). 9MTE/9MTX are
leakage-free proxies (released 2025-11-19); only 4 public HER2-VHH structures exist. The scorer is ambiguous (chain pooling, AP vs
trapezoid, best-of-5 vs last) and each ambiguity can change the winner. n=3 makes gaps < ~0.1 noise.
Structure decisions.
- KILL further NGS-representation research (q7): keep one utility worker for grouped-CV/family splits. The negative result is a headline finding for the dashboard ("why the data-representation hypothesis failed and what survives").
- MERGE q5+q8 into one execution sprint lane (outside the research hive, engineering agents): proxy table, validator, floor submission, co-fold consensus, ladder. Research is saturated; value now comes from measured runs.
- OPEN channels
proxy_labels(one shared Voronota-LT area>0 table for 5MY6/7QVK/9MTE/9MTX/8GZ5) andscorer_ambiguity(owner q4: ask organisers via the HF Space community tab). - Treat q9 as low-yield; fold its null spreads and decibans recipe into the runbook. No re-dispatch.
- Add one-off workers (engineering): ANARCI/BLASTp of ONU clones vs known VHHs and alpaca germline; reconcile the three versions of
her2_public_complexes.json; construct/chain-A-length reconciliation. - Close Track B/CAPRI and drug-history topics for this deadline.
Open contradictions carried forward. HER2 prevalence 0.126 vs 0.03-0.05; chain A length 618/624/630; HF dataset gated (401) vs q7 tables; Voronota version; official ranking variant; deadline 2026-12-15 vs 1 Dec.
Round 2 launched this tick (parallel lanes). Scribe (narrative, expert synthesis, glossary, facts), Predator audit, dashboard build (scaffold -> six page builders -> integrate + Vercel deploy), modeling lane CPU-only (validator, proxy labels, floor submission). GPU co-folding waits for measured seconds-per-seed and the user's go (Explorer public partitions, <= 2 GPUs).
Tick 2 — 2026-10-17 (round 2 closed: dashboard live, floor + -2 candidates)
Done. Dashboard live (http[local path] redeployed with model results). Boltz-2 multi-seed co-folding measured:
180 s/seed (A100-40GB), 3.2 GPU-h for 63 runs; antigen PR-AUC macro on 9MTE x2 + 9MTX 0.453 vs exposure prior 0.067 (null 0.054),
kill test (+0.02) passed by +0.387; VHH side 0.72 = floor (CDR prior already covers it). Per-complex spread large (0.96 / 0.32 / 0.076);
ipTM does not reliably say which maps to trust. ONU23 most self-consistent (domain I, ipTM 0.52); ONU11/13 low confidence (ipTM 0.39/0.37).
Shrinkage/log-odds blends did not beat raw by the 0.05 margin. Candidates validated: s1_floor, s2_cofold_candidate, s3_shrunk_blend.
Blocked. Chai-1 / OpenFold3 / extra Boltz proxies (5MY6, 7QVK) never got a GPU: sibling session's casmi_cre_* jobs held both GPUs under the
2-GPU rule. Environments, weights and scripts are ready (model/cofold/slurm/run_chai2.sh, run_of3.sh).
Decisions. No upload to Hugging Face without the user's account; organiser email drafted (research/_meta/organiser_email.md) but not sent
(no email tool, Chrome extension disconnected). Next: second engine when a GPU frees; ONU domain verification; user uploads S1 floor, then S2.
Tick 3 — 2026-10-18 (second engine measured: -1 does not help)
Chai-1 0.6.1, 3 seeds x 5 samples on 9MTE_27A05, 9MTE_47D05, 9MTX_29E09, ONU11/13/23 (job 11194640, public gpu, 1 A100, 1.4 h, finished clean; no jobs of ours left).
Proxy antigen AP macro: Chai-1 0.074 (null 0.053; vs exposure prior +0.007, CI -0.02..+0.05) vs Boltz-2 0.453. VHH side 0.59 vs Boltz-2 0.72 vs floor 0.72.
Mean ipTM 0.17-0.27 (Boltz-2 ~0.4), so Chai-1 as run here carries no antigen signal. Equal-weight Boltz-2+Chai-1 ensemble: antigen 0.368, VHH 0.63, both below Boltz-2 alone.
Decision: pre-registered ensemble rule FAILS; no s4 ensemble. S2 raw Boltz-2 stays the main candidate. Chai-1 scripts: model/cofold/chai_contacts.py, results model/cofold/results/{chai,ens}_*.
Caveat: n=3 proxies, one Chai config (no templates tuned); this is "no evidence it helps", not "Chai-1 is bad".
source
hive/EMPEROR.md
The predator audit
After round one, an auditor picked 30 claims that most affect decisions and re-checked each against the raw data or the original page. Verdicts: verified, imprecise, unsupported, unreachable (could not be opened), or fabricated.
The audit rechecked 30 decision-relevant claims from the research files against the real tables and original pages. Counts below are from the audit file. Files with fabricated or unsupported claims are quarantined below.
source
research/_meta/predator_audit.mdresearch/_meta/predator_audit.json
Audit date 2026-10-17.
Quarantined files
erbb_family_dimerisation_signalling_and_adjacent_proteins___.mdseverity high
- Why
- 1 fabricated claim: asserts 9MTE/9MTX contain Fab chains and that held-out ONU structures are probably Fab-assisted. RCSB and local mmCIF show only + .
- Action
- Do not cite the Fab statements; nothing else in the file was found contradicted among the claims checked.
- File
research/q1_her2_biology/erbb_family_dimerisation_signalling_and_adjacent_proteins_egfr_her3_her4.md
avida_hher2_and_avida_family_dataset_anatomy_columns_sizes_label_logic.mdseverity medium
- Why
- 1 unsupported claim ("a few hundred" labelled; true value 19,367) plus an unverifiable hIL6 split (20,980 / 552,911 / 38,599). Below the formal threshold (1 unsupported + 1 unreachable) but the headline label count is wrong.
- Action
- Use 19,367 (7,316 Yes / 12,051 Not) from q7.
- File
research/q3_assay_ngs_ground_truth/avida_hher2_and_avida_family_dataset_anatomy_columns_sizes_label_logic.md
source
research/_meta/quarantine.json
Every checked claim
#1 1,843,875 unique rowsverified
- Verdict
- verified
- Notes
- Row count of .csv = 1,843,875 (csv parse, 36 cols); _read_count_table.csv also 1,843,875 rows, 30 cols.
- Checked against
- local gated tables [local-path] (HF repo sepiq-2026/SEPIQ-2026-training-data, API opened: gated=manual, files AVIDa-hHER2.csv + NGS_read_count_table.csv)
- Claim made in
research/q7_data_representation_integration/clonotype_clustering_cdr3_families_and_enrichment_trajectories_as_epitope_bins.md
#2 Label Yes = 7,316verified
- Verdict
- verified
- Notes
- Counter on label column: Yes 7316 (binary_label 1.0 = 7316).
- Checked against
- local gated tables [local-path] (HF repo sepiq-2026/SEPIQ-2026-training-data, API opened: gated=manual, files AVIDa-hHER2.csv + NGS_read_count_table.csv)
- Claim made in
research/q7_data_representation_integration/negative_labels_pu_learning_noisy_labels_and_enrichment_to_label_denoising.md
#3 Label Not = 12,051verified
- Verdict
- verified
- Notes
- Not 12051 (binary_label 0.0 = 12051).
- Checked against
- local gated tables [local-path] (HF repo sepiq-2026/SEPIQ-2026-training-data, API opened: gated=manual, files AVIDa-hHER2.csv + NGS_read_count_table.csv)
- Claim made in
research/q7_data_representation_integration/negative_labels_pu_learning_noisy_labels_and_enrichment_to_label_denoising.md
#4 19,367 labelled clones (not "a few hundred")verified
- Verdict
- verified
- Notes
- 7,316 + 12,051 = 19,367. The BRIEF "few hundred" is stale.
- Checked against
- local gated tables [local-path] (HF repo sepiq-2026/SEPIQ-2026-training-data, API opened: gated=manual, files AVIDa-hHER2.csv + NGS_read_count_table.csv)
- Claim made in
research/q7_data_representation_integration/negative_labels_pu_learning_noisy_labels_and_enrichment_to_label_denoising.md
#5 noise 746; non-sig. 1,823,762verified
- Verdict
- verified
- Notes
- Both counts match exactly.
- Checked against
- local gated tables [local-path] (HF repo sepiq-2026/SEPIQ-2026-training-data, API opened: gated=manual, files AVIDa-hHER2.csv + NGS_read_count_table.csv)
- Claim made in
research/q7_data_representation_integration/negative_labels_pu_learning_noisy_labels_and_enrichment_to_label_denoising.md
#6 5,327,133 reads total across all columnsverified
- Verdict
- verified
- Notes
- Sum over all 29 count columns of _read_count_table.csv = 5,327,133.
- Checked against
- local gated tables [local-path] (HF repo sepiq-2026/SEPIQ-2026-training-data, API opened: gated=manual, files AVIDa-hHER2.csv + NGS_read_count_table.csv)
- Claim made in
research/q7_data_representation_integration/clonotype_clustering_cdr3_families_and_enrichment_trajectories_as_epitope_bins.md
#7 77.9% of sequences are singletonsverified
- Verdict
- verified
- Notes
- Fraction of rows whose total count over all columns equals 1 = 0.7789.
- Checked against
- local gated tables [local-path] (HF repo sepiq-2026/SEPIQ-2026-training-data, API opened: gated=manual, files AVIDa-hHER2.csv + NGS_read_count_table.csv)
- Claim made in
research/q7_data_representation_integration/negative_labels_pu_learning_noisy_labels_and_enrichment_to_label_denoising.md
#8 depleted >20x by ; and notverified
- Verdict
- verified
- Notes
- Understated. C3 37C columns, reads (per-million): 1 read (11.1 ppm) vs companion 184 (1,977 ppm) and 411 (4,663 ppm), i.e. roughly 180-420x. trast 5,385 ppm and 1,984 ppm are not depleted (ONU23 is higher than companion 312 ppm).
- Checked against
- local gated tables [local-path] (HF repo sepiq-2026/SEPIQ-2026-training-data, API opened: gated=manual, files AVIDa-hHER2.csv + NGS_read_count_table.csv)
- Claim made in
research/q7_data_representation_integration/deep_mutational_scanning_and_cdr_mutation_tolerance_as_paratope_evidence_in_public_data.md
#9 Only 49.5% of Yes clones are replicate-confirmed (3,624/7,316)unreachable
- Verdict
- unreachable
- Notes
- Arithmetic checked (3624/7316 = 49.54%) but the 3,624 count was not recomputed in this session.
- Checked against
- worker script output (not rerun)
- Claim made in
research/q7_data_representation_integration/negative_labels_pu_learning_noisy_labels_and_enrichment_to_label_denoising.md
#10 oracle macro 0.081 vs RSA prior 0.083 (5 complexes)unreachable
- Verdict
- unreachable
- Notes
- Internally consistent across the .md and digest; not recomputed. Worker itself notes effective n is 3-4.
- Checked against
- worker script output (not rerun)
- Claim made in
research/q7_data_representation_integration/mimotope_and_peptide_epitope_mapping_phage_sequences_and_epitope_inference_from_binding_alone.md
#11 3-mer LR AUROC 0.945 (93% cluster split) falls to 0.855 (family split)unreachable
- Verdict
- unreachable
- Notes
- Consistent between .md and digest; not recomputed. CI for 0.855 is wide (0.815-0.890).
- Checked against
- worker script output (not rerun)
- Claim made in
research/q7_data_representation_integration/negative_labels_pu_learning_noisy_labels_and_enrichment_to_label_denoising.md
#12 RCSB P04626 returns 63 entriesverified
- Verdict
- verified
- Notes
- total_count = 63; 9MTE and 9MTX are in the hit list.
- Checked against
- http[local path] (opened)
- Claim made in
research/q1_her2_biology/all_public_her2_complex_structures_in_pdb_with_epitope_tables_incl_nanobodies_and_scfvs.md
#13 9MTE released 2025-11-19, 2.66 A, 27A05 + 47D05 + verified
- Verdict
- verified
- Notes
- Release 2025-11-19; 2.66 A; entities: 27A05 (chain A, 146 aa), VHH 47D05 (chain B, 128 aa), (chain C, 630 aa).
- Checked against
- http[local path]
- Claim made in
research/q1_her2_biology/all_public_her2_complex_structures_in_pdb_with_epitope_tables_incl_nanobodies_and_scfvs.md
#14 9MTX released 2025-11-19, 3.06 A, 29E09verified
- Verdict
- verified
- Notes
- Release 2025-11-19; 3.06 A; 29E09 chain A (122 aa), chain B (630 aa). The ERBB file quotes "2.78-3.06"; RCSB shows 3.06 only.
- Checked against
- http[local path]
- Claim made in
research/q1_her2_biology/all_public_her2_complex_structures_in_pdb_with_epitope_tables_incl_nanobodies_and_scfvs.md
#15 9MTE = " ECD + Fabs", 9MTX = "+ Fab"; "9MTE/9MTX include a Fab"; ONU structures "probably Fab-assisted"fabricated
- Verdict
- fabricated
- Notes
- Contradicted. 9MTE has 3 polymer entities (two , ); 9MTX has 2 (VHH, HER2). No Fab chain in either file. Affects lines 107, 108, 140, 160 of the file.
- Checked against
- RCSB 9MTE/9MTX entity lists + local mmCIF ([local-path])
- Claim made in
research/q1_her2_biology/erbb_family_dimerisation_signalling_and_adjacent_proteins_egfr_her3_her4.md
#16 9MTE/9MTX primary citation doi 10.1158/1535-7163.MCT-25-0654 (Mol Cancer Ther 25:541)unreachable
- Verdict
- unreachable
- Notes
- DOI resolves (302) to aacrjournals.org/mct/article/25/4/541/... (slug: TPP-45142 anti- T-cell engager), consistent with 25(4):541, but the page returned 403 so content (/PDB mentions) was not read.
- Checked against
- http[local path]
- Claim made in
research/q1_her2_biology/all_public_her2_complex_structures_in_pdb_with_epitope_tables_incl_nanobodies_and_scfvs.md
#17 ~3-5% (5MY6 3.8%, 21/553)imprecise
- Verdict
- imprecise
- Notes
- Recount at 4 A: 5MY6 17/556 = 3.1% (matches Q1 count of 17); 9MTX 17/400 modelled = 4.3%; 9MTE union of 2 28/451 = 6.2%. Q3 value 21/553 uses area>0 and was not reproduced. Range 3-5% is fair for single-VHH epitopes; denominators are modelled , not the 630-aa ECD, and the union of 2 VHH can exceed 5%.
- Checked against
- local mmCIF 5MY6/9MTE/9MTX, my own 4 A heavy-atom contact count
- Claim made in
research/q3_assay_ngs_ground_truth/voronota_lt_contact_area_definition_and_how_epitope_labels_depend_on_the_cutoff.md
#18 "a few hundred" labelled out of 1,843,875 (line 98; table line 82)unsupported
- Verdict
- unsupported
- Notes
- Data have 19,367 labelled clones. The Q3 file repeats the stale BRIEF phrase; digest/Q7 correct it. Use 19,367.
- Checked against
- local table (see claims 2-4)
- Claim made in
research/q3_assay_ngs_ground_truth/avida_hher2_and_avida_family_dataset_anatomy_columns_sizes_label_logic.md
#19 AVIDa-hIL6: 573,891 antigen- pairsverified
- Verdict
- verified
- Notes
- Abstract page: 573,891 pairs, wild-type IL-6 + 30 mutants.
- Checked against
- http[local path]
- Claim made in
research/q3_assay_ngs_ground_truth/avida_hher2_and_avida_family_dataset_anatomy_columns_sizes_label_logic.md
#20 AVIDa-hIL6: 20,980 binders (3.7%), 552,911 non-binders, 38,599 unique unreachable
- Verdict
- unreachable
- Notes
- Neither page gives the binder/non-binder/unique- split. Sum 20,980 + 552,911 = 573,891 and 3.66% are arithmetically consistent; the paper body was not opened.
- Checked against
- arXiv 2306.03329 abstract page; HF card COGNANO/AVIDa-hIL6
- Claim made in
research/q3_assay_ngs_ground_truth/avida_hher2_and_avida_family_dataset_anatomy_columns_sizes_label_logic.md
#21 external-test 0.232 (AF input) vs 0.223 (solved), 24 antigen chainsverified
- Verdict
- verified
- Notes
- Table 2: AUC-PR 0.232 / 0.223, ROC 0.783 / 0.795, MCC 0.227 / 0.214; 6,788 , 436 residues (6.4%); SD +/-0.02 for both. The Q2 file writes 0.223 +/- 0.018 (imprecise: paper says 0.02).
- Checked against
- http[local path]
- Claim made in
research/q6_epitope_ml_methods/b_cell_epitope_predictors_bepipred3_discotope3_sema_abepitope_state_and_limits.md
#22 paper reports no -level metric, no 8GZ5, no ; 0.176 is organiser-sideverified
- Verdict
- verified
- Notes
- Page: pair-level binding classifier (AUC, F1, MCC...), datasets Cov-AbDab, LANL HIV, BioMap, MET; no . Caveat: last ~22k of 122k characters not read.
- Checked against
- http[local path]
- Claim made in
research/q6_epitope_ml_methods/antibinder_architecture_attention_epitope_extraction_and_why_it_scores_0_176.md
#23 Baseline 0.176 on 8GZ5 (P17 -RBD), fraction 26/206 = 0.126, RTX 3090 Tiverified
- Verdict
- verified
- Notes
- Quotes: "26/206 = 0.126", " of 0.176", "RTX 3090 Ti". The BRIEF "~30 min" runtime and the exact pretrain-then-finetune recipe were not confirmed in the fetched excerpt.
- Checked against
- http[local path]
- Claim made in
research/q6_epitope_ml_methods/antibinder_architecture_attention_epitope_extraction_and_why_it_scores_0_176.md
#24 AF3 -antigen success 31.6% (>0.23), n=60, single seedverified
- Verdict
- verified
- Notes
- 60 bound complexes; top-ranked decoy of first seed.
- Checked against
- http[local path] (Hitawala & Gray, mAbs 2025)
- Claim made in
research/q5_structure_prediction_docking/alphafold3_openfold3_boltz2_chai1_protenix_nanobody_antigen_benchmarks_and_success_rates.md
#25 -1 23.3%, -1 15.0% on the same 60 nanobodiesverified
- Verdict
- verified
- Notes
- Matches. Single seed; -2 / / were not benchmarked in this paper.
- Checked against
- same PMC12360200
- Claim made in
research/q5_structure_prediction_docking/alphafold3_openfold3_boltz2_chai1_protenix_nanobody_antigen_benchmarks_and_success_rates.md
#26 High-accuracy (>=0.80): AF3 13.3%, -1 5.0%, -1 3.3%verified
- Verdict
- verified
- Notes
- Matches. 13.3% is NOT the overall success rate; do not confuse with 31.6%.
- Checked against
- same PMC12360200 (PubMed snippet also shows 13.3 / 5 / 3.33)
- Claim made in
research/q5_structure_prediction_docking/alphafold3_openfold3_boltz2_chai1_protenix_nanobody_antigen_benchmarks_and_success_rates.md
#27 -v1 -antigen success 36.0 -> 42.9 -> 47.7% at 1/10/80 seeds; selected 40.3 vs best 54.2unreachable
- Verdict
- unreachable
- Notes
- PDF text streams could not be extracted (no pdf library available; WebSearch also could not confirm). Treat as unverified.
- Checked against
- PTX_V1 technical report PDF (URL resolves 200)
- Claim made in
research/q5_structure_prediction_docking/alphafold3_openfold3_boltz2_chai1_protenix_nanobody_antigen_benchmarks_and_success_rates.md
#28 window 2026-09-01 to 2026-10-31; results 2026-12-15verified
- Verdict
- verified
- Notes
- /detail: "submission (Sep 1-Oct 31, 2026)", "results announcement (Dec 15, 2026)". Home page also says "scheduled to begin in June 2026", judging 1-30 Nov. /guideline states no deadline.
- Checked against
- http[local path] ; http[local path]
- Claim made in
_meta/sepiq_verbatim.md (and BRIEF)
#29 Prizes $3,000/$2,000/$1,000; up to 5 submissions; 3 held-out complexes /13/23; macro- over 3; public-data not an issueverified
- Verdict
- verified
- Notes
- Prizes and 5-submission cap appear on /detail only; /guideline lists neither; home page lists no prizes (sponsor tiers "coming soon"). /guideline: over 3 antibodies; quote "we do not consider data ... to be an issue" (conditional on the answer being correct). HF Space has 42 likes (HF API); dataset is gated=manual.
- Checked against
- http[local path] ; http[local path]
- Claim made in
_meta/sepiq_verbatim.md (and BRIEF)
#30 Portal returns no score to participantsunsupported
- Verdict
- unsupported
- Notes
- Neither page says scores are or are not returned. /detail only mentions limiting submissions "to reduce leaderboard probing" and organiser-side "leaderboard reporting". Keep as "unknown, assume no feedback", not as a fact.
- Checked against
- http[local path] ; http[local path]
- Claim made in
_meta/king_world_KINGLOG.md
Patterns the audit sawWhere errors tend to come from
- Numbers computed by the hive from the real tables (counts, read totals, depletion) are accurate; the one understatement ( depletion is roughly 180-420x, not ">20x") errs conservative.
- Drift comes from stale inheritance: the BRIEF "few hundred labels" survived in q3, and "Fab" for 9MTE/9MTX was inferred rather than read from the entity list.
- Benchmark rates are right but easy to mislabel: 31.6% (>0.23) vs 13.3% (DockQ>=0.80) for AF3 nanobodies; single seed, n=60 (CI roughly 21-45%, worker arithmetic).
- Provenance traps: 0.176 comes from .uk, not the paper; "portal returns no score" is an absence of statement, not a stated rule; prizes appear on /detail only.
- Worker-run statistics (replicate precision, oracle, AUROC drops) were not rerun here and stay medium confidence. seed-scaling numbers are unverified.
- : 3-5% holds for single- epitopes on modelled (5MY6 3.1%, 9MTX 4.3%); the 9MTE two-VHH union is 6.2%.