AI's 94% Failure Rate: Why Most Computer-Designed Cancer Proteins Never Make It Past the Cell Door

Artificial intelligence has revolutionized protein design on paper, but a groundbreaking global competition has exposed a sobering reality: fewer than 6% of AI-designed therapeutic proteins actually function when tested in living human cells. The Bits to Binders challenge, which evaluated 12,000 computer-generated cancer-targeting proteins in CAR-T cells, reveals that the bottleneck isn't where researchers expected—most designs fail not because they can't bind their targets, but because cells simply can't produce them.

The study, published in Molecular Systems Biology, tested submissions from 28 teams across 42 countries who designed minibinders targeting the CD20 antigen for CAR-T cell therapy. While 98% of designs successfully passed DNA synthesis, only 56.8% survived cellular growth, and a mere 707 designs (5.9%) demonstrated meaningful CD20-specific activity. Team success rates varied dramatically, from 0.6% to 38.4%, suggesting that design strategy matters far more than computational power.

The ProteinMPNN Problem: When AI Creates Untranslatable Sequences

The competition uncovered a critical systematic failure in widely adopted AI tools. An overwhelming 98.9% of designs that failed cellular screening were generated by ProteinMPNN or SolubleMPNN—popular sequence design algorithms used by nearly 20 teams. These tools consistently produced alpha-helical structures (83% helical content) densely packed with lysine and glutamate residues that appear to block cellular translation machinery.

Failed designs averaged 35% of their residues in these problematic lysine-glutamate helices, compared to just 15% in successful designs. The culprit appears to be repetitive sequence patterns: failed designs contained approximately four glutamate-glutamate repeats versus 1.6 in recovered designs. Simple sequence statistics like the lysine-plus-glutamate helix ratio achieved 0.91 ROC-AUC for predicting recovery failure—far outperforming sophisticated structure-based confidence metrics.

Perhaps most troubling for the field, standard computational confidence scores from AlphaFold2 and related tools (ipTM, pLDDT, ipSAE, LIS, SAP, PDockQ) showed minimal predictive power for functional outcomes. The AI models confidently predicted structures for proteins that cells couldn't actually make.

What Worked: Context Matters More Than Confidence Scores

Among the top performers, seven of the ten leading designs outperformed controls in T cell proliferation and cytokine production, though only four achieved significant tumor cell killing. Interestingly, these successful AI designs showed weaker binding affinity (200-643 nM) compared to the traditional scFv control (1.75 nM)—a feature that may actually be optimal for CAR-T function by avoiding T cell exhaustion.

The competition's standout lesson came from design strategy rather than computational sophistication. Only one team folded their candidate binders within the full therapeutic context rather than in isolation, and this approach produced the top-performing design. Meanwhile, 18 teams used RFdiffusion, 17 used AlphaFold2 for validation, and 12 followed the default RFdiffusion-ProteinMPNN-AlphaFold2 pipeline—yet this popular workflow correlated with the translation-blocking failure mode.

Implications for Drug Discovery Pipelines

The findings carry immediate consequences for biotech companies and health AI platforms integrating these tools into therapeutic development. The study provides actionable sequence filters—including GC content, entropy measurements, adenosine/glutamate repeat detection, and lysine-glutamate helix ratios—that could dramatically improve hit rates. Companies relying on default AI workflows may be experiencing hidden 94%+ failure rates before reaching efficacy testing, wasting experimental resources on sequences that never produce functional proteins.

The competition methodology itself—using GeneWeld targeted integration into the TRAC locus of primary human T cells, pooled screening with DNA barcode tracking, and a two-week challenge with CD20+ Raji tumor cells—demonstrates a scalable validation framework that could become standard practice for evaluating computational designs before expensive downstream development.

For an industry increasingly dependent on AI-accelerated drug discovery, this large-scale validation study serves as both warning and roadmap: computational confidence must be supplemented with biological reality checks, and design tools must account for the full cellular context in which therapeutic proteins must function.

Sources

1. Global AI protein design contest puts 12,000 computer-made cancer binders to the test in living T cells — bioengineer.org