2023 Amylase Benchmark

A protein engineering competition to benchmark model performance with wetlab results.

In 2023, Align launched the inaugural Protein Engineering Tournament: a proof-of-concept competition that brought together academic, industry, and independent teams to test protein design models against real-world experimental validation. The theme: enzyme function prediction and design.

This pilot tested whether community challenges could generate rigorous, comparable benchmarks that the community would jump on. The result was a resounding yes.

7 finalist teams (3 academic, 2 industry, 2 mixed)

85 registered users across 28 teams

>40,000 data points across six protein datasets

1,200 unique sequences synthesized and tested

6 datasets contributed from 4 academic and 2 industry groups

The Challenge: Predict and Design Functional Enzymes

Participants competed in two rounds:

In silico round: Teams predicted properties of enzyme variants using provided sequences and (in some cases) training data.

In vitro round: Top teams designed new enzyme variants for experimental testing. Align partnered with International Flavors and Fragrances to synthesize and test submissions to determine the winner.

Six datasets were donated for this pilot Tournament, covering diverse enzymes including aminotransferase, α-amylase, imine reductase, and more. All data, code, and results are openly available on GitHub.

Winners and Highlights

Champion: TUM Rostlab

Awarded for the top-performing Amylase variant, measured by activity while maintaining thermostability and expression thresholds.

In Silico Zero-Shot Champion: Marks lab

In Silico Supervised Tie: ExaZyme and Nimbus

In Silico Overall Winner: Nimbus

In Vitro Winner & Overall Tournament Champion: TUM Rostlab

Zero-shot and Supervised Leaderboards Image (Align to get for Receptor, in progress)

Full Dataset and Results
View the Codebase
Results of the Protein Engineering Tournament: An Open Science Benchmark for Protein Modeling and Design
Read the Paper
Webinar: 2023 Community Recap
Watch the webinar

What We Learned

  • Benchmarking protein function models reveals strengths and gaps in current methods and fosters transparency in the community.
  • Clear, detailed reporting of experimental protocols is essential. Transparent communication about assay design and selection conditions helps participants reach better protein engineering outcomes.
  • Our challenges had a wide range of difficulty. We provided datasets on six different proteins, and we challenged teams to predict expression, melting point, thermostability, enzymatic activity, and substrate promiscuity with spearman accuracy of model predictions ranging from extremely good (<highest spearman>) to a grand challenge in the field (<lower spearmans, or closest to zero>).

This pilot set the stage for future tournaments with broader scope, bigger datasets, and more ambitious challenges—starting with the 2025 PETase Tournament now underway.

Contribute to the Next Tournament

If you have a dataset to share, want to host a challenge event, or are interested in sponsoring wet-lab evaluation, reach out at tournament@alignbio.org.

Ready to compete? Join the 2025 Tournament →

Interested in getting involved?