Protein Language Models: Powerful Black Boxes in Need of Explanation
A new perspective paper in Nature Machine Intelligence from researchers at the Centre for Genomic Regulation (CRG) warns that protein language models (pLMs)—powerful AI tools for engineering novel proteins—are operating as black boxes, creating serious concerns about their reliability, bias, and safety in real-world applications.
"Protein language models are moving fast but our understanding of fundamental biological processes... has not advanced alongside these breakthroughs."
The Promise and the Peril
Protein language models are used to design novel proteins, including enzymes that could absorb carbon dioxide or catalysts that reduce toxic waste in manufacturing. These tools hold immense potential for tackling global challenges, yet the authors argue that without understanding how they arrive at their predictions, they cannot be fully trusted.
Four Critical Areas for Explanation
The paper identifies four areas where explainability is essential for pLMs:
- Training Data: Understanding what biases exist in the data the model learns from.
- Input Sequences: Explaining predictions related to specific protein sequences.
- Model Architecture: Interpreting how the model's internal components work.
- Input-Output Behavior: Understanding how input changes affect predictions.
The Four Roles of Explainability
Through a literature review, the authors classified how explainability is currently used in protein research:
- Evaluator: Checking what the model has learned.
- Multitasker: Reapplying signals for protein annotation.
- Engineer/Coach: Optimizing model design (rarely used).
- Teacher: Revealing new biological principles (the ultimate goal).
"Most current use falls under Evaluator and Multitasker roles; very few studies use explainability for engineering or coaching purposes."
The Path Forward
The paper calls for community-wide action to establish robust benchmarks, evaluation frameworks, and open-source tools for explainability. Crucially, it emphasizes that any AI-derived insight must be validated experimentally in the lab.
"If we want protein language models to become a reliable partner in discovery and design, explainability must not be an afterthought," said Andrea Hunklinger, the study's first author.
Conclusion: From Pattern Recognizer to Trusted Partner
The authors argue that reaching a "Teacher" status—where pLMs can reveal new biological principles and provide mechanistic explanations—will require both reliability and validation. This would transform these models from simple pattern recognizers into truly trusted design partners.