NIST’s CAISI Signs New AI Safety Agreements with Google DeepMind, Microsoft, and xAI
The Center for AI Standards and Innovation (CAISI) at the National Institute of Standards and Technology has announced new agreements with Google DeepMind, Microsoft, and xAI. These pacts allow CAISI to conduct pre-deployment evaluations and targeted research on frontier AI models to assess capabilities and advance AI security.
These new agreements build on previously announced partnerships that have been renegotiated to align with directives from the Secretary of Commerce and the AI Action Plan. CAISI has been designated as the primary U.S. government point of contact for industry testing, collaborative research, and best practice development related to commercial AI systems.
"Independent measurement science is essential to understanding frontier AI and its national security implications."
CAISI evaluates AI models before public release and conducts post-deployment assessments. To date, CAISI has completed over 40 evaluations, including on unreleased state-of-the-art models.
Director Chris Fall stated that independent measurement science is essential to understanding frontier AI and its national security implications, noting that these collaborations help scale CAISI's work in the public interest.
Key aspects of the agreements include:
- Support for information-sharing and voluntary product improvements
- Enhanced government understanding of AI capabilities and international competition
- Access to models with reduced or removed safeguards for thorough evaluation of national security-related risks
- Participation from evaluators across government via the TRAINS Taskforce, an interagency group focused on AI national security
The agreements allow testing in classified environments and are designed to be flexible to respond to rapid AI advancements.