Partnership on AI Safety Standards
Industry consortium standards for AI safety including red teaming and safety evaluations.
| Category | AI Regulations |
|---|---|
| Status | Active |
| Latest development |
Analysis
The Partnership on AI Safety Standards is not a binding regulation; it is an industry-led consortium initiative. Based on the material provided, the closest official PAI document is PAI’s Guidance for Safe Foundation Model Deployment, which sets voluntary guidance for foundation and frontier model providers on safety, documentation, transparency, red teaming, and risk management.[1][2]
Because this is a consortium standard rather than a statute, there are no legal compliance deadlines in the way there would be for a government regulation; instead, organizations typically treat the guidance as a governance framework to align internal controls, evaluation processes, and disclosure practices.[1][2]
Key Requirements
- Pre-development disclosure and planning: PAI’s guidance calls for a “Pre-Systems Card” that discloses planned testing, evaluation, and risk-management procedures before development begins.[2] Source: PAI press release, NIST docket attachment summarizing PAI guidance
- Pre-deployment impact reporting: The guidance calls for public reporting of model impacts and a “key ingredient list” before deployment, along with downstream-use documentation.[2] Source: NIST docket attachment summarizing PAI guidance, PAI press release
- Post-deployment transparency: PAI recommends development of transparency reporting standards for disclosing foundation/frontier model usage and policy violations after deployment.[2] Source: NIST docket attachment summarizing PAI guidance, PAI press release
- Third-party inspection support: The guidance includes support for third-party inspection of models and training data in relation to societal risks.[2] Source: NIST docket attachment summarizing PAI guidance
- AI red-teaming enablement: PAI explicitly calls for establishing guidelines that enable developers of dual-use foundation models to conduct AI red-teaming tests.[2] Source: NIST docket attachment summarizing PAI guidance
Compliance Challenges
- Undefined minimum benchmark for “safe enough”: PAI guidance is high-level and multistakeholder, so organizations must translate broad safety and transparency expectations into concrete internal acceptance criteria.[1][2] Source: PAI press release, NIST docket attachment
- Operationalizing red teaming at scale: AI red-teaming requires specialized expertise, scenario design, and ongoing repetition as models change, which is hard to standardize across teams and model families.[2][8] Source: NIST docket attachment, NIST U.S. AI Safety Institute announcement
- Documentation burden across the lifecycle: The guidance expects documentation at research, pre-deployment, and post-deployment stages, which creates coordination challenges across product, legal, policy, and engineering teams.[2] Source: NIST docket attachment
- External validation and independent review: Supporting third-party inspection of models and training data can raise privacy, confidentiality, and security concerns, especially for frontier models.[2] Source: NIST docket attachment
- Real-world example of implementation complexity: NIST’s U.S. AI Safety Institute has formed formal AI safety research and evaluation collaborations with model developers including Anthropic and OpenAI, illustrating that frontier-model safety testing often requires structured partnerships rather than purely internal processes.[8] Source: NIST announcement
Implementation Best Practices
- Adopt a lifecycle control model: Map PAI’s expectations to a staged workflow covering research, pre-deployment, launch, and post-deployment review, using the guidance’s own lifecycle framing as the baseline.[2] Source: NIST docket attachment
- Create a formal red-teaming program: Define test objectives, threat scenarios, evaluator roles, escalation criteria, and remediation tracking, and align the process with broader AI risk-management practices.[2][4] Source: NIST AI Risk Management Framework materials, NIST docket attachment
- Implement standardized documentation: Use a consistent template for pre-systems cards, model cards, and post-deployment transparency reporting so product teams can reuse the same artifact set across releases.[2] Source: NIST docket attachment
- Use external reviewers where appropriate: Establish a process for independent testing or peer review of high-risk models, especially when model capabilities or societal impacts are difficult to assess internally.[2][8] Source: NIST docket attachment, NIST AI Safety Institute announcement
- Align with recognized risk frameworks: Pair PAI-style safety practices with the NIST AI Risk Management Framework for governance, mapping, measurement, and management of AI risks.[4] Source: NIST AI RMF docket material
- Useful tools and resources:
- PAI Guidance for Safe Foundation Model Deployment press release
- NIST AI Risk Management Framework docket material
- NIST U.S. AI Safety Institute collaboration announcement
- OWASP AI Exchange / AI Security and Privacy Guide
Recent Updates
- October 24, 2023: PAI released its Guidance for Safe Foundation Model Deployment for public comment.[1] Source: PAI press release
- January 15, 2024: The public comment period for the PAI model deployment guidance was scheduled to close.[1] Source: PAI press release
- Spring 2024: PAI said an updated version was expected in Spring 2024.[1] Source: PAI press release
- August 2024: NIST announced AI safety research and evaluation agreements with Anthropic and OpenAI through the U.S. AI Safety Institute, showing continued institutionalization of AI safety testing and evaluation work.[8] Source: NIST announcement
- Effective dates and deadlines: No legally binding compliance deadline is stated in the provided sources because PAI’s guidance is voluntary and was issued as a public-comment draft rather than a regulation.[1][2] Source: PAI press release, NIST docket attachment
Related Regulations
- NIST AI Risk Management Framework (AI RMF): This is the most directly related U.S. framework for identifying, assessing, and managing AI risks, and it complements PAI-style safety guidance by providing a broader governance structure.[4] Source: NIST AI RMF docket material
- OWASP AI Security and Privacy Guide: This is a practitioner-focused security resource that overlaps with PAI’s emphasis on red teaming, testing, and operational safety controls.[7] Source: OWASP AI Security and Privacy Guide
- U.S. AI Safety Institute initiatives: These are not regulations, but they overlap with PAI’s goals by advancing safety research, testing, and evaluation partnerships for frontier AI systems.[8] Source: NIST U.S. AI Safety Institute announcement
- How they interact: PAI guidance can serve as a voluntary operational standard, while NIST AI RMF and related safety-institute work provide a more formal risk-management and evaluation backbone for implementation.[2][4][8] Source: NIST docket attachment, NIST AI RMF docket material, NIST AI Safety Institute announcement
Industry Impact
- Higher baseline expectations for safety testing: PAI’s guidance helped normalize the idea that frontier-model providers should perform structured safety evaluations and red teaming before deployment.[1][2] Source: PAI press release, NIST docket attachment
- More emphasis on transparency artifacts: The requirement-like guidance for pre-systems cards, model-impact reporting, and downstream-use documentation pushes industry toward more standardized documentation practices.[2] Source: NIST docket attachment
- Greater use of external testing partnerships: NIST’s collaborations with Anthropic and OpenAI show a broader market trend toward independent or quasi-independent safety evaluation arrangements for frontier systems.[8] Source: NIST announcement
- Influence on standards ecosystems: OWASP notes that its AI security and privacy work contributes to international standards efforts, showing how industry consortium guidance can shape downstream standards development.[7] Source: OWASP AI Security and Privacy Guide
Sources
- PAI Releases Guidance for Safe Foundation Model Deployment
- NIST docket attachment summarizing PAI Model Deployment Guidance
- NIST AI Risk Management Framework docket material
- NIST U.S. AI Safety Institute signs agreements regarding AI safety research
- OWASP AI Security and Privacy Guide
- AI Safety Frameworks & Standards overview
Recent developments
- — Microsoft announced new agreements with the U.S. Center for AI Standards and Innovation and the U.K. AI Security Institute to advance AI testing and evaluation. The collaboration focuses on frontier-model assessments, safeguards, and methods to reduce national-security and public-safety risks. (source)
- — The U.S. launched the International Network of AI Safety Institutes with partners including the EU, U.K., Japan, and others. The network is intended to coordinate work on synthetic content risks, foundation-model testing, and advanced AI risk assessments. (source)
- — The U.S. AI Safety Institute signed agreements with Anthropic and OpenAI to collaborate on AI safety research. The memorandums allow pre- and post-release access to major new models and include feedback on possible safety improvements. (source)
Related regulations
- Internet Information Service Algorithmic Recommendation Management Provisions — China, Active
- National Artificial Intelligence Policy Bill — Chile, Active
- Artificial Intelligence Liability Directive — European Union, Active
- Federal Law on the Development and Use of Artificial Intelligence — Mexico, Active
- Presidential Regulation on the Framework for Artificial Intelligence Development — Indonesia, Active
- Law Establishing the Framework for the Development and Use of Artificial Intelligence — Peru, Active
- California Consumer Privacy Act AI Amendments — California, Active, effective 2020-01-01
- GPAI Framework for Responsible AI — AI Regulations, Active
Put it into practice
- Generate the policy: NIST AI RMF policy generator (generatepolicy.com)
- Buy the policy pack: NIST AI RMF Implementation Policy (cyberpolicy.shop)
- Build it yourself: Pillar 06 Companion — The 2026 AI Risk Register (ciso.diy)
Browse all regulations · Compliance deadlines · Latest updates