C3: Systemic Risk Analysis
Analyze identified risks through evaluation, modelling, and estimation.
Step 3 of 10
Commitment Progress0% (0/7)
REQ-3.1-01Measure 3.1 (Page 11)
Is model-independent information gathered to inform risk analysis?
"Signatories will gather model-independent information... using methods such as: web searches... literature reviews... market analyses... expert interviews."
REQ-3.2-01Measure 3.2 (Page 12)
Are state-of-the-art model evaluations conducted for capabilities and effects?
"Signatories will conduct at least state-of-the-art model evaluations in the modalities relevant to the systemic risk to assess the model’s capabilities, propensities, affordances, and/or effects."
REQ-3.2-02Measure 3.2 (Page 12)
Do evaluations include open-ended testing (e.g., red-teaming) to identify unexpected behaviors?
"Include open-ended testing of the model... with a view to identifying unexpected behaviours... Examples... red-teaming and other methods of adversarial testing."
REQ-3.3-01Measure 3.3 (Page 12)
Is systemic risk modelling conducted using state-of-the-art methods?
"Signatories will conduct systemic risk modelling... use at least state-of-the-art risk modelling methods."
REQ-3.4-01Measure 3.4 (Page 12)
Are the probability and severity of harm estimated for each risk?
"Signatories will estimate the probability and severity of harm for the systemic risk... expressed as a risk score, risk matrix, probability distribution, or in other adequate formats."
REQ-3.5-01Measure 3.5 (Page 13)
Is post-market monitoring conducted to assess risk acceptability?
"Signatories will conduct appropriate post-market monitoring to gather information relevant to assessing whether the systemic risk could be determined to not be acceptable."
REQ-3.5-02Measure 3.5 (Page 13)
Do independent external evaluators have access to the model?
"Signatories will provide an adequate number of independent external evaluators with adequate free access to the model's most capable versions."