We conducted benchmark evaluations across safety categories. We
report here on our Production Benchmarks, an evaluation set with
conversations representative of challenging examples from production
data. As we noted in previous system cards, we introduced these
Production Benchmarks to help us measure continuing progress given that
earlier Standard evaluations for these categories had become relatively
saturated.
These evaluations were deliberately created to be difficult. They
were built around cases in which our existing models were not yet giving
ideal responses, and this is reflected in the scores below. Error rates
are not representative of average production traffic. The primary metric
is safe completion rate, checking that the model’s response and actions
——
本文节选自 OpenAI 的《GPT-6 Sol and GPT-6 Luna October update》，原文链接：https://deploymentsafety.openai.com/gpt-6-october
