What OpenAI o1’s performance means for exams and expert benchmarks
Machine-assisted translation of the archived article. Read the Chinese original.

With o1, the author sees the industry’s focus shifting from parameter counts to time spent reasoning. The article presents that shift as a possible step towards artificial general intelligence.

As an early model, o1 had several feature limitations at launch.
The author recorded the following errors when testing unsupported features.
system field — HTTP 400
tools field — HTTP 400
Image input — HTTP 400
json_object output — HTTP 500
Structured output — HTTP 400
logprobs output — HTTP 403
Streaming output — HTTP 400
The article reports limits of 20 RPM and 150,000,000 TPM for o1. Requests over a limit could receive HTTP 429.
The temperature, top_p and n parameters were fixed at 1.
The presence_penalty and frequency_penalty parameters were fixed at 0.

The o1 family and o1-preview
OpenAI reported that its reasoning model performed at a level comparable to PhD students on challenging physics, chemistry and biology benchmarks.
On a qualifying exam for the International Mathematical Olympiad, GPT-4o solved 13% of the problems and the reasoning model solved 83%. In competitive programming, it reached the 89th percentile on Codeforces.
OpenAI noted that the early model lacked several ChatGPT features, including browsing and uploading files or images. It expected GPT-4o to remain the stronger option for many everyday tasks in the near term.
OpenAI described the result as a significant advance for complex reasoning, reset its model numbering and named the new series OpenAI o1.

The examples OpenAI gave for o1 included annotating cell sequencing data, generating mathematical expressions for quantum optics and helping developers carry out multi-step workflows.
At launch, developers in API usage tier 5 could prototype with o1 at 20 requests per minute. OpenAI said it would work to raise those limits after more testing.
