This is the repo for the paper A Comparative Study on Reasoning Patterns of OpenAI's o1 Model.
Enabling Large Language Models (LLMs) to handle a wider range of complex tasks (e.g., coding, math) has drawn great attention from many researchers. As LLMs continue to evolve, increasing the number of model parameters yields diminishing performance improvements and heavy computational costs. Recently, OpenAI's o1 model has shown that inference strategies (i.e., Test-time Compute methods) can also significantly enhance the reasoning capabilities of LLMs. However, the mechanisms behind these methods are still unexplored. In our work, to investigate the reasoning patterns of o1, we compare o1 with existing Test-time Compute methods (BoN, Step-wise BoN, Agent Workflow, and Self-Refine) by using OpenAI's GPT-4o as a backbone on general reasoning benchmarks in three domains (i.e., math, code and commonsense reasoning). Specifically, first, our experiments show that the o1 model has achieved the best performance on most datasets. Second, as for the methods of searching diverse responses (e.g., BoN), we find the reward models' capability and the search space both limit the upper boundary of these methods. Third, as for the methods that break the problem into many sub-problems, the Agent Workflow has achieved better performance than Step-wise BoN due to the domain-specific system prompt for planning better reasoning processes. Fourth, we summarize six reasoning patterns of o1 and provide a detailed analysis across different reasoning benchmarks.
Our filtered data can be found in this repo: data.
Besides, we also upload those data into hf dataset. The following code can be used:
from datasets import load_dataset
data = load_dataset('SiweiWu/o1_Reasoning_Patterns_Study')However, apart from the text data, Collie also provides functions in a special format that can be directly used to evaluate the generated responses of LLMs, but it cannot be uploaded to the HF dataset. Therefore, for the Collie dataset, we still suggest that you load the data from the GitHub folder data.
You can run the run.sh to reproduce the results on our main table.
bash run.shYou can change the '--N' and 'step_wise_type' and 'dataset_name' to obtain the results at different settings.
As for the evaluation of the Code task result (USACO), we are organizing this part of the code. As for Self-Refine, we use the code from Self-Refine. As for the Agent Workflow, we utilize the state-of-the-art agent framework Agents 2.0: Symbolic Learning Enables Self-Evolving Agents on the HotpotQA and Collie, and we use the GPTs for USACO and AIME.
The results of all settings in the main table are released in hf_dataset in a zip format: (data)[https://huggingface.co/datasets/SiweiWu/o1_analysis_results/blob/main/results.zip]. You can download it by using the following code:
wget https://huggingface.co/datasets/SiweiWu/o1_analysis_results/blob/main/results.zip@article{wu2024comparative,
title={A Comparative Study on Reasoning Patterns of OpenAI's o1 Model},
author={Wu, Siwei and Peng, Zhongyuan and Du, Xinrun and Zheng, Tuney and Liu, Minghao and Wu, Jialong and Ma, Jiachen and Li, Yizhi and Yang, Jian and Zhou, Wangchunshu and others},
journal={arXiv preprint arXiv:2410.13639},
year={2024}
}