--- id: test-time-compute-scaling title: "Test-Time Compute Scaling: Inference-Time Reasoning Paradigms from o1/o3 to Forest-of-Thought" schema_type: article category: ai language: en confidence: high last_verified: "2026-05-24" created_date: "2026-05-24" generation_method: ai_assisted ai_models: - claude-4.5-sonnet derived_from_human_seed: true conflict_of_interest: none_declared is_live_document: false data_period: static completeness: 0.85 atomic_facts: - id: af-test-time-compute-scaling-1 statement: >- OpenAI o1 (September 2024) and o3 (December 2024) introduced the test-time compute scaling paradigm — allocating additional computation during inference (not training) to generate and evaluate multiple reasoning chains — achieving PhD-level performance on GPQA Diamond (87.7% o3 vs. 65% human experts) and ELO 2727 on Codeforces, demonstrating that inference-time computation can substitute for model size scaling. source_title: OpenAI o1 System Card (2024) / OpenAI o3 Announcement (2024) — Test-Time Compute Scaling Paradigm source_url: https://openai.com/index/learning-to-reason-with-llms/ confidence: high - id: af-test-time-compute-scaling-2 statement: >- Forest-of-Thought (Li et al., arxiv 2412.09078, 2024) proposed a tree-structured reasoning framework that scales test-time compute by building a forest of reasoning paths and dynamically pruning branches via confidence-based and consistency-based criteria — achieving 15-20% improvement over chain-of-thought baselines on MATH and GSM8K benchmarks with equivalent compute budget. source_title: "Li et al., arxiv 2412.09078 (2024) — Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning" source_url: https://arxiv.org/abs/2412.09078 confidence: high primary_sources: - id: ps-test-time-compute-scaling-1 title: "OpenAI o1 System Card: Learning to Reason with Large Language Models" type: industry_report year: 2024 institution: OpenAI url: https://openai.com/index/learning-to-reason-with-llms/ - id: ps-test-time-compute-scaling-2 title: "Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning" type: academic_paper year: 2024 institution: arXiv url: https://arxiv.org/abs/2412.09078 known_gaps: - Optimal compute allocation between training-time and test-time budgets - Test-time compute strategies for non-reasoning tasks (creative writing, dialogue) disputed_statements: [] secondary_sources: - title: Scaling LLM Test-Time Compute Optimally Can Be More Effective Than Scaling Model Parameters (DeepMind) type: conference_paper year: 2024 authors: - Snell, Charlie - Lee, Jaehoon - Xu, Kelvin - Kumar, Aviral institution: Google DeepMind / ICLR 2025 url: https://arxiv.org/abs/2408.03314 - title: "OpenAI o1 System Card: Learning to Reason with Large Language Models" type: technical_report year: 2024 authors: - OpenAI institution: OpenAI url: https://openai.com/research/learning-to-reason-with-llms - title: Let's Verify Step by Step (Process Reward Models for Reasoning) type: conference_paper year: 2024 authors: - Lightman, Hunter - Kosaraju, Vineet - Burda, Yuri - et al. institution: OpenAI / ICLR url: https://arxiv.org/abs/2305.20050 - title: "Large Language Monkeys: Scaling Inference Compute with Repeated Sampling" type: conference_paper year: 2024 authors: - Brown, Bradley - Jurafsky, Dan - Hashimoto, Tatsunori institution: Stanford / NeurIPS url: https://arxiv.org/abs/2407.21787 updated: "2026-05-24" --- ## TL;DR Test-Time Compute Scaling represents a paradigm shift: instead of making models bigger during training, allocate more computation during inference for deeper reasoning. OpenAI o1/o3 demonstrated that "thinking longer" enables PhD-level scientific reasoning and competitive programming — reshaping the scaling landscape from pre-training to inference. ## Core Explanation Traditional scaling: increase model parameters (GPT-3→4) and training data to improve capability (pre-training scaling law). Limitation: diminishing returns; each 10x parameter increase yields ~5% benchmark improvement. Test-time compute scaling: allocate a fixed inference compute budget across strategies — (1) Best-of-N sampling — generate N independent chains, select best by verifier; (2) Chain-of-Thought (CoT) with sequential revision — generate, self-criticize, improve; (3) Tree search — build branching reasoning tree (Forest-of-Thought, Tree-of-Thoughts); (4) Process reward models (PRMs) — score intermediate steps, guide search. The key insight from OpenAI's research: test-time compute and model size follow a smooth substitutability curve — a smaller model with more inference time can match a larger model with less. ## Detailed Analysis OpenAI o1 architecture: internal chain-of-thought (hidden to users) + RL training with process reward models. The model "thinks" in a private reasoning space before producing the final answer. o3 (December 2024) scaled this further with fine-tuned compute allocation per problem difficulty. DeepSeek-R1 (January 2025) replicated the paradigm via pure RL without supervised CoT data, proving that reasoning emerges from reward optimization alone. Eight test-time compute strategies (from 2025 systematization surveys): (1) N-best sampling, (2) Majority voting, (3) CoT self-consistency, (4) Tree-of-Thoughts, (5) Forest-of-Thought, (6) Monte Carlo Tree Search (MCTS) reasoning, (7) Self-refinement loops, (8) Adaptive compute routing (allocate more inference time to harder problems). Practical considerations: latency vs. quality trade-off; for real-time applications (<1s latency), CoT with 2-3 revision steps outperforms complex tree search. Cost analysis (2025): o1-level reasoning at scale costs $0.10-1.00 per query vs. $0.001-0.01 for standard LLM inference — limiting deployment to high-value domains (scientific research, drug discovery, math education). ## Further Reading - DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL (2025) - Let's Verify Step by Step (OpenAI Process Reward Models, 2023) - Scaling LLM Test-Time Compute Optimally (Snell et al., NeurIPS 2024)