Chain-of-Thought (CoT): Forcing Step-by-Step Reasoning for Logic and Math Reliability
Standard autoregressive transformers generate tokens one at a time, where each token probability distribution is conditioned only on past tokens through $L$ feed-forward layers. When an LLM is asked to jump directly from a complex mathematical premise or multi-step logical deduction to the final answer in a single forward pass, it is forced to condense an exponential tree of intermediate states into a fixed depth computation.
Chain-of-Thought (CoT) prompting circumvents this fundamental architectural constraint. By instructing the model to generate intermediate reasoning tokens, CoT effectively gives the network dynamic recurrence and expanded computational budget, transforming the model into a sequential virtual machine.
Theoretical Mechanics: Marginalizing Over Reasoning Paths
Mathematically, the probability of generating the correct final answer $a$ given query $q$ is marginal over all latent reasoning paths $r = (r_1, r_2, \dots, r_m)$:
P(a | q) = \sum_{r \in \mathcal{R}} P(a | r, q) \cdot P(r | q)When an LLM is sampled using greedy decoding on direct prompting, it approximates $\arg\max_a P(a | q)$. In contrast, Chain-of-Thought decomposes this into a sequential generation of reasoning steps:
P(r_1, \dots, r_m, a | q) = \prod_{i=1}^{m} P(r_i | r_{<i}, q) \cdot P(a | r_{1:m}, q)Because each generated token $r_i$ becomes part of the key-value (KV) cache for subsequent steps, the attention heads in later layers can perform self-attention over the intermediate computations, verifying consistency and eliminating premature token collapse.
Production-Grade Self-Consistency CoT with Python & Async Aggregation
Single-path CoT is still vulnerable to single-point stochastic hallucinations. Self-Consistency CoT (Wang et al.) samples $N$ distinct reasoning trajectories at temperature $T=0.7$ and conducts a majority vote across parsed answers:
import asyncio
import re
from collections import Counter
from typing import List, Tuple
from openai import AsyncOpenAI
client = AsyncOpenAI()
COT_SYSTEM_PROMPT = """
You are a formal mathematical and algorithmic verification engine.
When given a problem, you must strictly follow this structure:
Break down the problem step-by-step. Number every step.
Calculate all intermediate values explicitly. Verify boundary conditions.
State the exact final scalar or structured outcome only.
"""
async def sample_reasoning_trajectory(query: str, temperature: float = 0.7) -> Tuple[str, str]:
response = await client.chat.completions.create(
model="gpt-4o",
temperature=temperature,
messages=[
{"role": "system", "content": COT_SYSTEM_PROMPT},
{"role": "user", "content": query}
]
)
content = response.choices[0].message.content or ""
# Extract intermediate thinking and final answer
thinking_match = re.search(r"(.*?) ", content, re.DOTALL)
answer_match = re.search(r"(.*?) ", content, re.DOTALL)
thinking = thinking_match.group(1).strip() if thinking_match else ""
answer = answer_match.group(1).strip() if answer_match else content.strip()
return thinking, answer
async def self_consistency_cot(query: str, num_samples: int = 5) -> str:
tasks = [sample_reasoning_trajectory(query) for _ in range(num_samples)]
results = await asyncio.gather(*tasks)
answers = [ans for _, ans in results]
vote_counts = Counter(answers)
majority_answer, count = vote_counts.most_common(1)[0]
print(f"Self-Consistency Consensus: {count}/{num_samples} agreed on: {majority_answer}")
return majority_answer