Layer 4 — Model Architecture · Chapter 4

Reasoning, Planning, and Test-Time Computation

Book chapterLayer 4

Many early language models generated an answer in one continuous pass. More capable systems allocate additional computation to difficult problems, examine intermediate steps, use tools, test alternatives, or revise an initial result. This is often called test-time computation or inference-time scaling.

Reasoning can be designed at several levels

  1. Internal computation. The model performs more transformations before producing an answer.
  2. Generated intermediate steps. The system creates a sequence of partial conclusions, subproblems, or scratch work.
  3. Search over alternatives. Several candidate solutions or plans are generated and compared.
  4. Tool-assisted reasoning. The model calls calculators, code, databases, simulators, or external verifiers.
  5. Feedback and revision. A critic, evaluator, or second pass identifies weaknesses and requests improvement.

Planning is different from prediction

Prediction asks what is likely to come next. Planning asks which sequence of actions is most likely to reach a goal. A planning architecture usually needs a representation of the current state, possible actions, constraints, expected consequences, and a way to compare future paths.

Search and verification

When a problem has a checkable answer, architecture can separate generation from evaluation. One component proposes solutions; another tests them. Search may explore trees, graphs, programs, or simulated trajectories. Verification can be learned, rule-based, or performed by an external tool.

Test-time computation creates an economic choice

A system may answer an easy question quickly and spend much more computation on a difficult one. This creates a new design variable: not only how large the model is, but how much thinking it performs for each task.

Fixed computation

Every request follows roughly the same computational path. Cost and latency are predictable.

Adaptive computation

The system spends more effort when uncertainty, complexity, or risk is high.

Persistent difficulties

More computation can improve an answer, but architecture must decide where to spend it, how to test it, and when to stop.