Layer 4 — Model Architecture · Chapter 1

What Is Model Architecture?

Book chapterLayer 4

Model architecture is the blueprint of an AI system. It specifies the major computational components, how information moves between them, what form of memory is available, how inputs and outputs are represented, and how learning changes the system’s internal parameters.

Architecture is a set of design decisions

Every architecture answers a collection of questions:

Architecture, parameters, training, and behavior

ElementMeaningAnalogy
ArchitectureThe structural design and information pathways.The design of a factory.
ParametersNumbers adjusted during training that encode learned patterns.The settings of its machines.
Training processThe procedure used to adjust parameters.How the factory is calibrated.
DataThe examples and signals from which the system learns.The materials and operating history.
BehaviorWhat the trained system actually does in use.The quality and reliability of its output.

Why architecture matters

Two models trained on similar data can behave differently because their architectures allocate attention, memory, and computation differently. One design may be excellent at long sequences but expensive to run. Another may be fast on a device but weak at multi-step reasoning. A third may understand images and language together but require enormous training resources.

Architecture is not destiny

Architecture strongly influences a model, but it does not determine performance by itself. Data quality, optimization, scale, evaluation, tool use, and post-training can matter just as much. Modern AI progress usually comes from combinations of architectural ideas and improvements elsewhere in the system.

Architecture defines the space of behaviors a model can learn efficiently; training determines where inside that space the model ends up.