Layer 4 — Model Architecture · Chapter 1
What Is Model Architecture?
Model architecture is the blueprint of an AI system. It specifies the major computational components, how information moves between them, what form of memory is available, how inputs and outputs are represented, and how learning changes the system’s internal parameters.
Architecture is a set of design decisions
Every architecture answers a collection of questions:
- How is text, sound, an image, a video frame, or a sensor reading converted into numbers?
- Which pieces of information are allowed to interact?
- How does the model preserve order, position, time, or spatial structure?
- How much computation is used for each input?
- Can the system remember information beyond the current input?
- How are several specialized components coordinated?
- How does the model produce a prediction, sequence, plan, or action?
Architecture, parameters, training, and behavior
| Element | Meaning | Analogy |
|---|---|---|
| Architecture | The structural design and information pathways. | The design of a factory. |
| Parameters | Numbers adjusted during training that encode learned patterns. | The settings of its machines. |
| Training process | The procedure used to adjust parameters. | How the factory is calibrated. |
| Data | The examples and signals from which the system learns. | The materials and operating history. |
| Behavior | What the trained system actually does in use. | The quality and reliability of its output. |
Why architecture matters
Two models trained on similar data can behave differently because their architectures allocate attention, memory, and computation differently. One design may be excellent at long sequences but expensive to run. Another may be fast on a device but weak at multi-step reasoning. A third may understand images and language together but require enormous training resources.
Architecture is not destiny
Architecture strongly influences a model, but it does not determine performance by itself. Data quality, optimization, scale, evaluation, tool use, and post-training can matter just as much. Modern AI progress usually comes from combinations of architectural ideas and improvements elsewhere in the system.
Architecture defines the space of behaviors a model can learn efficiently; training determines where inside that space the model ends up.