Layer 5 — AI Infrastructure · Chapter 3

Data Infrastructure and Knowledge Systems

Book chapterLayer 5

AI capability depends heavily on the information a system can learn from and access. Data infrastructure supplies that information in forms suitable for training, evaluation, retrieval, and real-time operation.

The data supply chain

Raw data is rarely ready for AI. It may be duplicated, mislabeled, outdated, private, biased, corrupted, or legally restricted. Data infrastructure transforms it through a controlled pipeline.

  1. Acquire: collect licensed, public, internal, generated, or user-provided data.
  2. Catalog: record source, ownership, permissions, date, sensitivity, and intended use.
  3. Clean: remove corruption, duplicates, irrelevant material, and obvious quality problems.
  4. Transform: parse documents, normalize formats, segment content, and create features or embeddings.
  5. Label: add human, programmatic, weak, or model-generated annotations.
  6. Filter: enforce privacy, safety, copyright, quality, and policy rules.
  7. Version: preserve the exact dataset used for an experiment or deployment.
  8. Evaluate: test coverage, contamination, representativeness, and downstream usefulness.

Training data and runtime knowledge are different

Training changes the model’s internal parameters. Retrieval supplies information at the time of a request without retraining the model. This distinction matters because many facts change quickly, some data is private, and some knowledge must be cited or access-controlled.

Learned knowledge

Compressed into model parameters during training. Broad and fast to use, but difficult to update precisely or trace to a source.

Retrieved knowledge

Fetched from documents, databases, or services during use. Easier to update, cite, restrict, and inspect.

Retrieval-augmented generation

A retrieval system normally receives a question, searches for relevant material, ranks the results, and inserts selected evidence into the model’s context. The model then uses that evidence when producing an answer. Effective retrieval may combine keyword search, vector similarity, metadata filters, graph relationships, database queries, and reranking models.

Embeddings and vector indexes

An embedding represents an item as a numerical vector that captures aspects of meaning or similarity. A vector index allows the system to find nearby items efficiently. This is useful for semantic document search, recommendations, clustering, deduplication, and memory systems.

Vector similarity is not sufficient by itself. Production retrieval often needs:

Synthetic data

Models can generate examples, labels, simulations, critiques, or edge cases. Synthetic data can reduce cost and expand coverage, but it may also reproduce the generating model’s errors, narrow the distribution, or make evaluation circular. Strong pipelines mix generation with filtering, diversity checks, expert review, and validation against real-world outcomes.

Data governance

Governance establishes who may use data, for what purpose, for how long, and with what safeguards. It includes consent, retention, deletion, geographic restrictions, contractual rights, lineage, audit trails, and procedures for correcting or removing problematic data.

In many domain-model businesses, the durable advantage is not the model alone. It is a lawful, high-quality, continuously improving data system.