Layer 5 — AI Infrastructure · Chapter 5

Evaluation, Observability, Security, and Governance

Book chapterLayer 5

AI systems can run without crashing and still fail their users. They may produce plausible errors, expose private data, follow malicious instructions, call the wrong tool, drift after a model update, or cost far more than expected. Layer 5 therefore needs measurement and control systems that go beyond ordinary uptime monitoring.

Evaluation before deployment

Evaluation asks whether a system is good enough for its intended use. A useful evaluation program combines several forms of evidence:

Production observability

Observability helps operators understand what happened inside a real request. A trace may include model version, prompts, retrieved documents, tool calls, intermediate decisions, token use, latency, policy checks, and final outcome. Sensitive content should be minimized, redacted, access-controlled, and retained only as long as necessary.

SignalWhat it reveals
Latency and queue timeWhether users are waiting because of model computation, retrieval, tools, or capacity limits.
Token and compute usageWhere cost is accumulating and whether prompts or contexts are unnecessarily large.
Groundedness and citation qualityWhether claims are supported by retrieved or approved evidence.
Tool success and action errorsWhether agents are executing the correct operations and recovering from failure.
User corrections and abandonmentWhere the system fails to satisfy real needs despite passing laboratory tests.
Safety and policy eventsWhere misuse, prompt injection, leakage, or prohibited content appears.

AI-specific security risks

Defense in depth

No single classifier or system prompt is enough. Protection may include input isolation, least-privilege credentials, tool allowlists, schema validation, separate approval for high-impact actions, content filtering, sandboxing, rate limits, anomaly detection, and auditable logs.

Governance

Governance assigns responsibility. It defines approved use cases, risk levels, data rules, evaluation standards, release authority, incident procedures, human oversight, vendor requirements, and documentation. Good governance is not paperwork added after development; it is the mechanism that tells infrastructure what policies to enforce.

Reliable AI requires a closed loop: evaluate before release, observe after release, investigate failures, and feed the lessons back into the system.