AWS Certified Generative AI Developer - Professional: Logging, monitoring, CloudWatch, X-Ray

Day 24, logging, monitoring, cloudatch,

x-ray. This is the day where AWS stops

caring whether you can build a Genai

system and starts caring whether you can

operate one after launch because real

systems don't fail politely. They fail

quietly, slowly, and expensively unless

you can see what's happening.

Imagine this. A country launches an AI

assistant for national emergency

coordination. On day one, everything

works. On day 10, response time spikes.

Costs double. One region receives

incorrect instructions. Leadership asks

a simple question. What happened when,

and why? If you don't have logs,

metrics, and traces, the honest answer

is silence. That's a failure in

production and on the exam. Let's start

with the foundation. Logging answers one

question. What happened in Genai

systems? Logging must be structured and

intentional, not random print

statements, not chat transcripts,

structured facts you can query later.

For every request, you should be able to

see a request or correlation ID, which

model was used, which prompt version

ran, what retrieval parameters were

applied, which tools were called, how

long it took, and whether it failed. For

tools like Lambda, you log the tool

name, sanitized inputs, output status,

latency, and retries. This is how

audits, post incident analysis, and

debugging actually work. Just as

important is knowing what not to log.

You do not log raw PII. You do not log

secrets. You do not dump full prompts

containing sensitive data. AWS expects

privacy aware logging. Logs tell you

what happened once. Metrics tell you

whether the system is healthy over time.

Metrics answer a different question. Is

something drifting? Is something getting

worse? The core Genai metrics AWS

expects you to care about are simple.

Latency, especially P95 and time to

first token, errors, model failures,

tool failures, throttling, cost signals,

tokens per request, model usage,

embedding volume, quality signals,

fallback rate, guardrail blocks,

retries. You don't read metrics line by

line. You watch trends. This is where

Cloudatch becomes your control room.

Cloudatch gives you logs for detail,

metrics for health, alarms for early

warning, and dashboards so ops teams see

everything in one place. AWS loves

answers that mention dashboards because

dashboards mean ownership.

Now comes the piece most people miss.

X-ray logs and metrics tell you what is

wrong. X-ray tells you why. X-ray gives

you end-to-end traces. A single user

request might pass through API gateway,

a lambda orchestrator, bedrock model

invocation, open search retrieval,

multiple tool lambdas. X-ray stitches

all of that into one timeline. You can

see which step was slow, which

dependency failed, and where time was

actually spent. That's impossible with

logs alone. Let's apply this to real

failures. If the system is slow, you

check Cloudatch for P95 latency, then

open X-ray to see which segment

dominates. Is it retrieval, the model, a

tool call? Now you know where to fix. If

costs explode, you check metrics for

token usage and model breakdown. Then

logs for repeated retries, agent loops,

or cache misses. You don't guess, you

prove. If answers are wrong, you inspect

logs for retrieved documents, top K

values, guardrail blocks, fallback

rates. Observability turns I think into

I know. AWS also likes subtle Geni

specific signals. Guardrail violation

rate, fallback response rate, agent step

count, tool calls per request, cache hit

ratio, embedding recomputation rate.

Mentioning these signals shows senior

level ownership. There are classic traps

here. Logs alone are not monitoring.

Chat history is not observability.

Errors alone are not enough. Tracing is

not optional for agents. The correct

mental model is all three together.

This triangle solves the exam. Logs tell

you what happened. Metrics tell you if

it's healthy. Traces tell you why it

happened. Miss one side and you're

guessing. Here's the one sentence to

lock this day into memory. If you can't

observe it, you don't own it. That is

AWS culture in a single line. Final self

test. A multi-step agent is slow and

sometimes fails. You need to find

exactly which step caused the problem.

What do you use? AWS X-ray combined with

structured logs and Cloudatch metrics.

That's day 24 mastered.