name: dd-debugger description: Live Debugger - inspect runtime argument/variable values in production by placing log probes on methods. Use when asked what values a function receives, what parameters look like at runtime, or to capture live data from running services without redeploying. metadata: version: "1.1.0" author: datadog-labs repository: https://github.com/datadog-labs/agent-skills tags: datadog,debugger,live-debugger,probes,dd-debugger globs: "" alwaysApply: "false"

Datadog Live Debugger

Place log probes on running services without redeploying. Create probes with custom templates and conditions, and stream captured events in real time.

Prerequisites

pup must be installed:

brew tap datadog-labs/pack && brew install pup

Authentication

Authenticate via OAuth2 (recommended) or API keys:

# OAuth2 (recommended)
pup auth login

# Or use API keys
export DD_API_KEY="key" DD_APP_KEY="key" DD_SITE="datadoghq.com"

Typical Workflow

Prefer --capture expressions over full snapshots. Capture expressions are lighter-weight, faster, and return exactly the data you need.

1. If you don't know the service, ask the user before proceeding.
2. Verify the service using pup debugger context <service> to list environments with active instances. If multiple environments exist, ask the user which one to target before proceeding.
3. Find a method using the dd-symdb skill (pup symdb search --view probe-locations)
4. Place a probe with capture expressions for the values you need
5. Watch events with --fields for compact output
6. Delete the probe when done

# 0. List environments (if multiple, ask user which to use)
pup debugger context my-service --fields service,language,envs

# 1. Create a probe with capture expressions (recommended)
#    Use --fields id to get just the probe ID back
pup debugger probes create \
  --service my-service \
  --env production \
  --probe-location "com.example.MyController:handleRequest" \
  --capture "request.id" \
  --capture "request.headers" \
  --ttl 1h \
  --fields id

# 2. Stream events with compact output
pup debugger probes watch <PROBE_ID> --timeout 60 --limit 10 \
  --fields "message,captures,timestamp"

# 3. Clean up
pup debugger probes delete <PROBE_ID>

Service Context

Always run this before creating probes. It returns JSON by default, showing environments with active instances, tracer versions, and supported probe features. If the service runs in multiple environments, ask the user which one to target — don't guess.

# Full JSON output (default, avoid)
pup debugger context my-service

# Compact: just the fields you need (preferably use this to avoid context bloat)
pup debugger context my-service --fields service,language,envs

# Filter to a specific environment
pup debugger context my-service --env production --fields service,envs,repo

Probe Management

List Probes

# All probes
pup debugger probes list

# Filter by service
pup debugger probes list --service my-service

Get Probe Details

pup debugger probes get <PROBE_ID>

Create a Log Probe

pup debugger probes create \
  --service my-service \
  --env staging \
  --probe-location "com.example.MyClass:myMethod" \
  --capture "user.name" \
  --capture "order.items[0].price"

Options:

Capture Expressions (Recommended)

Always prefer capture expressions over full snapshots. They are lighter-weight and return exactly the data you need.

# Capture specific fields using dot notation
--capture "user.name"
--capture "request.headers"

# Access array elements
--capture "orders[0].total"
--capture "items[len(items)].name"

# Chain member access
--capture "response.body.data.id"

Multiple --capture flags can be combined. Each expression is independently evaluated.

# Capture multiple specific values
pup debugger probes create \
  --service my-service --env prod \
  --probe-location "OrderService:processOrder" \
  --capture "order.id" \
  --capture "order.items[0].price" \
  --capture "customer.email"

Tip: Start with --depth 1 (the default) to see field names and types, then increase depth to drill into interesting subtrees. Use --depth 5 for deeply nested objects.

Full Snapshot (use sparingly)

Use bare --capture (no value) only when you don't know which fields to inspect:

# Full snapshot — captures all arguments, locals, return value
pup debugger probes create \
  --service my-service --env staging \
  --probe-location "com.example.MyClass:myMethod" \
  --capture

You can combine snapshot with capture expressions:

# Full snapshot + specific expressions
--capture --capture "user.name"

Templates

Templates can be used alongside capture expressions for custom log messages:

# Template with capture expressions
pup debugger probes create \
  --service my-service --env prod \
  --probe-location "MyClass:myMethod" \
  --capture "user.id" \
  --template "Processing request for user={user.id}, took {@duration}ms"

Template syntax uses {variable} placeholders:

--template "Processing order={orderId} for user={userId}"
--template "handleRequest took {@duration}ms"

Conditions filter when the probe fires:

--condition "status == 'error'"
--condition "@duration > 100"

Delete a Probe

pup debugger probes delete <PROBE_ID>

Watch Probe Events

Stream log events and status errors from a probe in real time.

pup debugger probes watch <PROBE_ID>

Options:

Behavior:

• Without --fields: outputs the trimmed debugger payload (snapshot, probe info, stack) — not the full log envelope
• With --fields: outputs only the requested fields as compact JSON per event
• Polls for new log events and probe status errors every 1s
• Default --from is now — only shows new events going forward
• Use --from 5m or --from 1h to include recent historical events
• Probe status errors (e.g., instrumentation failures) go to stderr
• Exit code 0 if events received, 1 if timed out with no events
• Use --wait <seconds> to retry probe existence check (handles create → watch pipeline); default is 0 (fail immediately if not found)

Using --fields for compact output

For most agent use cases, --fields gives you exactly what you need without jq:

# Message + captures + timestamp (most common)
pup debugger probes watch <ID> --fields "message,captures,timestamp" --limit 5

# Template message only (one line per event)
pup debugger probes watch <ID> --fields "message" --limit 10

# Just captures (expression values or snapshot data)
pup debugger probes watch <ID> --fields "captures" --limit 1

Extracting fields with jq (reference)

When using the default output (no --fields), the debugger payload is at the top level. Common jq patterns:

# Extract a specific captured expression
pup debugger probes watch <ID> --limit 1 \
  | jq '.snapshot.captures.return.captureExpressions'

# Get captured argument values
pup debugger probes watch <ID> --limit 1 \
  | jq '.snapshot.captures.return.arguments'

Default output structure (trimmed to debugger payload):

.snapshot.captures.return.arguments          → method arguments
.snapshot.captures.return.locals             → local variables
.snapshot.captures.return.captureExpressions → capture expression values
.snapshot.probe.id                           → probe ID
.snapshot.probe.location                     → source location

Supported Languages

Failure Handling

References

• Live Debugger Docs
• Log Probe Templates