Claude Code Assistant Guidelines

Go Code Style

• Follow the standard Go code style and conventions. Use gofmt for formatting and adhere to idiomatic Go practices.
• Follow best practices from the Effective Go guide:

Naming Conventions

• Use MixedCaps or mixedCaps rather than underscores for multi-word names
• Package names should be short, lowercase, single-word names
• Getters don't use "Get" prefix (use obj.Name() not obj.GetName())
• Interface names use "-er" suffix for single-method interfaces (e.g., Reader, Writer)

Formatting

• Use gofmt for consistent formatting (tabs for indentation, spaces for alignment)
• Line length: no strict limit, but keep lines reasonable
• Group related declarations together

Error Handling

• Return errors as the last return value
• Check errors immediately after the call
• Provide context with fmt.Errorf and error wrapping

Logging

• Use ctrl.Log for structured logging
• Keep log fields consistent and meaningful
• Avoid logging sensitive data

Documentation

• Every exported name should have a doc comment
• Start comments with the name being described
• Use complete sentences

Concurrency

• Share memory by communicating; don't communicate by sharing memory
• Use channels to orchestrate goroutines
• Always handle goroutine cleanup and cancellation properly

Project Structure

• Keep packages focused and cohesive
• Avoid circular dependencies
• Place tests in *_test.go files

Documentation

Prefer placing documentation in the docs/ directory.

There are 3 main types of documentation targeting different audiences:

1. Developer Documentation - For contributors and maintainers of this project

   • Architecture decisions
   • Development setup and workflow
   • Contributing guidelines
   • usually in the docs/developer-guide/ subdirectory

2. Administrator Documentation - For operators deploying and managing the autoscaler controller

   • Installation and configuration
   • Deployment guidelines
   • Monitoring and troubleshooting
   • usually located under the docs/user-guide/ directory (for example, in an admin-focused subdirectory)

3. End-User Documentation - For application developers creating applications that use the autoscaler

   • Usage guides and examples
   • API reference
   • Best practices and common patterns
   • usually located under the docs/user-guide/ directory (for example, in an end-user-focused subdirectory)

E2E Testing

• use make targets for running e2e tests (e.g., make test-e2e-smoke or make test-e2e-full) and document the process in docs/developer-guide/testing.md
• use make test for unit tests
• Never use images from docker.io in e2e tests. All container images must use fully-qualified registry paths (e.g., registry.k8s.io/, quay.io/, or a private registry). Do not rely on Docker Hub as a default registry.

CLI Tools

llm-d Inference Scheduler EPP CLI Reference

This section documents the command-line flags and environment variables supported by the llm-d inference scheduler EPP (Endpoint Picker). The EPP inherits its CLI from gateway-api-inference-extension.

Main Branch (Latest)

Uses gateway-api-inference-extension at commit fd30cb97714a (post-v1.3.0).

Command-Line Flags

Environment Variables

v0.5.0

Uses gateway-api-inference-extension v1.3.0.

Command-Line Flags

Environment Variables

Key Differences Between Main and v0.5.0

1. Metric Flags: In main branch, --total-queued-requests-metric, --total-running-requests-metric, --kv-cache-usage-percentage-metric, --lora-info-metric, and --cache-info-metric are deprecated and will error if explicitly set. In v0.5.0, these flags are functional.

2. Configuration: Main branch encourages using EndpointPickerConfig with engineConfigs for metrics configuration instead of CLI flags.

llm-d Inference Simulator CLI Reference

This section documents the command-line flags and environment variables supported by the llm-d inference simulator (llm-d-inference-sim). The simulator is a vLLM server simulator supporting OpenAI API endpoints.

Main Branch (Latest)

Command-Line Flags

Environment Variables

v0.5.0

Command-Line Flags

Environment Variables

Key Differences Between Main and v0.5.0

1. Duration Parameters: In main branch, latency-related parameters (--time-to-first-token, --inter-token-latency, etc.) use Go duration strings (e.g., "100ms", "1.5s"). In v0.5.0, these are integers representing milliseconds.

2. New Flags in Main: --max-waiting-queue-length, --global-cache-hit-threshold, --data-parallel-rank, --ssl-certfile, --ssl-keyfile, --self-signed-certs, --dataset-path, --dataset-url, --dataset-in-memory, --enable-sleep-mode, --enable-request-id-headers, --latency-calculator.

3. Environment Variables: Main branch adds POD_IP and VLLM_SERVER_DEV_MODE.