Skip to main content

Deployments

When specifying an operator preference for deployment planning, use English, such as prefer prefix cache, more GPU for performance, or use fewer GPU resources. The deterministic planner matches English preference keywords; Chinese keyword aliases are no longer supported.

Python domain contracts​

The models in llm_d_lens.deploy.contracts and the llm_d_lens.deploy.service.DeploymentService protocol define the internal Configuration-to-Deploy boundary. They are not additional HTTP endpoints. Configuration owns validation/generation of immutable configuration artifacts; Deploy owns provider execution and lifecycle evidence. Evaluation owns workload validation and benchmark execution; Simulation does not own deployment lifecycle.

Commands and immutable artifacts​

ContractFields and constraints
VersionedPayloadNonempty schema_version; object value defaults to {}. The envelope does not prescribe producer-owned keys.
ConfigurationArtifactartifact_id defaults to a UUID; nonempty schema_version, kind, provider_ref, checksum; media_type defaults to application/yaml; at least one of content or content_ref is required.
DeploymentCreateRequestUUID request_id; input_schema_version="v1"; nonempty configuration_artifacts; deployment_policy defaults to a non-null VersionedPayload(schema_version="v1", value={}); optional cluster_snapshot_ref/cluster_snapshot; object provenance.
DeploymentKillRequestUUID request_id; input_schema_version="v1"; required nonempty execution_id; optional expected_namespace/reason; object provenance.
DeploymentArtifactUUID artifact_id; nonempty artifact_hash and configuration_artifact_ids; optional source_ref, manifest_ref, manifest_checksum, values_checksum; required rendered_payload; created_at.

provider_ref selects the deployment provider; it does not transfer configuration ownership to Deploy. The current GuideAdapterDeploymentService is narrower than the generic DTO: each create call requires exactly one artifact containing an inline JSON object with media_type="application/json". The run planner splits the public run request into cases; do not pass a generic YAML/content-reference artifact directly to this adapter and assume it supports it.

create(request, *, model_token=None, on_initial_execution=None) returns DeploymentExecution. Runtime binding does not change configuration ownership. kill(request, execution) verifies matching execution IDs, the expected namespace when supplied, and provider state managed by that service before cleanup. It does not accept arbitrary resource selectors.

Execution, observation and evidence​

DeploymentExecution carries execution_id, request_id, status, artifact, configuration_artifacts, optional endpoint/namespace/resource_snapshot, provenance, evidence_refs, optional versioned diagnostics and timestamps. Additional current fields are forwarded_endpoint, monitoring_setup (setup audit, not live monitoring status), data_plane, and editable descriptive metadata (display_name, description), separate from origin provenance.

The complete run/case/execution state enums are listed below. A ready execution must contain DeploymentEndpoint: url, protocol="http" and optional service_ref, model_ref, baseline_url. Endpoint metadata must not carry credentials, kubeconfig or authorization values.

get(execution_id) returns DeploymentExecution, not another status DTO. The concrete service looks up its in-memory execution state and raises KeyError for unknown IDs; it is not itself a database query. The worker/DAO boundary owns durable records and restores execution context when needed. The protocol also exposes restore_execution, refresh_diagnostics, stop and clean; use the authenticated REST operations below for external clients.

get_logs(DeploymentLogRequest(...)) returns a DeploymentLogPage, separate from execution metadata:

  • Request: execution_id, optional nonnegative integer-string cursor, limit from 1 to 1000 (default 100), follow=False, and sources=[].
  • Current sources are workload and events, not arbitrary Kubernetes selectors. Entries contain timestamp, source, message and optional severity.
  • Response: execution_id, entries, next_cursor, complete, evidence_refs. follow=True sets complete=False; this method still returns one page, not a streaming iterator or a promise to wait for new logs.
  • Archived diagnostics use ExecutionLogArchive and deployment-log://<execution_id>/archive references. Preserve evidence separately from database records; see storage settings. Retained snapshots are partial diagnostics, not a complete cluster log history.

Consumer projections​

These are pure DTO helpers, not lifecycle calls or REST permission checks. benchmark_input() requires ready plus an endpoint and otherwise raises ValueError. It projects execution ID, artifact, endpoint, namespace and provenance. simulation_input() accepts any execution state and additionally projects deployment status and the optional resource snapshot/endpoint. It preserves execution evidence rather than substituting current mutable cluster state. The helpers exist, but current Evaluation/Simulation routes are not direct callers; do not describe these projections as their active request API.

from llm_d_lens.deploy.contracts import (
    BenchmarkDeploymentInput,
    DeploymentExecution,
    SimulationDeploymentInput,
)


def benchmark_projection(execution: DeploymentExecution) -> BenchmarkDeploymentInput:
    return execution.benchmark_input()


def simulation_projection(execution: DeploymentExecution) -> SimulationDeploymentInput:
    return execution.simulation_input()

Hardware rendering contract​

Deployment capability metadata describes provider behavior; its variants array is empty because variants belong to the selected source catalog. Rendering and Evaluation use the resolved hardware profile for images, accelerator resources and scheduling. Legacy configurations can resolve hardware from their rendered DRA or extended-resource requests; conflicting hardware evidence is rejected.

Python callers can inspect the same source entry points before rendering:

from pathlib import Path
from llm_d_lens.deploy.providers.hardware_profile import guide_overlays

variants = guide_overlays(
    Path("/path/to/llm-d"), "optimized-baseline", accelerator="nvidia",
)
print(sorted(variants))

Backend API​

Prepare Guide source (Node)​

POST /api/guide-planning/prepare accepts guide, accelerator, modelServer, guideVariant and the selected clusterId. It prepares the official source without deploying and returns its commit plus manifest-derived controls:

{"ready": true, "commit": "selected-source-commit", "capabilities": {"cacheCpuGiB": false, "rdmaNicCount": true}}

The booleans describe supported CPU-cache mutation and a matching profile NIC claim. They do not depend on variant names. This endpoint is owned by apps/server/src/routes/guidePlanning.ts; there is no Python callable for this Node operation. Python deployment providers use guide_overlays (above) and llm_d_lens.hardware.resolver.deployment_mode(profile, access_mode) for the corresponding source and hardware contracts.

Agentic create and refinement streams use llm_d_lens.agentic.streaming.planning_stream_response. The routes retain principal resolution, expected-error classification, and their existing generic error messages. Streams emit progress followed by complete or error; disconnect cleanup cancels and awaits unfinished planning. Gateway operation/log streams remain separate.

Recalculating an Agentic plan does not require an AI provider. Without one, AgenticDeploymentService.refine() refreshes planning facts and uses deterministic candidate generation and scoring with the new preference. If an available AI scorer fails, recalculation keeps the validated candidates and returns deterministically scored alternatives instead.

Agentic candidate planning and approval revalidation use the deployment request's max_model_len and gpu_memory_utilization for memory and KV-cache checks, even when the supplied planning_facts contains different values. Deterministic and AI-generated candidates use the same utilization as the deployed configuration. PD deployments apply that utilization to both prefill and decode vLLM roles. The estimate uses GPU capacity and model overhead, not vLLM's measured free KV cache at startup.

Deployments live under the FastAPI router in llm_d_lens/deploy/router.py with the /api/v1/deployments prefix.

Use the installation instructions and storage environment settings for the current monorepo and Lens storage roots. The old .llm-d-bench/deploy and host-specific /root/.llm_d_lens/run_store paths are not the current storage contract.

Execution discovery is read-only and uses execution_id as its cross-module reference; listing does not start port-forwards. Use the execution endpoint action to obtain a host-reachable URL rather than assuming a persisted in-cluster endpoint or old tunnel is reachable.

RecordLifecycle states
Runqueued, running, succeeded, partially_succeeded, failed, cleaned, cancelling, cancelled
Casequeued, rendering, deploying, ready, failed, stopped, cancelling, cancelled, cleaned, cleaned_up
Executiondraft, validated, rendered, deploying, ready, failed, rolling_back, rolled_back, cleaned, cleaned_up

A stopped case has a rolled_back execution; these are different state enums. Benchmarking requires a ready execution with an endpoint and an active cluster session. Python worker examples below run inside a configured Lens backend; external clients should use authenticated HTTP so route permissions and resource scope checks apply.

Optimized and precise routing capability metadata reuse shared_prefix_routing_workload() in llm_d_lens.deploy.capabilities. Each scenario and recommended workload receives independent nested values; provider-specific baselines and goals are unchanged. Precise-provider calibration and model discovery share lazy modelserver invocation traversal while retaining their respective last-value and first-model selection rules.

Agentic candidate generation requires its first tool batch to contain exactly get_cluster_overview and search_candidates. An invalid first batch is not executed; the generator gives the AI one corrective retry before using built-in candidates. If the retry also violates the tool contract, the fallback is reported as an invalid generator response rather than a provider connectivity failure. A tool-call limit failure retains the round, observed call count, and limit in generator_fallback_detail on the run and generator.fallback_detail in its planning snapshot; generator_fallback_reason remains the stable category.

Agentic planning estimates missing exact AIC predictions separately for each validated baseline-vllm, optimized-baseline, and pd-disaggregation candidate using its workload and GPU topology. The candidate's estimate is supplied to AI scoring as aic_estimate; an unavailable estimate remains unknown, not a performance or SLO guarantee. Agentic estimates do not pass the original search TTFT/TPOT targets; they verify the fixed topology and workload without an SLO filter. Generator fallback also estimates the top three built-in candidates, using the resolved workload when the failed generator left no tool trace. Unsupported or unavailable topologies remain unknown. Relaxed AIC search accepts complete configurations using no more than the available GPU budget. Its predictions do not establish compliance with the original SLO. Agentic preserves the relaxed anchor as a validated candidate even when the generator suggests only a smaller topology, then estimates its exact topology and workload. Candidates with known estimates that miss the original latency targets are marked not_satisfied; among comparable misses, smaller relative target violations rank ahead of GPU savings. Unknown estimates are not treated as failures. When an external planner scores multiple candidates, its selection rationale explains why each lower-scored returned candidate ranks below the top choice using the supplied planning facts. Only the returned top three are scored; no reason is inferred for candidates outside that set.

Planning-fact collection matches discovered accelerator models against registered hardware profiles to select one AIC system. Unknown, ambiguous, or mixed systems leave AIC evidence unavailable.

AIC support checks and estimates use the SDK release pinned in pyproject.toml (currently 0.12.0), including its bundled support matrix and performance data. An unsupported P/D result describes that installed matrix, not a hardware-level prohibition. The 0.12.0 matrix passes Qwen/Qwen3-0.6B on B60/vLLM for P/D; candidate availability still depends on the requested workload and GPU budget. After changing the dependency pin, reinstall project dependencies and restart the backend so support checks and searches use the same SDK release.

Estimate an AIC topology​

POST /api/v1/aic/estimate

Send model_name, scenario (inference_scheduling or pd_disaggregation), gpu_count, mean_input_tokens, mean_output_tokens, the AIC system/backend/database and the topology fields (tp and replicas, or prefill/decode TP and replicas). Optional ttft_target_ms and tpot_target_ms retain the constrained experiment path; omit both to estimate the fixed topology without an SLO filter.

{
  "model_name": "Qwen/Qwen3-0.6B",
  "scenario": "inference_scheduling",
  "gpu_count": 1,
  "mean_input_tokens": 1024,
  "mean_output_tokens": 256,
  "aic_system_name": "b60",
  "aic_backend_name": "vllm",
  "aic_database_mode": "SILICON",
  "tp": 1,
  "replicas": 1
}

The response includes mode, topology fields, ttft_ms, tpot_ms and throughput_tokens_per_sec when measurable. Estimates do not prove SLO compliance.

List deployment node labels​

List the distinct key/value labels on nodes of a cluster the caller can access. Advanced placement uses these values to populate its optional node-label selector.

GET /api/v1/deployments/node-labels?cluster_id=cluster-a

Query: cluster_id (required cluster ID). Response:

{"items": [{"key": "pool", "value": "accelerator"}]}

Start a deployment run​

Start a deployment run from one or more already-rendered deployable configurations.

For Router 0.11 endpoint-picker images, planning and installation adapt legacy P/D plugin settings: disagg-headers-handler is handled by disagg-profile-handler, and parameters.deciderPluginName becomes parameters.deciders.prefill. This also applies when retrying saved bundles. Source assets and their checksums remain unchanged; deployment evidence records the derived values actually installed. Conflicting old/new deciders and custom legacy header parameters are rejected. Other Router versions and custom image repositories retain their own settings.

The same pure Python transformation is shared by configuration validation and llm_d_bench.deploy.providers.deployment_bundle.install_deployment_bundle:

import yaml
from llm_d_bench.configuration.router_compatibility import compatible_router_values

values = yaml.safe_load(saved_router_values_yaml)
effective_values = compatible_router_values(values, chart_version="v0.11.0")

POST /api/v1/deployments/runs

Request body: DeploymentRunCreateRequest

  • configurations (DeployableConfiguration[], required): rendered deployment payloads to execute.
  • failure_policy ("stop" | "continue", default "continue"): whether later cases should continue after a failure.
  • provenance (object, optional): caller-owned origin metadata.
  • runtime_binding (RuntimeBinding, optional): environment overrides, cluster session binding, and placement ({"replicas":"prefer-same-node"|"same-node"|"balanced"|"spread"|"none","nodeLabel"?:{"key":string,"value":string}}). Model-server Deployments in the same rendered manifest share a placement group, including PD Prefill and Decode; router Pods are excluded. nodeLabel is accepted only with same-node and must exist on a node in the cluster identified by provenance.cluster_server_id. LeaderWorkerSet cannot use balanced or spread because its leader/worker Pods are not separate replicas. For optimized-baseline, placement is applied to the generated Kustomize overlay before deployment. Shared creation preflight requires a ready model and revision in registered Model Cache storage. Ordinary provider rendering sets Hugging Face offline flags; exact published manifests bypass that rendering and are not covered by the same final-manifest enforcement. Missing snapshots must be downloaded through Model Cache before deployment.
{
  "configurations": [
    {
      "schema_version": "deployable-configuration.v1",
      "type": "baseline",
      "format": "helm",
      "provider_ref": "optimized-baseline",
      "checksum": "sha256:0f4f7e4b",
      "content": {
        "model": {"name": "Qwen/Qwen3-8B"},
        "decode": {
          "replicaCount": 1,
          "tensorParallelSize": 1,
          "maxModelLen": 4096
        },
        "runtime": {
          "image": "ghcr.io/llm-d/llm-d-xpu:v0.9.0",
          "imageMode": "pull-existing"
        }
      },
      "provenance": {
        "deployment_name": "qwen3-8b-smoke",
        "description": "Deploy one optimized baseline for evaluation"
      }
    }
  ],
  "failure_policy": "continue",
  "provenance": {
    "cluster_session_id": "123e4567-e89b-12d3-a456-426614174000",
    "deployment_source": {"kind": "manual"}
  },
  "runtime_binding": {
    "schema_version": "runtime-binding.v1",
    "environment": {
      "HTTP_PROXY": "http://proxy.example:911"
    },
    "cluster_session_id": "123e4567-e89b-12d3-a456-426614174000",
    "placement": {"replicas": "same-node", "nodeLabel": {"key": "pool", "value": "accelerator"}}
  }
}

Response: DeploymentRun

{
  "id": "8c52df8b-5d31-4a36-8c01-3952abf2bf89",
  "status": "queued",
  "failure_policy": "continue",
  "provenance": {
    "cluster_session_id": "123e4567-e89b-12d3-a456-426614174000",
    "deployment_source": {"kind": "manual"}
  },
  "created_at": "2026-09-23T09:18:24.287000+00:00",
  "started_at": null,
  "finished_at": null,
  "source_configurations": [
    {
      "schema_version": "deployable-configuration.v1",
      "type": "baseline",
      "format": "helm",
      "provider_ref": "optimized-baseline",
      "checksum": "sha256:0f4f7e4b",
      "content": {
        "model": {"name": "Qwen/Qwen3-8B"},
        "decode": {"replicaCount": 1, "tensorParallelSize": 1, "maxModelLen": 4096},
        "runtime": {"image": "ghcr.io/llm-d/llm-d-xpu:v0.9.0", "imageMode": "pull-existing"}
      },
      "provenance": {"deployment_name": "qwen3-8b-smoke", "description": "Deploy one optimized baseline for evaluation"}
    }
  ],
  "cases": [
    {
      "id": "1f31e4de-5452-4b6c-a38f-d404ed5eff10",
      "run_id": "8c52df8b-5d31-4a36-8c01-3952abf2bf89",
      "ordinal": 0,
      "source_configuration_ordinal": 0,
      "provider_ref": "optimized-baseline",
      "component": "decode",
      "depends_on": [],
      "status": "queued",
      "attempt": 1,
      "execution_id": null,
      "failure": null,
      "create_request": {
        "request_id": "58b85f84-c8ef-47e1-8cb9-61831eb9d310",
        "input_schema_version": "v1",
        "configuration_artifacts": [
          {
            "artifact_id": "524a4178-7e9f-49ea-bccd-8ddb163ae2fd",
            "schema_version": "deployable-configuration.v1",
            "kind": "configuration",
            "provider_ref": "optimized-baseline",
            "media_type": "application/yaml",
            "checksum": "sha256:0f4f7e4b",
            "content": "{\"model\":{\"name\":\"Qwen/Qwen3-8B\"}}"
          }
        ],
        "deployment_policy": {"schema_version": "v1", "value": {"deployment_name": "qwen3-8b-smoke"}},
        "cluster_snapshot_ref": null,
        "cluster_snapshot": null,
        "provenance": {}
      }
    }
  ]
}

Start a standard vLLM deployment run​

Start a Standard Kubernetes Service deployment run using Lens's allowlisted vLLM contract. The generated namespace uses the backend's configured deployment prefix (for example, llmd-baseline-vllm-...). Standard vLLM deployments use the baseline adapter for rollout readiness and cleanup; they do not require a published Guide manifest's readinessDeployments contract. Only registered Model Cache storage is accepted. It requires a ready entry for the selected model and serves it offline using the entry's downloaded revision; a conflicting custom vLLM revision is rejected. Requests using local-cache or direct pvc storage fail validation. Shared creation preflight also revalidates model/revision readiness and local-disk node synchronization through llm_d_lens.deploy.providers.storage_mount.require_cached_model. Ordinary provider rendering mounts the registered cache read-only for serving; cacheOffline=false cannot disable its offline flags. Exact published manifests are not covered by those rendering guarantees. Missing or incomplete models must be downloaded or synchronized through Model Cache. The generated run defaults to preferring replicas on one node without making it mandatory. placement_replicas also supports required co-location, balanced distribution, or required separation across model-server Pods in the same rendered manifest. For required co-location, placement_node_label can restrict the Pods to a selected cluster node by its kubernetes.io/hostname label; the label is checked against that cluster before deployment. Tensor-parallel accelerator claims are requested by each model-server Pod, keeping its TP devices in that Pod. Agentic planning inherits the same fields and applies the selected preset on approval.

POST /api/v1/deployments/standard-vllm-runs

Request body: StandardKubernetesServiceRequest

  • model (string, required)
  • deployment_name (string, required)
  • description (string, optional)
  • cluster_session_id (string, required)
  • replicas (integer, required)
  • tensor_parallel_size (integer, required)
  • placement_replicas ("prefer-same-node" | "same-node" | "balanced" | "spread", default "prefer-same-node")
  • placement_node_label ({"key":"kubernetes.io/hostname","value":"node-name"} or null, optional; requires placement_replicas: "same-node")
  • storage_type ("model-cache", default "model-cache")
  • storage_volume_id (string, required, nonempty registered Model Cache volume ID)
  • model_cache_path, pvc_name (legacy fields, no longer used for deployment storage)
  • max_model_len, gpu_memory_utilization, max_num_seqs, max_num_batched_tokens
  • enable_prefix_caching, enforce_eager, vllm_arguments, use_cluster_proxy, http_proxy, https_proxy, no_proxy
{
  "model": "Qwen/Qwen3-8B",
  "deployment_name": "qwen3-standard",
  "description": "Standard baseline for smoke tests",
  "cluster_session_id": "123e4567-e89b-12d3-a456-426614174000",
  "replicas": 1,
  "tensor_parallel_size": 1,
  "storage_type": "model-cache",
  "storage_volume_id": "storage-model-cache",
  "max_model_len": 8192,
  "gpu_memory_utilization": 0.9,
  "max_num_seqs": 256,
  "max_num_batched_tokens": 8192,
  "enable_prefix_caching": true,
  "enforce_eager": false,
  "vllm_arguments": [
    {"name": "max-model-len", "value": "8192"}
  ],
  "use_cluster_proxy": true,
  "http_proxy": "",
  "https_proxy": "",
  "no_proxy": ""
}

Response: DeploymentRun

{
  "id": "73606e29-a348-4920-bf8c-3be5e038a8fe",
  "status": "queued",
  "failure_policy": "continue",
  "provenance": {
    "deployment_source": {"kind": "standard-kubernetes-service"},
    "cluster_session_id": "123e4567-e89b-12d3-a456-426614174000",
    "deployment_name": "qwen3-standard",
    "description": "Standard baseline for smoke tests",
    "model_market": {
      "deployment_name": "qwen3-standard",
      "description": "Standard baseline for smoke tests"
    }
  },
  "created_at": "2026-09-23T09:18:24.287000+00:00",
  "source_configurations": [
    {
      "schema_version": "deployable-configuration.v1",
      "type": "baseline",
      "format": "helm",
      "provider_ref": "baseline-vllm",
      "checksum": "sha256:9eb2d61d",
      "content": {
        "model": {"name": "Qwen/Qwen3-8B"},
        "decode": {"replicaCount": 1, "tensorParallelSize": 1, "maxModelLen": 8192},
        "runtime": {
          "image": "ghcr.io/llm-d/llm-d-xpu:v0.9.0",
          "imageMode": "pull-existing",
          "modelSource": "auto-cache",
          "storageVolumeId": "storage-model-cache",
          "cacheOffline": true
        },
        "customParameters": [
          {"target": "decode", "kind": "argument", "name": "gpu-memory-utilization", "value": "0.9"},
          {"target": "decode", "kind": "argument", "name": "enable-prefix-caching", "value": "true"}
        ]
      },
      "provenance": {
        "source": "standard-kubernetes-service.v1",
        "deployment_name": "qwen3-standard",
        "description": "Standard baseline for smoke tests",
        "storage_type": "model-cache"
      }
    }
  ],
  "cases": []
}

List deployment runs​

List deployment runs Lens has recorded.

GET /api/v1/deployments is the collection-root alias with the same query parameter and response. Results are returned newest first.

GET /api/v1/deployments/runs

Request: no body.

  • Query parameter: query (string, optional) searches by run id, provider ref, deployment name, or namespace.

Response: array of DeploymentRun records.

[
  {
    "id": "8c52df8b-5d31-4a36-8c01-3952abf2bf89",
    "status": "running",
    "failure_policy": "continue",
    "provenance": {
      "cluster_server_id": "cluster-1",
      "cluster_session_id": "123e4567-e89b-12d3-a456-426614174000"
    },
    "created_at": "2026-09-23T09:18:24.287000+00:00",
    "started_at": "2026-09-23T09:18:40.001000+00:00",
    "finished_at": null,
    "source_configurations": [],
    "cases": []
  }
]

Get a deployment run​

Get the full detail of one deployment run, including all cases and their current status.

GET /api/v1/deployments/runs/{run_id}

Request: no body.

  • Path parameter: run_id (string, required)

Response: DeploymentRun

{
  "id": "8c52df8b-5d31-4a36-8c01-3952abf2bf89",
  "status": "running",
  "failure_policy": "continue",
  "provenance": {
    "cluster_server_id": "cluster-1",
    "cluster_session_id": "123e4567-e89b-12d3-a456-426614174000"
  },
  "created_at": "2026-09-23T09:18:24.287000+00:00",
  "started_at": "2026-09-23T09:18:40.001000+00:00",
  "finished_at": null,
  "source_configurations": [
    {
      "schema_version": "deployable-configuration.v1",
      "type": "baseline",
      "format": "helm",
      "provider_ref": "baseline-vllm",
      "checksum": "sha256:9eb2d61d",
      "content": {"model": {"name": "Qwen/Qwen3-8B"}},
      "provenance": {"deployment_name": "qwen3-standard"}
    }
  ],
  "cases": [
    {
      "id": "1f31e4de-5452-4b6c-a38f-d404ed5eff10",
      "run_id": "8c52df8b-5d31-4a36-8c01-3952abf2bf89",
      "ordinal": 0,
      "source_configuration_ordinal": 0,
      "provider_ref": "baseline-vllm",
      "component": "decode",
      "depends_on": [],
      "status": "deploying",
      "attempt": 1,
      "execution_id": "execution-123",
      "failure": null,
      "create_request": {
        "request_id": "58b85f84-c8ef-47e1-8cb9-61831eb9d310",
        "input_schema_version": "v1",
        "configuration_artifacts": [],
        "deployment_policy": {"schema_version": "v1", "value": {"deployment_name": "qwen3-standard"}},
        "cluster_snapshot_ref": null,
        "cluster_snapshot": null,
        "provenance": {}
      }
    }
  ]
}

Cancel a deployment run​

Request cancellation of an active deployment run.

POST /api/v1/deployments/runs/{run_id}/cancel

Request: no body.

  • Path parameter: run_id (string, required)

Response:

{
  "id": "8c52df8b-5d31-4a36-8c01-3952abf2bf89",
  "status": "cancelling"
}

Rebind a deployment run's cluster session​

Rebind a deployment run to a different active cluster session.

POST /api/v1/deployments/runs/{run_id}/cluster-session

Request body: RebindClusterSessionRequest

  • Path parameter: run_id (string, required)
  • cluster_session_id (string, required)
{
  "cluster_session_id": "123e4567-e89b-12d3-a456-426614174111"
}

Response: HTTP 204 No Content. No JSON body is returned.

List deployment executions​

List deployment executions without creating new port-forwards.

This route reads stored records; it does not query the cluster itself. A background reconciler keeps those records current, re-checking every run that is running, succeeded, partially succeeded or failed (a failed rollout whose Pods later recover is promoted back to ready, and one whose namespace is gone becomes cleaned). Only cleaned and cancelled runs are terminal. Call POST /api/v1/deployments/runs/{run_id}/cases/{case_id}/refresh when you need an immediate re-check instead of waiting for that cycle.

GET /api/v1/deployments/executions

Request: no body.

  • Query parameters: status (DeploymentStatus, optional), statuses (DeploymentStatus[], optional), cluster_id (string, optional), query (string, optional), limit (integer, default 200).

Response:

{
  "items": [
    {
      "execution_id": "execution-ready",
      "status": "ready",
      "cluster_id": "cluster-1",
      "cluster_session_id": "session-1",
      "namespace": "namespace",
      "name": "qwen",
      "display_name": "",
      "description": "",
      "model": "Qwen",
      "backend": "vLLM",
      "guide": "optimized-baseline",
      "endpoint": "http://execution-ready.namespace.svc:8000",
      "forwarded_endpoint": null,
      "replicas": null,
      "tensor_parallel_size": null,
      "image": null,
      "preserve_deployment": true,
      "created_at": "2026-09-23T09:18:24.287000+00:00",
      "updated_at": null,
      "max_model_len": null,
      "gpu_memory_utilization": null,
      "enable_prefix_caching": null,
      "max_num_seqs": null,
      "max_num_batched_tokens": null,
      "storage_type": null,
      "storage_volume_id": null,
      "mount_path": null,
      "pvc_name": null,
      "failure": null,
      "custom_parameters": [],
      "baseline_endpoint": null,
      "service_ref": null
    }
  ]
}

Get a deployment execution​

Get one deployment execution by execution id.

GET /api/v1/deployments/executions/{execution_id}

Request: no body.

  • Path parameter: execution_id (string, required)

Response:

{
  "execution_id": "execution-ready",
  "status": "ready",
  "cluster_id": "cluster-1",
  "cluster_session_id": "session-1",
  "namespace": "namespace",
  "name": "qwen",
  "display_name": "",
  "description": "",
  "model": "Qwen",
  "backend": "vLLM",
  "guide": "optimized-baseline",
  "endpoint": "http://execution-ready.namespace.svc:8000",
  "forwarded_endpoint": null,
  "preserve_deployment": true,
  "created_at": "2026-09-23T09:18:24.287000+00:00",
  "custom_parameters": []
}

Get execution pod status​

Get live pod status for all pods in a deployment execution's namespace.

GET /api/v1/deployments/executions/{execution_id}/pods

Request: no body.

  • Path parameter: execution_id (string, required)

Response:

{
  "execution_id": "exec-123",
  "namespace": "prism-exec-123",
  "pods": [
    {
      "name": "pod-1",
      "namespace": "prism-exec-123",
      "phase": "Running",
      "ready": true,
      "pod_ip": "10.244.0.5",
      "node_name": "worker-1",
      "restarts": 0,
      "status_reason": "Running",
      "containers": [
        {
          "name": "vllm",
          "image": "vllm:latest",
          "ready": true,
          "restart_count": 0,
          "state": "running"
        }
      ],
      "creation_timestamp": "2026-09-06T00:00:00Z"
    }
  ]
}

Get pod logs for an execution​

Get logs for one pod inside a deployment execution's namespace.

GET /api/v1/deployments/executions/{execution_id}/pods/{pod_name}/logs

Request: no body.

  • Path parameters: execution_id (string, required), pod_name (string, required)
  • Query parameters: container (string, optional), tail (integer, default 200), previous (boolean, default false)

Response:

{
  "execution_id": "exec-123",
  "namespace": "prism-exec-123",
  "pod_name": "pod-1",
  "container": "vllm",
  "logs": "[INFO] Server started\n[INFO] Listening on :8000",
  "success": true
}

Connect to a deployment execution endpoint​

Create or reuse a reachable forwarded endpoint for a deployment execution and return the local URL.

POST /api/v1/deployments/executions/{execution_id}/endpoint

Request: no body.

  • Path parameter: execution_id (string, required)

Response:

{
  "success": true,
  "execution_id": "execution-ready",
  "id": "forward-1",
  "local_port": 38000,
  "endpoint": "http://127.0.0.1:38000/v1",
  "forwarded_endpoint": "http://127.0.0.1:38000/v1",
  "namespace": "namespace",
  "service": "qwen-router",
  "remote_port": 8000
}

Get a deployment case​

Get one deployment case within a deployment run, including its associated execution and deployment summary payloads.

GET /api/v1/deployments/runs/{run_id}/cases/{case_id}

Request: no body.

  • Path parameters: run_id (string, required), case_id (string, required)

Response:

{
  "case": {
    "id": "case-execution-ready",
    "run_id": "run-1",
    "ordinal": 0,
    "source_configuration_ordinal": 0,
    "provider_ref": "optimized-baseline",
    "component": "decode",
    "depends_on": [],
    "status": "ready",
    "attempt": 1,
    "execution_id": "execution-ready",
    "failure": null,
    "create_request": {
      "request_id": "58b85f84-c8ef-47e1-8cb9-61831eb9d310",
      "input_schema_version": "v1",
      "configuration_artifacts": [
        {
          "artifact_id": "524a4178-7e9f-49ea-bccd-8ddb163ae2fd",
          "schema_version": "deployable-configuration.v1",
          "kind": "configuration",
          "provider_ref": "optimized-baseline",
          "media_type": "application/yaml",
          "checksum": "sha256:0f4f7e4b",
          "content": "{\"model\":{\"name\":\"Qwen\"}}"
        }
      ],
      "deployment_policy": {"schema_version": "v1", "value": {"deployment_name": "qwen"}},
      "cluster_snapshot_ref": null,
      "cluster_snapshot": null,
      "provenance": {}
    }
  },
  "execution": {
    "execution_id": "execution-ready",
    "request_id": "58b85f84-c8ef-47e1-8cb9-61831eb9d310",
    "status": "ready",
    "artifact": {
      "artifact_id": "art-1",
      "artifact_hash": "sha256:ef35",
      "configuration_artifact_ids": ["524a4178-7e9f-49ea-bccd-8ddb163ae2fd"],
      "source_ref": null,
      "manifest_ref": "vllm/manifest.yaml",
      "manifest_checksum": null,
      "values_checksum": null,
      "rendered_payload": {"schema_version": "v1", "value": {}},
      "created_at": "2026-09-23T09:18:24.287000+00:00"
    },
    "endpoint": {
      "url": "http://execution-ready.namespace.svc:8000",
      "protocol": "http",
      "service_ref": null,
      "model_ref": null,
      "baseline_url": null
    },
    "forwarded_endpoint": null,
    "monitoring_setup": null,
    "namespace": "namespace",
    "configuration_artifacts": [],
    "resource_snapshot": null,
    "provenance": {"cluster_server_id": "cluster-1", "cluster_session_id": "session-1"},
    "metadata": {"display_name": "", "description": ""},
    "evidence_refs": [],
    "diagnostics": null,
    "created_at": "2026-09-23T09:18:24.287000+00:00",
    "updated_at": "2026-09-23T09:20:00.000000+00:00"
  },
  "deployment": {
    "run_id": "run-1",
    "case_id": "case-execution-ready",
    "execution_id": "execution-ready",
    "name": "qwen",
    "display_name": "",
    "description": "",
    "status": "ready",
    "namespace": "namespace",
    "endpoint": "http://execution-ready.namespace.svc:8000",
    "forwarded_endpoint": null,
    "guide": "optimized-baseline",
    "backend": "vLLM",
    "model_name": "Qwen",
    "replicas": null,
    "tensor_parallel_size": null,
    "image": null
  }
}

Get deployment case logs​

Get orchestration logs for one deployment case.

GET /api/v1/deployments/runs/{run_id}/cases/{case_id}/logs

Request: no body.

  • Path parameters: run_id (string, required), case_id (string, required)
  • Query parameter: limit (integer, default 200, max 1000)

Response: DeploymentLogPage

{
  "execution_id": "execution-ready",
  "entries": [
    {
      "timestamp": "2026-09-23T09:19:01.000000+00:00",
      "source": "provider",
      "message": "Rendered deployment manifest",
      "severity": "info"
    },
    {
      "timestamp": "2026-09-23T09:19:12.000000+00:00",
      "source": "provider",
      "message": "Deployment reached Ready status",
      "severity": "info"
    }
  ],
  "next_cursor": null,
  "complete": true,
  "evidence_refs": []
}

Clean a deployment case​

Clean up one deployment case's runtime resources and marks it cleaned.

POST /api/v1/deployments/runs/{run_id}/cases/{case_id}/clean

Request body: CleanCaseRequest

  • Path parameters: run_id (string, required), case_id (string, required)
  • preserve_rendered_overlay (boolean, default false)
{
  "preserve_rendered_overlay": true
}

Response:

{
  "case": {
    "id": "case-execution-ready",
    "run_id": "run-1",
    "ordinal": 0,
    "source_configuration_ordinal": 0,
    "provider_ref": "optimized-baseline",
    "component": "decode",
    "depends_on": [],
    "status": "cleaned",
    "attempt": 1,
    "execution_id": "execution-ready",
    "failure": null,
    "create_request": {
      "request_id": "58b85f84-c8ef-47e1-8cb9-61831eb9d310",
      "input_schema_version": "v1",
      "configuration_artifacts": [],
      "deployment_policy": {"schema_version": "v1", "value": {"deployment_name": "qwen"}},
      "cluster_snapshot_ref": null,
      "cluster_snapshot": null,
      "provenance": {}
    }
  },
  "execution": {
    "execution_id": "execution-ready",
    "request_id": "58b85f84-c8ef-47e1-8cb9-61831eb9d310",
    "status": "cleaned",
    "artifact": {
      "artifact_id": "art-1",
      "artifact_hash": "sha256:ef35",
      "configuration_artifact_ids": ["524a4178-7e9f-49ea-bccd-8ddb163ae2fd"],
      "source_ref": null,
      "manifest_ref": "vllm/manifest.yaml",
      "manifest_checksum": null,
      "values_checksum": null,
      "rendered_payload": {"schema_version": "v1", "value": {}},
      "created_at": "2026-09-23T09:18:24.287000+00:00"
    },
    "endpoint": {
      "url": "http://execution-ready.namespace.svc:8000",
      "protocol": "http",
      "service_ref": null,
      "model_ref": null,
      "baseline_url": null
    },
    "forwarded_endpoint": null,
    "monitoring_setup": null,
    "namespace": "namespace",
    "configuration_artifacts": [],
    "resource_snapshot": null,
    "provenance": {"cluster_server_id": "cluster-1", "cluster_session_id": "session-1"},
    "metadata": {"display_name": "", "description": ""},
    "evidence_refs": [],
    "diagnostics": null,
    "created_at": "2026-09-23T09:18:24.287000+00:00",
    "updated_at": "2026-09-23T09:22:00.000000+00:00"
  },
  "deployment": {
    "run_id": "run-1",
    "case_id": "case-execution-ready",
    "execution_id": "execution-ready",
    "name": "qwen",
    "display_name": "",
    "description": "",
    "status": "cleaned",
    "namespace": "namespace",
    "endpoint": "http://execution-ready.namespace.svc:8000",
    "forwarded_endpoint": null,
    "guide": "optimized-baseline",
    "backend": "vLLM",
    "model_name": "Qwen",
    "replicas": null,
    "tensor_parallel_size": null,
    "image": null
  }
}

Access-binding routes also exist for deployment runs and executions at GET/POST/DELETE /runs/{run_id}/access..., GET/POST/DELETE /executions/{execution_id}/access..., and GET .../access/options; see llm_d_lens/deploy/router.py for those exact handlers.

Additional endpoints exist for the deploy domain — see the full OpenAPI reference.

See the full request/response schemas in the FastAPI OpenAPI schema (regenerate with npm run docs:api).

Delete a deployment run​

DELETE /api/v1/deployments/runs/{run_id}

No request body. Requires deployment:execution:delete and access to the run. Returns 204 with no body. Missing runs return 404; active (queued, running, cancelling) runs or lifecycle conflicts return 409. Cancel active orchestration and wait for a terminal state before deleting.

This is not a metadata-only delete: remaining cases with stored executions are cleaned first, requiring their active cluster session, then stored run/execution records and associated shares are removed. Already-cleaned cases are skipped. By contrast, case clean removes runtime resources but retains the lifecycle record; preserve_rendered_overlay=true retains rendered files.

Example request: DELETE /api/v1/deployments/runs/run-1 (no JSON body or response).

Refresh a deployment case​

POST /api/v1/deployments/runs/{run_id}/cases/{case_id}/refresh

No body is required ({} is accepted). Requires deployment:case:execute. Returns 200 with the same case, execution, and deployment envelope as Get case, refreshed from provider diagnostics. Stopped or cleaned records are preserved rather than made ready again. Missing case/execution or refresh ValueError returns 404. Response excerpt:

{
  "case": {"id": "case-1", "status": "ready", "execution_id": "execution-1"},
  "execution": {"execution_id": "execution-1", "status": "ready"},
  "deployment": {"run_id": "run-1", "case_id": "case-1", "status": "ready"}
}

Stop a deployment case​

POST /api/v1/deployments/runs/{run_id}/cases/{case_id}/stop

No body is required ({} is accepted). Requires deployment:case:execute and an active cluster session. Invokes provider stop and retains the records; it is not deletion of the run. Returns 200 with the Get case envelope, or 409 for worker ValueError. Response excerpt:

{
  "case": {"id": "case-1", "status": "stopped", "execution_id": "execution-1"},
  "execution": {"execution_id": "execution-1", "status": "rolled_back"},
  "deployment": {"run_id": "run-1", "case_id": "case-1", "status": "stopped"}
}

Restart a deployment case​

:::warning Current restart limitation

The route exists, but the current worker cleans the old execution and marks its case cleaned before calling DeploymentPlanService.retry_case(), which accepts only the latest stopped or failed case. Consequently, successful cleanup is followed by 409, not a queued replacement. Cleanup has already taken effect; an error is not evidence that the old resources remain. Do not repeatedly retry this route expecting it to restore the deployment.

:::

POST /api/v1/deployments/runs/{run_id}/cases/{case_id}/restart

Requires deployment:case:execute, an active cluster session, and a stopped or failed case. Body: RestartCaseRequest; model_token is an optional string override, with empty/whitespace treated as absent. Send {} to omit it. The declared success shape is HTTP 200, { "case": ... }, but the current cleanup/retry conflict prevents that path after successful cleanup. Lifecycle ValueError becomes 409; invalid bodies return 422.

{}

Current conflict response excerpt (the application adds Problem Details fields):

{
  "status": 409,
  "detail": "only the latest stopped or failed deployment case may be retried"
}

Remote SSH deployment​

These legacy endpoints are implemented by the Node BFF in apps/server/src/routes/remoteDeploy.ts, not the Python Deploy service. They execute commands over SSH rather than create v1 deployment runs. Their current single-node Intel Data Center GPU/XPU restriction is specific to this legacy implementation, not the capability limit of the profile-driven v1 API.

SSH operations require PRISM_REMOTE_DEPLOY_ENABLED=true in production (enabled by default outside production). Production targets must also match PRISM_REMOTE_HOST_ALLOWLIST. Defaults remain readable when SSH operations are disabled, subject to authentication/permission checks.

OperationLens permission
Defaultsremote-deploy:target:read
Fingerprint, testremote-deploy:target:connect
Start, status, logsremote-deploy:deploy:execute
Teardownremote-deploy:deploy:teardown

Authenticated SSH operations require target.host, username, hostFingerprint and a nonempty cards array of numeric ID strings. port defaults to 22; authMethod defaults to agent and also accepts key or password. Agent authentication requires the BFF's SSH_AUTH_SOCK. Key authentication requires a server-side keyPath beneath PRISM_SSH_KEY_DIR (default $HOME/.ssh). Password authentication supplies credentials.password on each request; it is not stored or returned. Fingerprint discovery does not authenticate and does not require cards. Confirm its result through a trusted channel before using it; a returned fingerprint is not itself proof of host identity.

Route errors are { "error": "..." }: 400 for invalid input/credentials, 403 for a production host outside the allowlist, 503 for disabled SSH operations, and normally 502 for SSH/command failures. Gateway authentication and permission errors use the gateway's Problem Details format.

Python HTTP client for remote operations​

There is no Python service-layer equivalent for these Node endpoints. The Python tabs below use this HTTP helper instead. The environment variables in this example are client inputs, not new Lens server configuration. Set them from an authenticated Lens session; never commit their values.

import os
import httpx

def remote_request(method: str, path: str, body: dict | None = None) -> dict:
    with httpx.Client(
        base_url=os.environ["LENS_URL"],
        cookies={
            "prism_session": os.environ["LENS_SESSION"],
            "prism_csrf": os.environ["LENS_CSRF"],
        },
        headers={"x-prism-csrf": os.environ["LENS_CSRF"]},
        timeout=300,
    ) as client:
        response = client.request(method, path, json=body)
        response.raise_for_status()
        return response.json()

target = {
    "host": "192.0.2.20",
    "port": 22,
    "username": "operator",
    "authMethod": "agent",
    "hostFingerprint": os.environ["VERIFIED_SSH_FINGERPRINT"],
    "cards": ["0"],
}

REST request examples below show operation fields; for authenticated operations, add the complete target object above and optional per-request credentials. The Python examples demonstrate the complete request assembly.

Get remote deployment defaults​

GET /api/remote-deploy/defaults, no body. Returns 200:

{
  "repository": "", "branch": "", "httpProxy": "", "httpsProxy": "",
  "noProxy": "", "mountPath": "", "mountModelName": ""
}

Values come from the BFF environment. Repository uses LLM_D_ROOT, then LLM_D_REPOSITORY, otherwise empty; there is no host-specific default checkout. Branch uses LLM_D_BRANCH. Proxy fields prefer HTTP_PROXY, HTTPS_PROXY, NO_PROXY, then their LLM_D_ equivalents. Mount fields use LLM_D_MOUNT_PATH and LLM_D_MOUNT_MODEL_NAME.

Discover an SSH host fingerprint​

POST /api/remote-deploy/fingerprint. Request:

{"target": {"host": "192.0.2.20", "port": 22, "username": "operator"}}

Only host is required; port defaults to 22 and username to probe. Returns 200 with fingerprint, host, port. Illustrative response:

{"fingerprint": "SHA256:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", "host": "192.0.2.20", "port": 22}

Test a remote deployment target​

POST /api/remote-deploy/test. Body: target and optional credentials; no additional operation fields ({}). Probes hostname/OS, llm-d-bench, kubectl/cluster connectivity and xpu-smi. Returns 200 with ok and output (last 12,000 characters):

{"ok": false, "output": "kubectl could not reach the cluster"}

A completed probe with nonzero exit can return 200 and ok=false; HTTP success does not prove readiness. Connection failures use the error statuses above.

Start a remote deployment​

POST /api/remote-deploy/start. Add target and optional credentials to:

{
  "plan": {
    "namespace": "llm-d-optimized", "name": "llm-d-service",
    "accelerator": "Intel Data Center GPU", "nodes": 1,
    "guide": "optimized-baseline", "model": "Qwen/Qwen3-0.6B",
    "decodeReplicas": 1, "decodeTp": 1,
    "deploymentTarget": {"repository": "https://github.com/llm-d/llm-d.git", "branch": "main"}
  }
}

Namespace/name default to the values shown and must be Kubernetes-style names. Repository must be an absolute remote path, ~/ path, or GitHub HTTPS URL; branch defaults to main. URL checkouts use the remote host's LENS_CACHE_DIR (otherwise $XDG_CACHE_HOME/lens or $HOME/.cache/lens), under deploy-repos/<owner>/<repository>. The plan may set deploymentTarget.httpProxy, httpsProxy, noProxy, and a paired absolute mountPath / mountModelName.

The accelerator must match the legacy Intel label and nodes must be one. Decode TP/replicas default to 1 (replicas is a fallback for decodeReplicas); prefill TP/replicas default to zero. Dimensions are integers at most 128, decode dimensions at least one. Required topology cards cannot exceed selected card count. Card IDs are validated/count-checked; this endpoint does not promise physical-device pinning. An explicit guide must be a valid identifier; without one, source/topology chooses the guide, with epd unsupported.

Runs PRISM_REMOTE_DEPLOY_BIN (default llm-d-bench) with deploy custom --no-wait, then patches model-container mount/proxy settings. Returns 200, not 202, after commands complete; this is not a ready deployment. Response excerpt:

{
  "id": "remote-1700000000000", "name": "llm-d-service",
  "cluster": "operator@192.0.2.20:22", "remoteHost": "192.0.2.20",
  "nodes": 1, "gpusPerNode": 1, "selectedCards": ["0"], "replicas": 1,
  "namespace": "llm-d-optimized", "status": "deploying",
  "createdAt": "2026-10-13T00:00:00.000Z", "endpoint": null, "pods": [], "logTail": []
}

The response also includes candidate labels and supplied machine/accelerator metadata; replicas reflects plan.replicas or 1, not necessarily decodeReplicas. Poll status to determine rollout state.

Get remote deployment status​

POST /api/remote-deploy/status. Add target and optional credentials to:

{"namespace": "llm-d-optimized"}

Returns 200 with status, pods, podsReady, podsTotal, endpoint, error, logTail. States include deploying, ready, ready_no_endpoint, failed, not_found; inspect status, not just the HTTP code. Pod entries contain name, status, ready, restarts, node, gpuCount, reason, message. Endpoint discovery uses Gateway/LoadBalancer addresses or Gateway Service DNS; it does not establish a local tunnel. Example:

{"status": "deploying", "pods": [], "podsReady": 0, "podsTotal": 0, "endpoint": null, "error": null, "logTail": []}

Get remote deployment logs​

POST /api/remote-deploy/logs. Add target and optional credentials to:

{"namespace": "llm-d-optimized", "tail": 100}

Optional pod selects a pod. tail defaults to 100 and is bounded to 1..1000. Returns 200 with logs and truncated, retaining at most the final 32,000 JavaScript string code units, not a guaranteed 32 KiB byte limit:

{"logs": "Model server started\n", "truncated": false}

Tear down a remote deployment​

POST /api/remote-deploy/teardown. Add target and optional credentials to:

{"namespace": "llm-d-optimized"}

Runs remote deploy cleanup -n <namespace> -f. Returns 200 with success, namespace, and the final 100 output lines in logTail:

{"success": true, "namespace": "llm-d-optimized", "logTail": []}

Additional deployment APIs​

List orphan deployment namespaces​

List namespaces on a cluster that match the deployment namespace prefix but are not referenced by any Lens execution record. These are leftovers from records deleted without namespace cleanup, or from a rebuilt database; nothing else in Lens points at them.

GET /api/v1/deployments/orphans

Query parameters:

  • cluster_id (string, required): the cluster to scan.

Response: the deployment prefix and the orphan namespaces.

{
  "clusterId": "cluster-a",
  "prefix": "llmd-",
  "items": [
    { "name": "llmd-optimized-baseline-1-1-20260918032423139529", "createdAt": "2026-09-18T03:24:23Z", "phase": "Active" }
  ]
}

Clean orphan deployment namespaces​

Delete the given orphan namespaces (and the InferencePool/EPP/model server inside them). Names outside the deployment prefix, or still referenced by a Lens execution record, are refused rather than deleted.

POST /api/v1/deployments/orphans/clean

Request body (CleanOrphansRequest):

  • clusterId (string, required)
  • namespaces (string[], required): the orphan namespaces to delete.
{
  "clusterId": "cluster-a",
  "namespaces": ["llmd-optimized-baseline-1-1-20260918032423139529"]
}

Response: deleted names and any per-namespace failures.

{
  "clusterId": "cluster-a",
  "cleaned": ["llmd-optimized-baseline-1-1-20260918032423139529"],
  "failed": []
}

Configuration preparation APIs​

These Configuration-domain routes prepare artifacts for Deployments and Evaluation. Request and response models live in llm_d_lens.configuration.models. The examples use illustrative cluster/session IDs and guide commits; replace them with an active cluster session and real rendered guide provenance. Saving publishes an artifact; rendering alone does not deploy it.

The workflow is source selection/editing, resolve, render, save, then deployment. Resolve normalizes each source configuration before capability checking and validation, returning per-configuration feedback. The candidate source identifies input origin, not deployment type. Render binds candidate intent to actual Guide resources; save publishes immutable artifact content, identity and checksums. Configuration owns normalization, validation and artifact publication; Deploy owns runtime validation, resource lifecycle, teardown and rollback. Checksums alone cannot validate topology or resource semantics. These boundaries apply to both the REST endpoints and their Python service functions below.

The HTTP boundary uses application/problem+json errors. Shared DomainError instances retain their original status and code (for example a conflict remains 409); they are not converted into an operation-specific 500. Python service calls raise their original exceptions; the route adapters in llm_d_lens.configuration.router apply the HTTP mappings below.

Resolve source configurations​

POST /api/v1/configurations/resolve

Request: ResolveRequest. Success: HTTP 200, ResolveResponse. candidate_source and a nonempty configurations list are required; target.cluster_id is optional. Returns results, each with configuration_id, candidate_config, and validation (status, errors, warnings). Configuration errors escaping the resolver return 400; unexpected failures return 500 with Unable to resolve configurations.

{
  "candidate_source": {
    "name": "manual"
  },
  "configurations": [
    {
      "configuration_id": "example",
      "input_mode": "form",
      "payload": {
        "type": "baseline",
        "candidate_source": {
          "name": "manual"
        },
        "target": {
          "model": "New/Model"
        },
        "serving": {
          "replicas": 2,
          "tensor_parallel_size": 1
        },
        "runtime": {
          "image": "model:v1"
        },
        "guide_settings": {
          "routerValues": "router:\n  epp:\n    replicas: 2"
        }
      }
    }
  ],
  "target": {
    "cluster_id": "cluster"
  }
}

Render a deployable configuration​

POST /api/v1/configurations/render

Request: RenderRequest. Success: HTTP 200, RenderResponse. Requires candidate_config and render: format, guide/template references, pinned guide source, rendered manifest, active cluster reference, and deployment readiness/endpoint contract. Optional bundle, reference manifest, model secret and deployment name remain supported. Returns deployable_configuration and metadata. Configuration or value errors return 400; unexpected failures return 500 with Unable to render configuration.

{
  "candidate_config": {
    "type": "baseline",
    "candidate_source": {
      "name": "manual"
    },
    "target": {
      "model": "New/Model"
    },
    "serving": {
      "replicas": 2,
      "tensor_parallel_size": 1
    },
    "runtime": {
      "image": "model:v1"
    },
    "guide_settings": {
      "routerValues": "router:\n  epp:\n    replicas: 2"
    }
  },
  "render": {
    "format": "manifest",
    "guide_ref": "optimized-baseline",
    "template_ref": "guides/optimized-baseline/modelserver/xpu/vllm/kustomization.yaml",
    "guide_source": {
      "repository": "llm-d/llm-d",
      "requestedRef": "main",
      "commit": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
      "guide": "optimized-baseline",
      "accelerator": "xpu",
      "modelServer": "vllm",
      "variant": ".",
      "files": [
        "guides/optimized-baseline/modelserver/xpu/vllm/kustomization.yaml"
      ]
    },
    "rendered_manifest": "apiVersion: apps/v1\nkind: Deployment\nmetadata:\n  name: decode\nspec:\n  replicas: 2\n  template:\n    spec:\n      containers:\n      - args:\n        - New/Model\n        - --tensor-parallel-size=1\n        command:\n        - vllm\n        - serve\n        image: model:v1\n        name: modelserver\n---\napiVersion: v1\nkind: Service\nmetadata:\n  name: model\nspec:\n  ports:\n  - port: 8000\n",
    "deployment_bundle": {
      "schemaVersion": "guide-deployment-bundle.v1",
      "guide": "optimized-baseline",
      "sourceCommit": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
      "helm": {
        "chart": "oci://ghcr.io/llm-d/charts/llm-d-router-standalone",
        "version": "v0.9.0",
        "releaseName": "optimized-baseline",
        "values": [
          {
            "name": "router-base.yaml",
            "content": "router: {}",
            "checksum": "sha256:162352753bb2618923b371c7236fc0c69f6604abaceef33d769f7f1686f845f6"
          },
          {
            "name": "router-guide.yaml",
            "content": "router:\n  epp:\n    replicas: 1",
            "checksum": "sha256:bc3156ef61b87ae553e01848266fbe4f3dc526e0072f0d93f6dd9ede8c8784fd"
          },
          {
            "name": "router-effective.yaml",
            "content": "router:\n  epp:\n    replicas: 2",
            "checksum": "sha256:816e7b274c9e9c55442a5be90c74d252a517a62f49dfdcf6fba60ac8443d189a"
          }
        ]
      },
      "resources": []
    },
    "cluster_ref": {
      "id": "cluster",
      "name": "cluster",
      "session_id": "00000000-0000-0000-0000-000000000001",
      "connection": "managed-kubeconfig"
    },
    "deployment": {
      "readinessDeployments": [
        "decode"
      ],
      "endpoint": {
        "protocol": "http",
        "serviceName": "model",
        "port": 8000
      }
    }
  }
}

Save a configuration artifact​

POST /api/v1/configurations/save

Request: SaveRequest. Success: HTTP 200, SaveResponse. Requires deployable_configuration; file.name is optional. The configuration file URI is derived from the artifact ID and file name and is preserved when reading the saved artifact. Returns configuration_file (URI, type, file name, format, path, size, SHA-256) and artifact (ID, revision, status, deployable configuration, file metadata, creation time). Configuration, filesystem or value errors return 400; unexpected failures return 500 with Unable to save configuration.

{
  "deployable_configuration": {
    "schema_version": "deployable-configuration.v1",
    "type": "baseline",
    "format": "manifest",
    "content": {
      "deploymentType": "baseline",
      "model": {
        "name": "New/Model"
      },
      "serving": {
        "replicaCount": 2,
        "tensorParallelSize": 1
      },
      "runtime": {
        "image": "model:v1"
      },
      "guideSettings": {
        "routerValues": "router:\n  epp:\n    replicas: 2"
      },
      "officialGuide": {
        "source": {
          "repository": "llm-d/llm-d",
          "requestedRef": "main",
          "commit": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
          "guide": "optimized-baseline",
          "accelerator": "xpu",
          "modelServer": "vllm",
          "variant": ".",
          "files": [
            "guides/optimized-baseline/modelserver/xpu/vllm/kustomization.yaml"
          ]
        },
        "renderedManifest": "apiVersion: apps/v1\nkind: Deployment\nmetadata:\n  name: decode\nspec:\n  replicas: 2\n  template:\n    spec:\n      containers:\n      - args:\n        - New/Model\n        - --tensor-parallel-size=1\n        command:\n        - vllm\n        - serve\n        image: model:v1\n        name: modelserver\n---\napiVersion: v1\nkind: Service\nmetadata:\n  name: model\nspec:\n  ports:\n  - port: 8000\n",
        "manifestChecksum": "sha256:f0677cfd3ecd361a1e549b6a17fbf64c884d406152095c777b359d9d95f2b408",
        "cluster": {
          "id": "cluster",
          "name": "cluster",
          "session_id": "00000000-0000-0000-0000-000000000001",
          "connection": "managed-kubeconfig"
        },
        "deployment": {
          "readinessDeployments": [
            "decode"
          ],
          "endpoint": {
            "protocol": "http",
            "serviceName": "model",
            "port": 8000
          }
        },
        "modelSecret": {
          "mode": "none",
          "sourceNamespace": null,
          "sourceName": null
        },
        "deploymentBundle": {
          "schemaVersion": "guide-deployment-bundle.v1",
          "guide": "optimized-baseline",
          "sourceCommit": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
          "helm": {
            "chart": "oci://ghcr.io/llm-d/charts/llm-d-router-standalone",
            "version": "v0.9.0",
            "releaseName": "optimized-baseline",
            "values": [
              {
                "name": "router-base.yaml",
                "content": "router: {}",
                "checksum": "sha256:162352753bb2618923b371c7236fc0c69f6604abaceef33d769f7f1686f845f6"
              },
              {
                "name": "router-guide.yaml",
                "content": "router:\n  epp:\n    replicas: 1",
                "checksum": "sha256:bc3156ef61b87ae553e01848266fbe4f3dc526e0072f0d93f6dd9ede8c8784fd"
              },
              {
                "name": "router-effective.yaml",
                "content": "router:\n  epp:\n    replicas: 2",
                "checksum": "sha256:816e7b274c9e9c55442a5be90c74d252a517a62f49dfdcf6fba60ac8443d189a"
              }
            ]
          },
          "resources": []
        }
      }
    },
    "provider_ref": "optimized-baseline",
    "checksum": "sha256:da087b6024d1c8c6ef531f06d55e78d2a249594feb96b17580af140ac68c029f",
    "provenance": {
      "candidate_source": {
        "name": "manual",
        "run_id": null,
        "result_id": null
      },
      "guide_ref": "optimized-baseline",
      "template_ref": "guides/optimized-baseline/modelserver/xpu/vllm/kustomization.yaml",
      "guide_source": {
        "repository": "llm-d/llm-d",
        "requestedRef": "main",
        "commit": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
        "guide": "optimized-baseline",
        "accelerator": "xpu",
        "modelServer": "vllm",
        "variant": ".",
        "files": [
          "guides/optimized-baseline/modelserver/xpu/vllm/kustomization.yaml"
        ]
      },
      "cluster_ref": {
        "id": "cluster",
        "name": "cluster",
        "session_id": "00000000-0000-0000-0000-000000000001",
        "connection": "managed-kubeconfig"
      },
      "manifest_checksum": "sha256:f0677cfd3ecd361a1e549b6a17fbf64c884d406152095c777b359d9d95f2b408",
      "renderer_version": "1.0"
    }
  },
  "file": {
    "name": "example.yaml"
  }
}

Deployment access APIs (removed)​

Deployment sharing and deployment:run:share have been retired. Use model-service inference grants instead. Migration audits and removes existing deployment shares; it does not convert them into broader model grants.

The former GET/POST/DELETE /api/v1/deployments/executions/{execution_id}/access and /api/v1/deployments/runs/{run_id}/access routes, including /access/options and /access/{binding_id}, are no longer registered. They no longer grant, list or revoke access.