Observability

Backend API

Observability is split across five FastAPI routers under llm_d_lens/monitoring/: accelerator/router.py, cluster_stack/router.py, deployment/router.py, gpu_driver/router.py, and profiling/router.py.

List accelerator observability capabilities

List accelerator observability capabilities Lens supports, including access modes and default namespaces.

REST API

GET /api/v1/monitoring/accelerators

Request: no request body.

Response: returns an AcceleratorCapabilitiesResponse with supported accelerator families.

{
  "accelerators": [
    {
      "type": "intel_gpu",
      "display_name": "Intel GPU",
      "access_modes": ["dra", "plugin"],
      "release_name": "xpumd",
      "default_namespace": "intel-xpumd"
    },
    {
      "type": "nvidia_gpu",
      "display_name": "NVIDIA GPU",
      "access_modes": ["plugin", "dra"],
      "release_name": "dcgm-exporter",
      "default_namespace": "dcgm-exporter"
    }
  ]
}

Python API

from llm_d_lens.monitoring.accelerator.service import capabilities

response = capabilities()
print(response.model_dump())

Get accelerator observability status

Get observability status for one accelerator family, including release, component, and access-mode health.

REST API

GET /api/v1/monitoring/accelerators/{accelerator}/status

Request: no request body. Path param: accelerator (str, one of the registered families: intel_gpu or nvidia_gpu). Query params: namespace (str, defaults to intel-xpumd regardless of the selected accelerator — pass dcgm-exporter, or whatever namespace the NVIDIA release was installed into, explicitly when querying nvidia_gpu), access_mode ("dra" | "plugin", optional), and cluster_id (str, optional).

Response: returns an AcceleratorStatusResponse.

{
  "accelerator": "intel_gpu",
  "access_mode": "dra",
  "access_modes": [
    {
      "mode": "dra",
      "available": true,
      "detected": true,
      "message": null
    },
    {
      "mode": "plugin",
      "available": false,
      "detected": false,
      "message": "device plugin not installed"
    }
  ],
  "cluster_reachable": true,
  "context": "kind-test",
  "namespace": "intel-xpumd",
  "release": {
    "name": "xpumd",
    "status": "deployed",
    "chart": "xpumd-0.1.0",
    "revision": 1
  },
  "status": "ready",
  "message": "ready",
  "components": [
    {
      "name": "xpumd-controller",
      "status": "ready",
      "kind": "workload",
      "ready": 1,
      "desired": 1,
      "source": "xpumd",
      "diagnostics": []
    }
  ],
  "active_operation_id": null,
  "observed_at": "2026-09-23T08:00:00Z",
  "stale": false
}

Python API

import asyncio

from llm_d_lens.monitoring.accelerator.service import get_status, validate_namespace


async def main() -> None:
    status = await get_status(
        "intel_gpu",
        validate_namespace("intel-xpumd"),
        "dra",
        cluster_id="cluster-123",
    )
    print(status.model_dump())


asyncio.run(main())

Install accelerator observability

Install accelerator observability components for a supported accelerator family.

REST API

POST /api/v1/monitoring/accelerators/{accelerator}/installations

Request: path param accelerator (str). Optional header: Idempotency-Key (str). Optional query param: cluster_id (str). Body fields are accelerator, access_mode, and namespace.

{
  "accelerator": "intel_gpu",
  "access_mode": "dra",
  "namespace": "intel-xpumd"
}

Response: returns an AcceleratorOperationResponse for the queued install operation.

{
  "operation_id": "accel-op-123",
  "kind": "install",
  "status": "queued",
  "phase": "queued",
  "accelerator": "intel_gpu",
  "access_mode": "dra",
  "namespace": "intel-xpumd",
  "started_at": "2026-09-23T08:01:00Z",
  "finished_at": null,
  "exit_code": null,
  "logs": [],
  "error": null
}

Python API

import asyncio

from llm_d_lens.monitoring.accelerator.models import AcceleratorInstallRequest
from llm_d_lens.monitoring.accelerator.service import install


async def main() -> None:
    operation = await install(
        AcceleratorInstallRequest(
            accelerator="intel_gpu",
            access_mode="dra",
            namespace="intel-xpumd",
        ),
        idempotency_key="request-123",
        cluster_id="cluster-123",
    )
    print(operation.model_dump())


asyncio.run(main())

Get cluster monitoring stack status

Check whether the cluster-level Prometheus/Grafana monitoring stack (kube-prometheus-stack) is installed and ready.

REST API

GET /api/v1/monitoring/cluster-stack/status

Request: no request body. Query params: namespace (str, default llm-d-monitoring) and cluster_id (str, optional).

Response: returns a ClusterStackStatusResponse.

{
  "cluster": {
    "context": "kind-test",
    "reachable": true,
    "platform": "kubernetes"
  },
  "namespace": "llm-d-monitoring",
  "release": {
    "name": "llmd",
    "status": "deployed",
    "chart": "kube-prometheus-stack-65.0.0",
    "revision": 2
  },
  "status": "ready",
  "message": "ready",
  "components": [
    {
      "name": "grafana",
      "status": "ready",
      "kind": "workload",
      "ready": 1,
      "desired": 1,
      "source": "llmd",
      "diagnostics": []
    }
  ],
  "active_operation_id": null,
  "observed_at": "2026-09-23T08:00:00Z",
  "stale": false
}

Python API

import asyncio

from llm_d_lens.monitoring.cluster_stack.service import get_status, validate_namespace


async def main() -> None:
    status = await get_status(validate_namespace("llm-d-monitoring"), cluster_id="cluster-123")
    print(status.model_dump())


asyncio.run(main())

Run cluster monitoring stack preflight

Run cluster monitoring stack preflight checks without installing anything. Use this before starting a Prometheus/Grafana stack installation.

REST API

POST /api/v1/monitoring/cluster-stack/installations/preflight

Request: optional query param cluster_id (str). Body fields are namespace, mode, enable_tls, and reinstall.

{
  "namespace": "llm-d-monitoring",
  "mode": "central",
  "enable_tls": false,
  "reinstall": false
}

Response: returns a ClusterStackPreflightResponse with blocking and non-blocking checks.

{
  "allowed": true,
  "status": "ready",
  "checks": [
    {
      "name": "cluster-reachable",
      "passed": true,
      "message": "cluster is reachable",
      "blocking": true
    }
  ],
  "warnings": [],
  "command_preview": [
    "/path/to/llm-d/guides/recipes/observability/install-prometheus-grafana.sh",
    "--namespace",
    "llm-d-monitoring"
  ]
}

Python API

import asyncio

from llm_d_lens.monitoring.cluster_stack.models import ClusterStackInstallRequest
from llm_d_lens.monitoring.cluster_stack.service import preflight


async def main() -> None:
    result = await preflight(
        ClusterStackInstallRequest(
            namespace="llm-d-monitoring",
            mode="central",
            enable_tls=False,
            reinstall=False,
        ),
        cluster_id="cluster-123",
    )
    print(result.model_dump())


asyncio.run(main())

Install cluster monitoring stack

Install or reinstall the cluster-level monitoring stack. Use this when Prometheus/Grafana cluster monitoring should be provisioned or repaired.

REST API

POST /api/v1/monitoring/cluster-stack/installations

Request: optional header Idempotency-Key (str) and optional query param cluster_id (str). Body fields are namespace, mode, enable_tls, and reinstall.

{
  "namespace": "llm-d-monitoring",
  "mode": "central",
  "enable_tls": false,
  "reinstall": true
}

Response: returns a ClusterStackOperationResponse for the queued install or reinstall operation.

{
  "operation_id": "cluster-stack-op-123",
  "kind": "install",
  "status": "queued",
  "phase": "queued",
  "namespace": "llm-d-monitoring",
  "started_at": "2026-09-23T08:10:00Z",
  "finished_at": null,
  "exit_code": null,
  "logs": [],
  "error": null
}

Python API

import asyncio

from llm_d_lens.monitoring.cluster_stack.models import ClusterStackInstallRequest
from llm_d_lens.monitoring.cluster_stack.service import install


async def main() -> None:
    operation = await install(
        ClusterStackInstallRequest(
            namespace="llm-d-monitoring",
            mode="central",
            enable_tls=False,
            reinstall=True,
        ),
        idempotency_key="request-123",
        cluster_id="cluster-123",
    )
    print(operation.model_dump())


asyncio.run(main())

List deployment monitoring status

List deployment monitoring status for every deployment in a cluster. Use this to discover which executions have observability installed, enabled, or unhealthy.

REST API

GET /api/v1/monitoring/deployments

Request: no request body. Required query param: cluster_id (str).

Response: returns cluster-level deployment monitoring status with one item per execution namespace.

{
  "cluster_id": "cluster-123",
  "cluster_reachable": true,
  "stack_ready": true,
  "items": [
    {
      "execution_id": "deploy-exec-001",
      "namespace": "llm-d-run-001",
      "cluster_reachable": true,
      "stack_ready": true,
      "status": "enabled",
      "enabled": true,
      "epp_service": {
        "name": "demo-epp",
        "labels": {
          "app.kubernetes.io/name": "demo-epp"
        },
        "service_account": "demo-epp"
      },
      "servicemonitors": ["llm-d-prism-epp-monitor"],
      "podmonitors": ["llm-d-prism-modelserver-monitor"],
      "message": "monitoring enabled"
    }
  ]
}

Python API

import asyncio

from llm_d_lens.monitoring.deployment.service import get_cluster_status


async def main() -> None:
    status = await get_cluster_status("cluster-123")
    print(status)


asyncio.run(main())

Get deployment monitoring status

Get monitoring status for one deployment execution, optionally scoped to a cluster id.

REST API

GET /api/v1/monitoring/deployments/{execution_id}

Request: no request body. Path param: execution_id (str). Optional query param: cluster_id (str).

Response: returns one deployment monitoring status object.

{
  "execution_id": "deploy-exec-001",
  "namespace": "llm-d-run-001",
  "cluster_reachable": true,
  "stack_ready": true,
  "status": "enabled",
  "enabled": true,
  "epp_service": {
    "name": "demo-epp",
    "labels": {
      "app.kubernetes.io/name": "demo-epp"
    },
    "service_account": "demo-epp"
  },
  "servicemonitors": ["llm-d-prism-epp-monitor"],
  "podmonitors": ["llm-d-prism-modelserver-monitor"],
  "message": "monitoring enabled"
}

Python API

import asyncio

from llm_d_lens.monitoring.deployment.service import get_status


async def main() -> None:
    status = await get_status("deploy-exec-001", "cluster-123")
    print(status)


asyncio.run(main())

Manage deployment monitoring resources

Install, enable, disable, or uninstall monitoring resources for one deployment execution.

REST API

POST /api/v1/monitoring/deployments/{execution_id}/{action}

Request: no request body. Path params: execution_id (str) and action (str, one of install, enable, disable, or uninstall). Optional query param: cluster_id (str).

Response: returns the same deployment monitoring status shape after the requested action completes.

{
  "execution_id": "deploy-exec-001",
  "namespace": "llm-d-run-001",
  "cluster_reachable": true,
  "stack_ready": true,
  "status": "disabled",
  "enabled": false,
  "epp_service": {
    "name": "demo-epp",
    "labels": {
      "app.kubernetes.io/name": "demo-epp"
    },
    "service_account": "demo-epp"
  },
  "servicemonitors": [],
  "podmonitors": [],
  "message": "monitoring disabled"
}

Python API

import asyncio

from llm_d_lens.monitoring.deployment.service import disable, enable, install, uninstall


async def main() -> None:
    actions = {
        "install": install,
        "enable": enable,
        "disable": disable,
        "uninstall": uninstall,
    }
    result = await actions["disable"]("deploy-exec-001", "cluster-123")
    print(result)


asyncio.run(main())

Get GPU driver status

Get the current GPU driver installation status for a cluster, including readiness and access mode.

REST API

GET /api/v1/monitoring/gpu-driver/status

Request: no request body. Query params: access_mode ("dra" | "plugin", default dra), cluster_id (str, optional), and hardware (str, optional — a registered hardware profile id such as intel-xpu; omit it when only one driver provider is registered).

Response: returns a GpuDriverStatusResponse.

{
  "access_mode": "dra",
  "installed": true,
  "ready": true,
  "cluster_reachable": true,
  "message": "Intel GPU DRA driver is installed and ready."
}

Python API

import asyncio

from llm_d_lens.monitoring.gpu_driver.service import get_status


async def main() -> None:
    status = await get_status("dra", cluster_id="cluster-123", hardware="intel-xpu")
    print(status.model_dump())


asyncio.run(main())

Install GPU driver stack

Install the GPU driver/device-plugin stack Lens expects for the selected access mode.

REST API

POST /api/v1/monitoring/gpu-driver/install

Request: optional query params cluster_id (str) and hardware (str, a registered hardware profile id; omit when only one driver provider is registered). Body field: access_mode ("dra" | "plugin").

{
  "access_mode": "dra"
}

Response: returns a GpuDriverInstallResponse.

{
  "access_mode": "dra",
  "applied": true,
  "message": "Applied the Intel GPU DRA driver manifests. It may take a few minutes for pods to become ready."
}

Python API

import asyncio

from llm_d_lens.monitoring.gpu_driver.service import install


async def main() -> None:
    result = await install("dra", cluster_id="cluster-123", hardware="intel-xpu")
    print(result.model_dump())


asyncio.run(main())

Get deployment profiling flow map

Query the Prometheus-backed GPU idle/utilization flow-map for a specific deployment execution.

REST API

GET /api/v1/monitoring/profiling/flow-map

Request: no request body. Required query param: execution_id (str). Optional query param: cluster_id (str).

Response: returns a FlowMapResponse with component and edge metrics.

{
  "execution_id": "deploy-exec-001",
  "namespace": "llm-d-run-001",
  "prometheus_reachable": true,
  "message": null,
  "components": [
    {
      "role": "epp",
      "label": "EPP",
      "present": true,
      "instances": [
        {
          "name": "demo-epp-5d7d8c9b6f-abcde",
          "request_rate": 1.8,
          "queue_length": 0.2,
          "prefill_inflight": 0.1,
          "decode_inflight": 0.1
        }
      ],
      "request_rate": 1.8,
      "queue_length": 0.2,
      "process_time_ms": 18.4
    },
    {
      "role": "decode",
      "label": "Decode",
      "present": true,
      "instances": [
        {
          "name": "modelserver-decode-0",
          "request_rate": 1.7,
          "output_token_rate": 82.0,
          "kv_cache_usage_perc": 64.5
        }
      ],
      "request_rate": 1.7,
      "output_token_rate": 82.0,
      "kv_cache_usage_perc": 64.5
    }
  ],
  "edges": [
    {
      "id": "epp->decode",
      "source": "epp",
      "target": "decode",
      "request_rate": 1.7,
      "success_request_rate": 1.7,
      "failed_request_rate": 0.0,
      "input_token_rate": 41.0,
      "output_token_rate": 82.0
    }
  ]
}

Python API

import asyncio

from llm_d_lens.monitoring.profiling.service import build_flow_map


async def main() -> None:
    response = await build_flow_map("deploy-exec-001", "cluster-123")
    print(response.model_dump())


asyncio.run(main())

Additional accelerator endpoints exist for installation preflight, operation status, and resolved links; see the OpenAPI reference.

Additional cluster-stack endpoints exist for operation status and resolved Grafana/Prometheus links; see the OpenAPI reference.

See the full request/response schemas in the FastAPI OpenAPI schema (regenerate with npm run docs:api).

Shared monitoring behavior

Monitoring services share command JSON parsing, Service link lookup, Prometheus query transport and operation runtime mechanics. Existing Python entry points and REST request/response contracts below are unchanged; domain errors, installer phases, DAO records and verification callbacks remain with their respective accelerator or cluster-stack service.