LensThe ultimate control plane for AI inference: single-click deployment, automated benchmarking, and full-stack cluster monitoring.
Highlights
Deployments run on llm-d: prefill/decode disaggregation, KV-cache-aware routing, KV cache offloading to CPU/SSD tiers, and latency-predicted routing raise throughput and cut tail latency on the same GPUs.
Plan, approve and execute inference deployments across Kubernetes clusters from a single UI, with agentic, workload-guided placement.
Design benchmark experiments, run them against live deployments, and analyze and export the results without hand-written scripts.
Replay registered request traces against an inference endpoint to validate behavior and capacity before real traffic arrives.
Publish cluster deployments as OpenAI-compatible model services behind the cluster's shared Gateway (HTTPRoute → InferencePool → EPP), with per-model selection policies.
Role-based access control, per-cluster scoping and audit logging apply across every page, from clusters to model services.
Bootstrap and maintain the cluster monitoring stack (Prometheus/Grafana) to watch GPU utilization and deployment health.
Architecture
A single Lens server manages any number of Kubernetes clusters, on-premises or in the cloud, from one pane of glass: the console talks to an application layer over same-origin API routes, and that layer forwards requests to a control plane that owns the domain services, persists state to the database, and drives clusters, storage and the model-service data plane directly. Access control and audit logging are enforced in the control plane, so every page — deployments, evaluation, simulation and model services alike — respects the same per-cluster permissions.
Where to go next
Run the full stack locally.
Page-by-page guides to every Lens capability.
How the UI, BFF and Python backend fit together.
Publish a gateway-served, OpenAI-compatible model endpoint.
Fixes for the failure modes you're most likely to hit.
Guide-based performance evaluation.
Answers to common questions about roles and scope.