Skip to main content

Publish a model service

Start with a ready llm-d deployment. Use an account with model-service management access to publish it.

1. Add the model​

  1. Open Model services and click Add model to open the right-side drawer.
  2. Select the deployment's Cluster.
  3. Select a ready deployment in Provider. Deployments already added to a model are excluded.
  4. Under Model name mapping, confirm the read-only Provider model name.
  5. Below Exposed to users as, select an existing Public model name from the current cluster's dropdown or type a new name such as qwen-demo, then click Add. Use the question-mark icons beside field names for help.

To add another deployment to an existing model, enter that model's public name in the same cluster. Lens reuses its service and HTTPRoute, preserves existing weights, and adds the new provider at weight 1. Adjust its share in the three-line menu beside the model name → Traffic distribution. The provider model names must match.

2. Check the published model​

The model service and its model server are created together. In Service flow, inspect the status dots for the service.

If there is no eligible deployment, confirm it is in the same cluster and was created with an llm-d Guide that provides an InferencePool.

3. Check the gateway​

  1. Find the cluster's Gateway node in Service flow.
  2. If it is not installed, open its menu and click Install.
  3. If its settings need to be reapplied, click Reconcile.
  4. Read the operation log and wait for the gateway and service components to become healthy.

If the address is wrong, update it under Clusters → Edit cluster → Gateway host/IP. For a failed installation, see Troubleshooting.

4. Create an API key​

  1. Open API keys → New key.
  2. Enter a key name, such as first-call, and click Create.
  3. Click Copy and save to copy the full key to the clipboard and close the dialog.

The full key is shown only once. Do not put it in source code or share it with another user.

5. Send a request​

  1. Open My services and find qwen-demo.
  2. Click cURL, then Copy beside the chat request example.
  3. Paste it into your terminal. Replace lens-mk-... with your key, keeping the other headers and URL.
  4. Run the request and confirm it returns a model reply.

You can now benchmark this service. For application code, use the examples in Integrations.