Enterprise

This guide explains how to deploy HAMi Enterprise with Helm on a Kubernetes cluster that can access the required Chart and image registries. It does not cover air-gapped installation.

💡 Installation does not equal activation. The components can run after Helm installation, but GPU virtualization and scheduling require license activation before they can be used.

Installation itself does not require a license. Complete the software deployment first, then apply for and import the license.

Without activation, vGPU partitioning and scheduling are unavailable, and functional verification will fail.

Why Choose Online Installation

Online installation is intended for teams that can operate Kubernetes, Helm, and image registries and need greater configuration flexibility. Component versions, deployment order, and values can be managed according to the organization infrastructure and change process.

For network-isolated environments, follow the HAMi Enterprise Air-Gap Deployment Guide. This guide covers online Helm installation where the required Chart and image registries are accessible.

Deployment Scope and Prerequisites

Component Selection

Component	When to Install	Notes
HAMi Enterprise	Every cluster	Core scheduler and device plugin. Required.
NVIDIA GPU Operator	Optional when NVIDIA nodes need GPU Operator to manage their GPU software stack; skip it if the nodes are already prepared	The default GPU Operator device plugin must not run together with the HAMi device plugin.
Ascend device plugin	Ascend nodes are used	Install after HAMi and reuse the HAMi device configuration ConfigMap.
Prometheus monitoring stack	HAMi or GPU metrics are required	Do not install another stack if the cluster already has a compatible monitoring system.

Prerequisite Checks

Use a Kubernetes version supported by the selected Charts and provide an available default StorageClass. The administration host needs kubectl and Helm.
Prepare the NVIDIA driver and NVIDIA Container Toolkit on NVIDIA nodes.
Prepare the vendor driver, firmware and runtime on Ascend nodes.
Disable NVIDIA GPU Operator’s default device plugin when HAMi supplies the NVIDIA device plugin. Set devicePlugin.enabled=false in the GPU Operator Helm values.

kubectl cluster-info
kubectl version
kubectl get nodes -o wide
kubectl get storageclass

Online Installation

Before running the commands below, confirm that the current kubeconfig context points to the target cluster. Keep the values file for each Chart under version control. Contact Dynamia.ai technical support for access to commercial Charts, mainland China image registries, and production values.

Install HAMi Enterprise

helm install hami \
  oci://ghcr.io/dynamia-ai/charts/hami-enterprise \
  --version 2.10.0-r2 \
  --namespace hami-system \
  --create-namespace

See the HAMi Helm Chart Values Reference for the complete configuration.

For non-NVIDIA devices, enable the corresponding vendor under devices in the HAMi values. Ascend nodes require at least:

devices:
  ascend:
    enabled: true
    hamiVnpuCore: true

Install NVIDIA GPU Operator (NVIDIA Nodes Only)

NVIDIA GPU Operator is optional. If the nodes already have a compatible driver, NVIDIA Container Toolkit, and runtime configuration, you can skip it. See the HAMi NVIDIA GPU node prerequisites for the node setup and GPU Operator values. When installing or reusing GPU Operator, disable its bundled device plugin (devicePlugin.enabled=false) and CDI (cdi.enabled=false). This HAMi Enterprise deployment uses scheduler.useDownward, which is incompatible with CDI. Assess running GPU workloads before changing CDI settings on an existing cluster.

helm repo add nvidia https://helm.ngc.nvidia.com/nvidia && helm repo update

helm install --wait --generate-name \
  -n gpu-operator --create-namespace \
  nvidia/gpu-operator \
  --set devicePlugin.enabled=false \
  --set dcgmExporter.serviceMonitor.enabled=true \
  --set cdi.enabled=false \
  --version=v25.10.1

Install the Ascend Device Plugin (Ascend Nodes Only)

Complete the HAMi installation before this step. The following configuration reuses hami-scheduler-device so that multiple Charts do not manage the same device configuration.

nameOverride: "ascend-device-plugin"
fullnameOverride: "ascend-device-plugin"

image:
  repository: dynamia-ai-registry.cn-hangzhou.cr.aliyuncs.com/public/dynamia-ai/ascend-device-plugin
  tag: "v1.4.1"
  pullPolicy: IfNotPresent

config:
  create: false
  existingDeviceConfigMapName: hami-scheduler-device

helm install ascend-device-plugin \
  oci://ghcr.io/dynamia-ai/charts/ascend-device-plugin \
  --version 0.2.1 \
  --namespace hami-system \
  -f ascend-device-plugin.yaml

Enable GPU Nodes

The HAMi NVIDIA device plugin starts on nodes labeled gpu=on. Apply this label to NVIDIA nodes that HAMi should manage. Ascend components use their own nodeSelector; gpu=on is not their selection rule.

kubectl label nodes <node-name> gpu=on

Install the Monitoring Stack (When Needed)

Monitoring is optional. Install kube-prometheus-stack only if metrics are needed and the cluster does not already have a compatible Prometheus or VictoriaMetrics installation. Otherwise, connect the existing monitoring system as described below.

helm install prometheus \
  oci://ghcr.io/prometheus-community/charts/kube-prometheus-stack \
  --version 72.3.0 \
  --namespace monitoring \
  --create-namespace \
  --set alertmanager.enabled=false \
  --set grafana.enabled=false

Post-Installation Checks

Core Components

kubectl -n hami-system get pods
kubectl -n gpu-operator get pods
kubectl -n monitoring get pods

A Namespace for an optional component may not exist if that component was not installed. Pods for installed components should be Running or Completed.

Nodes and Monitoring

kubectl describe node <node-name>
kubectl api-resources --api-group=monitoring.coreos.com

For Prometheus, match ServiceMonitor.metadata.labels to Prometheus.spec.serviceMonitorSelector and check the namespace selector. For VictoriaMetrics Operator, VMAgent.spec.serviceScrapeSelector selects VMServiceScrape.metadata.labels; also check serviceScrapeNamespaceSelector and any ServiceMonitor conversion.

Check that the relevant scrape targets report up=1, then query the current NVIDIA metrics and confirm that they return data:

DCGM_FI_DEV_GPU_UTIL
hami_host_gpu_utilization_ratio
hami_gpu_core_allocated_ratio

License Application and Activation

Obtain License Request Information

Collect the license request information once the HAMi core is ready. You do not need to wait for optional GPU Operator, Ascend, or monitoring components. The administration host needs kubectl and jq. Use either method below.

Download and Run the Script

curl -fsSLO https://public.hami.run/collect-hami-license-info.sh
bash collect-hami-license-info.sh

Run a Local Copy

bash collect-hami-license-info.sh

The script outputs JSON in the following format:

{
  "esn": "96565d61-986a-4918-aafb-448ff6e3746b",
  "deviceInstances": [
    {
      "uuid": "GPU-ceee905d-48ac-93de-a81b-17c00e1e5e02",
      "deviceType": "NVIDIA A10"
    }
  ]
}

Send the JSON output to Dynamia.ai sales or technical support to obtain the license.

Activate HAMi Enterprise

After receiving the license file, save it on an administration host that can access the target cluster, then create the license Secret:

kubectl create secret generic hami-license \
  --from-file=license=/path/to/license-file \
  -n hami-system

kubectl label secret hami-license \
  hami.io/license="true" \
  -n hami-system

kubectl get secret hami-license -n hami-system
kubectl get events --field-selector involvedObject.name=hami-license -n hami-system

A LicenseValid event indicates that license validation succeeded. For NVIDIA nodes, you can also inspect license registration:

kubectl get nodes -o custom-columns='NODE:.metadata.name,LICENSE:.metadata.annotations.hami\.io/nvidia-license'

Post-Activation Verification

kubectl -n hami-system get pods
kubectl describe node <gpu-node>
kubectl get events --field-selector involvedObject.name=hami-license -n hami-system
kubectl get nodes -o custom-columns='NODE:.metadata.name,LICENSE:.metadata.annotations.hami\.io/nvidia-license'

A LicenseValid event indicates that license validation succeeded. Confirm that the selected component Pods are Running or Completed and that accelerator resources are registered on the managed nodes.

Online Sample Workload Verification

Before running this example, confirm that the target cluster can pull nvidia/cuda:12.4.0-base-ubuntu22.04. If the cluster uses a private registry, mirror the image first and update the image reference in the example.

kubectl delete pod hami-smoke --ignore-not-found

kubectl apply -f - <<'EOF'
apiVersion: v1
kind: Pod
metadata:
  name: hami-smoke
spec:
  restartPolicy: Never
  containers:
    - name: cuda
      image: nvidia/cuda:12.4.0-base-ubuntu22.04
      command: ["sh", "-c", "nvidia-smi && sleep 30"]
      resources:
        limits:
          nvidia.com/gpu: 1
          nvidia.com/gpumem: 2000
EOF

kubectl wait --for=condition=Ready pod/hami-smoke --timeout=180s
kubectl logs hami-smoke

If the Pod reaches Ready and the logs show GPU information, HAMi has scheduled the GPU and the NVIDIA runtime is available inside the container. Delete the test Pod after verification.

kubectl delete pod hami-smoke

Common Issues

Symptom	Checks and Resolution
`hami-device-plugin` is not running	Confirm that the node has the `gpu=on` label, then check `kubectl -n hami-system get pods`.
`hami-device-plugin` repeatedly restarts	Check whether NVIDIA GPU Operator still enables its default device plugin. Set devicePlugin.enabled=false in its Helm values.
Image pull fails	Inspect Pod events, the image reference and registry access. Verify the node can reach and authenticate to the configured image registry.
HAMi metrics are unavailable	Check whether the Prometheus or VictoriaMetrics selector matches the labels on the monitoring object.
A workload remains `Pending`	Check license activation, the `gpu=on` node label, available accelerator resources, and events from `kubectl describe pod`.

Support

Email: info@dynamia.ai
Sales / technical support: 400-026-7800
Customers with active commercial agreements should submit support requests through their dedicated support channel.

Deployment Scope and Prerequisites

Component Selection​

Prerequisite Checks​

Online Installation

Install HAMi Enterprise​

Install NVIDIA GPU Operator (NVIDIA Nodes Only)​

Install the Ascend Device Plugin (Ascend Nodes Only)​

Enable GPU Nodes​

Install the Monitoring Stack (When Needed)​