Skip to main content
Enterprise

T-Head PPU

PPU Device Plugin registers T-Head PPUs, such as Zhenwu 810E, as Kubernetes resources for memory and compute sharing. The HAMi scheduler selects devices; Pods that bypass it fail allocation. This guide uses Chart 0.0.4 and application v1.0.5 to configure licensing, install the plugin, and verify a workload.

Prerequisites​

  • Kubernetes 1.20 or later, and Helm 3.
  • PPU drivers are installed on each device node, which exposes /dev/alixpu, /dev/alixpu_ctl, and /dev/alixpu_ppu{N}.
  • The HAMi Enterprise scheduler is deployed, and its resource names match this plugin. See Installation for deployment options.
  • An Enterprise license covers the target PPU devices.

Install​

kubectl label node <node-name> alibabappu=on

helm upgrade --install ppu-vgpu-device-plugin \
oci://ghcr.io/dynamia-ai/charts/ppu-vgpu-device-plugin \
--version 0.0.4 \
-n hami-system --create-namespace

Chart 0.0.4 uses image ghcr.io/dynamia-ai/ppu-device-plugin:v1.0.5. To pull the image from another registry, set image.repository and image.tag.

Verify the install:

kubectl -n hami-system get ds -l app.kubernetes.io/instance=ppu-vgpu-device-plugin
kubectl get node <node-name> -o jsonpath='{.status.allocatable.alibaba\.com/ppu}{"\n"}'

allocatable equals the physical card count multiplied by deviceSplitCount.

Uninstall:

helm uninstall ppu-vgpu-device-plugin -n hami-system

Workload resources​

ResourceDescription
alibaba.com/ppuNumber of cards
alibaba.com/ppu-memoryMemory per card, in MiB
alibaba.com/ppu-memory-percentageMemory percentage per card (1–100)
alibaba.com/gpu-coreCompute percentage per card (1–100)

Requesting only the card count reserves the whole card. ppu-memory takes precedence over ppu-memory-percentage.

Configuration​

The following values belong to the plugin chart under config.data.alibaba:

ParameterDefaultDescription
deviceSplitCount10Maximum number of containers that share one card
deviceMemoryScaling1.0Memory oversubscription factor
deviceCoreScaling1.0Compute oversubscription factor
resourceCountName and related namesalibaba.com/*Resource names; keep them identical to the scheduler

nodeSelector defaults to alibabappu: "on" and selects the nodes that run the plugin. monitor.service.nodePort defaults to 31992.

Pod annotations:

AnnotationDescription
alibaba.com/use-pputype / nouse-pputypeSelect or exclude a model
alibaba.com/use-ppuuuid / nouse-ppuuuidSelect or exclude a UUID
hami.io/gpu-scheduler-policybinpack, spread, or topology-aware

Smoke test​

Run two vLLM instances on the same PPU. Each instance requests 50% memory and 50% compute. The test checks sharing, limit injection, and inference.

caution

This guide does not provide a test image. Prepare an image that includes the PPU SDK, PyTorch, and vLLM, and make sure the cluster can pull it.

export PPU_IMAGE=<ppu-vllm-image>

cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
name: ppu-smoke
spec:
replicas: 2
selector:
matchLabels:
app: ppu-smoke
template:
metadata:
labels:
app: ppu-smoke
annotations:
hami.io/node-scheduler-policy: binpack
hami.io/gpu-scheduler-policy: binpack
spec:
containers:
- name: vllm
image: ${PPU_IMAGE}
command:
- sh
- -c
- |
pip install modelscope &&
modelscope download --model Qwen/Qwen2.5-1.5B-Instruct --local_dir /models/qwen &&
exec python3 -m vllm.entrypoints.openai.api_server \
--model /models/qwen --served-model-name qwen \
--trust-remote-code --gpu-memory-utilization 0.8
readinessProbe:
httpGet:
path: /health
port: 8000
periodSeconds: 10
failureThreshold: 90
resources:
limits:
cpu: "8"
memory: 16Gi
alibaba.com/ppu: 1
alibaba.com/ppu-memory-percentage: 50
alibaba.com/gpu-core: 50
volumeMounts:
- name: models
mountPath: /models
volumes:
- name: models
emptyDir: {}
EOF

kubectl rollout status deployment/ppu-smoke --timeout=30m

Check the result:

for p in $(kubectl get pod -l app=ppu-smoke -o name); do
echo "== $p"
kubectl get $p -o jsonpath='{.metadata.annotations.hami\.io/vppu-devices-allocated}{"\n"}'
kubectl exec $p -- sh -c 'env | grep CUDA_DEVICE_'
kubectl exec $p -- python3 -c 'import torch; print("total", torch.cuda.mem_get_info()[1] >> 20, "MiB")'
kubectl exec $p -- curl -s localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model": "qwen", "messages": [{"role": "user", "content": "Hello"}]}'
echo
done

Pass conditions:

CheckExpected
Allocation annotationBoth Pods use the same device UUID
CUDA_DEVICE_MEMORY_LIMIT_0About 50% of the full card memory
CUDA_DEVICE_SM_LIMIT50
totalMatches CUDA_DEVICE_MEMORY_LIMIT_0
Inference responsechoices[0].message.content is non-empty

Clean up:

kubectl delete deployment ppu-smoke

Troubleshooting​

kubectl -n hami-system logs -l app.kubernetes.io/component=device-plugin --tail=200
SymptomCheck
No plugin PodNode label alibabappu=on
No alibaba.com/ppu resourceDriver, /dev/alixpu*, and plugin logs
device request not foundThe Pod schedulerName
License errorDevice UUIDs and expiry in node annotation hami.io/alibaba-license, license coverage for HGML-visible devices, workload timezone, and whether workload processes were restarted after renewal

Back to standalone device plugins