Kubernetes GPU Node Autoscaling & Pod Density Calculator
To maximize GPU density, ensure pod GPU requests equal limits (as required by NVIDIA device plugin). A node can host N pods where N ≤ total node GPU capacity. The scheduler calculates effective GPU requests as the sum of all non-init container GPU requests. If total pod requests exceed node capacity, the cluster autoscaler triggers scale-out. GPU requests and limits must be equal for nvidia.com/gpu resources.
Interactive Parameters & Live Verdict
Rule Diagnostics & Recommendations
Engineering Formula & Verification Notes
Calculations assume a 1:1 mapping of GPU requests to limits, as required by the NVIDIA device plugin. Node autoscaling is triggered when pod requests cannot be satisfied by existing allocatable capacity.
Worked Engineering Example
For a node with 4 GPUs, if you deploy 4 pods each requesting 1 GPU, you achieve 100% utilization. If you attempt to deploy 5 pods, the scheduler will mark the 5th pod as Pending until a new node is provisioned.
- GPU requests must equal limits.
- Autoscalers trigger based on requests, not actual usage.
- Fragmented requests lead to stranded GPU capacity.
Common Sizing & Compatibility Mistakes
Engineers often overlook the relationship between GPU requests and HPA calibration.
- Setting GPU requests lower than limits (invalid for NVIDIA plugin).
- Ignoring CPU/Memory overhead which can cause OOMKills before GPU limits are reached.
- Assuming autoscalers use actual usage metrics for provisioning.
GPU Node Scaling Benchmarks
| Node Type | GPU Capacity | Recommended Pod Density |
|---|---|---|
| Entry | 1 | 1 Pod |
| Standard | 4 | 4 Pods |
| High-Perf | 8 | 8 Pods |
Frequently Asked Technical Questions
Why must GPU requests equal limits?
The NVIDIA device plugin requires requests to equal limits to ensure deterministic scheduling and hardware isolation.
Does the autoscaler look at actual GPU usage?
No, the cluster autoscaler only evaluates resource requests against node allocatable capacity.
How do I prevent pod pending states?
Ensure your total pod GPU requests do not exceed the sum of all node capacities in your autoscaling group.