prod-inference-us
running
us-east-1 • NVIDIA H100 SXM • Created 10/5/2026
GPU Utilization
73%
Memory Util.
61%
VRAM Usage
468 / 640 GB
Queue Depth
12
Node Count
2
Monthly Cost
$28,400.00
Network Configuration
VPC CIDR
10.0.0.0/16
Subnet
10.0.1.0/24
Private Endpoint
https://prod-us.osi-cloud.internal:8443
Scaling Configuration
Total GPUs
8
Total Nodes
2
Deployed Models
| Name | Model | Replicas | GPU Alloc. | Latency (P50/P99) | RPM | Tokens Today | Status |
|---|---|---|---|---|---|---|---|
| deepseek-v4-pro-prod | DeepSeek V4 Pro | 3 | 4 | 142ms / 387ms | 1.2K | 48.7M | active |
| kimi-k3-longctx | Kimi K3 (1M Context) | 2 | 2 | 310ms / 890ms | 320 | 12.4M | active |