~/ shubham tatvamasi
GPU sharing, LLM serving and Kubernetes platforms, from a single node to fleets of clusters, engineered so models run fast, GPUs never sit idle and nothing goes dark.
Most people meet AI through a chat box. I live one layer down, where tokens become GPU cycles, GPU cycles become heat, and a single misrouted request can idle a rack. My work is making that layer boring in the best way: declarative, observable, self-healing, and secure enough that nobody has to think about it.
Making the whole stack, from the GPU die to the API gateway, behave like one calm, observable, self-healing system.
One control plane for models served through KServe, NVIDIA Dynamo, NIM, llm-d and Ray Serve, on engines from vLLM and SGLang to TensorRT-LLM, Triton and llama.cpp. Model-aware routing through Envoy AI Gateway and the Gateway API Inference Extension, keys and budgets via LiteLLM, GuideLLM and AIPerf benchmarks against TTFT and inter-token SLOs, and autoscaling that adds replicas before it adds GPUs.
Telling allocated apart from actually computing. DRA with MIG, MPS, time-slicing and fractional GPUs, Kueue and KAI queues and quotas, NVIDIA and AMD telemetry that catches idle, faulty (XID, ECC) and oversubscribed GPUs, and chargeback that puts a price on every GPU-hour.
Controllers, CRDs and operators in Go, product UI in React and TypeScript, Helm for everything in between. EKS, AKS and RKE2, multi-tenant isolation with RBAC and OIDC, and Rook-Ceph storage that heals itself.
Flux CD reconciling 80+ clusters, with Argo CD, GitHub Actions, Terraform and Ansible across AWS, Azure, GCP and OpenStack. If it isn't in Git, it doesn't exist.
ML-KEM key exchange, ML-DSA signatures and X.509 certificate lifecycles, automated as Kubernetes-native resources.
WireGuard tunnels from 900+ devices behind carrier-grade NAT to OpenStack and public clouds, with Gateway API at the front door. Fleets that stay reachable, observable and patched no matter where they live.
A short operating manual for the systems I build, and the way I build them.
Every cluster, model and secret should be reproducible from a commit. If a human has to remember it, it will be forgotten.
Allocated isn’t utilised. Ready isn’t responsive. I chase the metric that tells the truth, not the one that looks good on a dashboard.
Replicas before nodes, caches before GPUs, routing before hardware. Efficiency is a design decision, not a cost report.
Harvest-now, decrypt-later is real. Post-quantum crypto belongs in today’s pipelines, not tomorrow’s roadmap.
The best infrastructure ideas get sharper in public. Ship it, document it, let others break it.
A tiny, very real-feeling terminal. Type help to look around, or try nvidia-smi.