Hiren Patel

Software Engineer, AI Infrastructure · Knowledge Distillation & AI Production Systems

• Leading fleet-scale GPU inference migration for knowledge distillation models in an 84% over-subscribed accelerator fleet: $4M/yr realized, $12M/yr in flight, $50M/yr projected at full scale.
• Fit models that used to need 4+ RDMA-equipped hosts onto cheaper non-RDMA hardware by combining CPU and SSD embedding offloading with multi-host partitioning, across NVIDIA H100/A100 and AMD MI350X/MI300X.
• Designed and shipped a gang-level canary and automated-rollback system for KD inference releases, preventing 21 bad-package rollouts across 14 high-revenue models.
• Own release reliability for 4,000+ production AI models. Raised release config-test coverage from 0% to 90%+ with simulator-based validation.

Software Engineer, Systems · New York

• Tech lead for the enterprise SaaS-onboarding platform serving 1.6M Amazon employees, with event-driven user, group and permission sync handling 50M+ API calls a day.
• Eliminated $25M+/yr in operational costs by automating hardware-failure detection and routing.

Software Engineer · Fremont

• Designed and shipped TeamCI, a centralized deployment platform that replaced 1,000+ per-repo Jenkinsfiles across 4 regions, with self-service deploys and one-click rollback. Cut the deployment failure rate by 90%.