About the Role
Inference Platform is Confidential (Registered Users Only)'s multi-tier orchestration system for microservices, powering AI Inference across Confidential (Registered Users Only). Through a simple developer API, we deliver a "deploy and forget" runtime, hiding the complexity of scheduling, scaling, and reliably running services across our on-prem and multi-cloud footprint at global scale. As Senior Product Manager, Inference Platform, you'll take the helm at a defining moment - leading the charge as we scale the platform to become the default runtime for critical Confidential (Registered Users Only) services worldwide.
You Will
- Own Inference Platform end-to-end - set the multi-year vision for how Confidential (Registered Users Only) engineers deploy, run, and scale AI and other services across our Core and Edge Datacenters, and cloud.
- Power Confidential (Registered Users Only)'s AI future - build the platform that brings frontier models and next-gen AI workloads to life, with the primitives, scheduling guarantees, and resource classes AI teams need to move fast.
- Evolve the platform's technical core - define how Confidential (Registered Users Only) prioritizes, isolates, and preempts workloads across shared fleet, and shape the entire system across global control plane, multi-region orchestration, distributed systems primitives, capacity/state management, DR/failover, and operator-based Kubernetes extensions.
- Make developer experience the reason teams choose Jobs Platform - by delivering integrated tooling (CI/CD, telemetry, profiling, tracing) and managing robust SLOs via real-time fleet signals (health, queue depth, scheduling efficiency)—all while engineering a frictionless, effortless onboarding experience for teams.
- Architect intelligent, predictive scaling - advance Multi-Cloud Bursting from reactive to proactive with AI-powered prediction, and partner with Data Science and Finance on demand forecasting, cost-to-serve analytics, and AI-driven resource optimization.
- Own the trade-off calls across efficiency, latency, cost-to-serve and time-to-ship, while maintaining reliability.
- Lead cross-functionally, acting as the connective tissue between Jobs Platform Users, and Jobs Platform, AI, and infrastructure engineers.
You Have
- 7+ years of product management experience building resource orchestration, workload management platforms or large-scale distributed systems
- Track record of building developer-facing platforms that engineers genuinely love to use. You have a deep understanding of modern software development lifecycle and developer tooling with a history of abstracting complex infrastructure into clean, maintainable platform schemas.
- Built and operated Kubernetes and enterprise-grade service mesh at scale, and you understand the developer pain points around control plane mechanics like etcd scaling and API server bottlenecks. You've shipped custom Operators, Controllers, and CRDs, going beyond vanilla Kubernetes.
- Familiarity with GPU/accelerator architecture and the complex scheduling challenges that come with it.
- Background in AI model development, training, inference
- A builder mindset - you are passionate about prototyping, evolving products through rapid iteration, and leveraging AI for ideation and unlocking value for users.
- Experience building workload management or resource orchestration platforms in AWS, Google Cloud Platform (GCP), Azure or other cloud providers. (preferred)
- Kernel-level experience or familiarity with custom kernel drivers. (preferred)
- Experience building agentic systems for workload management or infrastructure. (preferred)