Software Engineer, Cloud
OllamaDeveloper Tools company
Palo Alto, United StatesSenior
Benchmark
Y Combinator
8VC
Theory Ventures
Garage Capital
GTMfund
Software Engineering
About the role
TL;DR
Build and scale Ollama's cloud inference platform for developers.
- •You'll build Ollama’s cloud, a scalable inference platform that lets developers run large, capable open models in their workflow.
- •Key Responsibilities Build and scale the inference platform that serves every request from ollama.com.
- •Design the routing and capacity layer that places workloads across GPUs and regions for cost, latency, and availability.
- •Own multi-tenant infrastructure: isolation, quotas, usage metering, billing, and Pro/Max/team/enterprise tiering.
- •Build the reliability, observability, and cost controls for our team and customers.
- •Requirements Deep experience with high-throughput, low-latency distributed systems
- •inference serving, traffic routing, real-time data pipelines, or large-scale APIs.
- •Comfortable with cost/performance tradeoffs at scale and have owned a production service end-to-end.
- •Experience with Kubernetes, GPU scheduling, or inference infrastructure.
- •Think in terms of reliability, SLOs, and honest capacity planning.
- •Bonus: experience building an inference platform, GPU fleet management, or billing/metering for an AI service.
Required skills
Kubernetes
Tech stack
Kubernetes