Software Engineer, Runtime
OllamaDeveloper Tools company
Palo Alto, United StatesSenior
Benchmark
Y Combinator
8VC
Theory Ventures
Garage Capital
GTMfund
Software Engineering
About the role
TL;DR
Develop and optimize the runtime for open models on various hardware.
- •You'll work on the heart of Ollama
- •the local runtime that runs open models on developers' own machines.
- •It loads models, manages memory, drives GPU acceleration across various hardware, and makes it feel instant.
- •Key Responsibilities Make open models run fast and reliably on consumer and enterprise hardware.
- •Own pieces of the runtime: model loading, scheduling, memory management, quantization, GPU hardware backends.
- •Integrate new model architectures and quantization formats so the latest open models work on day one.
- •Requirements Strong systems fundamentals and experience in Go, C, or C++.
- •Experience with GPU compute, inference, game engines, databases, OS, or networking.
- •Comfortable shipping to millions of users and handling various hardware and OS combinations.
Required skills
GoC++
Tech stack
GoCC++Linux