Senior Machine Learning Engineer, Inference
Inworld AIGenerative AI company
Mountain View, United States$250,000 - $300,000+Senior
Lightspeed Venture Partners
Intel Capital
Founders Fund
Kleiner Perkins
Section 32
BITKRAFT Ventures
Data & AINew
About the role
TL;DR
Senior ML Engineer to optimize real-time AI inference systems.
- •About Inworld Inworld is a research lab and inference provider focused on realtime AI for consumer-facing applications.
- •We build first-party speech models, serve LLMs, and run the inference behind modular APIs designed for high-volume, realtime workloads.
- •Hundreds of millions of users interact with Inworld powered apps every day and we serve over 10 trillion LLM tokens per month.
- •Our models and infrastructure support consumer applications across companions, healthcare, fitness, education, media, and more.
- •Our work spans model research, realtime inference, large-scale serving infrastructure, and the APIs developers use to bring these capabilities into production.
- •We’ve raised more than $125M from Lightspeed Venture Partners, Section 32, Kleiner Perkins, Microsoft’s M12 venture fund, Founders Fund, Meta, Stanford, and others.
- •Our technology has powered experiences from companies including NVIDIA, Microsoft Xbox, Niantic, Logitech Streamlabs, Wishroll, Little Umbrella, and Bible Chat.
- •Inworld has also been recognized by CB Insights as one of the 100 most promising AI companies globally and named one of LinkedIn’s Top 10 Startups in the USA.
- •Who We're Looking For A year ago, reliably working agentic systems and sub-second multimodal inference at scale barely existed.
- •Nobody has a decade of experience here.
- •So we're not screening for a resume template
- •we're looking for strong people from varied backgrounds who learn fast, thrive in ambiguity, and can show us what they've built, broken, and understood.
- •Experience We Find Useful You don't need all of this.
- •But you need enough to make a case.
- •Inference Optimization.
- •Deep understanding of modern serving frameworks and techniques like vLLM or TRT-LLM.
- •Model Acceleration.
- •Hands-on experience with quantization, distillation, caching strategies , continuous batching, paged attention, and speculative decoding.
- •High-Performance Systems.
- •Proficiency in C++, CUDA, Rust, or highly optimized Python.
- •You know how to profile code and squeeze every ounce of performance out of NVIDIA GPUs.
- •Distributed Systems & Scaling.
- •Experience with Kubernetes, Ray, custom load balancing, multi-GPU/multi-node inference, and reliably handling thousands of concurrent connections.
- •Non-trivial systems programming projects, open-source contributions to major inference engines, or deep-dive technical write-ups.
- •Full-cycle ownership.
- •You can take a model from the research team, containerize it, optimize its serving, and ensure it runs reliably in production.
- •PhD in CS, Physics, Math, or equivalent practical experience building backend or ML systems.
- •Who Thrives Here You don’t need a roadmap to start walking; you’re comfortable picking a direction and building the map as you go.
- •You believe engineering isn't finished until it’s shipped and stable.
- •You have a bias for impact over purely theoretical optimizations.
- •You don't just ship code; you obsess over the why.
- •You’re the first to question an architecture if you think there’s a better way to solve the core latency or throughput problem.
- •You aren't satisfied with "the PM said so." You thrive on deep context and want to understand the fundamental logic behind every decision we make.
- •What Working Here Is Like We hand you unclear problems and expect you to make them clear.
- •We value engineers who say "I don't know yet" and then design the benchmark or prototype that finds out.
- •We treat performance, latency, and reliability as first-class product features, not a box to check before launch.
- •Impact comes before everything else, though we support sharing work and open-source contributions that move the field forward.
- •Your work should be visible.
- •Flat structure, fast iterations, minimal process theater.
- •We believe in the power of in-person collaboration to solve the hardest problems and foster a strong team culture.
- •We offer relocation assistance and look forward to you joining us in our Mountain View office.
- •The base salary range for this full-time position is $250,000
- •$300,000+ bonus + equity + benefits.
- •Inworld Jobs Privacy
Required skills
PythonC++RustKubernetesLLMs
Domain expertise
ai
Benefits & perks
bonus, equity, benefits
Tech stack
PythonC++RustKubernetesLLMs
