Staff Applied AI Inference Engineer
CrusoeAI Infrastructure company
Denver, United States$185,000 - $225,000Lead
Valor Equity Partners
Founders Fund
Bain Capital Ventures
Fidelity Investments
G2 Venture Partners
Mubadala Capital
Data & AINew
About the role
TL;DR
Optimize large language models for faster, cheaper, and more reliable production inference.
- •You will spend your time making large language models run faster, cheaper, and more reliably in production.
- •This is core systems and performance work on some of the most demanding models in use today.
- •Key Responsibilities Bring current inference techniques into production and refine them.
- •Design and optimize serving architectures, including prefill and decode disaggregation, request routing, and related approaches.
- •Work down into the serving stack, from frameworks like vLLM and SGLang to the CUDA kernels underneath, profiling and running in-depth analysis to find and fix performance problems.
- •Adapt and scale optimization methods across many kinds of ML models, with an emphasis on large language models.
- •Profile and tune deployments against clear targets for latency, throughput, and cost, and keep them dependable under real traffic.
- •Requirements Hands-on experience shipping code in production with one or more general-purpose languages, such as Python or C++, with a strong preference for Python.
- •Familiarity with methods for optimizing LLMs for high throughput / low latency inference.
- •Comfort with modern LLM serving frameworks such as vLLM or SGLang, and with profiling and analyzing performance down to the kernel level.
- •A firm grasp of how GPUs are built and how they behave.
- •Clear interest and hands-on experience with large language models.
Required skills
PythonC++LLMsDockerKubernetes
Nice-to-have skills
MLflowKubeflowSageMakerVertex AIONNXJAX
Domain expertise
aideveloper-tools
Benefits & perks
Competitive compensation and equity packages, Restricted Stock Units, Paid time off, paid holidays & leave of absence programs, Comprehensive health, dental & vision insurance, Employer contributions to HSA account, Paid parental leave, Paid life insurance, short-term and long-term disability, Professional development & tuition reimbursement, Mental health & wellness support, Commuter benefits (parking & transit), Cell phone stipend, 401(k) Retirement plan with company match up to 4% of salary, Volunteer time off, Global travel insurance & emergency assistance, Daily meals allowance, Additional perks & programs specific to location
Tech stack
PythonC++DockerKubernetes