Staff Software Engineer, ML Performance & Systems
Fal.aiGenerative Media company
San Francisco, United StatesLead
Andreessen Horowitz
Meritech Capital Partners
Salesforce Ventures
Kindred Ventures
Notable Capital
Shopify Ventures
Software Engineering
About the role
TL;DR
Designs and implements high-performance ML serving and inference infrastructure.
- •Design and implement high-performance model serving and inference infrastructure to maximize throughput and minimize latency for generative media models, and build profiling and monitoring tools.
- •Key Responsibilities Design novel model serving architectures atop in-house inference engine.
- •Maximize throughput while minimizing latency and resource usage.
- •Develop performance monitoring and profiling tools.
- •Collaborate with Applied ML team and customers to optimize workloads.
- •Requirements Strong systems programming skills and bottleneck analysis.
- •Deep knowledge of ML infrastructure (model compilation, quantization, serving).
- •Familiarity with Nvidia hardware and lower-level accelerator programming.
- •Proficient in or willing to learn Triton and accelerator toolchains.
Required skills
PythonC++BashMLflowKubeflowSageMakerVertex AIONNXJAX
Nice-to-have skills
Computer VisionNLPLLMs
Domain expertise
aideveloper-tools
Tech stack
PythonC++Bash
