Member of Technical Staff - Research Software Engineer - Safety Evaluations Infrastructure
Reflection AIDeveloper Tools company
On-siteLead
Lightspeed Venture Partners
CRV
Endeavor Catalyst
Amex Ventures
Data & AI
About the role
TL;DR
Build secure infrastructure for sensitive AI model evaluations.
- •As a Research Software Engineer on the Safety team, you will design, build, and own the infrastructure used to run our most sensitive model evaluations
- •including evaluations in CBRN (chemical, biological, radiological, and nuclear), child safety, and other dangerous-capability domains.
- •These evaluations inform release decisions for our open models, so the systems you build must be secure, isolated, reproducible, and trustworthy under scrutiny.
- •Key Responsibilities Design and build secure, sandboxed infrastructure for running sensitive model evaluations.
- •Build controlled data pipelines and storage for sensitive evaluation material.
- •Partner with safety researchers and domain experts to translate evaluation designs into reliable systems.
- •Build eval-orchestration tooling and harnesses for high-throughput evaluations.
- •Implement guardrails, monitoring, and compartmentalization for sensitive work.
- •Requirements Strong software engineering skills, particularly in Python, with a track record of building reliable, scalable infrastructure.
- •Experience building sandboxed, isolated, or otherwise security-sensitive execution environments.
- •Solid grounding in security engineering fundamentals.
- •Experience building data pipelines and handling sensitive or restricted data.
- •Ability to own entire problems end-to-end.
Required skills
PythonDockerKubernetesCI/CDLinuxGitLLMs
Nice-to-have skills
AWSGoogle CloudAzureOpenAI API
Domain expertise
aideveloper-tools
Benefits & perks
Stock options, Health & wellness, Meals, Parental leave, Vacation days, Sponsorship support, Team building
Tech stack
PythonDockerKubernetesAWSGoogle CloudAzureCI/CDGitLinuxREST APIgRPCOpenAI APILLMs