AI Quality Engineer
XapienDue Diligence company
London, United KingdomSenior
YFM Equity Partners
D2 Fund
Deeptech Labs
Founders Factory
Software Engineering
About the role
TL;DR
Ensure the quality and reliability of AI-driven output through rigorous testing and evaluation.
- •Xapien is reinventing due diligence with cutting-edge AI.
- •This role focuses on ensuring the quality and reliability of their AI-driven output through rigorous testing and evaluation.
- •Key Responsibilities Enhance and run layered evaluation harnesses including automated checks and LLM-as-judge scoring.
- •Maintain versioned golden datasets for reproducible and auditable evaluation.
- •Track key signals for investigative output like groundedness and hallucination rate.
- •Run and improve a tiered CI model built on Python, Playwright and pytest.
- •Investigate and fix flaky tests and metrics.
- •Requirements Strong test-automation engineering with Python and modern stacks like Playwright, pytest.
- •Analytically sceptical and numerate, questioning baselines.
- •Ability to simplify complex concepts for diverse audiences.
- •Collaborative mindset, building trust across engineering teams.
- •Proven ability to learn complex skills quickly under pressure.
- •Genuine interest in AI quality and testing non-deterministic output. (Senior) Experience shipping LLM evaluations and raising quality bars.
Required skills
PythonPlaywrightPytestCI/CDLLMs
Domain expertise
deeptechai
Benefits & perks
Employee shares & equity programme, Private health insurance, Life Insurance, Unlimited holidays, Professional development fund
Tech stack
PythonPlaywrightPytestCI/CDLLMs
