Senior Data Scientist, AI Scoring & Evaluation
WorkeraSkills Intelligence company
On-siteSenior
New Enterprise Associates
AI Fund
AIX Ventures
IQT
Jump Capital
Owl Ventures
Data & AINew
About the role
TL;DR
Own AI scoring and evaluation mechanisms, ensuring accuracy and defensibility.
- •Workera is seeking a Senior Data Scientist to own the AI scoring and evaluation mechanisms.
- •You will build trust by ensuring accuracy, fairness, and defensibility in the measurement systems that power customer decisions about talent.
- •Key Responsibilities Own scoring quality end to end: evaluator design, rubric anchoring, calibration, and accuracy against expert benchmarks.
- •Build and run the continuous evaluation harness: gold sets, bias diagnostics, drift detection, and pre-release gates.
- •Define and publish assessment quality KPIs on company-wide dashboards.
- •Lead analytics investigations behind assessment decisions and ship measurement improvements.
- •Requirements 4+ years of independent production data science work, ideally in an AI startup.
- •Proficiency in Python and SQL for analysis.
- •Real statistical and analytical depth to design and execute studies quickly.
- •Experience working with LLM-based systems in production, including prompt design and output evaluation.
- •Ability to explain quantitative results to non-technical audiences.
Required skills
PythonSQLLLMsNLP
Nice-to-have skills
OpenAI APILangChainscikit-learn
Domain expertise
edtechaiproductivity
Tech stack
PythonSQLLLMsOpenAI APILangChainNLP
