AI Project Reviewer (CONTRACT)
MasterClassOnline Learning company
RemoteMid
IVP
New Enterprise Associates
Baillie Gifford
Fidelity Management & Research Company
Owl Ventures
Javelin Venture Partners
Data & AI
About the role
TL;DR
Review learner submissions against rubrics and provide feedback for a new AI product.
- •As a freelance AI Project Reviewer, you will evaluate learner submissions against detailed rubrics for a new product, providing consistent pass/fail judgments and constructive feedback.
- •You will collaborate with product and learning design teams to calibrate and validate assessment approaches.
- •Key Responsibilities Review and evaluate projects against detailed rubrics, providing consistent pass/fail judgments.
- •Write clear, actionable feedback that helps learners understand areas for improvement.
- •Collaborate with the learning design and product team to refine rubrics and assessment criteria.
- •Compare human and AI assessment outcomes to identify areas for calibration.
- •Document reasoning behind evaluation decisions to support assessment consistency.
- •Requirements 5+ years of experience in assessment, evaluation, or educational feedback roles.
- •Proven ability to consistently apply detailed rubrics across multiple submissions.
- •Strong written communication skills with the ability to articulate reasoning clearly.
- •Experience providing constructive feedback to support learner growth.
- •Willingness to refrain from using AI during evaluations and written rationales.
Required skills
OpenAI APIAnthropic APIGemini APILangChainHugging Face
Nice-to-have skills
PythonJavaScript
Domain expertise
edtechai
Tech stack
PythonJavaScriptOpenAI APIAnthropic APIGemini APILangChainHugging Face
