QA Analyst - LOIS for Word
FilevineLegal Case company
San Francisco, United StatesMid
Insight Partners
Accel
Halo Fund
Meritech
Stepstone
Album Ventures
Software EngineeringNew
About the role
TL;DR
Owns quality for LOIS for Word, Filevine's AI-powered legal drafting product.
- •This role owns quality for LOIS for Word, Filevine's AI-powered legal drafting product.
- •You'll help build and own the QA function: the test strategy, the process, and the automation.
- •You'll work at the intersection of legal AI and product quality, evaluating everything from AI output accuracy to UI behavior across the product.
- •Key Responsibilities Define and own the test strategy, process, and QA playbook for the product Evaluate AI output quality across dimensions like accuracy, citation reliability, and appropriately scoped reasoning Build automated guardrails integrated into CI/CD that cover critical flows Partner with legal domain experts to understand how different customer types use the product, and translate that into test coverage Approach QA with a product and design mindset.
- •Requirements 2+ years of QA experience, including ownership of test strategy for a product or major feature area Track record of building QA process and automation from scratch in a fast-moving environment Strong analytical and problem-solving skills, with attention to both functional correctness and AI output quality Ability to reason about non-deterministic systems and define what "correct" means when there's no single right answer Excellent written communication, with the ability to write clear, concise, actionable bug reports backed by evidence
Required skills
CI/CDOpenAI APILangChainPythonJavaScriptReactNode.jsAWSDockerKubernetesGitLinuxJiraPostman
Nice-to-have skills
LLMsNLPComputer VisionSeleniumCypressPlaywrightJestPytest
Domain expertise
legaltechaiproductivity
Tech stack
PythonJavaScriptReactNode.jsAWSDockerKubernetesCI/CDGitLinuxJiraConfluencePostmanREST APIOpenAI APILangChain