Data Engineer, Scientific Data Ingestion
MithrlGenomic Data company
San Francisco, United StatesSenior
Headline
Bonfire Ventures
Jenny Fielding
Scott Hartley
Agi House
Obvious Ventures
Data & AI
About the role
TL;DR
Develop and maintain an AI-powered data ingestion and normalization pipeline.
- •Build and own an AI-powered ingestion & normalization pipeline to import data from various sources.
- •Key Responsibilities Develop robust schema mapping, coercion, and conversion logic.
- •Use LLM-driven and classical data-engineering tools to structure messy tabular data.
- •Ensure all transformations execute during ingestion for clean, canonical data.
- •Requirements 5+ years of experience in data engineering with real-world tabular or semi-structured data.
- •Strong fluency in Python and data processing tools (Pandas, Polars, PyArrow).
- •Excellent experience dealing with messy Excel/CSV/spreadsheet-style data.
Required skills
PythonPandasAirflow
Nice-to-have skills
PrefectDagster
Domain expertise
biotech
Benefits & perks
Comprehensive PPO health coverage, 401(k) with top-tier plans
Tech stack
PythonPandasAirflowPrefectDagsterDatabricksApache KafkaKubernetesCI/CDGitHub Actions