Member of Technical Staff, Data Engineering
ParallelAI Agents company
San Francisco, United StatesLead
Khosla Ventures
Index Ventures
First Round Capital
Sequoia Capital
Data & AI
About the role
TL;DR
Build scalable data foundations for AI agent development and company analytics.
- •Parallel is seeking a Data Engineer to build the data foundations that power model training, product building, and analytics.
- •You will design and build scalable data pipelines, storage, and serving layers, ensuring data quality and reliability.
- •Key Responsibilities Design and build data pipelines for ingestion and transformation at scale.
- •Develop storage and serving layers for fast and reliable data querying.
- •Establish data quality, lineage, and observability systems.
- •Anticipate and address scaling bottlenecks in the data platform.
- •Make architectural decisions to support growing demand.
- •Requirements Deep intuition on distributed data processing, data modeling, and system reliability.
- •Experience reasoning about trade-offs between batch/streaming, storage cost/query performance, and build-it-yourself/off-the-shelf infrastructure.
- •Commitment to data accuracy and building dependable systems.
Required skills
PythonSQLSparkAirflowdbtAWSGoogle CloudAzureDockerKubernetesCI/CDGit
Nice-to-have skills
SnowflakeDatabricksPostgreSQLMySQLMongoDBRedisElasticsearchDynamoDB
Domain expertise
developer-toolsai
Benefits & perks
Competitive salary, Generous equity, Visa sponsorships, 401K plans, Daily lunch & office snacks, Dinner at the office, Unlimited vacation, Caltrain pass reimbursement
Tech stack
PythonSQLBashSparkAirflowdbtSnowflakeDatabricksAWSGoogle CloudAzurePostgreSQLMySQLMongoDBRedisElasticsearchDynamoDBDockerKubernetesCI/CDGit
