Senior Data Engineer
AlpacaAPI Brokerage company
RemoteSenior
Drive Capital
Spark Capital
Tribe Capital
Horizons Ventures
Portage Ventures
Social Leverage
Data & AI
About the role
TL;DR
Design and build the next generation of Alpaca's Data Platform.
- •We are seeking a Senior Data Engineer to help design and build the next generation of our Data Platform as we continue to scale to larger customers and new jurisdictions.
- •At Alpaca, Data Engineering encompasses financial transactions, customer data, API logs, system metrics, augmented data, and third-party systems that impact decision making for both internal and external stakeholders.
- •Key Responsibilities Design, build, and evolve the core data platform infrastructure.
- •Own our lakehouse infrastructure as code, managing deployments through Terraform and Ansible on Kubernetes.
- •Build and maintain low-latency streaming and CDC ingestion pipelines, as well as batch ingestion paths landing in Iceberg.
- •Develop and scale our BI landscape for performant, self-serve access to lakehouse data.
- •Enforce platform reliability best practices, including monitoring and alerting, on-call rotations, incident response, and SLAs.
- •Requirements 5+ years of experience in Data Engineering, including 2+ years building and operating scalable, low-latency data platforms handling > 100M events/day.
- •Strong hands-on experience running data infrastructure on Kubernetes, with cloud-native tooling like Docker and Helm.
- •Production experience with IaC: Terraform, Ansible, and ArgoCD (or equivalents).
- •Deep knowledge of distributed systems with hands-on experience operating open-source query engines like Trino or Presto.
- •Strong experience with object storage and open table formats, specifically Apache Iceberg.
Required skills
PythonSQLKubernetesDockerTerraformAnsibleArgoCDAirflowGoogle Cloud
Nice-to-have skills
dbtLooker
Domain expertise
fintechcryptodeveloper-tools
Benefits & perks
Competitive Salary & Stock Options, Health Benefits, New Hire Home-Office Setup: One-time USD $500, Monthly Stipend: USD $150 per month via a Brex Card
Tech stack
PythonSQLKubernetesDockerHelmTerraformAnsibleArgoCDAirflowGoogle ClouddbtLooker