Sr. Staff Platform/Data Reliability Engineer, Databricks
Shield AIAutonomous Systems company
RemoteLead
Andreessen Horowitz
L3Harris
Hanwha Asset Management
U.S. Innovative Technology Fund
Washington Harbour Partners
Riot Ventures
Software Engineering
About the role
TL;DR
Owns Databricks platform operational excellence, CI/CD, and reliability standards.
- •Shield AI is seeking a Sr.
- •Staff Platform/Data Reliability Engineer to own operational excellence for their Databricks platform.
- •This role involves ensuring the reliability, cost-efficiency, and security of data pipelines and platform services in a defense-tech environment.
- •Key Responsibilities Own operational excellence for the Databricks platform, including monitoring, alerting, and incident response.
- •Define and maintain CI/CD and promotion standards for Databricks assets.
- •Design and maintain platform standards for job orchestration, cluster policies, and production execution reliability.
- •Partner with Data Engineers to ensure observable, recoverable, cost-aware, and secure data ingestion and medallion patterns.
- •Enforce technical controls for data segregation, access boundaries, and operational compliance.
- •Requirements 12+ years of relevant experience in data platform engineering, platform operations, SRE, or cloud data infrastructure.
- •Hands-on experience with Databricks or a similar cloud data platform in production.
- •Experience designing or operating CI/CD, environment promotion, and deployment automation for data platforms.
- •Strong understanding of observability, monitoring, alerting, incident management, and reliability engineering concepts.
- •Experience with compute policy design, workload isolation, and secure production execution patterns.
Required skills
DatabricksCI/CDAWSPythonBashTerraformDockerKubernetesGit
Nice-to-have skills
Google CloudAzure
Domain expertise
defensedeveloper-toolsai
Tech stack
DatabricksPythonBashAWSGoogle CloudAzureTerraformCI/CDDockerKubernetesGit