Data Platform Engineer (Member of Technical Staff)
Cogent SecurityVulnerability Management company
San Francisco, United States$100K - $300KSenior
Data & AI
About the role
Build scalable data ingestion, pipelines, and indexing for GenAI-driven enterprise security use cases.
- •Design and implement Cogent's data platform and integration pipeline to ingest, normalize, and transform enterprise data for GenAI systems.
- •Key Responsibilities Build data connectors, pipelines, and ETLs to ingest enterprise data securely.
- •Design indexing, storage, and query strategies for semantic search and RAG.
- •Architect scalable data platform components and make build vs buy decisions.
- •Work with Applied AI teams to taxonomize, correlate, and deduplicate data.
- •Requirements 5+ years as a hands-on engineer and technical leader.
- •Experience building data pipelines for information retrieval systems.
- •Familiarity with ETL orchestration (Temporal, Airflow) and streaming (Kafka, Flink).
- •Expertise in SQL, database fundamentals, and distributed computing.
- •Experience with Kubernetes, Terraform, Docker, and multi-cloud deployments.
Tech stack
PythonSQLDatabricksApache KafkaApache FlinkAirflowKubernetesTerraformDocker
Match insights
Tech:Python, SQL, Databricks, Apache Kafka, Apache Flink
Level:Senior