About the role
Design, build, and maintain scalable data pipelines for an AI-driven platform.
- •As a Senior Data Engineer, you'll play a key role in building and scaling the data infrastructure that powers our AI-driven platform.
- •Key Responsibilities Design, build, and maintain scalable data pipelines that ingest, transform, and validate large volumes of data across multiple sources and channels.
- •Improve the scalability, reliability, and performance of our data pipelines to support rapidly growing workloads and new data streams.
- •Contribute to the design and implementation of our Data Lake architecture, enabling reliable data storage and reuse across teams.
- •Manage and optimize data ingestion workflows, including data collected from web scrapers, third-party vendors, and internal systems.
- •Monitor pipeline health, investigate incidents, and implement improvements to increase system reliability and observability.
- •Requirements Strong experience with data engineering and data pipeline design.
- •Proficiency in AWS services and data processing frameworks.
- •Experience with data warehousing and data lake architectures.
- •Excellent problem-solving skills and attention to detail.
Tech stack
AWSAirflowDatabricksApache KafkaApache FlinkPostgreSQLMySQLMongoDBRedisElasticsearchDynamoDBSnowflakeFivetranDagsterPrefectPandasNumPySparkdbtTableauPower BILooker
Match insights
Tech:AWS, Airflow, Databricks, Apache Kafka, Apache Flink
Level:Senior