About the role
TL;DR
Design, build, and maintain scalable data pipelines and real-time streaming solutions.
- •About Agile Defense At Agile Defense we know that action defines the outcome and new challenges require new solutions.
- •That’s why we always look to the future and embrace change with an unmovable spirit and the courage to build for what comes next.
- •Our vision is to bring adaptive innovation to support our nation's most important missions through the seamless integration of advanced technologies, elite minds, and unparalleled agility—leveraging a foundation of speed, flexibility, and ingenuity to strengthen and protect our nation’s vital interests.
- •Requisition #: 1569 Job Title: Data Engineer Location: Omaha, NE Clearance Level: Top Secret/SCI required Job Description Design, build, and maintain highly scalable, reliable, and efficient data pipelines for extracting, transforming, and loading (ETL/ELT) data from various sources into data warehouses, data lakes, and other storage systems.
- •Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and ensure the efficient flow of data across various platforms and systems.
- •Develop, deploy, and maintain real-time data streaming solutions to process and deliver data with low latency (using tools like Apache Kafka, Flink, or Spark Streaming).
- •Optimize and automate data workflows, ensuring that data pipelines are efficient, reliable, and capable of handling increasing data volumes.
- •Oversee the development of data models and schema designs to ensure accurate, accessible, and high-performance data storage for analytics and reporting.
- •Ensure data quality, integrity, and consistency by implementing data validation, monitoring, and error-handling mechanisms.
- •Monitor and troubleshoot performance bottlenecks in data systems, resolving issues with data processing and data access.
- •Stay current with emerging trends and best practices in data engineering, recommending new tools, technologies, and methods to enhance existing infrastructure.
- •Mentor and guide junior data engineers, promoting best practices in data engineering and fostering a collaborative, high-performance team environment.
- •Education and Background Typically has a Bachelor's degree in Computer Science, Information Systems, Data Engineering, or a related field (masters degree preferred)., and 5+ years of experience in data engineering or software engineering, with demonstrated experience in designing and managing complex data pipelines., or equivalent relevant work experience; e.g., each year of work experience may be substituted for each year of education required.
- •Years of Experience 5+ years Required Skills Strong understanding of data validation, model testing, and performance evaluation techniques.
- •Experience with data visualization and storytelling using tools such as Tableau, Plotly, or Matplotlib.
- •Excellent technical communication skills, with the ability to explain complex concepts to non-technical audiences.
- •Preferred Skills 4+ years of experience in applied data science, machine learning engineering, or data pipeline development.
- •Proficient in Python, SQL, and distributed data frameworks (e.g., Spark, Databricks, PySpark).
- •Experience developing ML models from training to deployment using industry-standard tools and libraries (e.g., scikit-learn, TensorFlow, XGBoost, MLflow).
- •Familiarity with MLOps, API development, and secure cloud-based environments (e.g., AWS, Azure, Palantir Foundry).
- •Working Conditions Onsite in a SCIF in STRATCOM
Required skills
PythonSQLSparkTableau
Nice-to-have skills
Apache KafkaApache FlinkDatabricksSparkscikit-learnTensorFlowMLflowAWSAzure
Domain expertise
defensegovernment
Tech stack
PythonSQLApache KafkaApache FlinkSparkDatabricksTableauscikit-learnTensorFlowMLflowAWSAzure
