Sr Staff/Sr Data Engineer

We are seeking an experienced Data Engineer to join our Mission Data Processing program. In this role, you will design, build, and maintain scalable ETL pipelines for processing terabyte-scale streaming data and architect databases optimized for machine learning on on-premises hardware using open-source software. The ideal candidate will have expertise in data design patterns such as the Medallion Architecture and data lakehouse technologies to ensure efficient and reliable data processing. You should be skilled at handling high-throughput, low-latency data ingestion, managing data bursts, and implementing features like time-based partitioning, versioning, auditing, and rollback for historical data replay and event reproducibility. Additionally, you will bring DevOps expertise for pipeline automation, Infrastructure as Code (IaC) skills with tools like Terraform and Ansible, and a strong understanding of DevSecOps practices for maintaining secure and compliant data...