Lead Data Engineer (Cloud)

patterns to support consistent, high-quality data processing. The position orchestrates end-to-end workflows in Airflow, progressing datasets through raw, bronze, silver, and gold layers. It also implements manifest-driven transformations using JSON configurations stored in MongoDB or CosmosDB. Responsibilities include maintaining and optimising Delta Lake tables while troubleshooting reliability and performance issues to ensure resilient, efficient, and well-governed data pipelines. What you'll be doing: Building and operating ingestion pipelines into the lakehouse Developing Spark transformation jobs and reusable pipeline patterns Orchestrating workflows in Airflow (raw > bronze > silver > gold) Implementing manifest-driven transformation using JSON configuration stored in MongoDB or CosmosDB Maintaining Delta Lake tables and trouble shoot pipeline reliability/performance Your skills and experiences: You will have deep understanding of modern data engineering principles, data

lifecycle, and enterprise-scale architecture You will have extensive knowledge of data governance, metadata management , and quality frameworks, with ability to enforce standards You will have extensive experience managing and scaling data across hybrid environments (on-prem and cloud), particularly Azure and Databricks Benefits: As well as a competitive pension scheme, BAE Systems also offers employee share plans, an extensive range of flexible discounted health, wellbeing and lifestyle benefits, including a green car scheme, private health plans and shopping discounts - you may also be eligible for an annual incentive. The Data team: This is an opportunity to join a new part of the business, undertaking a new piece of work, introducing new technology into the wider BAE Business. Joining from the ground up, this is a chance to make your mark and the role your own. This role will provide the opportunity to grow the role & your responsibility into a wider scope or even