Senior DataOps Engineer (18 Month Fixed-Term Contract)

We are seeking a senior Data Platform DataOps Engineer to serve as our first DataOps specialist in a growing team of Data Engineers and DevOps professionals. In this pivotal role, you will focus on operationalising and automating our data lifecycle to ensure that data workflows perform with reliability and efficiency. You will integrate CI/CD data pipelines, streamline deployment processes, enforce robust data governance, and optimise operational costs within our Microsoft Azure environment. Your work will be centred on proactive system monitoring, error resolution, and continuous improvements, while mentoring and guiding colleagues. Role Requirements: The role will:

• Oversee and automate the operational processes that support data workflows developed by the Data Engineering team while ensuring seamless coordination with the DevOps group.
• Spearhead the development, integration, and maintenance of CI/CD data pipelines for automated deployments.
• Integrate best practices for monitoring and observability to proactively detect, analyse, and resolve issues.
• Enforce robust data governance and security protocols through tools like Azure Key Vault, ensuring compliance with standards such as GDPR, and other regulatory frameworks.
• Collaborate closely with Data Engineering, Data Science, Analytics, and DevOps teams to align operational strategies with technical and business requirements.
• Optimize operational performance and cost management for services including Azure Data Factory, Azure Databricks, Delta Lake, and Azure Data Lake Storage.
• Serve as the domain expert in DataOps by providing strategic guidance, mentoring colleagues, and driving continuous process improvements.
• Evaluate and adopt emerging technologies and methodologies to further enhance operational efficiency and automation within our data ecosystem., We do not expect new joiners to have skills across all the technical areas we are looking for, but we do want you to enjoy learning and will support you to fill the gaps.
The job has flexibility in working pattern, but you will be part of a support rota so, at times, your working hours will be more rigid. If you are a passionate Data Platform DataOps Engineer, we encourage you to apply for this exciting position. Join our team and contribute to the success of the Bank's data and analytics strategy. Our Approach to Inclusion The Bank values diversity, equity and inclusion. We play a key role in maintaining monetary and financial stability, and to do that effectively, we believe we need a workforce that reflects the society we serve. At the Bank of England, we want all colleagues to feel valued and respected, so we're working hard to build an inclusive culture which supports people from all backgrounds and communities to be at their best at work. We celebrate all forms of diversity, including (but not limited to) age, disability, ethnicity, gender, gender identity, race, religion, sexual orientation and socioeconomic status. We believe that it's by drawing on different perspectives and experiences that we'll continue to make the best decisions for the public. We welcome applications from individuals who work flexibly, including job shares and part time working patterns. We've also partnered with external organisations to support us in making adjustments for candidates and employees in the recruitment process where they're needed. For most roles where work can be carried out at home, we aim for colleagues to spend half of their time in the office, with a minimum of 40% per month. Subject to that minimum requirement, individuals and managers should work together to find what works best for them, their team and stakeholders.


• Demonstrable experience in DataOps, Data Engineering, DevOps, or related roles focused on managing data operations in complex, data-centric environments.
• Proven experience working with agile teams and driving automation of data workflows within the Microsoft Azure ecosystem.
• Hands-on expertise with Azure Data Platform components including Azure Data Factory, Azure Databricks, Azure Data Lake Storage, Delta Lake, Azure SQL, Purview and APIM.
• Proficiency in developing CI/CD data pipelines and strong programming skills in Python, SQL, Bash, and PySpark for automation.
• Strong aptitude for data pipeline monitoring and an understanding of data security practices such as RBAC and encryption.
• Implemented data and pipeline observability dashboards, ensuring high data quality, and improving the efficiency of data workflows.
• Experience ensuring compliance with regulatory frameworks and implementing robust data governance measures.
• Demonstrated ability to implement Infrastructure as Code using Terraform, to provision and manage data pipelines and associated resources.
Essential Criteria In addition to your technical skills you will also:
• Enjoy sharing knowledge and training / upskilling others.
• Have excellent communication and collaboration skills, with both technical and business colleagues.
• Be motivated to learn, upskill and improve yourself in both technical and relevant complementary soft skills.
Desirable Criteria It would be beneficial to already have:
• Proven experience in optimizing and scaling high-volume data operations within enterprise Azure environments.
• Familiarity with Azure services that enhance data operational efficiency and support complex analytics.
• Exposure to Spark-based processing and advanced analytics techniques to further empower data-driven decision-making.
• A track record of successfully mentoring teams and implementing strategic improvements in data operations.
• Demonstrated expertise in cost optimisation and performance tuning within an Azure-based data infrastructure.
• Relevant certifications such as Azure Data Engineer Associate, Azure DevOps, Databricks Data Engineer Professional, or equivalent credentials.
• Ability to build and manage the DataOps processes in a newly formed team which is planned to continue to expand over the next few months and years.
  
  We encourage flexible working, part time working and job share arrangements. Part time salary and benefits will be on a pro-rated basis as appropriate.
We offer a salary as follows: Leeds circa £55,280 - £62,190 In addition, we also offer a comprehensive benefits package as detailed below:
• Currently a non-contributory, career average pension giving you a guaranteed retirement benefit of 1/80th of your annual salary for every year worked. There is the option to increase your pension (to 1/65th) or decrease (to 1/105th) in exchange for salary through our flexible benefits programme each year. The Bank has the discretion to vary standard accrual rates and dial up and dial down rates at any time and to withdraw dial up and dial down options at any time.
• A discretionary performance award based on a current award pool.
• A 8% benefits allowance with the option to take as salary or purchase a wide range of flexible benefits.
• 26 days' annual leave with option to buy up to 12 additional days through flexible benefits.
• Private medical insurance and income protection.
National Security Vetting Process