Data lake Engineer

We are excited to offer a fantastic permanent Data Lake Engineer opportunity offering Hybrid working with travel up to once a week across one of Amey's offices in Birmingham, London and Liverpool. HOURS OF WORK - Monday - Friday 37.5 Hours Join our vibrant, inclusive community in Group IT, lead and scale our Azure-based data platform. You will own the end-to-end design and optimisation of high-performance, cloud-native data pipelines - ensuring secure, cost-efficient delivery of high-quality datasets across our medallion architecture. What you will do:

• Lead the design and implementation of distributed data processing pipelines (batch and streaming) using PySpark/Spark and Python on Azure Databricks.
• Architect and evolve our medallion data platform - ensuring data quality, observability and reliability across all layers (Bronze/Silver/Gold).
• Define and enforce strong data quality controls, validation rules and monitoring strategies to maintain trusted, production-grade datasets.
• Make technical decisions on cluster strategy and cost optimisation (e.g., interactive vs. job clusters, autoscaling, serverless, pools, spot instances).
• Build and operationalise data workflows using Databricks Workflows, Azure Data Factory, or Fabric, ensuring efficient orchestration, scheduling and alerting.
• Drive API-based data integrations, ensuring secure, scalable and low-latency ingestion of data from third-party and internal systems.
• Champion DevOps standards across the team (CI/CD, TDD, Infrastructure-as-Code, Git-flow, Azure DevOps/GitHub).
• Embed and enforce security and governance best-practices, including data masking, encryption, and RBAC throughout the ETL lifecycle.
• Mentor junior engineers and contribute to technical standards and architecture roadmaps.
• Engage with data scientists, analysts and business stakeholders to deliver high-value, production-ready data products.
  
  Able to drive technical direction and influence architectural decisions.
Strong stakeholder management and communication skills. Mentoring mindset, collaborative and proactive. Operates with a product mindset and ownership mentality. Thrives in a fast-moving environment with ambiguity and growth. Extensive experience working with large-scale data engineering solutions in public cloud (Azure preferred).
• Deep hands-on experience with Databricks, ADLS, Delta Lake and distributed compute paradigms.
• Prior ownership of mission-critical production pipelines, including data quality monitoring, automated alerting, and remediation.
• Strong track record of cost and performance optimisation at scale, including cluster tuning and workload refactoring.
• Experience with leading orchestration tools (e.g. Databricks Workflows, ADF, Airflow).
• Proven experience integrating APIs and external data services into data platforms.
• Proficiency in SQL, Python, PySpark and software engineering best practices.
• Experience in DevOps, CI/CD, infrastructure automation and data security.
• Exposure to MLOps, ML model deployment or LLM-based pipelines is a plus.
  
  At Amey, we work on long-term stable contracts so you can plan a long-term career with us. We have been awarded Platinum status by Investors in People which underscores our commitment to supporting our people across every part of our organisation, enabling them to shine. Whether you are starting your career, returning to the workplace or can bring your professional expertise and skills, there is a place for you here., Amey is a global, integrated infrastructure consultancy and operations partner. We design, deliver, protect, and maintain critical assets and systems.
  
  Remuneration - Enjoy a competitive annual salary with the potential for yearly reviews to ensure you're rewarded for your contributions
• Career Growth: Shine in your career with advancement opportunities
• Training Opportunities: Unlock your potential with comprehensive training, including fully funded leadership programs tailored to your personal growth.
• Holidays - Enjoy at least 24 days of holiday plus bank holidays, and the opportunity to buy further days.
• Pension - Generous pension scheme, with extra contributions from Amey
• Flexible benefits - Customise your benefits with options such as insurance benefits, Cycle2Work scheme and access to discounted gym membership.
• Exclusive Discounts - Access our online portal filled with discounts from leading retailers, healthcare services, and more, helping you save on the things that matter.
• Give Back to community - Two Social Impact Days each year, for volunteering and fundraising opportunities
• Family friendly policies for new parents or if you provide care for a dependant
• Membership of our Affinity Networks who connect, support and inspire diverse communities within Amey