Upload your CV
Go to job search

Azure Databricks Engineer

Lloyd's, City of Westminster

Azure Databricks Engineer

Salary not available. View on company website.

Lloyd's, City of Westminster

  • Full time
  • Permanent
  • Onsite working

Posted 1 week ago, 3 Jun | Get your application in now before you miss out!

Closing date: Closing date not specified

Job ref: 81db1e1e1c5f4c2b938ce36542c87388

Location ref: City of Westminster

Full Job Description

  • Lead the development, optimisation and governance of large-scale data platforms using Azure Data Factory, Data Lake, Key Vault, Azure Functions, Databricks, Delta Lake, PySpark and Unity Catalog
  • Partner closely with underwriting, actuarial, delegated authority, bordereaux, exposure management, reinsurance, finance, Solvency II and risk teams to deliver solutions aligned to Lloyd's of London regulatory and operational requirements
  • Translate complex business needs into scalable cloud data architectures in partnership with architects, SMEs and product owners
  • Resolve complex technical challenges rapidly and decisively, consistently delivering high-quality outcomes
  • Champion engineering best practices including design patterns, data lifecycle management, CI/CD automation and cloud-native methodologies
  • Provide hands-on technical leadership, mentorship and uplift engineering capability across the team
  • Drive continuous improvement across performance, cost optimisation, reliability and resilience
  • Oversee end-to-end data engineering delivery ensuring quality, observability, robustness and compliance

    Expert-level knowledge of Azure Data Services and Databricks within enterprise environments
  • Deep understanding of Delta Lake, Medallion architecture, distributed compute, Lakehouse patterns, data modelling and performance optimisation
  • Strong Python, PySpark and SQL experience, with hands-on implementation of CI/CD for data solutions using Azure DevOps
  • Strong understanding of data governance, lineage, access management, FinOps and secure cloud engineering
  • Excellent communication skills with the ability to confidently engage senior stakeholders and simplify complex technical concepts
  • Proven ability to take ownership, deliver at pace and operate effectively in high-pressure, high-expectation environments
  • Experience working within or delivering solutions for the Lloyd's of London market or similarly regulated environments

    Lloyd's of London Market
    • Hybrid 2 days in London Office We are seeking an outstanding Lead Azure Databricks Engineer to own the design, build, optimisation and governance of our enterprise cloud data platforms. This is a senior, fully hands-on engineering position requiring deep technical expertise, strong delivery capability and experience operating within the Lloyd's of London market and its regulatory demands. You will shape the future of our cloud data strategy, enabling advanced analytics and building secure, scalable and highly performant data solutions across the organisation.

Direct job link

https://www.jobs24.co.uk/job/azure-databricks-engineer-126928581
Relevant jobs