Data Engineering Dev Lead

The Lead Data Engineer will be responsible for leading a team of developers and data engineers who build and deliver the components of modern Data platform -

• Data Lakehouse Architecture in Azure, leveraging data lake, delta Lake for data storage


• Utilising Azure Data Factory for data integration


• Databricks for data transformation


• Data governance using purview


• Profisee for MDM


• Synapse and Power BI for end-user data analytics layer and reporting


• In the pipeline - Co Pilot and Fabric


This role will play a critical part in the architectural design and implementation of these elements - capable of meeting a wide range of financial and underwriting reporting/analytics needs. This role will encompass a mix of hands-on technical development, thought leadership, team management, strategy development; supporting IQUW to continue its journey to develop its data platform on next-generation technology.,
• Mixture of hands on and lead a newly established data engineering team through both strategic development projects and day-to-day data management (BAU) activities.



• Defining and orchestrating tasks and deliverables in support of the project/scrum master.


• Setting the standard of what good looks like


Design, develop, and hold accountability for core data engineering services, encompassing the end-to-end data lifecycle journey from ingestion through to end user reporting.

Data acquisition from various data sources (API, XML, Json, parquet, flat file systems and relational data

Skills in data modelling (both structured and unstructured data) working directly with the business & data architects and SME's.

Administering CICD pipelines, completing peer/code reviews, ensuring the integrity of IQUW data, and holding accountability for access and security permission.

Design and develop metadata driven data ingestion (ETL/ELT) processes, data lake house architecture, data warehouse, data marts, reports, and dashboards; in-line with the company data management framework and enterprise data strategy.

Work in collaboration with Data Architects and act as a Senior contributor on the design, build, and management of the IQUW Data Platform; taking direct ownership of data engineering activities and responsibilities and ensure that all new data requirements are met using the most appropriate architecture components and engineering practices.

Design and develop models using delta lake both relational and dimensional to support reporting requirements.

Create reporting solutions in line with standards, architectural principles, and practices.

Work in an agile, proactive manner; delivering high quality solutions in agreed timescales.

Mentor and support data engineers/developers in the team.

Team leadership of both senior and general data engineers and BI developers.

Stay abreast of technology trends, state-of-the-art platforms, data engineering techniques/standards and industry trends to provide input on IQUW's strategic roadmap including Fabric and AI copilot

Develop Proof of Concepts to prove that business requirements can be met by technical solutions.

Required Skills & Knowledge

• Solid hands-on experience with Data Lake house-based data architecture using Delta Lake, Data Factory and Azure Databricks.


• Hands-on experience in building data platform components in delta lake.


• Proven experience of data engineering activities such as release management, environment controls, CICD pipeline orchestration.


• Experience working with Python, Py spark, delta lake, Spark SQL and SQL. · Strong data modelling, problem solving, information analysis, attention to detail, flexibility of approach. ·


• The ability to deal with business functions and interpret their thoughts into successful data & analytics solutions.


• The ability to design and develop complex BI/MI solutions in line with agreed architectural principles and toolsets.


• Strong interest in latest data engineering technologies, establishing best practises, techniques, and trends in the data industry.


Required Toolsets /Skills

• Azure Data Factory, Synapse Analytics Serverless


• Azure Databricks and Azure Data Lake storage


• PowerBI.


• Python, PySpark, Spark SQL, Delta Lake.


• Kimball / Inmon data modelling
  
  IQUW is a speciality (re)insurer at Lloyd's (Syndicate 1856) underwriting a diverse range of Property, Commercial and Speciality (re)insurance products from Cargo and Marine to Political Violence, Terror and War. We combine data, intelligent automation and human expertise to make smart decisions, fast.

ERS is the UK's largest specialist motor insurer with an A+ rating. We recognise that for some, motor insurance is more than just a must-have; it's a way of taking care of what stands at the heart of their passion or livelihood. For those people, standard insurance isn't enough. That's why we work exclusively with motor insurance brokers to help get under the skin of the most difficult insurance risks, helping build products to meet their customer's needs., Exciting Opportunity to join a Lloyds Market Syndicate that is embarking on a major Data & Analytics Transformation program using Modern Cloud Technologies to create a market leading data function that supports current and future data needs of a growing Lloyds Syndicate.


• Competitive Benchmarked Salary


• 25 days holiday


• Discretionary bonus scheme


• Employee assistance programme


• Annual holiday buy (up to 3 extra days)


• Salary sacrifice benefits


• Annual benefits reviews


• The option for professional qualifications and study support