Lead Data Engineer - IBCA Data Platform
Cabinet Office, Newcastle upon Tyne
Lead Data Engineer - IBCA Data Platform
Salary not available. View on company website.
Cabinet Office, Newcastle upon Tyne
- Full time
- Permanent
- Onsite working
Posted 1 week ago, 11 Jul | Get your application in now before you're too late!
Closing date: Closing date not specified
job Ref: 563cc43d519546818d5a4bf3807803a5
Full Job Description
The Infected Blood Compensation Authority (IBCA) is responsible for delivering a compensation scheme that has been long awaited by the infected blood community to provide financial compensation to victims of infected blood on a UK-wide basis.
This role will lead our data engineering capability within the engineering team of the Data Operations arm of the IBCA Data Directorate.
The Data Operations team is responsible for developing and running safe and secure data solutions that provide a single source of truth for those going through their compensation journey. They are building a new data platform using Amazon Web Services (AWS) and data management and intelligence products using Databricks, Quantexa and Tableau.
We are taking a product-centric approach treating data as a product and are building squads around our products, with a focus on paying compensation to those impacted by the infected blood scandal seamlessly.
You will focus on building robust data pipelines and data management processes (including master data management), ensuring data quality, and driving a culture of data-driven decision-making.
Working at IBCA gives you a huge opportunity to make an impact on those who deserve compensation, and this role suits a candidate who can lead a team of data engineers to deliver solutions from the ground up to take from ideation to reality so that data is an enabler to everything IBCA does., As the Lead Data Engineer, you will join a multidisciplinary team to design, build, and deliver high-impact, scalable AWS-based data solutions using Databricks and Quantexa technologies, while also driving data quality, mentoring junior colleagues, and fostering a strong data engineering community.
You will:
+ provide technical guidance for the development of robust, automated data pipelines and master data management processes for our data platform and products, encompassing DevSecOps best practices
+ ensure tools and techniques are scalable, secure and efficient
+ be responsible for ensuring the right data engineering practices are embedded consistently and to industry standard best practices within the data platform delivery teams
+ develop the data engineering capability by developing L&D routeways, shaping careers paths and recruiting talent to the organisation
+ provide leadership and guidance to a team of data engineers, further developing your own data engineering and leadership skills
+ work with business stakeholders and across digital service teams, understanding their needs and translating them into data development, + Data Solution Delivery: Designing, building, and delivering high-performance, scalable, and secure data solutions in a complex data environment within AWS, Databricks and Quantexa. This includes building robust ETL/ELT pipelines, and ensuring seamless integration between platforms.
+ Collaboration & Partnership: Work extensively with multidisciplinary teams across product delivery, architecture, engineering security and analytics to understand and meet data needs.
+ Quality & Operational Excellence: Driving high build and data quality, ensuring stability, robustness, and resilience of products, and embedding Agile, CI/CD, and DevOps practices.
+ Leadership & Community Building: Mentoring junior team members, fostering professional development, building a data engineering community, reviewing solutions, troubleshooting complex issues and proactively driving innovation and challenging existing methods.
+ Stakeholder Engagement & Problem Solving: Communicating complex data solutions and championing innovative approaches to diverse stakeholders, while also anticipating and resolving intricate data engineering challenges and building consensus to align data designs with organizational goals.
Opportunities to move to the new Infected Blood Compensation Authority (IBCA)
At the moment all IBCA roles are hosted by Cabinet Office within the Civil Service. However, once IBCA becomes operational, it is anticipated that most of these roles will transfer to IBCA, which is a new Arms Length Body (ALB) separate from the Cabinet Office. Roles that transfer to IBCA from Cabinet Office will maintain the same or substantially the same terms and conditions of employment that are overall no less favourable. It is anticipated that employees who move from Cabinet Office to IBCA will be able to continue to participate in the Civil Service Pension arrangements with no break in their pensionable service. Roles commencing after IBCA has become operational will be with IBCA itself on similar terms and conditions. All successful candidates will receive full details of the terms and conditions of employment for their role with their formal job offer.
Please note that the mission of IBCA means that it is likely to be operational for a period of approximately 5 to 7 years. When IBCA's work begins to wind down, IBCA employees will receive support and practical guidance to find a new role, whether in the Civil Service, another Arms Length Body (ALB), or an external employer., We'll assess you against these behaviours during the selection process:
+ Changing and Improving
+ Communicating and Influencing
Technical skills
We'll assess you against these technical skills during the selection process:
+ Data analysis and synthesis (Practitioner)
+ Data development process (Expert), If a person with disabilities is put at a substantial disadvantage compared to a non-disabled person, we have a duty to make reasonable changes to our processes.
If you need a change to be made so that you can make your application, you should:
+ Contact Government Recruitment Service via recruitment.grs@cabinetoffice.gov.uk as soon as possible before the closing date to discuss your needs.
+ Complete the 'Assistance required' section in the 'Additional requirements' page of your application form to tell us what changes or help you might need further on in the recruitment process. For instance, you may need wheelchair access at interview, or if you're deaf, a Language Service Professional.
Further Information
A reserve list may be held for a period of 3 months from which further appointments can be made.
Any move to IBCA from another employer will mean you can no longer access childcare vouchers. This includes moves between government departments. You may however be eligible for other government schemes, including Tax Free Childcare; for further information visit the Childcare Choices website.
If successful and transferring from another Government Department a criminal record check may be carried out.
In order to process applications without delay, we will be sending a Criminal Record Check to Disclosure and Barring Service/Disclosure Scotland on your behalf.
However, we recognise in exceptional circumstances some candidates will want to send their completed forms direct. If you will be doing this, please advise Government Recruitment Service of your intention by emailing Pre-EmploymentChecks.grs@cabinetoffice.gov.uk stating the job reference number in the subject heading.
For further information on the Disclosure Scotland confidential checking service telephone: the Disclosure Scotland Helpline on and ask to speak to the operations manager in confidence, or email Info@disclosurescotland.co.uk
Please note that this role requires SC clearance, which would normally need 5 years' UK residency in the past 5 years. This is not an absolute requirement, but supplementary checks may be needed where individuals have not lived in the UK for that period. This may mean your security clearance (and therefore your appointment) will take longer or, in some cases, not be possible.
For further information on National Security Vetting please visit the Demystifying Vetting website.
New entrants are expected to join on the minimum of the pay band.
Applicants who are successful at interview will be, as part of pre-employment screening, subject to a check on the Internal Fraud Database (IFD). This check will provide information about employees who have been dismissed for fraud or dishonesty offences. This check also applies to employees who resign or otherwise leave before being dismissed for fraud or dishonesty had their employment continued. Any applicant's details held on the IFD will be refused employment.
A candidate is not eligible to apply for a role within the Civil Service if the application is made within a 5 year period following a dismissal for carrying out internal fraud against government.
This role is full time only. Applicants who wish to work an alternative pattern are welcome to apply however your preferred working pattern may not be available and you should discuss this with the vacancy holder before applying.
Please note terms and conditions are attached. Please take time to read the document to determine how these may affect you.
If you are experiencing accessibility problems with any attachments on this advert, please contact the email address in the 'Contact point for applicants' section.
Feedback will only be provided if you attend an interview or assessment.
Security
Successful candidates must undergo a criminal record check.
Successful candidates must meet the security requirements before they can be appointed. The level of security needed is security check (opens in a new window).
See our vetting charter (opens in a new window).
People working with government assets must complete baseline personnel security standard (opens in new window) checks.
Nationality requirements
This job is broadly open to the following groups:
+ UK nationals
+ nationals of the Republic of Ireland
+ nationals of Commonwealth countries who have the right to work in the UK
+ nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities with settled or pre-settled status under the European Union Settlement Scheme (EUSS) (opens in a new window)
+ nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities who have made a valid application for settled or pre-settled status under the European Union Settlement Scheme (EUSS)
+ individuals with limited leave to remain or indefinite leave to remain who were eligible to apply for EUSS on or before 31 December 2020
+ Turkish nationals, and certain family members of Turkish nationals, who have accrued the right to work in the Civil Service
Further information on nationality requirements (opens in a new window)
Working for the Civil Service
The Civil Service Code (opens in a new window) sets out the standards of behaviour expected of civil servants.
We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission's recruitment principles (opens in a new window).
The Civil Service embraces diversity and promotes equal opportunities. As such, we run a Disability Confident Scheme (DCS) for candidates with disabilities who meet the minimum selection criteria.
The Civil Service also offers a Redeployment Interview Scheme to civil servants who are at risk of redundancy, and who meet the minimum requirements for the advertised vacancy.
Diversity and Inclusion
The Civil Service is committed to attract, retain and invest in talent wherever it is found. To learn more please see the Civil Service People Plan (opens in a new window) and the Civil Service Diversity and Inclusion Strategy (opens in a new window).
Apply and further information
This vacancy is part of the Great Place to Work for Veterans (opens in a new window) initiative.
The Civil Service welcomes applications from people who have recently left prison or have an unspent conviction. Read more about prison leaver recruitment (opens in new window).
Once this job has closed, the job advert will no longer be available. You may want to save a copy for your records.
+ Strong experience of cloud-native data engineering in AWS and building and maintaining complex data pipelines (both ETL and ELT) in a rapid delivery setting that covers data quality & transformation processes, data matching and master data management.
+ Experience working with structured and unstructured data, and data lakes to service operational and analytical business needs.
+ Proficiency in writing clear parameterised code in two or more of the following - Python, Databricks, Apache Spark (Pyspark, Spark SQL), NoSQL, Scala.
+ Experience of delivering through Agile/DevOps working practices in multi-disciplinary teams - CI/CD, Scrum, Automation.
+ Strong problem-solving skills, including assessing and mitigating risks while identifying opportunities for innovation.
+ Experience in the full end to end data lifecycle for design, build and test, with knowledge of the interactions and dependencies with data architecture, data modelling and testing engineering.
+ Strong experience in managing team dynamics and facilitating difficult discussions within the team or with a diverse set of stakeholders.
+ Demonstrable experience of setting up data engineering processes from scratch or of implementing large changes to existing processes within an organisation that operate in a cloud DevSecOps environment.
Desirable Criteria:
+ Management of contracts and 3rd party suppliers for software and professional services.
Alongside your salary of £55,403, Cabinet Office contributes £16,050 towards you being a member of the Civil Service Defined Benefit Pension scheme. Find out what benefits a Civil Service Pension provides.
+ Learning and development tailored to your role.
+ An environment with flexible working options.
+ A culture encouraging inclusion and diversity.
+ A Civil Service Pension which provides an attractive pension, benefits for dependants and employer contributions of 28.97%.
+ A minimum of 25 days of paid annual leave, increasing by one day per year up to a maximum of 30.
Direct job link
Relevant jobs
- Tech / Digital / IT Jobs in Alnwick, Northumberland
- Tech / Digital / IT Jobs in Ashington, Northumberland
- Tech / Digital / IT Jobs in Berwick-upon-Tweed, Northumberland
- Tech / Digital / IT Jobs in Bishop Auckland, County Durham
- Tech / Digital / IT Jobs in Consett, County Durham
- Tech / Digital / IT Jobs in Darlington
- Tech / Digital / IT Jobs in Durham
- Tech / Digital / IT Jobs in Gateshead
- Tech / Digital / IT Jobs in Hartlepool
- Tech / Digital / IT Jobs in Hexham, Northumberland
- Tech / Digital / IT Jobs in Malton, County Durham
- Tech / Digital / IT Jobs in Middlesbrough
- Tech / Digital / IT Jobs in Morpeth, Northumberland
- Tech / Digital / IT Jobs in Newcastle upon Tyne
- Tech / Digital / IT Jobs in North Shields, North Tyneside
- Tech / Digital / IT Jobs in Peterlee, County Durham
- Tech / Digital / IT Jobs in Redcar, Redcar and Cleveland
- Tech / Digital / IT Jobs in South Shields, South Tyneside
- Tech / Digital / IT Jobs in Stockton-on-Tees
- Tech / Digital / IT Jobs in Sunderland, Cumbria
- Tech / Digital / IT Jobs in Tynemouth, North Tyneside
- Tech / Digital / IT Jobs in Washington, Sunderland
- Tech / Digital / IT Jobs in Whitley Bay, North Tyneside
Similar jobs for you
Senior DevOps Engineer (Infected Blood Compensation Authority)
Cabinet Office,
- Full time
- Permanent
Apply on company site