Data Engineer

Can you design and lead data flows connecting operational, analytics and business intelligence (BI) systems? Reuse existing data flows, lead data streaming builds, optimise code and lead database management? Lead complex, secure and scalable data solutions and drive continuous improvement? Select integration techniques that ensure security, efficiency and compliance? Do you lead the design and delivery of complex reporting data solutions whilst establishing reusable data engineering standards? Can you build ETL (Extract Transform Load) pipelines across Azure, AWS and modern cloud platforms? Have you designed and maintained data pipelines across cloud platforms such as Azure and AWS? Can you provide technical leadership and mentoring whilst promoting consistent data engineering standards? If so, we'd love to hear from you! The Data Engineering Team (DET) is responsible for the Business as Usual (BAU) activities of the organisation's Data Platform (EDH). We build and maintain data pipelines, troubleshoot incidents, and support data consumers across the organisation, from Data Analysts and Data Scientists to Data Architects. Beyond day-to-day operations, we build efficient data solutions for sharing data externally, ensuring data is delivered securely and in line with organisational standards. We continually improve the performance of the platform, enabling teams to access trusted data when they need it. Joining our department comes with many benefits, including:

• Employer pension contribution of 28.97% of your salary. Read more about Civil Service Pensions here
• 25 days annual leave, increasing by 1 day each year of service (up to a maximum of 30 days annual leave), plus 8 bank holidays a privilege day for the King's birthday
• Flexible working options where we encourage a great work-life balance.
Read more in the Benefits section below! Find out more about what it's like working at Driver and Vehicle Standards Agency - Department for Transport Careers A DVSA Senior Data engineer is responsible for the design and implementation of numerous complex data flows to connect operational systems, data for analytics and BI systems. DVSA Senior Data Engineers:
• lead the build of data streaming systems
• optimize the code to ensure processes perform optimally
• lead work on database management
• recognize and share opportunities to re-use existing data flows
• coordinate project teams and set best practice and standards
• apply knowledge of systems integration to their work.
Your responsibilities will include, but aren't limited to:
• Understands and can help teams apply a range of techniques for data profiling and source system analysis from a complex single source. Able to bring multiple data sources together in a conformed model for analysis.
• Senior Data Engineer develops service capabilities and the adoption of frameworks for better operational practice. Supports development of Data Ops capabilities and supporting individual's professional growth
• Responsibility for the design build and deployment of information and data management process-flow systems, working collaboratively with delivery partners to manage process flow implementations.
• Integrates and separates data feeds to map, produce, transform, and test new data products. Establishes enterprise-scale data integration procedures across the data development lifecycle and ensures that teams adhere to these.
• Manages resources to ensure that data services work effectively at an enterprise & project level.
• Working with peers and the head of profession to develop a professional development program and providing coaching and mentoring to others within the data engineering profession.
• Execute management responsibilities effectively, working with, leading and acting as a role model for team members and colleagues, ensuring capabilities to achieve successful delivery of business performance whilst maximising the efficiency and effectiveness of the financial resources delegated in line with the Civil Service values!