Senior Data Engineer

HOURS: 36

ROLE PURPOSE
We are looking for a Senior Data Engineer to design and build the data pipelines and infrastructure that transform complex, heterogeneous data into reliable, well-structured resources. Initially, you will focus on the data layer underpinning analyst-facing tools, including curated datasets, data models, serving layers and query interfaces. As the team's machine learning capabilities grow, this foundation will also support feature pipelines, dataset versioning and evaluation data.

You will establish and maintain the team's data platform across cloud and on-premise environments, covering storage, compute, orchestration and access control. Working within Defence and partner environments, you will design solutions that meet technical and security constraints while enabling effective data use.

This is a senior individual contributor role suited to an experienced data engineer who has built and operated data systems at scale, can set technical direction, and is comfortable working within operationally constrained environments.

HOW YOU WILL MAKE AN IMPACT

• Design, build and operate robust data pipelines that ingest, validate, clean and structure large volumes of varied data, ensuring quality, lineage and reproducibility from the outset.
• Develop the data products that support the team's tools, including data models, curated datasets, serving layers and query interfaces used by analysts and engineers.
• Establish and maintain the team's data platform across cloud and on-premise environments, covering storage, compute, orchestration, access control and infrastructure-as-code.
• Work closely with applied engineers and partner teams to understand how data will be consumed and design solutions that operate effectively within technical and security constraints.
• Set and maintain the team's data engineering standards, including testing, schema and contract management, documentation and secure data handling, supported by effective code review.
• Help shape the team's data architecture and ways of working, mentoring colleagues and advising on the future development of data and platform engineering capabilities.

ABOUT YOU

You will bring:

• Significant experience designing, building and operating production data pipelines across large, complex and poorly structured datasets, including batch and streaming data.
• Strong Python engineering skills, including testing, packaging, code review and CI/CD practices.
• Experience deploying and operating data infrastructure, including storage, compute, orchestration and access control, using infrastructure-as-code in cloud and/or on-premise environments.
• Expertise in designing data models and developing trusted data products, such as curated datasets, serving layers and query interfaces.
• Experience handling data under security, access-control and audit constraints, and designing systems that respect them by default.
• The ability to collaborate effectively with engineers, data scientists, analysts and partner organisations, balancing technical ambition with practical delivery constraints.

Terms and Conditions

This position is offered on a fixed-term basis with a salary of £57,000-£69,000 per annum, plus an excellent benefits package, including flexible working and family-friendly policies.

Application Procedure

Please visit our jobs portal for full details of the role and application process.

Eligibility for Developed Vetting (DV) clearance is an essential requirement for this role. Further information on eligibility and the clearance process can be found on the UK Government security vetting  website.

EQUALITY, DIVERSITY AND INCLUSION
We value diversity of background, experience, and perspective, and are proud to be an inclusive employer. We warmly encourage applications from all backgrounds, particularly from groups currently under-represented in our sector.  If you feel passionate about this role but don’t meet every single requirement, please apply – we recognise that great candidates may bring strengths beyond the criteria listed. 

Advert information