Data Engineering Lead

About the role

AstraZeneca's Drug Development Data Platform team is looking for a Data Engineering Lead to join its Drug Discovery Data Platform leadership team. This is a hands-on technical leadership role leading multidisciplinary teams that deliver trusted data products, analytical products and AI-enabled solutions across the drug discovery lifecycle, from discovery and architecture through engineering, deployment, adoption, monitoring and continuous improvement.

Key responsibilities

• Define technical direction for data products, analytics platforms and AI-enabled capabilities, and design secure, scalable, resilient and cost-effective architectures for batch, streaming, event-driven, API and AI workloads.
• Set standards for data engineering, APIs, analytics, machine learning and generative AI, and establish reusable patterns for data contracts, semantic models, metadata, lineage and self-service.
• Contribute to production-quality code, prototypes and investigations, build and review data pipelines, transformation frameworks, APIs, analytical products and AI components, and develop cloud-native solutions using containers, serverless services, managed platforms and infrastructure as code.
• Establish practices for Git-based development, automated testing, CI/CD, release management, observability and operational support, and apply data quality checks, monitoring and secure software supply-chain practices.
• Identify and engineer production AI solutions covering data preparation, model or foundation-model integration, retrieval, tool use, evaluation, deployment, monitoring and cost management.
• Translate user and business needs into product outcomes and delivery plans, and lead delivery across globally distributed teams (UK, Chennai, Barcelona, Guadalajara).
• Coach senior engineers and technical leads, promote responsible and well-governed use of AI, and manage technical debt, platform risks, performance, dependencies and cloud costs.

Required profile

• Experience as a Data Engineering Lead, Software Engineering Lead, Platform Engineering Lead, AI Engineering Lead or equivalent.
• Strong hands-on background building and operating production data, software or AI products in the cloud.
• Experience partnering with product, architecture, security, governance, data science and business teams.
• Ability to make pragmatic architectural decisions and communicate complex technical topics clearly.
• Experience coaching engineers and raising engineering standards.
• Desirable: experience in pharmaceutical, biotechnology, healthcare or clinical research, and understanding of drug discovery or scientific data.

Required skills

• Python and SQL, including testing, code review, debugging and optimisation.
• Data modelling, batch and streaming pipelines, orchestration, data quality, metadata, lineage, data contracts and observability.
• AWS, Azure or comparable platforms, including CI/CD, infrastructure as code, containers and production operations.
• Desirable: Terraform, Spark, dbt, Power BI, LLM APIs, RAG, data mesh, third normal form and dimensional modelling.

The role works primarily with the AWS data and analytics ecosystem while collaborating with teams using Azure. On average, three days per week are spent in the office, with flexibility for individuals.