Lead Engineer - Data Platform

About the role

DWS's data infrastructure team builds the secure, scalable and trustworthy foundation for storing, processing and distributing enterprise data. As Lead Engineer - Data Platform, you will drive the design, build and continuous improvement of a hybrid data platform, lead a small but high-impact team, set technical direction, mentor team members and contribute directly to architecture and code.

Key responsibilities

• Own the end-to-end architecture, design and evolution of enterprise data platform capabilities across hybrid cloud and on-premises environments
• Define the technical roadmap, engineering standards and reusable patterns for workflow orchestration, scheduling, dependency management and data processing
• Lead architectural decisions, proofs of concept and adoption plans based on scalability, interoperability, security, operability and total cost of ownership
• Establish reliability, observability and operational standards, including monitoring, alerting, failure recovery, capacity management and service-level objectives
• Drive platform automation through infrastructure as code, CI/CD, automated testing and self-service engineering
• Partner with architecture, security, governance and application teams to meet enterprise standards and regulatory requirements
• Stay hands-on in architecture and code, lead design and code reviews, mentor engineers and foster engineering excellence

Required profile

• 12+ years of experience in data or platform engineering, including designing and operating enterprise-scale data platforms
• A relevant degree is advantageous
• Proven technical leadership: setting engineering direction, reviewing designs and code, mentoring engineers and communicating with technical and business stakeholders
• Fluent written and spoken English
• Proven ability to leverage AI tools to improve productivity and workflows, with responsible and ethical use of data and AI outputs

Required skills

• Deep hands-on Snowflake expertise, including ingestion, transformation, workload management, performance, cost optimisation, secure data sharing and governance
• Production experience with Apache Airflow, including DAG patterns, scheduling, testing and monitoring
• Apache Kafka and event-driven architectures, including topic design, schema management, Kafka Connect and delivery semantics
• Batch and streaming pipeline integration across Snowflake, Airflow and Kafka
• Advanced Python and SQL; Java for Kafka-based services is beneficial
• Terraform, Git, CI/CD and automated testing; OpenShift or Kubernetes is desirable
• Understanding of reliability, observability, data quality, security, access management and compliance in regulated environments

What we offer

• A personalised benefits plan covering healthcare, company perks and retirement planning
• A hybrid working model balancing office collaboration and working from home, with the split clarified during the application and interview process