Data Engineer (Scala/Spark)

About the role

Inetum, an international digital consultancy, is hiring a Data Engineer in Madrid under a hybrid work arrangement. The role centres on migrating the existing Hadoop infrastructure to cloud-based services and building reliable, automated data pipelines.

Key responsibilities

• Migrate the existing Hadoop infrastructure to cloud infrastructure on Kubernetes Engine, COS, Spark as a service and Airflow as a service.
• Implement data transformation and quality checks to ensure data consistency and accuracy, using Scala, SQL and Spark.
• Set up CI/CD pipelines to automate deployments, unit testing and development management.
• Write and run unit and validation tests to ensure code accuracy and integrity.
• Automate data pipelines and streamline data ingestion with orchestrators and schedulers, mainly Airflow as a Service.
• Write technical documentation (specifications, operational documents).
• Collaborate with cross-functional teams to understand data requirements and deliver solutions, and foster continuous learning within the team.

Required profile

• English at C1 level; French is optional.
• Knowledge of banking industry and processes.
• Problem-solving, results-driven approach and ability to support change.
• Teamwork with data squads and business teams.

Required skills

• Spark on Scala and strong SQL and NoSQL database knowledge.
• CI/CD tools such as GitLab and Jenkins.
• HDFS, Apache Airflow, S3 storage, COS and Parquet/ORC formats.
• Full understanding of Oozie, shell scripting, SDLC and Agile principles.
• Some knowledge of Kubernetes containerization, Dremio, Elasticsearch/Kibana, Kafka and Vault is a plus.