Senior Databricks Administrator / Architect (Remote)

About the role

A major European institution is looking for a fluent English-speaking Senior Databricks Administrator/Architect to work remotely. For the first assignment, the selected candidate joins the Development team and works closely with a major client's IT team.

Key responsibilities

• Architect and implement scalable enterprise data platform solutions with Azure Databricks, Delta Lake, Unity Catalog, Azure Data Lake Storage Gen2, Azure Data Factory, Azure Event Hub, Azure SQL and related Azure services.
• Design and develop data engineering frameworks for batch, streaming, ELT/ETL and real-time analytics workloads.
• Administer Azure Databricks workspaces, including clusters, jobs, Unity Catalog, secrets, compute and cluster policies and workspace governance.
• Implement enterprise security: Microsoft Entra ID integration, RBAC, PIM, encryption, private networking, managed identities and Key Vault.
• Set up data governance with Unity Catalog, covering lineage, quality, metadata, access policies and audit.
• Build CI/CD pipelines with Azure DevOps, Databricks Asset Bundles and IaC (Terraform/Bicep), and manage releases and environment promotion.
• Tune workloads: Spark optimization, cluster sizing, caching, partitioning, Delta optimization, Photon and cost optimization.
• Set up monitoring, logging, dashboards and alerts with Azure Monitor, Log Analytics, Application Insights and Databricks system tables.
• Create reusable patterns, standards, documentation and support migrations to the Lakehouse architecture.
• Provide troubleshooting for production incidents on Databricks, Spark pipelines and infrastructure.

Required profile

• University degree plus 9 years of IT professional experience, including at least 8 years designing enterprise data platforms.
• At least 5 years of hands-on Azure Databricks experience in enterprise environments.
• Experience with Lakehouse solutions, Unity Catalog, Spark (PySpark/Spark SQL), Terraform or Bicep, Azure DevOps and enterprise security architectures.
• Desirable: experience with large international organizations, NGOs or UN agencies, and the listed certifications.
• Fluent English.

Required skills

• Azure Databricks, Delta Lake, Unity Catalog, Azure Data Lake Storage Gen2, Azure Data Factory, Azure Event Hub, Azure SQL, Azure Key Vault, Azure Monitor, Microsoft Entra ID, Azure networking.
• SQL, Python and PySpark.
• Dimensional modelling, Data Vault, medallion architecture and data warehousing concepts.
• AI/ML workloads, Databricks ML, Feature Store, Model Serving and Generative AI are an asset; Power BI, Microsoft Fabric or Synapse Analytics is highly desirable.

What we offer

• Competitive remuneration, either on contract basis or with a full benefits package, depending on qualifications and experience.
• Work within a team of a major European institution in an international, multilingual and multicultural environment.