Senior AI Test Automation Engineer (100%)

About the role

Julius Baer is hiring a Senior Test Automation Engineer (100%, f/m/d) for its ML & AI ART in Madrid. You will own the technical test automation solution supporting the bank's AI and MLOps platform, which covers RAG systems, LLM-based assistants, agentic workflows and ML pipelines. You will define the automation approach, build reusable patterns, run the tests in CI/CD and act as the Test Automation Ambassador within the ART.

Key responsibilities

• Define and evolve the test automation approach and framework architecture, aligned with the Bank's Test Strategy and Test Policy
• Design reusable, scalable patterns (page objects, API clients, test data builders) adopted across squads
• Analyse requirements and features for testability, derive test cases from risk analysis and automate them with Python-based frameworks (Playwright-Python, requests + pytest, Behave or pytest-bdd)
• Build AI/ML test cases such as LLM evaluation pipelines and contract tests (Pact) for APIs and microservices
• Integrate tests into GitLab CI/CD pipelines, use Docker and Kubernetes for test environments, and schedule runs across DEV, INT, UAT and pre-PROD
• Monitor execution health, quarantine flaky tests and raise defects in Jira with evidence
• Produce audit-ready execution evidence and support traceability from stories to tests to defects
• Write and maintain BDD scenarios in Gherkin, prepare synthetic or anonymised test data, and refactor tests when models, APIs or prompts change
• Take part in SAFe ceremonies and collaborate with Product Owners, Scrum Masters, ML and MLOps engineers

Required profile

• Bachelor's or Master's degree in Computer Science, Engineering or a related field
• 5-7 years in test automation with substantial hands-on Python experience and framework ownership
• Experience in a regulated environment (financial services, healthcare, pharma) strongly preferred
• ISTQB Foundation certification; SAFe or DevOps certifications are a plus
• Fluent English; German is a plus

Required skills

• Python, Playwright-Python, Behave or pytest-bdd, requests + pytest
• GitLab CI/CD, Docker, Kubernetes basics
• Jira and test management tools such as Xray or Agile Hive
• Reporting tools such as Allure or pytest-html
• Understanding of SAFe, DevOps and testing AI/ML systems