Senior AI Test Automation Engineer

Tasks

• Define and further develop the technical test automation approach and test framework architecture for ML and AI-ART
• Designing reusable, scalable test automation patterns (page objects, API clients, test data builders)
• Analyzing and evaluating requirements, features, and user stories in terms of testability
• Deriving test cases from technical and risk analyses of functional and non-functional requirements
• Automating test cases using Python-based frameworks (Playwright/Python, requests + pytest, Behave, or pytest-bdd)
• Designing and implementing A/ML-specific test cases (evaluation pipelines for LLM outputs)
• Creating and maintaining contract tests (e.g., Pact) for platform APIs and microservice boundaries
• Integrating and Orchestrating Automated Test Phases in GitLab CI/CD Pipelines
• Planning, scheduling, and triggering automated test runs across environments
• Monitoring execution health, investigating and quarantining flaky test cases, and maintaining a low false-positive rate
• Prioritizing execution results, reporting errors in Jira with supporting evidence, and communicating quality signals
• Generating execution evidence (run metadata, artifacts, reports) for audit and release governance

Requirements

• Proven expertise in Python-based test automation (Playwright/Python, Behave, or pytest-bdd, requests + pytest)
• Proven ability to design and maintain test automation frameworks, including reusable utilities and maintenance patterns
• Practical experience with integrating and running automated tests in CI/CD pipelines
• Experience with test execution, parameterization, flaky test management, and modern reporting tools (Allure, pytest-html, or equivalent)
• Solid understanding of Git and version control workflows, ability to write clean code, and a culture of code review
• Practical knowledge of Docker, familiarity with Kubernetes fundamentals (pods, namespaces)
• Experience with AI/ML systems gained through testing, development, or applied projects is a strong plus; the ability to develop and demonstrate AI/ML testing expertise is essential
• Fluency in English; Spanish is a plus

We offer