Senior AI Test Automation Engineer

Tasks

• Define and further develop the test automation approach and test framework architecture for ML and AI-ART
• Designing reusable, scalable test automation patterns (page objects, API clients, test data builders)
• Analyzing and evaluating requirements, features, and user stories for testability during PI planning, backlog refinement, and iteration planning
• Deriving test cases from technical and risk analyses of functional and non-functional requirements
• Automating identified test cases using Python-based frameworks (Playwright/Python, requests + pytest, Behave, or pytest-bdd)
• Designing and Implementing AI-Specific Test Cases: Evaluation Pipelines for LLM Outputs
• Creating and maintaining test contracts (e.g., Pact) for platform APIs and microservice boundaries
• Integrating and Orchestrating Automated Tests in GILab CI/CD Pipelines
• Use Docker and Kubernetes to provide isolated, reproducible test environments

Requirements

• Proven expertise in Python-based test automation: Playwright/Python (UI), Behave or pytest-bdd (BDD), requests + pytest (API)
• Proven ability to design and maintain test automation frameworks, not just write test scripts
• Practical experience integrating and running automated tests in CI/CD pipelines, ideally GitLab or equivalent enterprise platforms
• Experience with test execution, parameterization, flaky test management, and modern reporting tools (Allure, pytest-html, or equivalent)
• Solid knowledge of GILab and version control workflows, ability to apply clean code principles and foster a culture of code review
• Practical experience with Docker, familiarity with the basics of Kubernetes (jobs, namespaces)
• Experience in testing AI/ML systems, whether through testing, development, or applied projects, is a strong asset; the ability to develop and possess AI/ML testing expertise is essential
• Fluency in English; Spanish is a plus

We offer