Senior Test Manager & Quality Engineer, Full-Time (f/m/d)

Tasks

• Ensure that the quality of the data and the proper execution of tests for AI use cases are guaranteed
• Define and further develop the technical testing approach and framework architecture for ML and AI-ART, in alignment with the bank's testing strategy and testing guidelines
• Design reusable, scalable test patterns (page objects, API clients, test data builders) that can be adopted by other engineers in different squads to ensure technical consistency of tests across the ART
• Analyze and evaluate requirements, features, and user stories in terms of their testability during PI planning, backlog refinement, and iteration planning
• Derive test cases from technical and risk-based analyses of functional and non-functional requirements (reliability, performance, security, usability, robustness), and select appropriate test techniques and levels of automation based on risk, coverage goals, and ROI
• Automate identified test cases using Python-based frameworks—Playwright-Python for UI, requests + pytest-bdd for BDD/Gherkin—while applying clean code best practices to ensure reusability, readability, and stability
• Design and implement AI/ML-specific test cases: evaluation pipelines for LLM outputs
• Integrate and orchestrate automated tests into GitLab CI/CD pipelines, including merge request pipelines and GitLab Runners
• Plan, schedule, and trigger automated test runs in various environments (DEV, INT, UAT, pre-PROD), including regression suites, smoke tests, release runs, and on-demand runs that are linked to merge requests and PI milestones
• Compile the results of the runs, report bugs in JIRA with supporting evidence (logs, traces, screenshots, videos), and communicate quality signals to the squad and the product owner
• Actively contribute to PI planning, system demos, Inspect & Adapt, and other SAFe ceremonies as part of the ML and AI ART
• Prepare test data, and ensure that synthetic or anonymized data is used whenever possible to meet confidentiality expectations

Requirements

• Practical experience in integrating and conducting tests for AI solutions and/or big data projects
• A solid understanding of Git and version control workflows, clean code principles, and code review culture
• Practical experience with Docker; familiarity with the basics of Kubernetes (jobs, namespaces)
• Exposure to testing AI/ML systems or a strong motivation to develop this expertise: evaluating LLM outputs, handling non-deterministic responses, and assessments for RAG and agentic workflows
• Understanding of API design, microservices, event-driven architectures, and authentication layers
• A collaborative team player with a strong sense of personal responsibility who takes the lead on automation challenges—from analysis through implementation to resolution—with minimal supervision