DataOps Engineer

Job Description

The DataOps team is responsible for the seamless integration, release, and monitoring of mission-critical data pipelines and workflows.

Tasks and responsibilities

The duties and responsibilities include:

• Assumption of operational responsibility for specific data services (change management, quality assurance, SLA, SOP)
• Monitoring and ensuring the operational continuity of critical data pipelines within the group during Asian business hours, using a follow-the-sun support model
• Design, implementation, and optimization of data pipelines in batch and streaming modes
• Using CI/CD practices to automate and optimize deployments and integrations
• Development, maintenance, and improvement of documentation and operating procedures as part of operational responsibilities
• Collaborating with cross-functional teams to review governance and improve integration operations
• Maintaining high standards of data quality, reliability, and performance
• Stay up to date with advancements in data engineering and DataOps technology to support continuous improvement

Requirements and qualifications

The requirements and qualifications include:

• Experience in data engineering and data operations roles
• Practical experience with SQL, Python, PySpark, and DataBricks
• Knowledge of microservice architectures, API management, and observability is highly desirable
• Solid understanding of CI/CD platforms and processes (e.g., Azure DevOps, GitLab CI, GitHub Actions)
• Experience with cloud data platforms (Azure preferred)
• Demonstrated ability to create rigorous, clear, and comprehensive documentation and procedures
• Familiarity with database design
• Excellent problem-solving skills and attention to detail
• Effective communication and collaboration skills

We offer

We offer a supportive environment where your contributions are valued and recognized. Our values include promoting diversity and inclusion to foster better decision-making and greater innovation.