Data Operations Engineer

Job Description

The DataOps Platform Team’s mission is to ensure a reliable service level across all technological components of the EFG data platform (Databricks, Kafka, Spark, Elasticsearch, Apache MQ, Sidences Stack). We build and develop the CI/CD platform that supports data pipeline deployments and work closely with project teams to deliver results.

Main tasks

As a Data Ops Engineer, your responsibilities will include:

• Take over operational responsibility for specific data services (change management, quality management, SLA, SOP)
• Monitor the smooth operation of critical data pipelines during Asian business hours and serve as the primary point of contact for critical issues in a follow-the-sun support model
• Design, implement, and optimize data pipelines in batch and streaming modes
• Use CI/CD practices to automate and streamline deployments and integrations
• Create, maintain, and improve documentation and operating procedures for the services provided
• Work with cross-functional teams to improve and refine governance around integrations
• Maintain high standards of data quality, reliability, and performance
• Stay up to date on advancements in the data engineering and DataOps technology stack to support continuous improvement

Requirements and qualifications

We are looking for a candidate with:

• At least 5 years of experience in data engineering and data pipeline implementation using DevOps practices
• Proficiency in Python, SQL, PySpark, and Databricks
• Extensive experience with leading CI/CD platforms and processes in hybrid environments (e.g., Azure DevOps, GitLab CI, GitHub Actions)
• Familiarity with microservices architecture, API management, and observability is a plus
• Experience with Azure Databricks
• Strong communication and collaboration skills
• Thorough and methodical in documentation and process development
• Interest in convergence, normalization, and reusability
• Curious, enthusiastic, and proactive
• Passionate about data processing technologies, with a strong desire to learn and grow in a challenging environment
• Fluency in spoken and written English; French is a plus

Our values

We value:

• Responsibility: Taking ownership of tasks and challenges and striving for continuous improvement
• Practical: Be proactive to deliver high-quality results quickly
• Passionate: Be committed and excellent
• Solution-oriented: Focusing on client results and treating clients fairly and with an awareness of risks
• Partnership-oriented: Fostering collaboration and teamwork and working with an entrepreneurial spirit