Data Operations Engineer

Job Description

The DataOps Platform Team's mission is to ensure a reliable service level across all technological components of the EFG data platform (Databricks, Kafka, Spark, Elasticsearch, Apache MQ, and the uServices stack).

Main tasks

As our platform and data landscape continue to evolve, we’d like to strengthen our team with an experienced CICD/CV engineer who specializes in data pipeline implementations. - We’re looking for someone who is comfortable integrating Kafka, Spark, and Databricks operations using Python. - By leveraging our team of experienced data services specialists (change management, quality management, SLA, SOR), you will play a key role in ensuring the continuity and quality of our operations:

• Daily monitoring of pipeline services (change management, quality management, SLA, SOR).
• Ensuring the smooth operation of critical data pipelines during Asian business hours, serving as the primary point of contact for critical issues in a "follow-the-sun" support model.
• Design, implement, and optimize data pipelines in batch and streaming modes.
• Using CI/CD practices to automate and streamline deployments and integrations.
• Create, maintain, and improve documentation and operating procedures, ensuring that the company retains ownership of them.
• Collaborate with cross-functional teams to improve and review governance related to integrations.
• Maintain high standards of data quality, reliability, and performance.
• Stay up to date with advances in data engineering and the DataOps technology stack to support continuous improvement.

Requirements and qualifications

- At least 5 years of hands-on experience in data engineering and data pipeline deployments based on DevOps practices. - Experience with Python, SQL, PySpark, and Databricks. - Solid experience with leading CI/CD platforms and processes in hybrid environments (e.g., Azure DevOps, GitLab CI, GitHub Actions). - Familiarity with microservices architecture. Experience with API management and observability is a definite plus. - Experience with Azure Databricks. - Strong communication and collaboration skills. - Rigorous and methodical in documentation and process creation. - Strong interest in convergence, standardization, and reusability. - Curious, enthusiastic, and proactive. - Passionate about innovative technologies, with a strong desire to learn and grow in a challenging environment. - Fluent in spoken and written English; French is a plus.

Our values

- Responsibility: Taking on tasks and challenges, and striving for continuous improvement. - Hands-on: Proactive and quick to deliver high-quality results. - Passionate: Committed and striving for excellence. - Solution-oriented: Focus on customer outcomes and fair treatment of customers, with a risk-aware mindset. - Partnership-oriented: Fostering collaboration and teamwork. Working together with an entrepreneurial spirit.