Data Operations Engineer

Job Description

The DataOps Platform Team’s mission is to ensure a reliable service level across all technological components of the EFG data platform (Databricks, Kafka, Spark, Elasticsearch, Apache MQ, and the uServices stack). We build and develop the CI/CD platform that supports data pipeline deployments and work closely with project teams to get releases over the line.

Main tasks

As our platform and data landscape evolve, we’d like to strengthen our team with an experienced CI/CD engineer who specializes in data pipeline implementations. - Expectations for data quality are constantly rising, and there’s a need for stable service delivery. We must ensure that our delivery processes are controlled and acceptable to our stakeholders. - We are looking for someone who is comfortable managing Kafka, Spark, and Databricks operations using Python. - By strengthening our team, you will make a significant contribution to the continuity and quality of our operations. - This position involves taking ownership of specific data services (change management, health management, SLA, SORP). - Monitor and ensure the smooth operation of critical data pipelines during Asian business hours and serve as the primary point of contact for critical issues in a follow-up support model. - Design, implement, and optimize data pipelines in batch and streaming modes. - Follow and apply CI/CD practices to automate and streamline deployments and integrations. - Create, maintain, and improve documentation and operational procedures, ensuring operational ownership. - Collaborate with cross-functional teams to improve and review governance around integrations. - Maintain high standards for data quality, reliability, availability, and performance. - Stay up to date with advancements in data engineering technology and the DataOps technology stack to support continuous improvement.

Requirements and qualifications

- At least 5 years of hands-on experience in data engineering and data pipeline implementation based on DevOps practices. - Experience with Python, SQL, PySpark, and Databricks. - Solid experience with one of the leading CI/CD platforms and processes in hybrid environments (e.g., Azure DevOps, GitLab CI, GitHub Actions). - Familiarity with microservices architecture, API management, and observability is a definite plus. - Experience with Azure Databricks. - Strong communication and collaboration skills. - Rigorous and methodical in documentation and process creation. - Strong interest in convergence, standardization, and reusability. - Curious, enthusiastic, and proactive. - Passionate about data processing technologies, with a strong desire to learn and grow in a challenging environment. - Fluent in spoken and written English; French is a plus.

Our values

-

• Responsibility: Taking on tasks and challenges and striving for continuous improvement.
• Practical: Practical implementation to deliver high-quality results quickly.
• Passionate: Dedicated and committed to excellence.
• Solution-oriented: Focus on client results and fair treatment of clients, guided by a risk-conscious approach.
• Relationship-oriented: För