Data Operations Engineer

Job Description

The DataOps Platform Team’s mission is to ensure a reliable service level across all technological components of the EFG data platform (Databricks, Kafka, Spark, Elasticsearch, Apache MQ, and the uServices stack). We build and develop the CI/CD platform that supports data pipeline deployments and work closely with project teams to deliver results.

Main Responsibilities

As our platform and data landscape continues to evolve, we are looking to strengthen our team with an experienced CI/CD engineer who specializes in data pipeline implementations. - Expectations for data quality are constantly rising, and we need to ensure that our delivery processes are controlled and transparent to our stakeholders. - We are looking for someone who is comfortable integrating Kafka, Spark, and Databricks operations with Python. - By reducing our technical debt, you will make a significant contribution to the continuity and quality of our operations. - Take responsibility for data services (change management, quality assurance management, SLA, SOP, tracking, and ensuring the smooth operation of critical data pipelines during Asian business hours, serving as the primary point of contact for critical issues in a 24/7 follow-the-sun support model. - Design, implement, and optimize data pipelines in batch and streaming modes. - Follow and apply CI/CD practices to automate and streamline deployments and integrations. - Create, maintain, and improve documentation and operational procedures for which operational responsibility is ensured. - Collaborate with cross-functional teams to improve and review governance over integration operations. - Maintain high standards for data quality, reliability, and performance. - Stay up to date with advancements in data engineering and data pipeline technology to support continuous improvement.

Requirements and Qualifications

- At least 5 years of hands-on experience in data engineering and data pipeline implementation based on DevOps practices. - Experience with Python, SQL, PySpark, and Databricks. - Solid experience with leading CI/CD platforms and processes in hybrid environments (e.g., Azure DevOps, GitLab CI, GitHub Actions). - Familiarity with microservices architecture, API management, and observability is a plus. - Experience with Azure Databricks. - Strong communication and collaboration skills. - Rigorous and methodical in documentation and process creation. - Strong interest in convergence, standardization, and reusability. - Curious, enthusiastic, and proactive. - Passionate about data processing technologies, with a strong desire to learn and grow in a challenging environment. - Fluent in spoken and written English; French is a plus.

Our Values

- Accountability: Taking responsibility for tasks and challenges, and striving for continuous improvement. - Hands-on: Proactive in delivering high-quality results quickly. - Passionate: Committed and striving for excellence. - Solution-oriented: Focus on customer outcomes and fair treatment of customers with a risk-aware mindset. - Partnership-oriented: Promoting collaboration and teamwork. Collaborating with an entrepreneurial spirit.