Data Operations Engineer

Job Description

The DataOps Platform Team’s mission is to ensure a reliable service level across all technological components of the EFG data platform (Databricks, Kafka, Spark, Elasticsearch, Apache MQ, and the uServices stack). We build and develop the CI/CD platform that supports data pipeline deployments and work closely with project teams to deliver results.

Main Responsibilities

As our platform and data landscape continues to evolve, we are looking to strengthen our team with an experienced CI/CD engineer who specializes in data pipeline implementations. - Expectations for data quality are constantly rising, along with the need for extremely stable service delivery. We must ensure that our delivery processes are controlled and transparent to our stakeholders. - We are looking for someone who is comfortable integrating Kafka, Spark, and Databricks operations with Python. - By remotely monitoring our sites, you will contribute to the overall continuity and quality of our operations. - Take responsibility for EFG data services (change management, quality management, SLA, SOP), monitor and ensure that critical data pipelines run smoothly during Asian business hours, and act as the primary point of contact for critical issues in a follow-the-sun support model. - Design, implement, and optimize data pipelines in batch and streaming modes. - Follow and apply CICD practices to automate and streamline deployments and integrations. - Create, maintain, and improve documentation and operational procedures for which operational responsibility is ensured. - Collaborate with cross-functional teams to improve and review governance over integration operations. - Maintain high standards for data quality, reliability, and performance. - Stay up to date with advancements in data engineering and data pipeline technologies to support continuous improvement.

Requirements and Qualifications

- At least 5 years of verifiable hands-on experience in data engineering and data pipeline deployments based on DevOps practices. - Experience with Python, SQL, PySpark, and Databricks. - Solid experience with leading CI/CD platforms and processes in hybrid environments (e.g., Azure DevOps, GitLab CI, GitHub Actions). - Familiarity with microservices architecture, API management, and observability is a plus. - Experience with Azure Databricks. - Strong communication and collaboration skills. - Rigorous and methodical in documentation and process creation. - Strong interest in convergence, standardization, and reusability. - Curious, enthusiastic, and proactive. - Passionate about data processing technologies, with a strong desire to learn and grow in a challenging environment. - Fluent in spoken and written English; French is a plus.

Our Values

- Responsibility: Taking ownership of tasks and challenges, and striving for continuous improvement. - Hands-on: Proactive in delivering high-quality results quickly. - Passionate: Committed and striving for excellence. - Solution-oriented: Focus on customer outcomes and fair treatment of customers with a risk-aware mindset. - Partnership-oriented: Promoting collaboration and teamwork. Working together with an entrepreneurial spirit.