Data Operations Engineer

Tasks

• Cloud operations for a data platform: platform deployment, configuration management, and processing
• Full data operations in real time
• Participate in the design and implementation of centralized daily ETL pipelines used for batch processing activities required to support data analytics needs.
• Monitor the performance of ETL sessions and operations, and identify the root causes of performance issues to ensure that key business processes are scheduled according to the calendar.
• Map the respective operational requirements and establish data quality and standardization processes and KPIs.
• Assist the data team in rolling out corporate services and delivering daily and periodic releases, RAs, and KPIs.
• Administer and monitor UAT testing and application testing for the cloud/data platform.
• Work with the Data Science team and the Product Management team in partnership with the business, and be responsible for daily data operations (ETL jobs, validations, monitoring).
• Develop and maintain data integration infrastructure to support dashboards and visualization tools.
• Gather and analyze cloud and data operations requirements, and design and implement data integrations.
• Create, validate, and update documentation, including technical requirements, design, and implementation plans.
• Identify and manage issues related to operational activities.
• Monitor data platform operations, including monitoring performance, availability, and issue management.

Requirements

• A Level 1 Cloud Operations specialist is an excellent team player and has strong communication and problem-solving skills.
• Candidates must have experience with Big Data, SQL, Python, and Apache Spark, as well as AWS, Azure, etc.
• Self-motivated, flexible, and able to write clear documentation relevant to the tasks at hand (code, DBAs, SLAs, SQL, ELK Stack).
• Knowledgeable in data science methods, algorithms, and machine learning libraries (MLlib, etc.).
• Experience with Azure DevOps
• Knowledge of distributed systems and scalability
• Experience with relational and non-relational databases using SQL
• A high degree of complexity, vulnerability, and sensitivity
• Ability to maintain confidentiality and keep records of assets
• Eager to learn and grow in a challenging environment.
• Passionate about data, learning, and technology, with a strong belief that data and products can be both business-driven and user-driven.

We offer

• Accountability: Taking ownership of safe and efficient data engineering and reporting systems
• Flexibility: Being responsive to project changes and meeting deadlines
• Dedication: Demonstrating responsibility and commitment to project deliverables and stakeholder initiatives
• Empowerment: Promoting inclusivity and training others to take responsibility for mission-critical operations