Job offer
Senior Data Engineer
We are looking for an experienced Senior Data Engineer for a 12-month contract project in Singapore to lead the implementation of methods and solutions aimed at improving data quality and reliability. The ideal candidate has extensive experience in data engineering, data modeling, and the development of data pipelines and services.
Tasks
The main tasks include:- Implementation of methods and solutions to improve data reliability and quality
- Collaborate with data scientists and data analysts to develop data models and pipelines for research, reporting, and machine learning
- Takeover and maintenance of the data platform to ensure stability, performance, and continuous improvement
Requirements
The requirements for this position are:- 5+ years of experience in data processing and workflow management tool integration in pipeline development
- 5+ years of experience in ETL/ELT, data warehousing, and/or business intelligence development
- 5+ years of experience in building and maintaining end-to-end data systems and supporting services in Python, Scala, or similar languages
- 5+ years of experience with SQL to understand data, investigate data issues, and provide solutions
- 2+ years of experience with cloud data technologies
- Experience with structured, semi-structured, and unstructured data
- Experience with various databases, including data warehouses, RDBMSes, in-memory caches, and searchable document databases
- Experience with large data sets in SQL (Gatabricks/PySpark)
- Experience with Spark Streaming and Delta (Live) tables
- Experience with storage systems such as Azure Storage/Data Lakes
- Experience with data modeling
- Strong design, implementation, and testing skills
- Experience with continuous integration and automated deployment
- Experience with cloud platforms (preferably Azure) in a continuous delivery environment
Preferred qualifications
Preferred qualifications are:- Experience with microservices platforms (Kubernetes, Docker, Helm Charts)
- Experience with event-driven streaming systems (Kafka, Event Hub, Event Grid, Apache Flink)
- Knowledge and application of OBT
- Experience with data validation as well as
- Experience with BI tools
- 1+ years of software development with languages such as JavaScript, Java, or C#
- Ability to provide technical leadership to other developers
We offer
We offer a challenging and interesting work environment where you can use your skills and experience to contribute to the success of our company.Job details
Training opportunities powered by skillaware.