Job offer
Senior Data Engineer
We are looking for an experienced Senior Data Engineer for a 12-month contract in Singapore to implement methods and solutions for improving data quality and reliability and to collaborate with data scientists and data analysts. The ideal candidate will have extensive experience in data engineering, data modeling, and the development of data pipelines and platforms.
Tasks
The main tasks include:- Implementation of methods and solutions to improve data reliability and quality
- Collaborate with data scientists and data analysts to develop data models and pipelines for research, reporting, and machine learning
- Takeover and maintenance of the data platform to ensure stability, performance, and continuous improvement
Requirements
The requirements for this position are:- 5+ years of experience in data processing and workflow management tool integration into pipeline design
- 5+ years of experience in ETL/ELT, data warehousing, and/or business intelligence development
- 5+ years of experience in building and maintaining end-to-end data systems and supporting services in Python, Scala, or similar
- 5+ years of experience with SQL to understand data, investigate data issues, and provide solutions
- 3+ years of experience with cloud data technologies
- Experience with structured, semi-structured, and unstructured data
- Experience with various databases, including data warehouses, RDBMSes, in-memory caches, and searchable document databases
- Experience with large data sets in SQL (Gatabricks/PySpark)
- Experience with Spark Streaming and Delta (Live) tables
- Experience with storage systems such as Azure Storage/Data Lakes
- Experience with data modeling
- Strong design, implementation, and testing skills
- Experience with continuous integration and automated deployment
- Experience with cloud platforms (preferably Azure) in a continuous delivery environment
Preferred qualifications
Preferred qualifications are:- Experience with microservices platforms (Kubernetes, Docker, Helm Charts)
- Experience with event-driven streaming systems (Kafka, Event Hub, Event Grid, Apache Flink)
- Knowledge and application of OBT
- Experience with data validation as well as
- Experience with BI tools
- 1+ years of software development with languages such as JavaScript, Java, or C#
- Ability to provide technical leadership to other developers
We offer
We offer a challenging and interesting work environment where you can use your skills and experience to contribute to the success of our company.Job details
Training opportunities powered by skillaware.