Principal, Cloud Data Engineer

Job Overview

The Principal Cloud Data Platform Engineer is responsible for designing, building, and maintaining scalable, secure, and high-performance cloud-based data platforms that support analytics, machine learning, and enterprise data needs.

Responsibilities

The tasks include:

• Monitoring and optimization of data pipelines for the collection, transformation, processing, and structuring of structured, semi-structured, and unstructured data.
• Creation of scalable frameworks and infrastructures for data workflows, monitoring, and scaling.
• Ensuring that data storage and computing solutions are optimized for performance, security, and cost efficiency.
• Integration of cloud data services with enterprise applications, APIs, and third-party tools.

Data Management & Governance

The tasks include:

• Implementation of data quality, line, and catalog solutions across the data platform.
• Enforcement of governance, security, and compliance standards (e.g., GDPR, HIPAA, SOX).
• Collaborate with architects and security teams to implement data at rest and in motion encryption and audit control.
• Support for metadata management, data discoverability, and enterprise-wide data democratization.

Operations & Reliability

The tasks include:

• Management of the operational status of the cloud data platform, including monitoring, alerting, and troubleshooting.
• Setup of CICD/CI/CD pipelines for data services and platform components.
• Implementation of disaster recovery, backup, and failover strategies.
• Continuous optimization of platform performance, costs, and scalability.

Collaboration & Innovation

The tasks include:

• Collaborate with data scientists, BI teams, and application developers to enable advanced analytics and AI/ML workloads.
• Evaluate emerging cloud data services and recommend adoption based on business needs.
• Provision of technical expertise in the design of cloud-based data architectures and solutions.
• Mentoring junior engineers and contributing to best practices in cloud engineering and data development.

Qualifications

The requirements include:

• Degree in computer science, data engineering, or a related field (master's degree preferred).
• 10+ years of experience in technology infrastructure.
• 10+ years of experience in data engineering, cloud engineering, or platform engineering.
• Practical experience with at least one major cloud provider (AWS or Azure).

Further requirements

The requirements include:

• Experience in regulated environments (financial services, healthcare, etc.).
• Knowledge of SQL and at least one programming language (Python, Java, or Scala).
• Deep understanding of cloud-native data services (e.g., AWS Redshift, Glue, S3, Athena, Azure Synapse, Data Factory, GCP BigQuery/Spanner).
• Familiarity with containerization and