Job Description

17 July 2023

A great opportunity as a Data Engineer to work in cloud based data engineering and data management within the financial sector.

Mandatory Skill(s)

• Bachelor\'s Degree in Computer Science or equivalent;
• Minimum 3 years of experience in data engineering or software engineering;
• Possess a solid foundation in Computer Science concepts such as Cloud Computing Architecture, Distributed Computing, High-Velocity Data Processing, and Lambda Architecture;
• Proficient in Data modeling and managing Distributed Computing Platforms for efficient Data Processing;
• Strong hands-on experience on SQL and Pyspark;
• Thorough understanding of Kubernetes clusters and their scalability capabilities;
• Demonstrated implementation of manual or automated tools for Data Quality, Catalog, and Lineage;
• Excellent interpersonal and presentation skills to effectively communicate and advocate for Best Practices across the organization, catering to both technical and non-technical stakeholders.

Desirable Skill(s)

• Certifications in AWS.

Responsibilities

• Extensive expertise in constructing Data Pipelines utilizing Spark on AWS EMR or Glue;
• Orchestrating Batch Pipelines through Apache Airflow;
• Deploying jobs on AWS EKS (Kubernetes);
• Implementing a Code Repository Pipeline for automating Continuous Integration and Continuous Deployment (CI/CD);
• Enhancing Cost and Performance Benefits by structuring tables with Partitioning and Clustering;
• Providing guidance to Data Analysts and Data Scientists to optimize queries and workloads;
• Enabling secure Data Sharing with scalable On-Demand Encryption/Decryption capabilities;
• Executing Containerized ETL workflows at a large scale.

If you are interested in this role, click on the \xe2\x80\x9cApply to this job\xe2\x80\x9d button below or you could also write in with your CV to Valorie Chu at quoting the job title.

EA Reg No.: R1984038
Lead Consultant
Let\xe2\x80\x99s connect via

Sciente Consulting