Job Description

We are seeking a Senior Data Engineer to

work directly with agencies, enabling them to reach their desired development delivery efficiency and system resiliency through optimized Continuous Integration/Continuous Deployment (CI/CD) and Site Reliability Engineering (SRE) practices.

As a forward-deployed engineer, you'll

partner with agencies to solve complex challenges, gather insights, and drive continuous improvement in our product offerings . This role

combines deep technical expertise with a commitment to customer enablement and feedback, ensuring that our Core Engineering Products evolve to address the real-world needs of our customers.

As a Senior Data Engineer, you will be working on:

Translate data requirementsfrom business users into technical specifications.

Collaborate with partner agency's IT teams on technology stack, infrastructure and security alignment.*

Build out data productas part of a data team:

Architect and build ingestion pipelines to collect, clean, merge, and harmonize data from different source systems.*

Day-to-day monitoring of databases and ETL systems, e.g., database capacity planning and maintenance, monitoring, and performance tuning; diagnose issues and deploy measures to prevent recurrence; ensure maximum database uptime;

Construct, test, and update useful and reusable data modelsbased on data needs of end users.

Design and build secure mechanisms for end users and systemsto access data in data warehouse.

Research, propose and develop new technologies and processesto improve agency data infrastructure.

Collaborate with data stewards to establish and enforce data governance policies, best practices and procedures.*

Maintain data catalogueto document data assets, metadata and lineage.

Implement data quality checks and validation processesto ensure data accuracy and consistency.

Implement and enforce data security best practices,including access control, encryption, and data masking, to safeguard sensitive data.

What we are looking for:

A Bachelor's Degree, preferably in Computer Science, Software Engineering, Information Technology, or related disciplines. minimum

7 years of relevant experience

Deep understanding of system design, data structure and algorithms, data modelling, data access, and data storage.*

Demonstrated ability in using cloud technologiessuch as AWS, Azure, and Google Cloud.

Experience with Databricks.

Experience in designing, building, and maintaining batch and real-time data pipelines.*

Experience with orchestration frameworkssuch as Airflow, Azure Data Factory.

Proficiency in working with Python, Shell Scripts, and SQL.*

Preferred requirements:

Familiarity with building and using CI/CD pipelines.*

Familiarity with DevOps toolssuch as Docker, Git, Terraform.

Experience with implementing technical processes to enforce data security, data quality, and data governance.*

Familiarity with government systems and government's policies relating to data governance, data management, data infrastructure, and data security.*

Experience in Climate and Weather domainswill be an advantage.


Working hours : 8.30am - 6pm

We regret to inform that only shortlisted candidates will be notified. Personal data collected will be used for recruitment purposes