Job Description

Key Responsibilities

Data Pipeline Development

:

Design, implement, and optimize ETL/ELT pipelines to ingest, transform, and store structured and unstructured data.


Automate data processing workflows and monitor data pipeline health.

Data Architecture

:

Develop and maintain scalable data architectures to support analytics and machine learning.


Collaborate with stakeholders to define data storage, access, and integration strategies.

Data Modeling

:

Create and maintain robust data models for data warehouses, lakes, and marts.


Design efficient schemas for transactional and analytical use cases.

Cloud Infrastructure

:

Work with cloud platforms (e.g., AWS, Azure, GCP) to design and manage data solutions.


Leverage cloud-native tools and services for data integration, processing, and storage.

Performance Optimization

:

Monitor and improve data processing performance, storage efficiency, and query response times.


Implement best practices for data management and system scalability.

Collaboration

:

Partner with data scientists, analysts, and other engineers to meet data needs.


Ensure data availability, reliability, and accessibility for all stakeholders.

Data Governance

:

Ensure compliance with data governance and security standards.


Implement data quality checks and maintain data lineage documentation.

Required Skills & Qualifications

Bachelor's degree in Computer Science, Information Systems, or a related field. 10+ years of experience in data engineering, with a focus on building and managing large-scale data systems. Expertise in programming languages such as

Python, Java, Scala, or SQL

. Hands-on experience with data frameworks like

Apache Spark, Kafka, Hadoop

, or similar technologies. Strong knowledge of database systems (e.g.,

PostgreSQL, MySQL, MongoDB, Cassandra

). Proficiency in working with cloud platforms (e.g., AWS, Azure, GCP) and their data services (e.g.,

AWS Redshift, BigQuery, Snowflake

). Experience with data orchestration tools (e.g.,

Apache Airflow, Prefect

). Strong understanding of data modeling, normalization, and schema design. Proven ability to work with APIs, data lakes, and RESTful services.

Preferred Qualifications

Certifications in cloud platforms (e.g.,

AWS Certified Big Data Specialty, Google Professional Data Engineer

). Experience with real-time data processing frameworks (e.g.,

Apache Flink, Storm

). Knowledge of machine learning and AI workflows. * Familiarity with data visualization tools (e.g.,

Tableau, Power BI

).