Job Description

Key Responsibilities:

Design, build, and maintain scalable and robust data pipelines (ETL/ELT) for batch and real-time processing. Develop and manage data lake, data warehouse, and data mart solutions. Collaborate with data scientists, analysts, and business teams to understand data needs and deliver relevant solutions. Optimize data workflows for performance, scalability, and cost efficiency. Implement data governance, security, and privacy best practices. Automate data validation and monitoring to ensure high data quality. Lead the evaluation and implementation of new data tools and technologies. Mentor junior data engineers and support technical leadership in architecture decisions.

Required Skills & Qualifications:

Bachelor's or Master's degree in Computer Science, Engineering, or a related field. 10+ years of experience in data engineering, data architecture, or software development. Strong programming skills in Python, Java, or Scala. Expertise in SQL, data modeling, and database optimization. Deep knowledge of data warehousing solutions like

Snowflake, Redshift, BigQuery

, or

Synapse

. Hands-on experience with big data tools:

Spark, Hadoop, Kafka, Flink

. Experience with cloud platforms:

AWS, GCP, or Azure

. Familiarity with orchestration tools like

Airflow, Luigi, or Prefect

. Solid understanding of CI/CD, Git, Docker, and Kubernetes.

Preferred Skills:

Experience with machine learning pipelines and MLOps. Knowledge of data security, encryption, and compliance (GDPR, HIPAA, etc.). * Certification in cloud or big data technologies (e.g., AWS Certified Data Analytics, Google Cloud Data Engineer).