Job Description

? We're Hiring: Data Engineering Lead! ?
Key Responsibilities

• Lead the design, development, and maintenance ofETL/ELT pipelines usingDatabricks Notebooks & Workflows
• Architect, implement, and manageDelta Lake andData Lakehouse solutions
• Manage and govern data usingUnity Catalog, ensuring proper access control and lineage
• Tune and optimizeDatabricks jobs for cost, performance, and reliability
• Develop and optimizePython and SQL code for data transformation and analysis
• Integrate and manage data withAzure Synapse
• Ensure scalability, availability, and security of data pipelines
• Establish data engineering best practices, standards, and operating procedures
• Support data quality checks, monitoring, and alerting
• Collaborate with data scientists, analysts, architects, and business stakeholders
• Provide guidance and mentorship to junior data engineers (for Lead role)

Required Technical Skills

• Strong hands-on experience withDatabricks (Notebooks & Workflows)
• Expertise inETL / ELT pipeline management
• Strong experience withDelta Lake and Data Lakehouse architecture
• Experience managingUnity Catalog (data governance & access control)
• Strong hands-on coding experience inPython and SQL
• Experience integrating withAzure Synapse Analytics
• Experience in performance tuning & cost optimization of Databricks jobs

Preferred / Nice-to-Have Skills

• Experience inbanking or regulated industries
• Knowledge ofAzure Data Services (Data Factory, Data Lake Storage Gen2, Event Hub)
• Experience with streaming data and real-time pipelines
• Knowledge of data quality frameworks and monitoring tools
• Experience in CI/CD for data pipelines (Azure DevOps / GitHub)

Qualifications

• Bachelor's Degree in Computer Science, Engineering or related field
• 5-10 years of experience in Data Engineering (3-5 for non-lead role)
• Proven experience delivering enterprise-grade data platforms

Personal Attributes

• Strong analytical, problem-solving and leadership skills
• Excellent communication and stakeholder management abilities
• Highly organised with attention to detail
• Ability to work under pressure and manage multiple priorities

Skills Required