Job Description

The Engineer, Data should have a proven track record of delivering data pipeline solutions and architecture. He/She should also understand business requirement and able to build reliable data infrastructure using big data technologies. Ideally, you are someone who enjoys optimizing data pipeline, automating and building from scratch.

Job Responsibilities :

Contribute to the full data engineering lifecycle, including conceptualization, data modeling, implementation, and operational management of data systems. Build and maintain robust, scalable data infrastructure, including real-time and batch ETL/ELT data pipelines, using orchestration tools like Apache Airflow. Support the development and management of the organization's data lake and data warehouse by implementing data models and applying modern data transformation practices with tools like dbt. Automate data processes by integrating different systems, APIs, and third-party services to ensure smooth data flow and synchronization. Adhere to and help enforce best practices for data governance, quality, and lineage, while implementing monitoring and alerting to ensure system reliability. Collaborate with cross-functional teams to help translate business requirements into technical specifications and production-ready solutions. Develop, deploy, and maintain internal automation solutions and tools by applying AI, web scraping, and custom logic to create effective data products. Any ad hoc duties as assigned

Job Requirements:

Bachelor's degree in computer science, Software Engineering, or related field is preferred. Minimum 2 years of similar experiences Solid understanding of software engineering principles, system architecture, and object-oriented design, with experience building applications. Strong proficiency in Python, including experience with data manipulation libraries (e.g., Pandas, NumPy) and frameworks for web scraping (e.g., Scrapy) or AI/ML (e.g., TensorFlow, PyTorch), is a plus. Experience with workflow orchestration tools like Apache Airflow for managing data pipelines. Familiarity with data transformation tools like dbt and an understanding of modern ETL/ELT design patterns. Experience with data modeling techniques (conceptual, logical, physical) and data warehouse schemas. Hands-on experience with any cloud platform (AWS, GCP, or Azure). Proficient in SQL with experience in designing and optimizing relational databases (e.g., PostgreSQL, Oracle, SQL Server). Experience with containerization and CI/CD is a strong plus. Experience in building AI agents or developing solutions using Generative AI frameworks (e.g., LangChain, LlamaIndex) is a plus. Good communication skills with the ability to explain technical concepts to various stakeholders. A proactive and analytical approach to problem-solving with attention to detail. * Strong sense of ownership with the ability to work independently and manage assigned tasks and projects effectively.