Job Description

DataSpark was created from a vision to transform Singtel’s rich and unique repository of data into business value and social impact. Our data products and services provide powerful insights and advanced analytics capabilities to businesses, government agencies, and other telecommunication companies. We strive for our analytics to be trustworthy and relevant to our clients while adhering to high standards of data privacy. We are looking for an experienced Data Engineer to join us to build robust data platforms and applications that incorporate data science and machine learning algorithms and models that solve problems in telco network management, transportation, urban planning, real time crowd management, and retail intelligence, to name a few. This is a great opportunity to apply your established expertise in data science and machines learning. At Dataspark, you get to work with rich and diverse datasets, cutting edge technology, and you get to see the impact of your results in real business and government decisions, which in turn provide positive social benefit for consumers at a large scale. As a startup that is part of Singtel, DataSpark provides an enviable work environment with spirited trailblazing and industrial countenance. Working alongside creative, energetic and passionate teammates from around the world, you get to be a part of our exciting growth journey as we build the company to the next level. Responsibilities

• design and implement scalable and robust software platform for ingesting and transforming telco network datasets in (near) real-time using a variety of open-source and proprietary Big Data technologies

• recommend and implement ways to improve data reliability, efficiency and quality

• collaborate with product management, sales and marketing, and solution delivery teams to support the objectives that customer requirements are well managed and reflected in product releases

• lead and mentor development team

• support the deployment of DataSpark software within clients' IT environment

• working closely with stakeholders to ensure high standards of data governance during implementation

• serve as technical subject matter expert in latest big data technologies

Requirements

• 7+ years of superior software development experience building commercial large-scale software systems and database systems

• Excellence in algorithms, data structure, discrete math, data base and data warehousing

• Expert knowledge in data management technologies and software engineering tools to efficiently process large volume of data

• Demonstrated clear and thorough logical and analytical thinking, as well as problem solving skills

• Experience of data warehouses in excess of 10TB

• Experience of Web UI, middle tier, and data back end development

• Production coding experience in choice of programming languages and development frameworks

• Proven professional experience in processing large-scale commercial data. Experience with telco data a plus.

• Superior and proactive communications skills, including verbal, written, and presentation.

• A proven team player and contributor.

• Self-directed, ability to work independently and research innovative solutions to business problems

• Aptitude of working on multiple projects in parallel

• Attention to details and data accuracy

• Mandatory:
• Spark [Intermediate]

- Data Management - file format, compression, partitions, data orchestration [Intermediate]

• Any NoSQL DB [Advance]

• Good to have
• Airflow