Big Data Engineer
Singapore, Singapore
Job Description
b'
As a Lead you will be responsible for end-to-end development of Data Analytics and AI/ML use cases. The role must be hands on and willingness to work on ML Ops. The role is for a self-motivated individual with software engineering skills and expertise with Big Data technologies. The candidate will be extensively involved in hands-on activities including POCs, design, documentation, development, and test of new functionality. Candidate must be agile and flexible with changing priorities based on team\xe2\x80\x99s needs. You will be working with
- Massive data : You will source / examine, analyze, engineer data pipelines for gigabytes/terabytes of structured and unstructured data with our platform to create value for customers. You will also be working with Enterprise data.
- Pushing the limits : This role will be on the cutting edge of our Data / Machine Learning platform. As we push to solve more of our customer challenges, you will be prototyping new features, tools and ideas. Innovate at a very fast pace to maintain our competitive edge.
- Linux hacking: You will be masterfully using the command line, including tools like vi/emacs and understanding beyond basics of grep, bash, awk, sed, etc to aggressively dive into data, systems, and compute platforms to get the results you are seeking.
- Production deployment : You will be responsible for integration and deployment of the machine learning pipelines into production where your ideas can come to life.
- Coordinate and work with cross functional teams, sometimes located at different geo locations.
- CS fundamentals : You have earned at least a B.S. / MS in Computer Science, or related degree AND you have a strong ethos of continuous learning.
- Commercial software engineering : You have 6 to 10 years of professional software development experience with languages and systems such as Python, PySpark, Java and version control (git), with good analytical & debugging skills.
- Big data : You have extensive experience with data analytics and working knowledge of big data infrastructure such as Google Cloud, Big Query, Data Flow, Hadoop Eco System, HDFS, Spark. You\'ve routinely built data pipelines with gigabytes/terabytes of data and understand the challenges of manipulating such large datasets.
- Data Modeling : Flair for data, schema, data model, PL/SQL, Star & snowflake schema, how to bring efficiency in data modeling for efficient querying data for analysis, understands criticality TDD and develops data validation techniques.
- Real Time Systems : Understands evolution of databases for in-memory, NoSQL & indexing technologies along with experience on real-time & stream processing systems like Google pub/sub, GCP technologies, Kafka, Storm, Spark Streaming.
- Strong design skills : with a proven track record of success on large/highly complex projects preferably in Enterprise Apps and Integration.
- Project management : You demonstrate excellent project and time management skills, exposure to scrum or other agile practices in JIRA.
- Excellent verbal and written communication skills : Must be able to effectively communicate & work with fellow team members and other functional team members to coordinate & meet deliverables.
Beware of fraud agents! do not pay money to get a job
MNCJobz.com will not be responsible for any payment made to a third-party. All Terms of Use are applicable.
Related Jobs
Job Detail
-
Job IdJD1291720
-
IndustryNot mentioned
-
Total Positions1
-
Job Type:Full Time
-
Salary:Not mentioned
-
Employment StatusPermanent
-
Job LocationSingapore, Singapore
-
EducationNot mentioned