Job Description

• Design, develop, and implement Spark Scala applications and data processing pipelines to process large volumes of structured and unstructured data
• Integrate Elasticsearch with Spark to enable efficient indexing, querying, and retrieval of data
• Optimize and tune Spark jobs for performance and scalability, ensuring efficient data processing and indexing in Elasticsearch.
• Collaborate with data engineers, data scientists, and other stakeholders to understand requirements and translate them into technical specifications and solutions
• Implement data transformations, aggregations, and computations using Spark RDDs, Data Frames, and Datasets, and integrate them with Elasticsearch
• Develop and maintain scalable and fault-tolerant Spark applications, adhering to industry best practices and coding standards
• Troubleshoot and resolve issues related to data processing, performance, and data quality in the Spark-Elasticsearch integration
• Monitor and analyze job performance metrics, identify bottlenecks, and propose optimizations in both Spark and Elasticsearch components
• Stay updated with emerging trends and advancements in the big data technologies space to ensure continuous improvement and innovation

Requirements

• Bachelor\'s or Master\'s degree in Computer Science, Software Engineering, or a related field
• Strong experience in developing Spark applications. Experience with Spark Streaming is a plus
• Proficiency in Scala programming language and familiarity with functional programming concepts
• In-depth understanding of Apache Spark architecture, RDDs, Data Frames, and Spark SQL
• Experience integrating and working with Elasticsearch for data indexing and search applications
• Solid understanding of Elasticsearch data modeling, indexing strategies, and query optimization
• Experience with distributed computing, parallel processing, and working with large datasets
• Familiarity with big data technologies such as Hadoop, Hive, and HDFS
• Proficient in performance tuning and optimization techniques for Spark applications and Elasticsearch queries
• Strong problem-solving and analytical skills with the ability to debug and resolve complex issues
• Familiarity with version control systems (e.g., Git) and collaborative development workflows
• Excellent communication and teamwork skills with the ability to work effectively in cross-functional teams

Benefits

What we can offer you

• Diverse and dynamic work environment
• Work-life balance and support for career development
• An amazing life inside the element!
• Annual Leave. Medical Leave, Insurance Coverage, Yearly appraisal

Additional Information

Career Level

Junior Executive

Qualification

Bachelor\'s Degree, Post Graduate Diploma, Professional Degree, Master\'s Degree

Years of Experience

1 year

Job Type

Full-Time

Job Specializations

,

Company Overview

At Quess Singapore, we are a leading business services provider, transforming enterprises with digital-first staffing solutions and innovative strategies.

With over 20 years of experience, we leverage our rich industry knowledge and future-ready digital platforms to reinforce our clients\' businesses with modern, world-class staffing and recruitment solutions that are on-demand. A B2B provider focusing on powering staffing through advanced solutions, we deliver a host of services such as IT Staff Augmentation, General Staff Augmentation, MSP, Selection and Services, and HR Outsourcing solutions. We achieve this with a growing team of skilled associates and a wide presence across APAC, reshaping companies to be agile and competitive.

Additional Company Information

Registration No.

199801439d

Company Size

1001 - 2000 Employees

Average Processing Time

30 days

Industry

Human Resources Management / Consulting

Benefits & Others

Education support, Miscellaneous allowance, Medical, Regular hours, Mondays - Fridays, Business (e.g. Shirts)

Search Index