Job Description

• Design, develop, and implement Spark Scala applications and data processing pipelines to process large volumes of structured and unstructured data
• Integrate Elasticsearch with Spark to enable efficient indexing, querying, and retrieval of data
• Optimize and tune Spark jobs for performance and scalability, ensuring efficient data processing and indexing in Elasticsearch

\xc2\xb7 Collaborate with data engineers, data scientists, and other stakeholders to understand requirements and translate them into technical specifications and solutions

• Implement data transformations, aggregations, and computations using Spark RDDs, DataFrames, and Datasets, and integrate them with Elasticsearch
• Develop and maintain scalable and fault-tolerant Spark applications, adhering to industry best practices and coding standards
• Troubleshoot and resolve issues related to data processing, performance, and data quality in the Spark-Elasticsearch integration
• Monitor and analyze job performance metrics, identify bottlenecks, and propose optimizations in both Spark and Elasticsearch components
• Stay updated with emerging trends and advancements in the big data technologies space to ensure continuous improvement and innovation

Requirements

• Bachelor\'s or Master\'s degree in Computer Science, Software Engineering, or a related field
• Strong experience in developing Spark applications. Experience with Spark Streaming is a plus
• Proficiency in Scala programming language and familiarity with functional programming concepts
• In-depth understanding of Apache Spark architecture, RDDs, DataFrames, and Spark SQL
• Experience integrating and working with Elasticsearch for data indexing and search applications
• Solid understanding of Elasticsearch data modeling, indexing strategies, and query optimization
• Experience with distributed computing, parallel processing, and working with large datasets
• Familiarity with big data technologies such as Hadoop, Hive, and HDFS
• Proficient in performance tuning and optimization techniques for Spark applications and Elasticsearch queries
• Strong problem-solving and analytical skills with the ability to debug and resolve complex issues
• Familiarity with version control systems (e.g., Git) and collaborative development workflows
• Excellent communication and teamwork skills with the ability to work effectively in cross-functional teams

Benefits

What we can offer you

• Diverse and dynamic work environment
• Work-life balance and support for career development
• An amazing life inside the element!
• Annual Leave. Medical Leave, Insurance Coverage, Yearly appraisal

Additional Information

Career Level

Senior Executive

Qualification

Bachelor\'s Degree, Post Graduate Diploma, Professional Degree, Master\'s Degree

Years of Experience

8 years

Job Type

Full-Time

Job Specializations

,

Company Overview

At Quess Singapore, we are a leading business services provider, transforming enterprises with digital-first staffing solutions and innovative strategies.

With over 20 years of experience, we leverage our rich industry knowledge and future-ready digital platforms to reinforce our clients\' businesses with modern, world-class staffing and recruitment solutions that are on-demand. A B2B provider focusing on powering staffing through advanced solutions, we deliver a host of services such as IT Staff Augmentation, General Staff Augmentation, MSP, Selection and Services, and HR Outsourcing solutions. We achieve this with a growing team of skilled associates and a wide presence across APAC, reshaping companies to be agile and competitive.

Additional Company Information

Registration No.

199801439d

Company Size

1001 - 2000 Employees

Average Processing Time

30 days

Industry

Human Resources Management / Consulting

Benefits & Others

Education support, Miscellaneous allowance, Medical, Regular hours, Mondays - Fridays, Business (e.g. Shirts)

Search Index