Job Description

Posted on 25 August 2023

Group: M3S

Project Overview

We are seeking one or more Research Engineers in computer science or computer engineering to join the \xe2\x80\x9cSupporting Interactive Online Learning\xe2\x80\x9d project. This project is part of Task \xe2\x80\x9cT4\xe2\x80\x9d, titled AI for Human Capital Development in Future of Work, of a five-year, ambitious research program on "Mens, Manus and Machina\xe2\x80\x94How AI Empowers People, Institutions and Cities in Singapore (M3S)."

Successful applicants for this position will have the opportunity, in partnership with other post-doctoral colleagues and PhD students, to work on cutting-edge research that builds new capabilities for AI-enhanced interaction between assistive agents/chatbots and humans, with a special focus on supporting multi-modal natural interactions for learning tasks. The end goal is to build exciting new capabilities for scalable adult pedagogy and learning, that allow adult learners to avail of the advantages that self-paced, online learning offers while still benefit from the higher levels of engagement generated by the use of smart, multi-modal chatbots.

This research will specifically look at the integration of verbal queries, natural gestures (e.g., pointing to specific diagrams or content in a book or screen) and 2D (RGG) and 3D point (LIDAR) inputs, to provide richer contextualized understanding of doubts and queries issued by an adult learner. The work will build and optimize AI models that perform such integration of multi-sensory input to perform \xe2\x80\x9cimmersive visual grounding of human queries\xe2\x80\x9d. Such AI models will be integrated into prototypes of multi-modal chatbots, that can be deployed as plug-ins to a variety of laptop/mobile-based learning platforms. Later, as the work progresses, such chatbots can be integrated with state-of-the-are LLMs and generative AI technologies to generate, using a combination of text and images/video, responses to the issued queries.

The T4 team is led by distinguished scholars, including Profs. Jinhua Zhao and Alex (Sandy) Pentland from MIT; and Prof. Archan Misra from Singapore Management University. The proposed research described above will be spearheaded by Prof. Misra.

• Implement, train and test embedded AI models for spatially-aware multi-modal instruction comprehension.
• Develop mobile/laptop-based prototypes for learning-driven chatbots, that integrate AI models and multi-modal sensory input.
• Development, implement and evaluate experiments to characterize the performance of such prototypes
• Assist the overall team to publish research results in top-tier conferences and journals, with focus on venues associated with pervasive/ubiquitous computing applications, mobile/embedded systems, AI and education science.

Job Requirements

• Bachelor\xe2\x80\x99s or Master\xe2\x80\x99s (preferred) degree in Computer Science/Engineering or Electronics Engineering
• 1-2 years project-level experience with embedded AI implementations (e.g., for vision or speech applications) on platforms such as Raspberry Pi or Nvidia Jetson is essential.
• High proficiency with ML libraries and frameworks (e.g., Tensorflow, PyTorch) is essential.
• Experience with implementing signal processing algorithms over data streams from sensors, such as IMU, microphone or LIDAR, is highly desired.
• Record of co-authored publications in top-tier conferences and journals is a plus.
• Strong communication and collaboration skills.

This role is primarily based in Singapore and appointment is on a two-year contract, with the possibility of extension.

If you want to find out more about the role, please contact Professor Archan Misra (archanm@smu.edu.sg).

Interested applicants are invited to send in their full CV/resume, cover letter and list of three references (to include reference names and contact information). We regret that only shortlisted candidates will be notified.

Singapore-MIT Alliance for Research and Technology