Job Description

Having witnessed the achievements of the large models in 2023, there has been a new surge of machine learning and AI models in recent days, which again puts up the public concerns about the explosively increasing energy consumption during the model training phase. According to Dr. Sajjad Moazeni, an AI researcher at University of Washington, the training of GPT-3 roughly consumes 10 gigawatt-hour of electricity ^([1]), approximately equivalent to the daily electricity consumption of 440,000 average Singapore households in 2021 ^([2]).Significant efforts have been devoted to improving the efficiency of the model training phase, in terms of hardware, model architecture, networking, and algorithm design, etc. The PI (Dr. Zhang Junyu) of this project would like to tackle this problem from the algorithmic perspective, by developing theoretical understanding of the open questions on the training algorithms, in terms of the convergence, complexity, local landscape properties, and potential algorithmic improvement based on the new theoretical findings.[1] Data from UW News. [2] Data from StatistaQualificationsInterested applicants are required to possess a PhD in 2024. He/she should have a good understanding in1. convergence and complexity analysis for (nonconvex) optimization algorithms2. stochastic process and martingale theory3. semi-algebraic and subanalytic geometry4. stochastic approximation methods5. dynamical systemsThe applicant should also be experienced in MATLAB and Python coding. In particular, he/she should have the ability to adapt base codes of PyTorch to implement new algorithms instead of calling built-in functions.More InformationLocation: Kent Ridge CampusOrganization: College of Design and EngineeringDepartment : Industrial Systems Engineering and ManagementEmployee Referral Eligible: NoJob requisition ID : 23720

National University of Singapore