Course Title

High Performance Computing in Machine Learning

Course Code

SDS 427

Course Type

Elective

Level

Master’s

Year / Semester

2^nd Semester

Instructor’s Name

Prof. Constantine Dovrolis (Lead Instructor), Asoc. Prof. Mihalis Nicolaou, Prof. Vangelis Harmandaris

ECTS

5

Lectures / week

1

Laboratories / week

1

Course Purpose and Objectives

• Provide the students with the necessary computer programming and software engineering background to solve complex problems using high performance computing (HPC), and Machine Learning (ML) methods.
• Introduce the use HPC infrastructures to get the best performance out of different machine learning tools with several hands-on sessions

Learning Outcomes

This course combines high performance computing (HPC) and artificial intelligence (AI). This emerging trend combines the principles and practices of distributed training, which is critical for the success of both deep learning and machine learning disciplines. Students will develop the skills necessary to write their own codes and will be trained in common practices used to implement numerical methods that exploit HPC in ML applications.

Prerequisites

None

Course Content

• Week-1: Revision/overview of HPC and ML fundamentals

• Week-2: Workload characterization: how does ML training differ from scientific computing?

• Week-3: Performance bottlenecks and optimizations in ML (mostly deep learning)

• Week-4: Distributed ML algorithms (including federated learning)

• Week-5: The application of ML in scientific computing: how is it different than other ML applications?

• Week-6: Physics-informed ML and related methods

• Week-7: Future trends in HPC architectures, hardware accelerators and ML algorithms

Teaching Methodology

• Lectures interleaved with hands-on practical exercises.
• Laboratory work will include completing or fixing codes provided by the instructor, running the codes on an HPC cluster, and visualizing or summarizing the results

Bibliography

• Tutor’s lecture notes and slides

Optional bibliography:

• “The Art of HPC, volumes 1 & 2”, V. Eijkhout, ISBN: 9781387387892

Assessment

• Weekly exercises and lab participation
• Three interim homework assignments
• Final homework assignment

Language

English