"Parallel Processing (Parallelverarbeitung)" (Lecture
+ Lab) Winter 2023/24
- Contents: Basics of parallel computer architecture, principles and practice of parallel programming\, with focus on OpenMP and MPI
- Prerequisites in terms of content: Algorithms
and Data Structures, Objekt oriented and functional Programming,
Operating Systems I
- Details: Module description in unisono (tab 'Contents'; mostly in German), inofficial (and probably outdated) description in English
- Examination: written electronic exam (60 minutes, in
German or in English);
for students in PO 2012, admission to the examination requires the successful participation in the lab - Link to further information in unisono (date/time etc.): Lecture, Lab
- Link to the moodle course.
Announcements
10.10.2023 | For questions and help on the lab assignments, please use the Discord server set up for this course. | ||
20.09.2023 | The next exam will take place in Feb./March 2024. Exam duration will be 60 minutes. It will be an open book exam, i.e., you are allowed to use printed materials like lecture slides, scripts, books, own notes, etc., but no electronic devices or communication means. You also will get a PDF Dodument with the most important OpenMP pragmas and MPI routines. The exam will be conducted electronically using the Q-Exam system. The computers (laptops) will be provided by the university. To get acquainted with the system, please have a look at the demo exams (select the exam named "Demo-Prüfung Parallelverarbeitung") and the screen casts provided in the moodle course! If you have questions or problems, please contact the e-assessment support team (e-klausuren@uni-siegen.de). You can also ask questions immediately before the exam starts. Please notice that the laptops provide a German keyboard. The '#' is located left of the 'Enter' key, '{', '}', '[', and ']' can be entered by pressing the 'AltGr' key (right of the 'Space' key) together with '7', '8', '9', or '0'. The 'Ctrl' key is named 'Strg'. Please acquaint yoursef with that layout! See here for an accurate image or here for an interactive simulation (which actually differes slightly from the laptop keyboards). | ||
20.09.2023 | The practical programming exercises should be done using a Linux operating system. However, most exercises can also be done in a Windows environment. You can find some technical information about this here. If you want to install the Scalasca performance analysis tool on your Linux computer, you can use this script which contains all the necessary commands. Please be sure to also look at the documentation. | ||
20.09.2023 | You can find old slides in German (from WiSe 15/16!) here: original size, 2-on-1. |
||
20.09.2023 | IMPORTANT: In order to
pass this module, you MUST
register for the practical exercises in unisono until 05.01.2024 (for Exam Regulations 2012: "822120-SL -
Parallelverarbeitung", see this
screen dump; for Exam Regulations 2021: "4INFMA024-S - Studienleistung Parallelverarbeitung", see this screen dump). You must do this in this semester even
if you want to take the exam in a later semester! |
||
|
20.09.2023 | |
Start of the lecture: 16.10.2023; Start of the Lab: 10.10.2023 (with lecture!) |
Course Materials
- Complete set of slides (date: 09.10.23
): original
size, 2
on 1
- Slides for individual chapters (updated during the semester):
0. Organisation 10.10.23
original size
1. Basics
13.11.23
original size 2 on 1
2. Parallel Programming with Shared Memory 04.12.23
original size 2 on 1 code 3. Parallel Programming with Message Passing 04.12.23
original size 2 on 1 code 4. Optimisation Techniques 20.09.23
original size 2 on 1 code 5. Appendix (Tutorials, Information related to the practical exercises)
04.12.23
original size 2 on 1 code 6. Summary / Important Topics
20.09.23
original size 2 on 1 - The slides in original size also include all animations, the 2 on 1 versions include additional notes and explanations.
Exercises
Solutions for compulsory exercises must be submitted via the moodle course.No. | Date of issue | Deadline | Presentation/ Discussion | Exercise sheet/code | |
1 | 30.10. |
21.11., 10:00 |
Tuesday, 21.11., 10:15, Room H-C 6321 | Exercise sheet, Code |
|
2 | 13.11. |
12.12., 10:00 | Tuesday, 12.12., 10:15, Room H-C 6321 | Exercise sheet, Code |
|
3 | 04.12. |
16.01., 10:00 | Tuesday, 16.01., 10:15, Room H-C 6321 | Exercise sheet, Code |
|
4 | 18.12. |
30.01., 10:00 | Tuesday, 30.01., 10:15, Room H-C 6321 | |
General Literature
- A. Grama, A. Gupta, G. Karypis, V. Kumar: Introduction to Parallel Computing, 2. ed., Pearson Education, 2003.
- Barry Wilkinson, Michael Allen: Parallel Programming, internat. ed., 2. ed., Pearson Education international, 2005.
- R. Trobec, B. Slivnik, P. Bulic, B. Robic: Introduction to Parallel Computing. Springer, 2018.
- Thomas Rauber, Gudula Rünger: Parallele und verteilte Programmierung. Springer, 2000.
- Theo Ungerer: Parallelrechner und parallele Programmierung, Spektrum, Akad. Verl., 1997.
- Ian Foster: Designing and Building Parallel Programs, Addison-Wesley, 1995.
- Seyed Roosta: Parallel Processing and Parallel Algorithms, Springer, 2000.
- Rainer Oechsle: Parallele Programmierung mit Java Threads, Fachbuchverlag Leipzig, 2001.
- S. Hoffmann, R.Lienhart: OpenMP, Springer, 2008.
- W. Gropp, E. Lusk, A. Skjellum: Using MPI, MIT Press, 1994.
- Michael Wolfe: Optimizing Supercompilers for Supercomputers, MIT Press, 1989.
Further Reading and Links
- Synchronisation
- Linux
- C/C++
- Introduction to C and C++ (in German)
- C++-Tutorial
- C++-Reference (cppreference.com), C++-Reference (cplusplus.com)
- OpenMP
- OpenMP home page
- OpenMP standard (version 3.0 for C/C++/FORTRAN, PDF, 1,2MB; version 2.0 for C/C++, PDF, 270KB)
- OpenMP introduction
- OpenMP tutorial
- MPI
- MPI forum home page
- MPI standards
- Collection of MPI tutorials
- Seminar talk on MPI (in German)
- Online book: "MPI: The Complete Reference"
- Parallel computers
- Talks on BlueGene/L (short, more detailed)
Sample codes (see also 'Course Materials' above!)
- Threads using PThreads and Java
- "Hello World" programm in C with PThreads
- "Hello World" programm in Java (by subclassing Thread, by implementing Runnable)
- Sample codes for the Monitor concept (C with PThreads, Java)
- Example for executing a function with a new thread (C with PThreads)