Home  -   Teaching  -   Research  -   Publications  -   People  -   Meetings  -   Media  -  

Since the very beginning of computers, processors were build with ever-increasing clock frequencies and instruction-level optimizations for faster serial code execution, such as ILP, caches, or speculative engines. Software developers and industry got used to the fact that applications get faster by just exchanging the underlying hardware. For several years now, these rules are proven to be no longer valid. Moore's law about the ever-increasing number of transistors per die is still valid, but decreased structural sizes and increased power consumption demand stalling, or even reduced, clock frequencies. Due to this development, serial execution performance no longer improves automatically with the next processor generation.

In the 'many-core era' that happens now, additional transistors are used not to speed up serial code paths, but to offer multiple execution engines ('cores') per processor. This changes every desktop-, server-, or even mobile system into a parallel computer. The exploitation of additional transistors is therefore now the responsibility of software, which makes parallel programming a mandatory approach for all software with scalability demands.

In this course, we want to discuss the relevant theoretical and practical solutions available for parallel software development. The following topics are covered:

• Block 1: Introduction
  • Introduction
• Block 2: Formal aspects of concurrency
  • Theory of Concurrency - Shared Memory
  • Theory of Concurrency - Multicomputer
• Block 3: Parallel hardware
  • Parallel Hardware and Interconnects
  • GPU Compute Devices
• Block 4: Parallel programming
  • Programming models
  • GPU Computing with OpenCL
  • Multi-Tasking
  • Message Passing
  • Implicit Parallelism & Mixed Approaches
  • Parallel Algorithms
• Block 5: Examples

Assignments

Please make sure that your group is registered in the assignment submission system. You can use your HPI login credentials (information from the lecture was incorrect).

• Assignment 1: Shared memory programming with basic synchronization primitives
• Assignment 2: Data-Parallel Programming (OpenCL example code)
• Assignment 3: Wator with Scala

Rules

There are three mandatory assignments, to be solved alone or as team of two persons. The oral exam admittance is achieved if two out of three assignments are solved correctly.

Recommended Readings (permanently updated)

Journal Article

[BibTex] [Download] [Link]

[BibTex] [Abstract]

Using graphics hardware to enhance CPU-based standard desktop applications is a question not only of programming models but also of critical optimizations that are required to achieve true performance improvements

[Link]

[BibTex] [Link]

[BibTex]

[BibTex]

[BibTex]

[BibTex]

[BibTex] [Download]

[BibTex]

[BibTex] [Download]

Book

[BibTex]

[BibTex]

[BibTex] [Link]

[BibTex]

[BibTex]

[BibTex]

[BibTex]

[BibTex]

[BibTex]

[BibTex]

[BibTex]

Book Section

[BibTex] [Download]

[BibTex]

[BibTex] [Download]

[BibTex] [Download]

[BibTex]

Conference Paper

[BibTex] [Download]

[BibTex] [Abstract]

Today's largest systems have over 100,000 cores, with million-core systems expected over the next few years. This growing scale makes debugging the applications that run on them a daunting challenge. Few debugging tools perform well at this scale and most provide an overload of information about the entire job. Developers need tools that quickly direct them to the root cause of the problem. This paper presents AutomaDeD, a tool that identifies which tasks of a large-scale application first manifest a bug at a specific code region and specific program execution point. AutomaDeD statistically models the application's control-flow and timing behavior, grouping tasks and identifying deviations from normal execution, which significantly reduces debugging effort. In addition to a case study in which AutomaDeD locates a bug that occurred during development of MVAPICH, we evaluate AutomaDeD on a range of bugs injected into the NAS parallel benchmarks. Our results demonstrate that AutomaDeD detects the time period when a bug first manifested with 90% accuracy for stalls and hangs and 70% accuracy for interference faults. It identifies the subset of processes first affected by the fault with 80% accuracy and 70% accuracy, respectively and the code region where the fault first manifested with 90% and 50% accuracy, respectively.

[BibTex] [Abstract]

The increasing demand for portable computing has elevated power consumption to be one of the most critical design parameters. A high-level synthesis system, HYPER-LP, is presented for mini- mizing power consumption in application specific datapath intensive CMOS circuits using a variety of architectural and computational transformations. The synthesis environment consists of high-level esti- mation of power consumption, a library of transformation primitives, and heuristic/probabilistic opti- mization search mechanisms for fast and efficient scanning of the design space. Examples with varying degree of computational complexity and structures are optimized and synthesized using the HYPER-LP system. The results indicate that more than an order of magnitude reduction in power can be achieved over current-day design methodologies while maintaining the system throughput; in some cases this can be accomplished while preserving or reducing the implementation area.

[BibTex] [Download]

[BibTex]

[BibTex] [Download]

[BibTex] [Link]

Slide

[BibTex] [Download]

Web Page

[BibTex]

[BibTex]

[BibTex] [Link]