Distributed Systems: Paradigms and models

Official lesson register

Lessons

Day	Hours	Argument	Blackboard	Multimedia
Sept. 19	2	Course introduction. Real life examples of parallel and distributed computing. Motivations. (Partial audio video support due to laptop substitution)	blackboard (2nd hour only)	(1st hour only)
Sept. 20	2	Motivations: different kind of (parallel & distributed) devices available. Technological improvements.	PDF
Sept. 21	2	More on clusters: homogeneous and heterogeneous configurations, MTBF, effect of initial program deployement. Measures (latency, service time, speedup, scalability)	PDF
Sept. 26	2	Measures (efficiency, Amdahal & Gustafson law). Application parallelization principles.	PDF
Sept. 27	2	Application parallelization principles: more applications. Concurrent activity mechanisms (processes and threads)	PDF
Sept. 28	2	Mechanisms (and code) to set up threads and processes under Posix/Linux.	PDF tarball of the code used.
Oct. 3	2	Techniques to contain interaction overheads. Parallel/distributed models: data parallelism, task graph.	PDF
Oct. 4	2	Parallel/distributed models: Work pool, Master/slave, Client/server, Pipeline (producer/consumer). Granularity of parallel computations. Mapping.	PDF
Oct. 5	2	Exercise: assignment of Xeon PHI account, instructions for remote access to the machine, assignment of the exercise with creation of n threads and fair termination	PDF Instructions to access PHI	(partial)
Oct. 10	2	Load balancing: static and dynamic techniques. Data and recursive decomposition, speculative decomposition.	PDF
Oct. 11	2	Mechanisms for thread programming: mutex, condition_variable, (async/futures and thread pool (only concepts)). RPC (rpcgen).	PDF
Oct. 12	2	Exercises on threads.	Exercise text Blackboard
Oct. 17	2	Design patterns: introduction. Finding concurrency: Task and Data decomposition patterns. Group, Ordered tasks patterns.	PDF
Oct. 18	2	Design patterns: Finding concurrency: Data sharing and design evaluation patterns. Algorithm structure space: Organize by task, by data decomposition and by flow of data patterns.	PDF
Oct. 19	2	Tools and methodology for experiments. Assignment: finish second exercise given last on Oct 12 lesson	PDF sources
Oct. 24	2	Design patterns: Supporting structure pattern space	PDF
Nov. 7	2	Design patterns: Distributed arrays (+ PGAS). Design patterns general picture and introduction of algorithmic skeletons. Cilk Plus with sample code	PDF
Nov. 8	2	Introduction to algorithmic skeletons	PDF
Nov. 9	2	Sort (Cilk and C++ Thread versions)	PDF
Nov. 14	2	Algorithmic skeletons: RISC and CISC approach, template based implementation techniques, assigment of templates to skeleton trees. Optimization of the parallelism degree.	PDF
Nov. 15	1	Optimization of skeleton / template composition as space search problem. Data flow based implementation of skeletons.	PDF
Nov. 15	1	Introduction to FastFlow. First example: 3-stage pipeline.	PDF
Nov. 16	2	ClassWork1 and FastFlow farm pattern.	PDF
Nov. 21	2	Optimizations (overhead reduction) in macro data flow implementations of design pattern/skeleton frameworks	PDF
Nov. 22	2	More on the FastFlow farm pattern. Discussed different configurations. Task scheduling, feedback-channels.	PDF
Nov. 23	2	Assigned ClassWork3 (see slides of Nov. 22). Proposed and discussed 2 possible implementations.
Nov. 28	2	Vectorization	PDF
Nov. 29	2	Data parallel computations using the FastFlow parallel-for pattern. Iterations' scheduling, and load-balancing. Some examples.	PDF
Nov. 30	2	Discussion on possible variants of the ClassWork3. Assigned ClassWork4.	PDF
Nov. 28	2	Sample code discussion (vectorization). Options to drive the parallelization process in the Intel compiler suite.	PDF
Dec. 07	2	Discussion about ClassWork4. Assigned ClassWork5.	PDF
12 dic	2	Rewriting rules and optimizations	PDF
13 dic	2	More optim rules: map to stream parallel, pipeline stage collapse, farm worker increase/decrease. SPM application development methodology. Presentation of the final project.	PDF
14 dic	2	Discussion about ClassWork5. Introduction of the Divide and Conquer parallel pattern in FastFlow. Assigned ClassWork6.	PDF
15 dic	1	Discussion on 2016-17 project text	PDF
15 dic	1	Discussion on ClassWork6. Introduction to debugging and profiling tools. Dynamic memory allocations: jemalloc, Intel TBB Allocator, Hoard Allocator, FastFlow Allocator.	PDF