The University of Western Ontario
London, Canada

Department of Computer Science

Distributed and Parallel Systems

Course Web Site -- Winter 2018

Course Description

To best utilize parallel and distributed systems (multi-core, many-core and cluster) is nowadays an essential task for computer scientists.

This course studies the fundamental aspects of parallel systems and aims at providing an integrated view of the various facets of software development on such systems: hardware architectures, programming languages and models, software development tools, software engineering concepts and design patterns, performance modelling and analysis, experimenting and measuring, application to scientific computing.

Course topics may include but are not limited to:

Follow this link for various resources (software tools and tutorials, hardware documentation, conferences, other HPC course web sites, etc.) regarding this course and HPC in general (

Prerequisites for Undergraduate Students


Name:Marc Moreno Maza
Office:MC 327
Office Hours: Tuesday 1:30-3:20 pm in MC 327
Phone:661-2111 x3741

Lecture Notes and Textbook

Notes of each lecture will be available on the course website, approximately one or two days after the oral presentation. There is no textbook.

Course Website

The course web site is accessible from:

Please check the site often for updates on lecture notes and errata. Also be aware that the course website is not a substitute for actual classroom attendance!

Course outline

Please find the course outline here.

Lecture Topics

The list of topics will be something in the order of:

Relation of CS4402 to CS3101

CS3101 is a new course which has started in the 2012-2013 academic year. Its web site is This course is meant to provide the necessary theoretical background (architectures, models of computations, algorithms) to understand and practice high-performance computing.

CS3101 can be seen as an extension of other CS courses such as

thus providing the parallel dimension of Today's Computer Science.

Students who are taking CS4402 the academic year 2017-2018 are likely not to have taken CS3101 in previous years since it was not offered last year. Therefore, for the academic year 2017-2018, CS4402 is a self-contained course (as in previous years) where a sufficient subset of this theoretical background is presented.

Lecture Notes from 2016-2017

  • An introduction to parallel and distributed computing slides (long version) and slides (short version).
  • Parallel and distributed computing with Julia slides
  • The Fork-Join Model and its Implementation in Cilk slides. Some examples of CilkPlus programs and a few more here.
  • Analysis of Multithreaded Algorithms. slides
  • Cache memories: complexity analysis and practical issues. slides and handouts and C programs.
  • Parallel Prefix Sum slides
  • MetaFork: A Compilation Framework for Concurrency Platforms Targeting Multicores (by Xiaohui Chen). slides
  • Many-core Computing with CUDA. slides and handouts. and simple CUDA programs.
  • High-Performance Computing with CUDA. slides and handouts.
  • Parallel Random-Access Machines. slides and handouts.
  • A Many-core Machine Model for Designing Algorithms with Minimum Parallelism Overheads.
  • Synchronizing without locks. slides and handouts.
  • Issues in Parallelism (by Matteo Frigo). slides
  • Problem Sets from 2016-2017

  • Problem set 1
  • Problem set 2
  • Readings for Problem Set 1

  • All-Pair Shortest Paths: course notes.
  • All Pair Shortest Paths and Matrix Multiplication: book chapter
  • Parallel Algorithms for the All-Sources Generalized Shortest Paths Problem: article by Jeffrey D. Oldham and Vaughan Pratt.
  • Quiz corrections from 2014-2015

  • Quiz 1: elements of corrections for a 2014 quiz.
  • A CUDA quiz and its elements of correction.
  • 2015 Quiz 2 with elements of corrections.
  • Quiz corrections from 2016-2017

  • Quiz 1 with elements of corrections.
  • Quiz 2 with elements of corrections.
  • Class Schedule

    Lectures: 3 hours (Tuesdays 15:30 - 16:20 in MC 320 and Wednesdays15:30 - 17:20 in MC-320.

    Each student is expected to attend the lectures. In particular, quizzes (short written tests) may take place without notice.

    Student Evaluation

    Assignment/Project/Quiz Schedule

    All dates are tentative and currently subject to change, although it is doubtful by any significant amount.

    Evaluation Technique Weight Posted Date (tentative!) Due Date (tentative!) Workload
    Assignment One1/6Tu, Jan. 23Fr, Feb. 16regular
    Assignment Two1/6Tu, Mar. 7Tu, Mar. 21regular
    Project 1/3Fr, Feb. 14Wed, Apr. 6heavy
    Quizzes1/9 eachN/AvariousN/A

    If for any reason the schedule given above cannot be adhered to, the assignment, project and quiz marks will be pro-rated. For instance, if an assignment has to be cancelled for any reason, the remaining assignment weight will be prorated to add up to 1/3.)

    Every effort will be made to have assignments, projects and quizzes marked and handed back within 3 weeks of the hand-in date, preferably sooner.


    Quizzes may be held without being announced in advance.

    Quizzes will be closed book.


    Assignments will be due on the (tentative) dates listed above. The assignment will be sent by email to the instructor.

    Extensions will be granted only by the course instructor. If you have serious medical or compassionate grounds for an extension, you should take supporting documentation to the office of the Dean of your faculty, who will contact the instructor.

    Computing Facilities

    Each student will be given an account on the Computer Science Department senior undergraduate computing facility, GAUL. In accepting the GAUL account, a student agrees to abide by the department's

    Note: After-hours access to certain Computer Science lab rooms is by student card. If a student card is lost, a replacement card will no longer open these lab rooms, and the student must bring the new card to a member of the Systems Group in Middlesex College Room 346.

    Marc Moreno Maza
    Last modified: Mon Jan 10 EDT 2017