The University of Western Ontario
London, Canada

Department of Computer Science

CS4402B/CS9635B -- Parallel and Distributed Computing

Course Outline -- Winter 2022

Course Description

The efficient usage of parallel and distributed systems (multi-processors and computer networks) is nowadays an essential task for computer scientists.

This course studies the fundamental aspects of parallel systems and aims at providing an integrated view of the various facets of software development on such systems: hardware architectures, programming languages and models, software development tools, software engineering concepts and design patterns, performance modelling and analysis, experimenting and measuring, application to scientific computing.

Course topics may include but are not limited to: hierarchical memory, cache complexity, multi-core and many-core architectures, fork-join parallelism, scheduling, scalability, GPU computing, data parallelism, pipelining, message passing (MPI), parallel and distributed data-structures, and applications of parallel and distributed computing.

Follow this link for various resources (software tools and tutorials, hardware documentation, conferences, other HPC course web sites, etc.) regarding this course and HPC in general.

Prerequisites for undergraduate students

Instructor

Name:Marc Moreno-Maza
Office:MC 327
Office Hours: the half-hour before each lecture, or by appointment
Email:moreno@csd.uwo.ca
Phone:661-2111 x3741

Lecture Notes and Textbook

Notes of each lecture will be available on the course website, approximately one or two days after the oral presentation.

The following textbooks are recommended but not required:

  1. Structured Parallel Programming by Michael McCool, Arch Robison, James Reinder.
  2. C++ Concurrency in Action, practical multithreading by Anthony Williams
  3. Programming language pragmatics (Chapter 12 only) by Michael L. Scott
  4. Models of Computation Exploring the Power of Computing by John E. Savage.

Course and OWL Websites

The course web site is accessible here For both CS4402 and CS9635, the OWL web site is here.

Please check the course and OWL sites often for updates on lecture notes and errata, as well as the forum discussions and announcements.

Also be aware that the course website is not a substitute for the actual classroom attendance!

Polices regarding email and contact

For all questions related to the course materials (lectures, assignments, and quizzes) students must use the forum of the OWL web sites (see above) instead of contacting the instructor or the TA by email. Indeed a question about the lectures or an assignment problem is likely to be of interest to many students in the class.

Writing to the instructor or the TAs should only be done for handling a personal matter, like individual projects.

Delivery mode

The format of the course lectures and tutorials will be public presentations in the lecture room TC 343

However, due to the exceptional circumstances created by the COVID 19 pandemic, lectures will be on-line, synchronously, during the month of January 2022 here on Tuesdays and on there on Thursdays.

The slides presented during the lectures will be posted on the course web site.

The lectures will not be recorded.

Protection of online content

The instructor owns the intellectual property (IP) of the lecture materials even when such materials are posted online. Students are not to post lecture materials including (assignment statements, assignment solutions, etc.) to any other websites or platforms or use the lecture recording or materials for any other purpose.

Participants in this course are not permitted to record the sessions, except where recording is an approved accommodation, or the participant has the prior written permission of the instructor.

Computer requirements

Students must have a reliable internet connection and computer that are compatible with online learning (including working microphone and webcam) and testing system requirements.

As for web browser, using Mozilla Firefox v20.0 or Higher is required.

Lecture Topics

The list of topics will be something on the order of:

  1. Overview of parallel and distributed computing
  2. Programming patterns in parallel and distributed computing; illustration with Julia
  3. Hierarchical memories, cache complexity
  4. The fork-join model and the cilk concurrency platform
  5. Multi-threading parallelism and performance
  6. Pipelining
  7. Scheduling and Synchronizing; parallelism overheads
  8. Parallel Random-Access Machines.
  9. Many-core programming (GPGPUs)
  10. Code optimization techniques for multi-cores and GPGPUs
  11. High-Performance Computing with CUDA
  12. Multiprocessed parallelism, message passing (MPI)
  13. Other concurrency platforms (Open MP, Threading Building Blocks)

Class Schedule

Lectures: 3 hours (Tuesdays 12:30 - 13:30 and Thursdays 11:30 - 13:30)

When lectures are in-person, all lectures take place in TC 343.

When lectures are on-like follow this link on Tuesdays and that link on Thursdays.

Each student is expected to attend the lectures. In particular, quizzes (short written tests) may take place without notice.

Student Evaluation

Assignment/Project/Quiz Schedule

All dates are tentative and currently subject to change, although it is doubtful by any significant amount.

Evaluation Technique Weight Posted Date (tentative!) Due Date (tentative!) Workload
Assignment One1/6Jan. 27Feb. 18regular
Assignment Two1/6Feb. 18March 7regular
Project 1/3March 7See aboveheavy
Quizzes1/9 eachN/AvariousN/A

If for any reason the schedule given above cannot be adhered to, the assignment, project and quiz marks will be pro-rated. For instance, if an assignment has to be cancelled for any reason, the remaining assignment weight will be prorated to add up to 1/3.

Every effort will be made to have assignments, projects and quizzes marked and handed back within 3 weeks of the hand-in date, preferably sooner.

Quizzes

Quizzes may be held without being announced in advance.

Quizzes will be closed book.

No electronic devices may be in the possession of students during quizzes.

Assignments

Assignments will be due on the (tentative) dates listed above.

Extensions will be granted only by the course instructor. If you have serious medical or compassionate grounds for an extension, you should take supporting documentation to the office of the Dean of your faculty, who will contact the instructor.

The submission of assignments and quizzes is electronic using OWL.

Any changes, updates, and clarifications to assignments will also be posted on the course and/or OWL websites. It is your responsibility to monitor these pages closely.

When a student submits an assignment for evaluation, this student automatically certifies that the material she/he has handed in is exclusively her/his own work.

It is your responsibility to keep up-to-date backups of assignment disk files in case of system crashes or inadvertently erased files. Retain disk copies of all material handed in, as well as the actual graded assignment, to guard against the possibility of lost assignments or errors in recording marks. It is not safe to discard these materials until you are satisfied that your final mark for the course has been computed properly.

Lecture attendance

All students are expected to attend all classes. A student found to be missing a large number of classes without acceptable reasons risks being denied a passing grade.

Academic accommodation for medical illness

If you are unable to meet a course requirement (assignment due date) due to illness or other serious circumstances, you must provide valid medical or supporting documentation to the Academic Counselling Office of your home faculty as soon as possible. This rule does not apply for quizzes; see below.

For policy on Academic Consideration for Student Absences, follow this link, and for the Student Medical Certificate (SMC), follow this other link.

There are no make-up quizzes. If your faculty's Academic Counselling Office has approved your circumstances, the value of the missed test or quiz will be reallocated.

Public health and impact of the COVID-19 pandemic

Mask wearing

Students will be expected to wear triple layer non-medical masks at all times in the classroom as per Western policy and public health directives.

Students who are unable to wear a mask for medical or religious reasons must seek formal accommodation through Accessible Education.

Students with accommodations approved by Accessible Education must make arrangements with the instructor before coming to class.

Eating and drinking are prohibited in the classroom to facilitate proper mask wearing.

Students not wearing masks will be asked to leave the classroom.

Students unwilling to wear a mask as stipulated by Western policy and public health directives will be referred to the Department, and such actions will be considered a violation of the student Code of Conduct.

Daily return to campus questionnaire

Students must complete the symptom checklist. every day.

Students, who are found to be on campus and who have not completed the symptom checklist or have failed the daily symptom check, will be referred for investigation under the Student Code of Conduct.

Failing the daily Return to Campus Questionnaire does not automatically excuse them from in-class assessments and assignment deadlines: students must still either use a Self-Reported Absence or seek Academic Considerations through their Academic Counsellors.

Any student who is unable to attend in-person due to COVID-19 symptoms will be asked to obtain a test. That test result – positive or negative – should be submitted to Academic Counselling.

Students are expected to carry their student cards to campus each day in anticipation of spot checks by Building Ambassadors.

All instruction (classes in particular) do not need social distancing.

More information about public health and impact of the COVID-19 pandemic, in particular regarding vaccination requirements, can be found here.

Accommodation policies

Students with disabilities work with Accessible Education (formerly SSD) which provides recommendations for accommodation based on medical documentation or psychological and cognitive testing. The Academic Accommodation for Students with Disabilities policy can be found here.

Religious accommodation

Students should consult the University's list of recognized religious holidays, and should give reasonable notice in writing, prior to the holiday, to the Instructor and an Academic Counsellor if their course requirements will be affected by a religious observance. Additional information is given in the Western Multicultural Calendar.

You may also be eligible to write the Special Exam if you are in a “Multiple Exam Situation”, follow the link.

Academic policies

The website for Registrarial Services is here.

In accordance with this policy. the centrally administered e-mail account provided to students will be considered the individual’s official university e-mail address. It is the responsibility of the account holder to ensure that e-mail received from the University at his/her official university address is attended to in a timely manner.

All of the remote learning sessions for this course may be recorded. The data captured during these recordings may include your image, voice recordings, chat logs and personal identifiers (name displayed on the screen). The recordings will be used for educational purposes related to this course, including evaluations. The recordings may be disclosed to other individuals participating in the course for their private or group study purposes. Please contact the instructor if you have any concerns related to session recordings.

As mentioned above, participants in this course are not permitted to record the sessions, except where recording is an approved accommodation, or the participant has the prior written permission of the instructor.

Ethical conduct

Scholastic offences are taken seriously and students are directed to read the appropriate policy, specifically, the definition of what constitutes a Scholastic Offence, at this website.

Computer-marked, multiple-choice tests and exams may be subject to submission for similarity review by software that will check for unusual coincidences in answer patterns that may indicate cheating. In particular, checking software, such as MOSS , Turnitin or others will be used randomly for detecting similarity in assignment submission.

All assignments are individual assignments. You may discuss approaches to problems among yourselves. However, the actual details of the work (assignment coding, answers to concept questions, etc.) must be your individual effort. Assignments that are judged to be the result of academic dishonesty will, for the student's first offence. You are responsible for reading and respecting the Computer Science Department's policy on Rules of Ethical Conduct and Scholastic Offenses.

Support services

Please visit the Science & Basic Medical Sciences Academic Counselling we bpage for information on add/drop courses, academic considerations for absences, appeals, exam conflicts, and many other academic related matters.

Please contact the course instructor if you require lecture in an alternate format or if any other arrangements can make this course more accessible to you. You may also wish to contact Student Accessibility Services (SAS) at (519) 661-2147 if you have any questions regarding accommodation.

Western University is committed to a thriving campus as we deliver our courses in the mixed model of both virtual and face-to-face formats. We encourage you to check out the Digital Student Experience website to manage your academics and well-being. .

Learning-skills counsellors at the Student Development Centre. are ready to help you improve your learning skills. They offer presentations on strategies for improving time management, multiple-choice exam preparation/writing, textbook reading, and more. Individual support is offered throughout the Fall/Winter terms in the drop-in Learning Help Centre, and year-round through individual counselling. See also the services provided by the University Students’ Council.


Marc Moreno Maza
Last modified: Mon Jan 10, 2022