AMS 530: Principles of Parallel Computing
|
Instructor |
Professor Yuefan Deng |
|
Room Number |
A-135 |
|
Phone Number |
|
|
Email Address |
|
|
Web URL |
|
|
Class Time |
Mondays &
Wednesday 2:20-3:40 |
|
Office Hours |
Mondays & Wednesday 11-12 |
|
Classroom |
Room S235 SINC Site |
|
|
|
|
TA |
Peng Zhang (Email: penzhang@ams.sunysb.edu) |
|
Office |
Physics A-134 |
|
Office Hours |
Tuesday and Thursday 4-6 |
Notes for spring 2008:
Update 5/20/2008
10:41AM:
(1)The
grades will be sent to you by Peng individually.
(2)I would
not go to the classroom today (5/12), but would post the lecture on Load
Balance and several papers on my website.
(3)Please
make sure your Project 4 is submitted successfully to Peng.
I would have his final grading and send you the ABCs.
(4)Today (4/11)’s presentations are
excellent! I like this format of projects. Please send Peng
the latest PPT and essay that I might find use for the lecture notes (and even
my future presentations.) Thank you for taking the time to make the project3 so
much value. I hope someone in this class would give Project 5 a shot at.
(5)We will continue the lecture on parallel linear
algebra today (4/2).
(6)Optional Project 5 is posted. This project has
no effects on the final grades for most students. I would love to see one or
two students with strong mathematics background to try this.
(7)Peng has already finished
grading Project 1 and his grading reports have been sent to you individually
while a common report that’s useful for the entire class is posted here:
“
Dear Prof Deng: The
following is my comments on Project 1 of AMS 530.
1. A MPI-based parallel program is a multiprocess program so when you run “mpirun –np 4 your_prog” where 4 implies 4 processes instead of 4 processors. Thus, when plotting the speedup figures, it is better to state you use 2/4/6/8 processes, rather than 2/4/6/8 processors. To better illustrate this, you can do the following experiment: you allocate 4 processors while you can run a 16-process program. The commands are:
a. qsub –I –v nodes=2:ppn=2
b. mpirun –np 16 your_prog
c. It will run well. PBS handles this job when the number of required processes is more than the number of allocated processors.
2. To measure the running time of a multiprocess program, you can use MPI_Wtime(). Before calling MPI_Wtime(), you might call MPI_Barrier() firstly since it can help block all of the processes in the communicator until all of them reach this routine.
3. Pay attention to the usage of MPI_Barrier(communicator). This routine will block all of the processes in the communicator. If one or more processes in the communicator do not call this routine, a deadlock will occur. By the way, you can consider the implementation of MPI_Barrier() and the definition of MPI_COMM_WORLD.
4. Considering the performance, you should avoid using some collective communication routines such as MPI_Bcast and MPI_Reduce in programming. You should try to send/receive the messages just in time and just you need. For example, you can consider some non-blocking communication routines before you’ve to use the collective messaging routins.
5. In plotting the speedup figures, you should use the data measured by your programs. Also, in discussing the theoretical speedup, you cannot ignore the communication portion which actually plays a key role in affecting the total performance.
6. As for the topic to evaluate the latency and bandwidth on the same switch and different switches, you should see that if two processes run on the same node, the latency between them would be comparatively smaller since they communicate via the same switch without using the external network. Thus, when evaluating latencies between different switches, the latency on the same switch should be eliminated.
“
(8)I revised the Project 3 presentation format:
a) Presentation order is
posted. Click “Students” to find out;
b) Grading guideline is
posted, Click “Students” to find out;
c) Essay and PPT must be
sent to Peng Zhang no later than midnight of April 9
(Wednesday), i.e., the last minute of April 9;
d) Presentations will
start at 9AM on April 11 (Friday) in the regular classroom;
e) Each presentation can
last no longer than 20 minutes;
f)
Lectures on Wednesdays (4/16 and 4/23) are merged to this one
presentation.
(9)Project 4 is posted.
(10)
Please follow this link to login to Seawulf:
http://ringo.ams.sunysb.edu/index.php/Getting_a_Seawulf_Account.
Please inform Peng or myself if you have any
questions.
(11)
Problem assignment for Project 3 is completed. Please click
“Students” above to find out what problem you are assigned. We will
stick to this assignment unless we have major issues.
(12)
Additional information from Yan so please pay careful
attention to Yan’s emails:
|
From: |
"Yan Yu" <yan2000@ams.sunysb.edu> |
|
Subject: |
Re: ams530 students |
|
Date: |
Tue, 26 Feb 2008 15:06:52 -0500 |
Their username on seawulf. Six of them already have accounts so I didn't put their usernname below…. Their galaxy accounts will be deleted tomorrow after 5pm. Thanks!
|
Balius |
Trent |
|
|
Barsegian |
Garen |
|
|
Jiang |
Tao |
tjiang |
|
Liverpool |
Robert |
rliver |
|
Lou |
Zhihao |
zhlou |
|
Nievesgonzalez |
Aniel |
aniegon |
|
Ortiz |
Joseph |
jortiz |
|
Patsalo |
Vadim |
|
|
Rao |
Abhishek |
|
|
Sun |
Tao |
tsun |
|
Zuber |
James |
jzuber |
(13)
We have serious complaints about one student’s way of
using Galaxy. They will move all students to Seawulf
so please be prepared for that. Please see this message sent by Yan and also,
please contact Peng, on how to access Seawulf:
|
From: |
"Yan Yu" <yan2000@ams.sunysb.edu> |
|
To: |
|
|
Subject: |
ams530 students |
|
Date: |
Tue, 26 Feb 2008 12:40:17 -0500 |
Yuefan, can you talk to xxx about his job on
galaxy? It was messing up the queue again. Also, I am going to move all your
ams530 students to seawulf. Please ask them to get data
off galaxy ASAP and wait for the accounts on seawulf.
They should all access seawulf through one machine.
You can specify what that machine could be and send me the IP address. This
machine should not be a highly used multi-user machine for security reasons. Seawulf is key login only. So please ask them to follow the
instruction (at the end of this message) on generating the key pair and
send us the PUBLIC key. Thanks! –Yan
(14)
Reworked Project 3 is posed. Please check and respond
accordingly.
(15)
Due to machine problems (both Galaxy and Seawulf),
I would like to extend the collection dates for both Projects 1&2 for one
week. Please check “Projects” for such extensions.
(16)
Project 3 is to be posted today and, likely, I may add problems. Please
follow the instruction to select the problem. Please be reminded that you may
design your own research problem as your project.
(17)
Project 2 is posted.
(18)
Lecture for “Performance Analysis” is posted.
(19)
We will reserve the lecture time of Wednesday 2/13 for working on Project
1 already posted on the web. Our TA Peng Zhang will
come to the classroom to help with the project.
(20)
The Galaxy accounts for all are ready. Call my office (2-8614) for your
password. Better still, come to my office so that I
can give it to you. Please IMMEDIATELY login and change your passport by yppasswd. Galaxy account names are (students whose names
are not here already have obtained Galaxy account earlier.)
Saad Arif: sarif
Trent Balius: tbalius
Garen Barsegian: gbarse
Tao Jiang: tjiang
Robert Liverpool: rliver
Joseph Ortiz: jortiz
Abhishek Rao: arao
Tao Sun: tsun
Yu Wu: ywu
James Zuber: jzuber
(21)
Welcome to the parallel computing class for the spring 2008.
(22)
The accounts on the iMac (Linux) boxes and on the Galaxy computer have
been requested. I expect them to be set up soon and I will communicate to you
on the progress shortly.
(23)
You can now create your accounts on the iMac by following the procedure
posted in the classroom as Pat has already added the roster at ~1PM on
1/30/2008.
(24)
Students asked about prerequisites…knowing one sequential
computer language is necessary or some crash course to make it up.
Notes for Spring 2007: No Class due to sabbatical leave.
Notes for Spring 2006:
(1) More notes are posted
here.
(2) Today (April 17), I
will lecture on Lectures 11 (posted on web). See you in class!
(3) People who have not
emailed Project 3 should do so.
(4) Project 4 is posted.
(5) Today (March 13), I
will lecture on Lectures 5 and 7 and Project 2.
(6) Project 1 grading will
be done later this week.
(7) Project 3 is also ready.
Its due date is April 5 (three weeks after Project
(8) Project 2 is posted.
Good luck!
(9) Project 1 was due on
Wednesday (March 1) so if you have not sent it in, please do it ASAP.
(10)
Sorry for my inability to be with the class regularly. I hope Mr. Xin Chen and Sebastian Okser were
of help to you during my absence.
(11)
Due to snow, I need to reschedule the class. If I don’t post it
here again, please consider Monday’s class is rescheduled. The
rescheduled class will be posted here later.
(12)
Project 1 is ready. Please read and work on it and then feedback.
(13)
Please download each lecture notes from the links I posted here because
I still don’t have a full list of emails to send lecture notes.
a)
Introduction:
presentation
I gave in China
b)
Lecture01B-Hardware (Link disabled)
c)
Lecture02-Software
d)
lecture03-performance-measure
e)
lecture04-pp-algorithm-intro
f)
lecture05-MPI-1
g)
lecture07-Complete List of MPI
h)
lecture08-Collective Operations
i)
lecture09-summary-collective-operations
j)
lecture10-process-mapping
k)
lecture11-12-linear-algebra
l)
lecture13-matrix-matrix-multiply
p)
lecture24-monte-carlo-methods
(14)
Please email directly to staff@galaxy.ams.sunysb.edu for Galaxy
account if you don’t have it. Please do this yourself. Let me know if you
have difficulties.
(15)
Due to my heavy travel schedule, I’ll lecture more whenever I am
on campus. This means that I need to “lump” the project classes
together and I’ll assign the first three projects during the second week.
You will have regular submission date.
(16)
Project 1 will be ready on web on Wednesday.
Notes for spring 2005:
(1) 3/25-A: Hope your
spring break is re-charging…
(2) 3/23-A: Sorry for long
absence.
(3) 3/23-B: I have started
grading Project-1 and Project-2. Please let me know if you don’t receive
grading reports in ~ 1 week from today.
(4) 3/23-C: Lectures will
resume on Monday (3/28) and a lot of catch-up is necessary. Hope to see you on
Monday at the regular time and place (SINC site.)
(5) 2/11-F: Sorry for the following changes (C, D, E) to cause inconvenience.
(6) 2/11-E: We will not meet on 3/14 and
3/16. We make up these two lectures by two Fridays in April (The precise dates
will be determined in class.)
(7) 2/11-D: We will not
meet on 3/7 and 3/9. These two lectures are reserved for your project-I.
(8) 2/11-C: We will not
meet on 2/28 and 3/2. We make up these two lectures on Fridays (2/11 and 2/18).
(9) 2/11-B: Project-2
assignment is ready on the web. Its due date is 3/23.
(10)
2/11-A: Lecture-5 has been mailed.
(11)
2/9-A: I have received multiple emails about missing lecture notes.
Please don’t worry too much about the past notes. I have added all
registered students (and others who request) to my mailing list. You will
receive the notes.
(12)
2/9-B: My parallel computing “book” that has most of the
lecture material will be distributed to all registered students and other who
request it. Currently I got help from two talented students to finish the book
format.
(13)
2/2-B: Today’s lecture is emailed.
(14)
2/2-A: Project One is ready on the web. Please view it and I lecture on
it on Monday (2/7)
(15)
1/31-E: Project One will be posted on Wednesday (2/2). Please view the
project.
(16)
1/31-D: I have posted MANY parallel code examples in
“References”; please go there to check out if you are interested in
such examples.
(17)
1/31-C: I’ll post the presentation
I gave in China here as it’s relevant to the class. All lecture notes will be
emailed to people who register.
(18)
1/31-B: I would like to repeat announcement 1/24-B. Please get your
Galaxy account. The latest Students
list can
be found.
(19)
1/31-A: Several more students are registering for the class so we will
stay in the current classroom for one more week and then we will move to
(20)
1/24-A: First lectures will meet in the assigned classroom in SINC site
(S235). Will move to
(21)
1/24-B: Please send me your
“name, email, and preferred Galaxy account login” ASAP so that I
can (a) get the Galaxy accounts setup and (b) send you the lecture notes.
Notes for spring 2004:
(1)Projects 1-3 are all posted on my web. Please
click “Projects” above to see these projects. The Project 3
collection date is April 14 (You have three weeks to work on it after finishing
Project 2 whose collection date is march 23.)
(2)Your accounts on nkstars.nankai.edu.cn are ready.
I have the passwords. Will bring them to class on Monday (2/16). If you need
your account earlier please email or see me.
(3)Nankai-Stars real-time status could be viewed
here:
http://www.nku-isc.org/nkstar/ganglia/?c=Nankai%20Stars&m=&r=week&s=descending&hc=4
(4)Please send me your “name, email, and
preferred Galaxy account login” ASAP so that I can (a) get the Galaxy
accounts setup and (b) send you the lecture notes.
(5)I’ve requested accounts for people on Students
list. Lecture notes are also mailed.
(6)Hope to see you at the seminar on 2/
Partial list
of old announcements made in 2003.
(1)Project I is assigned. Please check on this web
(at “Projects” above). The Due date is 2/28/’03 for Project
I.
(2)There are many examples that are helpful for
completing your projects. Please access such examples at
“References” above.
(3)All lecture notes will be sent to registered
students on Mondays (for Tuesday and Thursday’s lectures.)
(4)Please send me your “name, email, and
preferred Galaxy account login” ASAP so that I can (a) get the Galaxy
accounts setup and (b) send you the lecture notes.
(5)This course does not require prior knowledge of
C++; all students interesting in programming multiple processors for research
are encouraged to take this course. It’s a project-oriented course, i.e.,
learning through practice of projects.
(6)The load of each project is similar to that of
a typical take-home exam which can be done within 3-5 hours. For graduate
students who have own research projects, we can discuss to count the research
project as one of the course projects.