AMS 530: Principles of Parallel Computing

Spring 2008

 

Instructor

Professor Yuefan Deng

Room Number

A-135 Physics Tower, Stony Brook, NY 11794-3600

Phone Number

632-8614

Email Address

Yuefan.Deng@StonyBrook.edu

Web URL

http://www.ams.sunysb.edu/~deng

Class Time

Mondays & Wednesday 2:20-3:40

Office Hours

Mondays & Wednesday 11-12

Classroom

Room S235 SINC Site

 

TA

Peng Zhang (Email: penzhang@ams.sunysb.edu)

Office

Physics A-134

Office Hours

Tuesday and Thursday 4-6

Notes for spring 2008:

Update 5/20/2008 10:41AM:

(1)The grades will be sent to you by Peng individually.

(2)I would not go to the classroom today (5/12), but would post the lecture on Load Balance and several papers on my website.

(3)Please make sure your Project 4 is submitted successfully to Peng. I would have his final grading and send you the ABCs.

(4)Today (4/11)’s presentations are excellent! I like this format of projects. Please send Peng the latest PPT and essay that I might find use for the lecture notes (and even my future presentations.) Thank you for taking the time to make the project3 so much value. I hope someone in this class would give Project 5 a shot at.

(5)We will continue the lecture on parallel linear algebra today (4/2).

(6)Optional Project 5 is posted. This project has no effects on the final grades for most students. I would love to see one or two students with strong mathematics background to try this.

(7)Peng has already finished grading Project 1 and his grading reports have been sent to you individually while a common report that’s useful for the entire class is posted here:

Bottom of Form

Dear Prof Deng: The following is my comments on Project 1 of AMS 530.

 

1.        A MPI-based parallel program is a multiprocess program so when you run “mpirunnp 4 your_prog” where 4 implies 4 processes instead of 4 processors. Thus, when plotting the speedup figures, it is better to state you use 2/4/6/8 processes, rather than 2/4/6/8 processors. To better illustrate this, you can do the following experiment: you allocate 4 processors while you can run a 16-process program. The commands are:

a.       qsub –I –v nodes=2:ppn=2

b.      mpirunnp 16 your_prog

c.       It will run well. PBS handles this job when the number of required processes is more than the number of allocated processors.

2.      To measure the running time of a multiprocess program, you can use MPI_Wtime(). Before calling MPI_Wtime(), you might call MPI_Barrier() firstly since it can help block all of the processes in the communicator until all of them reach this routine.

3.      Pay attention to the usage of MPI_Barrier(communicator). This routine will block all of the processes in the communicator. If one or more processes in the communicator do not call this routine, a deadlock will occur. By the way, you can consider the implementation of MPI_Barrier() and the definition of MPI_COMM_WORLD.

4.      Considering the performance, you should avoid using some collective communication routines such as MPI_Bcast and MPI_Reduce in programming. You should try to send/receive the messages just in time and just you need. For example, you can consider some non-blocking communication routines before you’ve to use the collective messaging routins.

5.      In plotting the speedup figures, you should use the data measured by your programs. Also, in discussing the theoretical speedup, you cannot ignore the communication portion which actually plays a key role in affecting the total performance.

6.      As for the topic to evaluate the latency and bandwidth on the same switch and different switches, you should see that if two processes run on the same node, the latency between them would be comparatively smaller since they communicate via the same switch without using the external network. Thus, when evaluating latencies between different switches, the latency on the same switch should be eliminated.

(8)I revised the Project 3 presentation format:

a)  Presentation order is posted. Click “Students” to find out;

b) Grading guideline is posted, Click “Students” to find out;

c)  Essay and PPT must be sent to Peng Zhang no later than midnight of April 9 (Wednesday), i.e., the last minute of April 9;

d) Presentations will start at 9AM on April 11 (Friday) in the regular classroom;

e)  Each presentation can last no longer than 20 minutes;

f)   Lectures on Wednesdays (4/16 and 4/23) are merged to this one presentation.

(9)Project 4 is posted.

 

(10)       Please follow this link to login to Seawulf: http://ringo.ams.sunysb.edu/index.php/Getting_a_Seawulf_Account. Please inform Peng or myself if you have any questions.

(11)       Problem assignment for Project 3 is completed. Please click “Students” above to find out what problem you are assigned. We will stick to this assignment unless we have major issues.

(12)       Additional information from Yan so please pay careful attention to Yan’s emails:

From:

"Yan Yu" <yan2000@ams.sunysb.edu> 

Subject:

Re: ams530 students

Date:

Tue, 26 Feb 2008 15:06:52 -0500

Top of Form

Bottom of Form

 

Their username on seawulf. Six of them already have accounts so I didn't put their usernname below…. Their galaxy accounts will be deleted tomorrow after 5pm. Thanks!

 

Balius

Trent

Barsegian

Garen

Jiang

Tao

tjiang

Liverpool

Robert

rliver

Lou

Zhihao

zhlou

Nievesgonzalez

Aniel

aniegon

Ortiz

Joseph

jortiz

Patsalo

Vadim

Rao

Abhishek

Sun

Tao

tsun

Zuber

James

jzuber

(13)       We have serious complaints about one student’s way of using Galaxy. They will move all students to Seawulf so please be prepared for that. Please see this message sent by Yan and also, please contact Peng, on how to access Seawulf:

From:

"Yan Yu" <yan2000@ams.sunysb.edu> 

To:

deng@ams.sunysb.edu

Subject:

ams530 students

Date:

Tue, 26 Feb 2008 12:40:17 -0500

Top of Form

Bottom of Form

Yuefan, can you talk to xxx about his job on galaxy? It was messing up the queue again. Also, I am going to move all your ams530 students to seawulf. Please ask them to get data off galaxy ASAP and wait for the accounts on seawulf. They should all access seawulf through one machine. You can specify what that machine could be and send me the IP address. This machine should not be a highly used multi-user machine for security reasons. Seawulf is key login only. So please ask them to follow the instruction (at the end of this message) on generating the key pair and send us the PUBLIC key. Thanks! –Yan

           

(14)       Reworked Project 3 is posed. Please check and respond accordingly.

(15)       Due to machine problems (both Galaxy and Seawulf), I would like to extend the collection dates for both Projects 1&2 for one week. Please check “Projects” for such extensions.

(16)       Project 3 is to be posted today and, likely, I may add problems. Please follow the instruction to select the problem. Please be reminded that you may design your own research problem as your project.

(17)       Project 2 is posted.

(18)       Lecture for “Performance Analysis” is posted.

(19)       We will reserve the lecture time of Wednesday 2/13 for working on Project 1 already posted on the web. Our TA Peng Zhang will come to the classroom to help with the project.

(20)       The Galaxy accounts for all are ready. Call my office (2-8614) for your password. Better still, come to my office so that I can give it to you. Please IMMEDIATELY login and change your passport by yppasswd. Galaxy account names are (students whose names are not here already have obtained Galaxy account earlier.)

Saad Arif: sarif

Trent Balius: tbalius

Garen Barsegian: gbarse

Tao Jiang: tjiang

Robert Liverpool: rliver

Joseph Ortiz: jortiz

Abhishek Rao: arao

Tao Sun: tsun

Yu Wu: ywu

James Zuber: jzuber

(21)       Welcome to the parallel computing class for the spring 2008.

(22)       The accounts on the iMac (Linux) boxes and on the Galaxy computer have been requested. I expect them to be set up soon and I will communicate to you on the progress shortly.

(23)       You can now create your accounts on the iMac by following the procedure posted in the classroom as Pat has already added the roster at ~1PM on 1/30/2008.

(24)       Students asked about prerequisites…knowing one sequential computer language is necessary or some crash course to make it up.

 

Notes for Spring 2007: No Class due to sabbatical leave.

 

Notes for Spring 2006:

(1) More notes are posted here.

(2) Today (April 17), I will lecture on Lectures 11 (posted on web). See you in class!

(3) People who have not emailed Project 3 should do so.

(4) Project 4 is posted.

(5) Today (March 13), I will lecture on Lectures 5 and 7 and Project 2.

(6) Project 1 grading will be done later this week.

(7) Project 3 is also ready. Its due date is April 5 (three weeks after Project 2’s due date).

(8) Project 2 is posted. Good luck!

(9) Project 1 was due on Wednesday (March 1) so if you have not sent it in, please do it ASAP.

(10)               Sorry for my inability to be with the class regularly. I hope Mr. Xin Chen and Sebastian Okser were of help to you during my absence.

(11)               Due to snow, I need to reschedule the class. If I don’t post it here again, please consider Monday’s class is rescheduled. The rescheduled class will be posted here later.

(12)               Project 1 is ready. Please read and work on it and then feedback.

(13)               Please download each lecture notes from the links I posted here because I still don’t have a full list of emails to send lecture notes.

a)    Introduction: presentation I gave in China

b)   Lecture01B-Hardware (Link disabled)

c)    Lecture02-Software

d)   lecture03-performance-measure

e)    lecture04-pp-algorithm-intro

f)    lecture05-MPI-1

g)   lecture07-Complete List of MPI

h)   lecture08-Collective Operations

i)     lecture09-summary-collective-operations

j)     lecture10-process-mapping

k)   lecture11-12-linear-algebra

l)     lecture13-matrix-matrix-multiply

m) lecture14-integration-PDF

n)   lecture15-2D-Poisson

o)   lecture16-fft

p)   lecture24-monte-carlo-methods

q)   lecture27-load-balance

r)     lecture28-load-balance-II

 

(14)               Please email directly to staff@galaxy.ams.sunysb.edu for Galaxy account if you don’t have it. Please do this yourself. Let me know if you have difficulties.

(15)               Due to my heavy travel schedule, I’ll lecture more whenever I am on campus. This means that I need to “lump” the project classes together and I’ll assign the first three projects during the second week. You will have regular submission date.

(16)               Project 1 will be ready on web on Wednesday.

 

Notes for spring 2005:

(1) 3/25-A: Hope your spring break is re-charging…

(2) 3/23-A: Sorry for long absence.

(3) 3/23-B: I have started grading Project-1 and Project-2. Please let me know if you don’t receive grading reports in ~ 1 week from today.

(4) 3/23-C: Lectures will resume on Monday (3/28) and a lot of catch-up is necessary. Hope to see you on Monday at the regular time and place (SINC site.)

(5)  2/11-F: Sorry for the following changes (C, D, E) to cause inconvenience.

(6)  2/11-E: We will not meet on 3/14 and 3/16. We make up these two lectures by two Fridays in April (The precise dates will be determined in class.)

(7) 2/11-D: We will not meet on 3/7 and 3/9. These two lectures are reserved for your project-I.

(8) 2/11-C: We will not meet on 2/28 and 3/2. We make up these two lectures on Fridays (2/11 and 2/18).

(9) 2/11-B: Project-2 assignment is ready on the web. Its due date is 3/23.

(10)                    2/11-A: Lecture-5 has been mailed.

(11)                    2/9-A: I have received multiple emails about missing lecture notes. Please don’t worry too much about the past notes. I have added all registered students (and others who request) to my mailing list. You will receive the notes.

(12)                    2/9-B: My parallel computing “book” that has most of the lecture material will be distributed to all registered students and other who request it. Currently I got help from two talented students to finish the book format.

(13)                    2/2-B: Today’s lecture is emailed.

(14)                    2/2-A: Project One is ready on the web. Please view it and I lecture on it on Monday (2/7)

(15)                    1/31-E: Project One will be posted on Wednesday (2/2). Please view the project.

(16)                    1/31-D: I have posted MANY parallel code examples in “References”; please go there to check out if you are interested in such examples.

(17)                    1/31-C: I’ll post the presentation I gave in China here as it’s relevant to the class. All lecture notes will be emailed to people who register.

(18)                    1/31-B: I would like to repeat announcement 1/24-B. Please get your Galaxy account. The latest Students list can be found.

(19)                    1/31-A: Several more students are registering for the class so we will stay in the current classroom for one more week and then we will move to AMS seminar room.

(20)                    1/24-A: First lectures will meet in the assigned classroom in SINC site (S235). Will move to AMS seminar room for future lectures. The exact date will be announced in class and here.

(21)                    1/24-B: Please send me your “name, email, and preferred Galaxy account login” ASAP so that I can (a) get the Galaxy accounts setup and (b) send you the lecture notes.

 

Notes for spring 2004:

(1)Projects 1-3 are all posted on my web. Please click “Projects” above to see these projects. The Project 3 collection date is April 14 (You have three weeks to work on it after finishing Project 2 whose collection date is march 23.)

(2)Your accounts on nkstars.nankai.edu.cn are ready. I have the passwords. Will bring them to class on Monday (2/16). If you need your account earlier please email or see me.

(3)Nankai-Stars real-time status could be viewed here: http://www.nku-isc.org/nkstar/ganglia/?c=Nankai%20Stars&m=&r=week&s=descending&hc=4

(4)Please send me your “name, email, and preferred Galaxy account login” ASAP so that I can (a) get the Galaxy accounts setup and (b) send you the lecture notes.

(5)I’ve requested accounts for people on Students list. Lecture notes are also mailed.

(6)Hope to see you at the seminar on 2/4 in our classroom.

 

Partial list of old announcements made in 2003.

(1)Project I is assigned. Please check on this web (at “Projects” above). The Due date is 2/28/’03 for Project I.

(2)There are many examples that are helpful for completing your projects. Please access such examples at “References” above.

(3)All lecture notes will be sent to registered students on Mondays (for Tuesday and Thursday’s lectures.)

(4)Please send me your “name, email, and preferred Galaxy account login” ASAP so that I can (a) get the Galaxy accounts setup and (b) send you the lecture notes.

(5)This course does not require prior knowledge of C++; all students interesting in programming multiple processors for research are encouraged to take this course. It’s a project-oriented course, i.e., learning through practice of projects.

(6)The load of each project is similar to that of a typical take-home exam which can be done within 3-5 hours. For graduate students who have own research projects, we can discuss to count the research project as one of the course projects.