AMS 526: Numerical Analysis I (Numerical Linear Algebra)
Fall 2022
Time: Monday & Wednesday 4:25--5:45 pm ET
Location: Earth and Space Sciences (ESS) 177

Lecture Schedule and Slides

Instructor: Prof. Xiangmin (Jim) Jiao
Phone: 631-632-4408
Office hours: Mon. 1:15pm--2:15pm ET &
                      Wed. 10:15am--12:15pm ET
Office: 1-117 Math Tower or Join via Zoom

TA: Chengpeng Sun
Office hours: Tue. & Thur. 4:30pm--5:30pm
Office: Join via Zoom

[ Description | Outline | Delivery Mode | Tech Requirements | Grading Policy | Homework and Exams | References | University Policy ]

Course Description (back to top)

Direct and iterative methods for solving simultaneous linear equations. Matrix factorization, conditioning, stability, sparsity, and efficiency. Computation of eigenvalues and eigenvectors. Singular value decomposition.

Required Textbook


  • AMS 510 (linear algebra portion as co-requisite) or equivalent undergraduate-level linear algebra course. Familiarity of the following concepts is assumed: Vector spaces, Gaussian elimination, Gram-Schmidt orthogonalization, and eigenvalues/eigenvectors.
  • AMS 595 (co-requisite for students without programming experience).

Learning Objectives

The objective of this course is to introduce the key concepts and algorithms in numerical linear algebra, including direct and iterative methods for solving simultaneous linear equations, least squares problems, computation of eigenvalues and eigenvectors, and singular value decomposition. The key learning outcomes include the following:

  1. Master concepts and numerical methods for solving systems of linear equations
    • Gaussian elimination and its variants
    • Cholesky and LDL' factorizations
  2. Master concepts of orthogonality and numerical methods for linear least squares problems
    • Orthogonal matrices, projectors, and linear least squares
    • QR factorization using Gram-Schmidt orthogonalization, Householder reflectors, and Givens rotation
  3. Build fundamental understanding of conditioning and stability
    • Norms, condition numbers, and effect of rounding errors
    • Stable and backward stable algorithms
    • Backward error analysis of fundamental algorithms
  4. Master concepts and analyses based on eigenvalues and singular values and their numerical computations
    • Singular value decomposition and eigenvalue decomposition
    • Power method and similarity transformations
    • Reduction to Hessenberg and tridiagonal forms
  5. Build fundamental understanding of iterative methods for solving large sparse linear systems and computing eigenvalues
    • Conjugate gradient method, GMRES, and other Krylov subspace methods
    • Lanczos and Arnoldi iterations
    • Preconditioners for iterative methods
  6. Demonstrate programming skills for numerical methods using the abstractions of linear algebra

Course Outline (back to top)

  • Fundamentals (matrix notation and basic operations; vector spaces; algorithmic considerations; norms and condition numbers; decomposition of matrices)
  • Linear systems (triangular systems; Gaussian elimination; accuracy and stability; Cholesky factorization; sparse linear systems)
  • QR factorization and least squares (Gram-Schmidt orthogonalization; QR factorization with Householder reflection; updating QR factorization with Givens rotation; stability of QR factorization; least squares problems; rank-revealing QR factorization; SVD and low-rank approximations)
  • Eigenvalue problems (eigenvalues and invariant spaces; classical eigenvalue methods; QR algorithms; two-stage methods; Arnoldi and Lanczos iterations)
  • Iterative Methods for linear systems (basic iterative methods; conjugate gradient methods; minimal residual style methods; bi-Lanczos iterations; preconditioners)
  • Special topics (multigrid methods; under-determined linear systems etc., if time permits)

Lecture Schedule and Slides

Course Delivery Mode and Structure (back to top)

This course will be in person. We will use Brightspace ( for posting and submission of assignments, posting grades, and discussion forums. For personal/private issues, my preferred method of contact is email, as listed at the top of this syllabus. I strive to respond to your emails as soon as possible, but please allow between 24-48 hours for a reply. All email communication will be sent to your Stony Brook University email account. You must have an active Stony Brook University email account and access to the Internet. Please plan on checking Brightspace regularly and your SBU email account for course-related messages or set up your SBU email account to forward to your email account. To log in to Stony Brook Google Mail, go to and sign in with your NetID and password.

Important announcements will be sent on Brightspace. These will be posted in the class and may or may not be sent by email. I will participate and post regularly on the discussion board in Brightspace and provide feedback on assignments within a week. Regular communication is essential in online classes. When participating in class discussions, the expectation is that you will regularly respond to your peers and questions posed to your responses. Logging in regularly, checking the discussion board, and participating with your colleagues ensure that you can remain an active member of the class.

Grading Policy (back to top)

Assignment Policy

Homework assignments are typically due two weeks after they are assigned. You can discuss course materials and homework problems in small groups but only discuss general ideas. You must write your solutions entirely independently. Under no circumstances may you copy solutions from any source, including but not limited to other studentsí solutions, official solutions distributed in past terms, and solutions from courses taught at other universities. Violation of these rules may result in disciplinary actions.


The exams (including two tests and the final exam) are closed-book, but you are allowed to bring a single-sided, one-page, letter-size cheat sheet, which you must prepare by yourself. The exams will be in person.


All students are expected to attend all the lectures and exams.


  • Assignments: 30%

  • Two midterm exams: 40%
  • Final exam: 30%

Homework and Exams (back to top)


All the assignments will be posted on Brightspace and should be submitted through Brightspace. You are encouraged to typeset your homework solutions using LaTeX or LyX (a very easy-to-use document processor, using LaTeX in the backend), or write your answers by hand and submit a scanned copy. However, Microsoft Word is not recommended.


The two midterms are tentatively scheduled to be held on
  • September 28th in class
  • November 2nd in class
Specific times of the midterms are subject to change. The final exam will be on
  • December 7th between 8:30pm and 11pm Eastern Time

Sample tests will be posted on Brightspace about one week before the exam.

References and Other Resources (back to top)

References on Undergraduate-Level Linear Algebra

The following is an excellent textbook for reviewing fundamental concepts and some applications of linear algebra.

Other References on Numerical Linear Algebra

The following book is a graduate-level textbook on numerical linear algebra, similar to the main textbook for this course.

References on Iterative and Multigrid Methods

The following books are for additional readings on iterative methods and multigrid methods, which are increasingly important but not covered in this course due to time constraint.

Linear Algebra for Data Science

References on MATLAB Programming

References on NumPy Programming

References on C/C++ Programming

There are many reference on C and C++. For example,

  • M. Banahan, D. Brady and M. Doran, The C Book, 2nd Edition, Addison Wesley, 1991.
  • B. Stroustrup, A Tour of C++, 2nd Edition, 2018.

References on Technical Writing

University Policies and Academic Integrity (back to top)

Student Accessibility Support Center Statement

If you have a physical, psychological, medical, or learning disability that may impact your course work, please contact the Student Accessibility Support Center, Stony Brook Union Suite 107, (631) 632-6748, or at They will determine with you what accommodations are necessary and appropriate. All information and documentation is confidential.

Academic Integrity Statement

Each student must pursue his or her academic goals honestly and be personally accountable for all submitted work. Representing another person's work as your own is always wrong. Faculty is required to report any suspected instances of academic dishonesty to the Academic Judiciary. Faculty in the Health Sciences Center (School of Health Technology & Management, Nursing, Social Welfare, Dental Medicine) and School of Medicine are required to follow their school-specific procedures. For more comprehensive information on academic integrity, including categories of academic dishonesty please refer to the academic judiciary website at

Critical Incident Management

Stony Brook University expects students to respect the rights, privileges, and property of other people. Faculty are required to report to the Office of Student Conduct and Community Standards any disruptive behavior that interrupts their ability to teach, compromises the safety of the learning environment, or inhibits students' ability to learn. Faculty in the HSC Schools and the School of Medicine are required to follow their school-specific procedures. Further information about most academic matters can be found in the Undergraduate Bulletin, the Undergraduate Class Schedule, and the Faculty-Employee Handbook.