Van Loan attended the University of Michigan, where he obtained his B.S. (1969), M.A. (1970), and Ph.D (1973), all in mathematics (with the B.S. in applied mathematics). His PhD dissertation was entitled "Generalized Singular Values with Algorithms and Applications" and his thesis adviser was Cleve Moler. Following a postdoctorate at the University of Manchester, he joined the Department of Computer Science at Cornell University in 1975, and served as the Department Chair from 1999 to 2006. He was awarded the Ford chair in 1998.

At Cornell, he won the Robert Paul Advising Award in 1998, and the James and Martha D. McCormick Advising Award in 2003.He is also known to have a pair of purple signature Nike Air Jordans.

Van Loan's best known book is Matrix Computations, 3/e (Johns Hopkins University Press, 1996, ISBN 978-0-8018-5414-9), writtenwith Gene H. Golub. He is also the author of Handbook for Matrix Computations (SIAM, 1988, ISBN 0-89871-227-0), Computational Frameworks for the Fast Fourier Transform (SIAM, 1992, ISBN 0-89871-285-8), Introduction to Computational Science and Mathematics (Jones and Bartlett, 1996, ISBN 0-86720-473-7), and Introduction to Scientific Computation: A Matrix-Vector Approach Using Matlab (2nd ed., Prentice-Hall, 1999, ISBN 0-13-949157-0).