I'm comfortable with general programming, but I'm relatively new to scientific computing. I've started doing research with a professor, and much of my work involves simulations, modeling physical systems and mathematical equations, and writing algorithms to analyze or reproduce those systems. I'd like to become substantially better at numerical methods, computational physics, mathematical modeling, simulation, and scientific programming. What textbooks, courses, libraries, or projects would you recommend for moving beyond programming fundamentals and learning the mathematics and computational techniques behind this work?
5 Answers
A broad progression would be linear algebra first: direct and iterative solvers, LU and QR factorizations, SVD, eigenvalues, conditioning, norms, and eventually preconditioning. Then study analysis topics such as Taylor expansions, regularity, integration, sequences, and series. After that, move into finite differences and time-stepping methods such as Euler, upwind schemes, and Runge–Kutta, paying attention to consistency, stability, and convergence order. Interpolation, numerical quadrature, and optimization methods such as Newton’s method are also useful across many applications.
Numerical methods are probably the best place to focus. Numerical Recipes is an approachable starting point, and a numerical analysis textbook such as Burden and Faires can provide a more systematic foundation. It’s also worth learning how Taylor series and fixed-point iteration appear in root finding, interpolation, and ODE solvers. You should understand floating-point round-off and numerical stability well enough to recognize when tolerances and results may be unreliable.
If you want to specialize in larger-scale scientific computing, combine the mathematics with practical performance skills. Learn memory layout and cache behavior, then explore multithreading, MPI, GPU programming, and optimized linear algebra. These skills are especially valuable because a mathematically correct algorithm may still be impractical if it is poorly parallelized or handles data inefficiently.
Build small projects and keep careful records of both successful and failed approaches. Implement a root finder, an ODE solver, a numerical integration routine, and a simple PDE model, then test them against problems with known solutions. Use manufactured solutions and convergence tests to check whether your implementation is behaving as expected. Once you are comfortable with the basics, try finite-element or finite-volume tools and learn enough mesh generation to work with realistic models.
Your professor may be able to point you toward an introductory mathematical-modeling or numerical-analysis course. Since you’re already working on research, try connecting each topic directly to your project: identify the governing equations, determine what assumptions are being made, choose a discretization, and validate the result against analytical, experimental, or benchmark data. That feedback loop is usually more effective than studying algorithms in isolation.
You don’t necessarily need to master every detail of floating-point arithmetic immediately, but you should know that computations have inaccuracies and can become unstable. In practice, carefully chosen tolerances and tests are often enough at first, while deeper knowledge becomes important for difficult or high-precision problems.