x = A b; in Matlab

x = A b;

  1. Is A square?
    no => use QR to solve least squares problem.
  2. Is A triangular or permuted triangular?
    yes => sparse triangular solve
  3. Is A symmetric with positive diagonal elements?
    yes => attempt Cholesky after symmetric minimum degree.
  4. Otherwise
    => use LU on A (:, colamd(A))