User:Scut723/sandbox/Singular Value Decomposition

Singular Value Decomposition

The singular value decomposition (SVD) is one of the most powerful tools in theoretical and numerical linear algebra. The utility comes from three basic properties:

Every matrix has an SVD. The SVD provides an orthonormal resolution for the four invariant subspaces. The SVD provides an ordered list of singular values.

The Singular Value Decomposition Theorem

Existence

Every matrix has a singular value decomposition. Given a matrix $\mathbf {A} \in \mathbb {C} _{\rho }^{m\times n}$, that is, with $m$ rows, $n$ columns, and rank $\rho$, the SVD can be written as

$\mathbf {A} =\mathbf {U} \Sigma \mathbf {V} ^{*}$,

where

• $\mathbf {U} \in \mathbb {C} ^{m\times m}$ resolves the column space,
• $\mathbf {V} \in \mathbb {C} ^{n\times n}$ resolves the row space,
• $\mathbf {\Sigma } \in \mathbb {R} _{\rho }^{m\times n}$ contains the singular values.

The domain matrices are unitary:

$\mathbf {U} \mathbf {U} ^{*}=\mathbf {U} ^{*}\mathbf {U} =\mathbf {I} _{m}$

$\mathbf {V} \mathbf {V} ^{*}=\mathbf {V} ^{*}\mathbf {V} =\mathbf {I} _{n}$

Uniqueness

The singular values are unique, therefore the matrices $\mathbf {S}$ and $\Sigma$ are unique. Typically the domain matrices are not unique. For example, there could be two different decompositions such that

$\mathbf {A} =\mathbf {U} _{1}\mathbf {\Sigma } \mathbf {V} _{1}^{*}=\mathbf {U} _{2}\mathbf {\Sigma } \mathbf {V} _{2}^{*}$

References

• Golub, Gene H.; Van Loan, Charles F. (1996). Matrix Computations (3rd ed.). Johns Hopkins. ISBN 978-0-8018-5414-9.
• GSL Team (2007). "§14.4 Singular Value Decomposition". GNU Scientific Library. Reference Manual.
• Trefethen, Lloyd N.; Bau III, David (1997). Numerical linear algebra. Philadelphia: Society for Industrial and Applied Mathematics. ISBN 978-0-89871-361-9.
• Demmel, James; Kahan, William (1990). "Accurate singular values of bidiagonal matrices". Society for Industrial and Applied Mathematics. Journal on Scientific and Statistical Computing. 11 (5): 873–912. doi:10.1137/0911052.

External links

• Mathworld
• Online SVD calculator
• Wikipedia
• MIT Tutorial