Singular Value Decomposition and Its Visualization
Lingsong Zhang^{1}, J. S. Marron, Haipeng Shen and Zhengyuan
Zhu
Abstract
Singular Value Decomposition (SVD) is a useful tool in Functional
Data Analysis (FDA). Comparing to Principal Component Analysis
(PCA), SVD is more fundamental, because SVD simultaneously provides
the PCAs in both row and column spaces. We compare SVD and PCA from
the FDA view point, and extend the usual SVD to variations by
considering different recenterings. A general scree plot is proposed
to select an appropriate recentering in practice. Several matrix
views of the SVD components are introduced to explore different
features in data, including SVD surface plot, rotation movie, curve
movie and image plot. These methods visualize both column and row
information of a twoway matrix simultaneously, relate the matrix to
relevant curves, show local variations and interactions between
columns and rows. Several toy examples are designed to compare as
well as reveal the different variations of SVD, and real data
examples are used to illustrate the usefulness of the visualization
methods.
Key words: Exploratory Data Analysis, Functional
Data Analysis, Principal Component Analysis.
Introduction
Functional Data Analysis (FDA) is the study of curves (and more
complex objects) as data [Ramsay and Silverman, 1997,Ramsay and Silverman, 2002]. Methods related to
Principal Component Analysis (PCA) have provided many insights.
Comparing to the PCA method, Singular Value Decomposition (SVD) is
more fundamental, because SVD not only provides a direct approach to
calculate the principal components (PCs), but also derives the PCAs
in row and column spaces simultaneously. In this paper, we view a
set of curves as a twoway data matrix, explore the connections and
differences between SVD and PCA from a FDA view point, and propose
several visualization methods for the SVD components.
SVD provides a useful factorization of the data matrix X, while
PCA provides a nearly parallel factoring, via eigenanalysis of the
covariance matrix, i.e. X^{T}X, when X is column centered at 0.
The eigenvalues for X^{T}X are then the squares of the singular
values for X, and the eigenvectors for X^{T}X are the singular
rows for X. In this paper, we extend the usual (column centered)
PCA method into a general SVD framework, and consider four types of
SVDs based on different recenterings: SVD, column SVD (CSVD), row
SVD (RSVD) and double SVD (DSVD) (cf. Section for
details). Several criteria are discussed in this paper for model
selection, i.e. selecting the appropriate type of SVD, including
model simplicity, approximation performance, and interpretability
etc. We introduce a general scree plot, which combines model
simplicity and approximation performance together, and provides a
visual aid for model selection. See Section for
details. Several toy examples in Section are
designed to illustrate the general scree plot and the differences of
the four types of recenterings.
Visualization methods always help to find underlying features of a
data set. In the context of PCA or SVD, common visualization methods
include the biplot [Gabriel, 1971], scatter plots between singular
columns or singular rows [Shen and Huang, 2005], etc. The biplot
shows the relations between the rows and columns, and the scatter
plot can be used to cluster the rows or columns. However these plots
fail to show the functional curves.
In the FDA field, besides the above plots, it is common to plot
singular columns or singular rows as curves [Ramsay and Silverman, 1997].
Marron et al., [2004] provided a visualization method for functional data
(using functional PCA, basically CSVD in our paper), which shows the
functional objects (curves), projections on the PCs and the
residuals. These methods can also be applied in the SVD framework,
but to understand all the structure in the data, they need to be
applied twice, once for the rows and once for the columns. When the
data set is a time series of curves, the method in Marron et al., [2004] uses
different colors to show the time ordering. However, if the time
series structure is complicated, color coding curves might not be
enough to reveal the time effect.
In our paper, we propose several matrix views of the SVD components,
which provide new underlying features of the data set. The major
visualization tool is a set of surface plots. Related to the surface
plot, we introduce SVD rotation movie, SVD curve movie, and SVD
image plot. The surface plots for the SVD components show the
functional curves of rows and columns simultaneously. SVD rotation
movie shows the surface plots from different view angles, which
makes it easier to find underlying information. In fact, the
visualization method in Marron et al., [2004] (for example, the first column
in Figure 3 of their paper) can be viewed as the surface plots from
a special angle. Another special view angle, the overlook view of
the surface plots, becomes the image plot, which can show the
relative variation of the surface, and highlight the interaction
between columns and rows. Motivated by the definition of SVD
components (cf. Section 3), we introduce the SVD curve
movie which can show the time varying features of the functions. One
motivating example is an Internet traffic data set, which is
discussed in Section 2 to illustrate the usefulness
of the surface plot, the rotation movie and the curve movie. Two
other real applications are reported in Section
6 as well. One chemometrics data set is used to show that a zoomed
version of the surface plot and the SVD movies highlight local
behaviors. A Spanish mortality data set is analyzed to illustrate
that the image plot highlights the cohort effect (i.e., the
interaction between age groups and years).
The rest of the paper is organized as follows. The motivating
example in network traffic analysis is in Section 2.
Section 3 gives a brief introduction of SVD, and
compares it with PCA. Section 4 describes in detail
about the generation of the plots. Section 5 shows
several toy examples to illustrate the four types of recenterings.
Two more real applications in chemometrics and demography are
reported in Section 6.
Discussions
In this paper, we propose four types of recenterings for SVD, and
describe a general scree plot to help decide which recentering is
appropriate in practice. Various SVD visualization tools, such as
the SVD surface plot, the image plot, the SVD curve movie and the
SVD rotation movie, are proposed to find insightful features in a
two way data matrix, especially a functional data set. These
visualization methods simultaneously show the rows, columns, local
variations, and interactions, which help to find underlying
features. Furthermore, when the data set has time series structure
in one direction, our method shows the time varying features
clearly.
With noisy data, we notice that the SVD components contain a lot of
noise. If one prefers, one can presmooth the data matrix using his
favorite smoother, and then apply our programs to gain some
insights, and then apply our program to get additional insights. We
are working on incorporating some regularized SVD method
[Huang et al., 2005] into our program, so that the
exploration of noisy data will be easier.
Our notion of the twoway data matrix requires that the data are
observed on common grid points. In cases with irregularly observed
data points or missing data, one could preprocess the data using
some smoothing or interpolation methods to obtain regular observed
data matrices, before applying our visualization methods.
Acknowledgement
The authors gratefully acknowledge Dr. Andrés M. Alonso
Fernández of Departamento de Estadística in Universidad Carlos
III de Madrid, for providing the Spanish mortality data to us.
Thanks also go to two referees and an Associate Editor for their
helpful comments and constructive suggestions.
References
 [Gabriel 1971]

Gabriel, K. R. (1971), "The biplot graphic display of matrices with
application to principal component analysis," Biometrika, 58,
453467.
 [Huang et al. 2005]

Huang, J. Z., Shen, H., and Buja, A. (2005), Analysis of TwoWay
Functional Data Using Regularized Singular Value Decomposition, working
paper.
 [Marron et al. 2004]

Marron, J. S., Wendelberger, J. R., and Kober, E. M. (2004), Time
Series Functional Data Analysis, Los Alamos National Lab, no. LAUR043911.
 [Ramsay and Silverman 1997]

Ramsay, J. O. and Silverman, B. W. (1997), Functional Data Analysis,
SpringerVerlag.
 [Ramsay and Silverman 2002]

 (2002), Applied Functional Data Analysis, Methods and Case
Studies, SpringerVerlag.
 [Shen and Huang 2005]

Shen, H. and Huang, J. Z. (2005), "Analysis of Call Center Arrival Data
Using Singular Value Decomposition," Applied Stochastic Models in
Business and Industry, 21, 251263.
Footnotes:
^{1}Correspondence to Lingsong Zhang,
Department of Statistics and Operations Research, University of
North Carolina, Chapel Hill, NC, 275993260. Email:
lszhang@email.unc.edu
File translated from
T_{E}X
by
T_{T}H,
version 3.67.
On 25 Apr 2006, 02:30.