Start Submission Become a Reviewer

Reading: Nonlinear principal component analysis by neural networks

Download

A- A+
Alt. Display

Original Research Papers

Nonlinear principal component analysis by neural networks

Author:

William W. Hsieh

Oceanography/EOS, University of British Columbia, Vancouver, BC, V6T1Z4, CA
X close

Abstract

Nonlinear principal component analysis (NLPCA) can be performed by a neural network model which nonlinearly generalizes the classical principal component analysis (PCA) method. The presence of local minima in the cost function renders the NLPCA somewhat unstable, as optimizations started from different initial parameters often converge to different minima. Regularization by adding weight penalty terms to the cost function is shown to improve the stability of the NLPCA. With the linear approach, there is a dichotomy between PCA and rotated PCA methods, as it is generally impossible to have a solution simultaneously(a) explaining maximum global variance of the data, and (b) approaching local data clusters. With the NLPCA, both objectives (a) and (b) can be attained together, thus the nonlinearity in NLPCA unifies the PCA and rotated PCA approaches. With a circular node at the network bottleneck, the NLPCA is able to extract periodic or wave modes. The Lorenz (1963) 3-component chaotic system and the monthly tropical Pacific sea surface temperatures (1950-1999) are used to illustrated the NLPCA approach.

How to Cite: Hsieh, W.W., 2001. Nonlinear principal component analysis by neural networks. Tellus A: Dynamic Meteorology and Oceanography, 53(5), pp.599–615. DOI: http://doi.org/10.3402/tellusa.v53i5.12230
  Published on 01 Jan 2001
 Accepted on 5 Mar 2001            Submitted on 27 Sep 2000

References

  1. Barnston , A. G. and Livezey , R. E . 1987 . Classification, seasonality and persistence of low-frequency atmo-spheric circulation patterns. Mon. Wea. Re v . 115 , 1083 – 1126 .  

  2. Cybenko , G . 1989 . Approximation by superpositions of a sigmoidal function . Mathematics of Control, Signals, and Systems 2 , 303 – 314 .  

  3. Diamantaras , K. I. and Kung , S. Y . 1996 . Principal com-ponent neural networks . New York : Wiley .  

  4. Hoerling , M. P. , Kumar , A. and Zhong , M. 1997 . El Nirio, La Nina and the nonlinearity of their teleconnections . J. Climate 10 , 1769-178 6 .  

  5. Hsieh , W. W . 2000 . Nonlinear canonical correlation ana-lysis by neural networks . Neural Networks 13 , 1095 – 1105 .  

  6. Hsieh , W. W . 2001 . Nonlinear canonical correlation ana-lysis of the tropical Pacific climate variability using a neural network approach . J. Climate , in press .  

  7. Hsieh , W. W. and Tang , B . 1998 . Applying neural net-work models to prediction and data analysis in met-eorology and oceanography . Bull. Amer. Meteor. Soc . 79 , 1855 – 1870 .  

  8. Jollife , I. T . 1986 . Principal component analysis . New York : Springer .  

  9. Kaiser , H. F . 1958 . The varimax criterion for analytic rotation in factor analysis . Psychometrika 23 , 187 – 200 .  

  10. Kirby , M. J. and Miranda , R . 1996 . Circular nodes in neural networks . Neural Comp . 8 , 390 – 402 .  

  11. Kramer , M. A . 1991 . Nonlinear principal component analysis using autoassociative neural networks . AlChE Journal 37 , 233 – 243 .  

  12. Lorenz , E. N . 1963 . Deterministic nonperiodic flow. J. Atmos. Sc i . 20 , 130 – 141 .  

  13. Monahan , A. H . 2000 . Nonlinear principal component analysis by neural networks: theory and application to the Lorenz system . J. Climate 13 , 821 – 835 .  

  14. Monahan , A. H . 2001 . Nonlinear principal component analysis: tropical Indo-Pacific sea surface temperature and sea level pressure . J. Climate 14 , 219 – 233 .  

  15. Oja , E . 1982 . A simplified neuron model as a principal component analyzer . J. Math. Biology 15 , 267 – 273 .  

  16. Preisendorfer , R. W . 1988 . Principal component analysis in meteorology and oceanography . New York : Elsevier .  

  17. Reynolds , R. W. and Smith , T. M . 1994 . Improved global sea surface temperature analyses using optimum inter-polation . J Climate 7 , 929 – 948 .  

  18. Richman , M. B . 1986 . Rotation of principal components . J. Climatology 6 , 293 – 335 .  

  19. Rumelhart , D. E. , Hinton , G. E. and Williams , R. J . 1986 . Learning internal representations by error propaga-tion . In D. E. Rumelhart , J. L. McClelland , and P. R. Group (Eds.) , Parallel distributed processing ( vol. 1 , pp. 318 – 362 ). Cambridge, MA : MIT Press .  

  20. Smith , T. M. , Reynolds , R. W. , Livezey , R. E. and Stokes , D. C . 1996 . Reconstruction of historical sea surface temperatures using empirical orthogonal functions . J. Climate 9 , 1403 – 1420 .  

  21. Von Storch , H. and Zwiers , F. W . 1999 . Statistical ana-lysis in climate research . Cambridge : Cambridge Univ. Pr .  

comments powered by Disqus