Beyond the PCA : a comprehensive review of dimension reduction techniques

Search for this publication on Google Scholar
Kenjale, T. (2022). Beyond the PCA : a comprehensive review of dimension reduction techniques. Unc Charlotte Undergraduate Research Conference.
Download PDF

Analytics

37 views ◎
11 downloads ⇓


Description

Regression analysis is a key tool in modeling the relationship between predictor variables and the conditional mean of the response variable. However, as the number of predictors increases, existing estimation methods become challenging. Dimension reduction techniques are utilized to reduce the number of predictors while maintaining the same amount of information. Therefore, the purpose of this work is to provide a thorough analysis of several dimension reduction techniques and provide recommendations regarding the decision-making process. This project considers two categories of techniques: unsupervised and supervised. Unsupervised techniques focus on finding a low-dimensional predictor that maintains the maximum possible variability of the original data. In contrast, supervised techniques focus on finding a low-dimensional predictor that contains all the information necessary to describe the conditional distribution. For this work, we consider the principal component analysis (PCA), the sliced inverse regression (SIR), and the sliced average variance estimation (SAVE). We also consider the kernel PCA (KPCA) and the kernel SIR (KSIR), which extract nonlinear features. The performances of the various techniques are demonstrated through extensive simulations and real data applications. The findings show that (a) supervised techniques typically perform better than unsupervised techniques in regression settings; (b) nonlinear techniques can perform better than linear techniques but have higher computation times; (c) PCA can perform well and has the shortest computation time, although it may not always be adequate in regression; (d) SAVE has strict assumptions that results in lower versatility; and (e) SIR provides the best combination of performance and computation time.

Details
Title

Beyond the PCA : a comprehensive review of dimension reduction techniques

Author(s)

Kenjale, Tanmay

UNC Charlotte Affiliate Type

Undergraduate

Extent

1 online resource

Date Created

2022

Material Type

Conference poster

Genre

conference proceedings

Resource Type

text

Abstract

Regression analysis is a key tool in modeling the relationship between predictor variables and the conditional mean of the response variable. However, as the number of predictors increases, existing estimation methods become challenging. Dimension reduction techniques are utilized to reduce the number of predictors while maintaining the same amount of information. Therefore, the purpose of this work is to provide a thorough analysis of several dimension reduction techniques and provide recommendations regarding the decision-making process. This project considers two categories of techniques: unsupervised and supervised. Unsupervised techniques focus on finding a low-dimensional predictor that maintains the maximum possible variability of the original data. In contrast, supervised techniques focus on finding a low-dimensional predictor that contains all the information necessary to describe the conditional distribution. For this work, we consider the principal component analysis (PCA), the sliced inverse regression (SIR), and the sliced average variance estimation (SAVE). We also consider the kernel PCA (KPCA) and the kernel SIR (KSIR), which extract nonlinear features. The performances of the various techniques are demonstrated through extensive simulations and real data applications. The findings show that (a) supervised techniques typically perform better than unsupervised techniques in regression settings; (b) nonlinear techniques can perform better than linear techniques but have higher computation times; (c) PCA can perform well and has the shortest computation time, although it may not always be adequate in regression; (d) SAVE has strict assumptions that results in lower versatility; and (e) SIR provides the best combination of performance and computation time.

Subjects--Topics

Statistics
Dimension reduction (Statistics)
Regression analysis

Is Part Of

UNC Charlotte undergraduate research conference

Language

English

Permalink

http://hdl.handle.net/20.500.13093/work:284

Rights

This item may be protected by copyright and other related rights. Atkins Library provides access to this item for educational and research purposes only; other uses require the permission of the copyright holder.