Principal Component Analysis
Principal component analysis is a statistical technique for reducing the dimensionality of a data set while preserving as much of its variation as possible. It transforms a set of possibly correlated variables into a set of uncorrelated variables, called principal components, which are ordered so that the first few retain most of the variation in the original data.
Method
The analysis computes the eigenvectors and eigenvalues of the covariance or correlation matrix of the data. The eigenvectors define the directions of maximum variance, and the eigenvalues indicate the amount of variance explained by each. The first principal component is the linear combination of the original variables that has the greatest variance, subject to a normalisation constraint.
By retaining only the first few components, analysts reduce the number of variables while preserving the dominant patterns in the data. The components are uncorrelated, which simplifies subsequent analysis, and they can be interpreted in terms of the underlying structure, though the interpretation is not always straightforward.
Applications in Economics
Principal component analysis is widely used in economics and finance. In finance, it is used to analyse the term structure of interest rates, where the first three components are typically interpreted as level, slope, and curvature. It is also used to construct factors from large sets of asset returns and to reduce the dimensionality of risk models.
In macroeconomics, the technique is used to extract common factors from large data sets, as in factor-augmented models and nowcasting, where many indicators are summarised by a few components. It is also used in the analysis of survey data and in the construction of composite indicators.
Factor Models and Forecasting
Principal component analysis underlies factor models, in which the covariation among many variables is attributed to a small number of common factors. These models are used to forecast, to identify structural shocks, and to analyse the transmission of disturbances across variables.
The approach is related to dynamic factor models, which allow the factors to evolve over time and are used in central banks for nowcasting and for the analysis of business cycle dynamics. The methods are computationally intensive but have become standard with the availability of large data sets and computing power.
Limitations
Principal component analysis assumes linear relationships among variables and identifies components by variance rather than by economic meaning. The components may be difficult to interpret, and the method is sensitive to the scaling of variables, which determines the relative importance of each in the components.
The technique identifies correlation but not causation, and the components need not correspond to economically meaningful factors. It is therefore often used alongside economic theory, which guides the interpretation of the components and the specification of the models in which they are used.
Mathematics of the Method
Principal component analysis proceeds by computing the covariance or correlation matrix of the data and finding its eigenvalues and eigenvectors. The eigenvectors are the principal components, and the eigenvalues indicate the variance explained by each. The components are orthogonal, so they are uncorrelated with one another.
The choice of the covariance or correlation matrix matters. The covariance matrix retains the original units and is sensitive to the scaling of variables, while the correlation matrix standardises the variables and treats them equally. The choice affects the results and should reflect the objectives of the analysis.
Dimensionality Reduction
The principal use of the method is dimensionality reduction. By retaining the first few components, analysts summarise a large set of variables with a small number of factors, preserving the dominant patterns while discarding noise. The number of components to retain is chosen by criteria such as the proportion of variance explained or the size of the eigenvalues.
The reduction is useful for visualisation, for forecasting, and for the construction of composite indicators. It also facilitates the analysis of high-dimensional data where the number of variables is large relative to the number of observations.
Applications in Finance
In finance, principal component analysis is used to model the term structure of interest rates. The first three components are typically interpreted as the level, the slope, and the curvature of the yield curve, and they explain most of the variation in yields. This representation is used in risk management and in the pricing of interest rate derivatives.
The method is also used to construct factors from large sets of asset returns, to reduce the dimensionality of risk models, and to analyse the comovement of international markets. Its application has grown with the availability of high-frequency data and computing power.
Applications in Macroeconomics
In macroeconomics, the method is used to extract common factors from large data sets, as in factor-augmented vector autoregressions and in nowcasting, where many indicators are summarised by a few components to estimate current conditions. Dynamic factor models, which allow the factors to evolve over time, are used by central banks to analyse the business cycle.
The approach addresses the problem of information overload, in which the number of potentially relevant indicators is large relative to the length of the sample. By reducing the data to a few factors, it makes the analysis tractable while retaining the information most relevant to the question at hand. The interpretation of the factors, however, requires economic theory and judgment, since the method itself is agnostic about meaning.