# Different kinds of PLS weights, loadings, and what to look at?

##### Sep 12, 2009

The age of Partial Least Squares (PLS) regression (as opposed to PLS path modeling) began with the SIAM publication of Svante Wold *et. al.* in 1984 [1]. Many of us learned PLS regression from the rather more accessible paper by Geladi and Kowalski from 1986 [2] which described the Nonlinear Iterative PArtial Least Squares (NIPALS) algorithm in detail.

For univariate **y**, the NIPALS algorithm is really more sequential than iterative. There is a specific sequence of fixed steps that are followed to find the weight vector **w** (generally normalized to unit length) for each PLS factor or Latent Variable (LV). Only in the case of multivariate **Y** is the algorithm really iterative, a sequence of steps is repeated until the solution for **w** for each LV converges.

Whether for univariate **y** or multivariate **Y** the calculations for each LV end with a *deflation* step. The **X** data is projected onto the weight vector **w** to get a score vector, **t** (**t** = **Xw**). **X** is then projected onto the score **t** to get a loading, **p**, (**p** = **X’t**/**t’t**). Finally, **X** is deflated by **tp’** to form a new **X**, **X**_{new} with which to start the procedure again, **X**_{new} = **X** – **tp’**. A new weight vector **w** is calculated from the deflated **X**_{new} and the calculations continue.

So the somewhat odd thing about the weight vectors **w** derived from NIPALS is that each one applies to a different **X**, *i.e.* **t**_{n+1} = **X**_{n}**w**_{n+1}. This is in contrast to Sijmen de Jong’s SIMPLS algorithm introduced in 1993 [3]. In SIMPLS a set of weights, sometimes referred to as **R**, is calculated, which operate on the original **X** data to calculate the scores. Thus, all the scores **T** can be calculated directly from **X** without deflation, **T** = **XR**. de Jong showed that it is easy to calculate the SIMPLS **R** from the NIPALS **W** and **P**, **R** = **W**(**P’W**)^{-1}. (Unfortunately, I have, as yet, been unable to come up with a simple expression for calculating the NIPALS **W** from the SIMPLS model parameters.)

So the question here is, “If you want to look at weights, which weights should you look at, **W** or **R**?” I’d argue that **R** is somewhat more intuitive as it applies to the original **X** data. Beyond that, if you are trying to standardize outputs of different software routines (which is actually how I got started on all this), it is a simple matter to always provide **R**. Fortunately, **R** and **W** are typically not that different, and in fact, they start out the same, **w**_{1} = **r**_{1}, and they span the same subspace. Our decision here, based on relevance and standardization, is to present **R** weights as the default in future versions of PLS_Toolbox and Solo, regardless of which PLS algorithm is selected (NIPALS, SIMPLS or the new Direct Scores PLS).

A better question might be, “When investigating a PLS model, should I look at weights, **R** or **W**, or loadings **P**?” If your perspective is that the scores **T** are measures of some underlying “latent” phenomena, then you would choose to look at **P**, the degree to which these latent variables contribute to **X**. The weights **W** or **R** are merely regression coefficients that you use to estimate the scores **T**. From this viewpoint the real model is **X** = **TP’** + **E** and **y** = **Tb**+ **f**.

If, on the other hand, you see PLS as simply a method for identifying a subspace within which to restrict, and therefore stabilize, the regression vector, then you would choose to look at the weights **W** or **R**. From this viewpoint the real model is **Y** = **Xb** + **e**, with **b** = **W**(**P’W**)^{-1}(**T’T**)^{-1}**T’y** = **R**(**T’T**)^{-1}**T’y** via the NIPALS and SIMPLS formulations respectively. The regression vector is a linear combination of the weights **W** or **R** and from this perspective the loadings **P** are really a red herring. In the NIPALS formulation they are a patch left over from the way the weights were derived from deflated **X**. And the loadings **P** aren’t even in the SIMPLS formulation.

So the bottom line here is that you’d look at loadings **P** if you are a fan of the latent variable perspective. If you’re a fan of the regression subspace perspective, then you’d look at weights, **W** or preferably **R**. I’m in the former camp, (for more reasons than just philosophical agreement with the LV model), as evidenced by my participation in S. Wold *et. al.*, “The PLS model space revisited,” [4]. Your choice of perspective also impacts what residuals to monitor, etc., but I’ll save that for a later time.

BMW

[1] S. Wold, A. Ruhe, H. Wold, and W.J. Dunn III, “The Collinearity Problem in Linear

Regression. The Partial Least Square Approach to Generalized Inverses”, SIAM J. Sci.

Stat. Comput., Vol. 5, 735-743, 1984.

[2] P. Geladi and B.R. Kowalski, “PLS Tutorial,” Anal. Chim. Acta., 185(1), 1986.

[3] S. de Jong, “SIMPLS: an alternative approach to partial least squares regression,” Chemo. and Intell. Lab. Sys., Vol. 18, 251-263, 1993.

[4] S. Wold, M. Høy, H. Martens, J. Trygg, F. Westad, J. MacGregor and B.M. Wise, “The PLS model space revisited,” J. Chemometrics, pps 67-68, Vol. 23, No. 2, 2009.