NIPALS versus Lanczos Bidiagonalization

In 2007, Randy Pell, Scott Ramos and Rolf Manne (PRM) ignited a controversy when they published “The model space in PLS regression.” Their paper pointed out that the X-block residuals in different PLS packages were not the same. Specifically, packages which use the NIPALS or SIMPLS method for PLS (including PLS_Toolbox/Solo, Unscrambler and SIMCA-P) produce different residuals than those that use Lanczos Bidiagonalization (primarily Pirouette). PRM claimed that that residuals in NIPALS were “inconsistent” and made the rather inflammatory statement that NIPALS “amounted to giving up mathematics.”

As you might imagine, this has resulted in a considerable amount of activity in the chemometrics community. And it really has been useful because many of us, including myself, have learned quite a bit about PLS, a subject we thought we already understood pretty well.

There will be a crop of articles in the upcoming issue of Journal of Chemometrics on this subject. This will include a letter to the editor by Svante Wold et. al., “The PLS model space revisited,” which takes a theoretical/philosophical look at how PLS via NIPALS is derived and shows that, in this light, it is not inconsistent. Rasmus Bro and Lars Eldén’s contribution, “PLS Works,” shows that while the PLS NIPALS residual space is orthogonal to the model scores, and thus the fitted y-values, this is not true of Bidiag. I understand that there will also be a paper in the upcoming issue from Rolf Ergon, though I don’t know the title yet.

The work of Bro and Eldén served as a launching point for an investigation of my own regarding how and why Bidiag residuals are correlated with scores. The result is a poster which I will show at CAC-2008 next week, “Properties of PLS, and Differences between NIPALS and Lanczos Bidiagonalization.” The poster shows why and when NIPALS and Bidiag residuals are different, and shows some examples of when Bidiag residuals are strongly correlated with the scores. This includes the main example given in PRM, where, as it turns out, the main difference in the residuals is due to the 3rd factor in the Bidiag model being quite correlated with the residuals.

If you are attending CAC, please drop by and talk to me during the poster presentation. I’m sure we’ll have a lively discussion!

BMW

References:
R. J. Pell, L. S. Ramos and R. Manne, “The model space in PLS regression,” J.Chemometrics, Vol. 21, pps 165-172, 2007. 
R. Bro and L. Eldén, “PLS Works,” J. Chemometrics, in press, 2008.
S. Wold, M. Høy, H. Martens, J. Trygg, F. Westad, J. MacGregor and B.M. Wise, “The PLS model space revisited, J. Chemometrics, in press, 2008.
B.M. Wise, “Properties of PLS, and Differences between NIPALS and Lanczos Bidiagonalization,” CAC-2008, Montpellier, France, 2008.