Eigenvector University returns to Seattle, USA May 12-16, 2025 Complete Info Here!

On Turning 30

Jan 6, 2025

Eigenvector Research was founded on January 1, 1995, which means that we just turned 30. When I mentioned writing a piece for the occasion of our 30th anniversary, our Donal O’Sullivan replied “I don’t know if you want people to know we’re that old!” And I understand where he’s coming from. In the software business, especially in data science, maybe you don’t want to advertise that you’ve been around for 3+ decades. Perhaps better to look like the bright shiny new thing. 

But I think that 30 years of experience counts for something. We’ve seen quite a few shiny new things get misused and abused, mostly by people that don’t appreciate the basics that you just can’t get around. Things like if your model is purely based on data (rather than physics and chemistry) you can’t expect it to work outside the range and subspace of the calibration data. And the more effects you have contributing to the variance in your data, the smaller the unique part of the signal you are looking for will be, until it disappears all together. And if you are fitting your data better than your ability to predict it you’re fitting the noise or clutter. And, and….

The downside of turning 30 in the software business is what you might call technical debt, except most definitions of technical debt focus on the cost of implementing short term work-arounds instead of long term solutions. But if you’ve been in the business as long as we have, you know that, in spite of the advantages they initially offered, most software frameworks are eventually abandoned. (Remember ActiveX?) Learning to utilize new technologies to add new methods and features to software is fun, but replacing old architecture is hard work. Over the last couple years we’ve put a lot of effort into updating the infrastructure that supports our users behind the scenes. In the coming year we’ll be focused on updating some of the technologies behind our end user software PLS_Toolbox, Solo and their variants while still improving the existing methods. It is going to be challenging!

Looking back, 2024 was a great year. It was definitely the “year of training.” Between our in person classes (Eigenvector University in Seattle and Rome) our online classes (both open and for specific companies) and our recorded courses we reached more students than ever. And we also had our biggest year of software sales. Associations with our instrument company partners and software resellers was a big part of that. We were also happy to help many new users transition from other software packages. And of course our students, users and consulting clients all benefited from that 30 years experience.

Our biggest software development of 2024 was the release of Diviner, our semi-automated Machine Learning tool for accelerating the development of calibration models. Diviner automates the construction of regression models and keeps the analyst in the loop so that they can learn from the process. We have lots of plans for improving and expanding the use of Diviner: even as useful as it is now, there is still much to do!

Example output from Diviner. Each point represents a model with different preprocessing, variable selections and meta-parameters. The best models are the ones that have the best predictive ability but are not overfit.

When young people turn 30 they are often a bit depressed to be leaving their 20’s behind. (A search on “turning 30” reveals lots of this, along with a lot of stuff that is really, really not useful!) But generally that feeling is soon replaced with the realization that they’ve entered a very productive period of their life, where they can make real progress on careers and relationships, both business and personal. As a company we feel the same way. Here’s to life beyond 30!

BMW