Search Results for: ridge regression

cross-validation, ridge regression, the LASSO, Uncategorized

Estimation and Variable Selection with Ridge Regression and the LASSO

2 Comments

I’ve posted on ridge regression and the LASSO (Least Absolute Shrinkage and Selection Operator) some weeks back. Here I want to compare them in connection with variable selection  where there are more predictors than observations (“many predictors”). 1. Ridge regression does not really select variables in the many predictors situation. Rather, ridge regression “shrinks” all … Continue reading Estimation and Variable Selection with Ridge Regression and the LASSO →

Big Data, data mining, data science, kernel ridge regression

Kernel Ridge Regression – A Toy Example

Kernel ridge regression (KRR) is a promising technique in forecasting and other applications, when there are “fat” databases. It’s intrinsically “Big Data” and can accommodate nonlinearity, in addition to many predictors. Kernel ridge regression, however, is shrouded in mathematical complexity. While this is certainly not window-dressing, it can obscure the fact that the method is … Continue reading Kernel Ridge Regression – A Toy Example →

Big Data, kernel ridge regression, kernel trick, predictive analytics, ridge regression

The Problem of Many Predictors – Ridge Regression and Kernel Ridge Regression

2 Comments

You might imagine that there is an iron law of ordinary least squares (OLS) regression – the number of observations on the dependent (target) variable and associated explanatory variables must be less than the number of explanatory variables (regressors). Ridge regression is one way to circumvent this requirement, and to estimate, say, the value of … Continue reading The Problem of Many Predictors – Ridge Regression and Kernel Ridge Regression →

bagging, Big Data, complete subset regression, data science, financial forecasts, variable selection

Complete Subset Regressions

A couple of years or so ago, I analyzed a software customer satisfaction survey, focusing on larger corporate users. I had firmagraphics – specifying customer features (size, market segment) – and customer evaluation of product features and support, as well as technical training. Altogether, there were 200 questions that translated into metrics or variables, along … Continue reading Complete Subset Regressions →

accuracy of forecasts, new proximity model, predictive analytics, random walk, stock market forecasts

Thoughts on Stock Market Forecasting

Here is an update on the forecasts from last Monday – forecasts of the high and low of SPY, QQQ, GE, and MSFT. This table is easy to read, even though it is a little” busy”. One key is to look at the numbers highlighted in red and blue (click to enlarge). These are the … Continue reading Thoughts on Stock Market Forecasting →

Big Data, complete subset regression, kernel ridge regression, many predictors, three pass regression filter (3PRF), variable selection

Primer on Method – Some Perspectives from 2014

As Hal Varian writes in his popular Big Data: New Tricks for Econometrics the wealth of data now available to researchers demands new techniques of analysis. In particular, often there is the problem of “many predictors.” In classic regression, the number of observations is assumed to exceed the number of explanatory variables. This obviously is … Continue reading Primer on Method – Some Perspectives from 2014 →

macroeconomic forecasting, mixed data frequencies, nowcasting, predicting recessions, predicting turning points

Forecasting the Downswing in Markets

I got a chance to work with the problem of forecasting during a business downturn at Microsoft 2007-2010. Usually, a recession is not good for a forecasting team. There is a tendency to shoot the messenger bearing the bad news. Cost cutting often falls on marketing first, which often is where forecasting is housed. But … Continue reading Forecasting the Downswing in Markets →

Big Data, gasoline price forecasts, geopolitical risk, predictive analytics

When the Going Gets Tough, the Tough Get Going

Great phrase, but what does it mean? Well, maybe it has something to do with the fact that a lot of economic and political news seem to be entering kind of “end game.” But, it’s now the “lazy days of summer,” and there is a temptation to sit back and just watch it whiz by. … Continue reading When the Going Gets Tough, the Tough Get Going →

accuracy of forecasts, bagging, Big Data, boosting, sales forecasting

Business Forecasting – Some Thoughts About Scope

In many business applications, forecasting is not a hugely complex business. For a sales forecasting, the main challenge can be obtaining the data, which may require sifting through databases compiled before and after mergers or other reorganizations. Often, available historical data goes back only three or four years, before which time product cycles make comparisons … Continue reading Business Forecasting – Some Thoughts About Scope →

accuracy of forecasts, Bayesian methods, bias in forecasts, regression diagnostics, ridge regression, the LASSO

Some Ways in Which Bayesian Methods Differ From the “Frequentist” Approach

I’ve been doing a deep dive into Bayesian materials, the past few days. I’ve tried this before, but I seem to be making more headway this time. One question is whether Bayesian methods and statistics informed by the more familiar frequency interpretation of probability can give different answers. I found this question on CrossValidated, too … Continue reading Some Ways in Which Bayesian Methods Differ From the “Frequentist” Approach →