Tag Archives: predictive analytics

Deep Questions, predictive analytics, probability theory

The Arc Sine Law and Competitions

There is a topic I think you can call the “structure of randomness.” Power laws are included, as are various “arcsine laws” governing the probability of leads and changes in scores in competitive games and, of course, in winnings from gambling.

I ran onto a recent article showing how basketball scores follow arcsine laws.

Safe Leads and Lead Changes in Competitive Team Sports is based on comprehensive data from league games over several seasons in the National Basketball Association (NBA).

“..we find that many …statistical properties are explained by modeling the evolution of the lead time X as a simple random walk. More strikingly, seemingly unrelated properties of lead statistics, specifically, the distribution of the times t: (i) for which one team is leading..(ii) for the last lead change..(and (iii) when the maximal lead occurs, are all described by the ..celebrated arcsine law..”

The chart below shows the arcsine probability distribution function (PDF). This probability curve is almost the opposite or reverse of the widely known normal probability distribution. Instead of a bell-shape with a maximum probability in the middle, the arcsine distribution has the unusual property that probabilities are greatest at the lower and upper bounds of the range. Of course, what makes both curves probability distributions is that the area they span adds up to 1.

arcsine

So, apparently, the distribution of time that a basketball team holds a lead in a basketball game is well-described by the arcsine distribution. This means lead changes are most likely at the beginning and end of the game, and least likely in the middle.

An earlier piece in the Financial Analysts Journal (The Arc Sine Law and the Treasure Bill Futures Market) notes,

..when two sports teams play, even though they have equal ability, the arc sine law dictates that one team will probably be in the lead most of the game. But the law also says that games with a close final score are surprisingly likely to be “last minute, come from behind” affairs, in which the ultimate winner trailed for most of the game..[Thus] over a series of games in which close final scores are common, one team could easily achieve a string of several last minute victories. The coach of such a team might be credited with being brilliantly talented, for having created a “second half” team..[although] there is a good possibility that he owes his success to chance.

There is nice mathematics underlying all this.

The name “arc sine distribution” derives from the integration of the PDF in the chart – a PDF which has the formula –

f(x) = 1/(π (x(1-x).5)

Here, the integral of f(x) yields the cumulative distribution function F(x) and involves an arcsine function,

F(x) = 2/(π arcsin(x.5))

Fundamentally, the arcsine law relates to processes where there are probabilities of winning and losing in sequential trials. The PDF follows from the application of Stirling’s formula to estimate expressions with factorials, such as the combination of p+q things taken p at a time, which quickly becomes computationally cumbersome as p+q increases in size.

There is probably no better introduction to the relevant mathematics than Feller’s exposition in his classic An Introduction to Probability Theory and Its Applications, Volume I.

Feller had an unusual ability to write lucidly about mathematics. His Chapter III “Fluctuations in Coin Tossing and Random Walks” in IPTAIA is remarkable, as I have again convinced myself by returning to study it again.

Feller

He starts out this Chapter III with comments:

We shall encounter theoretical conclusions which not only are unexpected but actually come as a shock to intuition and common sense. They will reveal that commonly accepted motions concerning chance fluctuations are without foundation and that the implications of the law of large numbers are widely misconstrued. For example, in various applications it is assumed that observations on an individual coin-tossing game during a long time interval will yield the same statistical characteristics as the observation of the results of a huge number of independent games at one given instant. This is not so..

Most pointedly, for example, “contrary to popular opinion, it is quite likely that in a long coin-tossing game one of the players remains practically the whole time on the winning side, the other on the losing side.”

The same underlying mathematics produces the Ballot Theorem, which states the chances a candidate will be ahead from an early point in vote counting, based on the final number of votes for that candidate.

This application, of course, comes very much to the fore in TV coverage of the results of on-going primaries at the present time. CNN’s initial announcement, for example, that Bernie Sanders beat Hillary Clinton in the New Hampshire primary came when less than half the precincts had reported in their vote totals.

In returning to Feller’s Volume 1, I recommend something like Sholmo Sternberg’s Lecture 8. If you read Feller, you have to be prepared to make little derivations to see the links between formulas. Sternberg cleared up some puzzles for me, which, alas, otherwise might have absorbed hours of my time.

The arc sine law may be significant for social and economic inequality, which perhaps can be considered in another post.

accuracy of forecasts, forecasting research, M-competition

Business Forecasting – Practical Problems and Solutions

Forecasts in business are unavoidable, since decisions must be made for annual budgets and shorter term operational plans, and investments must be made.

And regardless of approach, practical problems arise.

For example, should output from formal algorithms be massaged, so final numbers include judgmental revisions? What about error metrics? Is the mean absolute percent error (MAPE) best, because everybody is familiar with percents? What are plus’es and minus’es of various forecast error metrics? And, organizationally, where should forecasting teams sit – marketing, production, finance, or maybe in a free-standing unit?

The editors of Business Forecasting – Practical Problems and Solutions integrate dozens of selections to focus on these and other practical forecasting questions.

Here are some highlights.

In my experience, many corporate managers, even VP’s and executives, understand surprisingly little about fitting models to data.

So guidelines for reporting results are important.

In “Dos and Don’ts of Forecast Accuracy Measurement: A Tutorial,” Len Tashman advises “distinguish in-sample from out-of-sample accuracy,” calling it “the most basic issue.”

The acid test is how well the forecast model does “out-of-sample.” Holdout samples and cross-validation simulate how the forecast model will perform going forward. “If your average error in-sample is found to be 10%, it is very probable that forecast errors will average substantially more than 10%.” That’s because model parameters are calibrated to the sample over which they are estimated. There is a whole discussion of “over-fitting,” R2, and model complexity hinging on similar issues. Don’t fool yourself. Try to find ways to test your forecast model on out-of-sample data.

The discussion of fitting models when there is “extreme seasonality” broke new ground for me. In retail forecasting, there might be a toy or product that sells only at Christmastime. Demand is highly intermittent. As Udo Sglavo reveals, one solution is “time compression.” Collapse the time series data into two periods – the holiday season and the rest of the year. Then, the on-off characteristics of sales can be more adequately modeled. Clever.

John Mello’s “The Impact of Sales Forecast Game Playing on Supply Chains,” is probably destined to be a kind of classic, since it rolls up a lot of what we have all heard and observed about strategic behavior vis a vis forecasts.

Mello describes stratagems including

  • Enforcing – maintaining a higher forecast than actually anticipated, to keep forecasts in line with goals
  • Filtering – changing forecasts to reflect product on hand for sale
  • Hedging – overestimating sales to garner more product or production capability
  • Sandbagging – underestimating sales to set expectations lower than actually anticipated demand
  • Second-guessing – changing forecasts to reflect instinct or intuition
  • Spinning – manipulating forecasts to get favorable reactions from individuals or departments in the organization
  • Withholding – refusing to share current sales information

I’ve seen “sand-bagging” at work, when the salesforce is allowed to generate the forecasts, setting expectations for future sales lower than should, objectively, be the case. Purely by coincidence, of course, sales quotas are then easier to meet and bonuses easier to achieve.

I’ve always wondered why Gonik’s system, mentioned in an accompanying article by Michael Gilliland on the “Role of the Sales Force in Forecasting,” is not deployed more often. Gonik, in a classic article in the Harvard Business Review, ties sales bonuses jointly to the level of sales that are forecast by the field, and also to how well actual sales match the forecasts that were made. It literally provides incentives for field sales staff to come up with their best, objective estimate of sales in the coming period. (See Sales Forecasts and Incentives)

Finally, Larry Lapide’s “Where Should the Forecasting Function Reside?” asks a really good question.

The following graphic (apologies for the scan reproduction) summarizes some of his key points.

TTable

There is no fixed answer, Lapide provides a list of things to consider for each organization.

This book is a good accompaniment for Rob Hyndman and George Athanasopoulos’s online Forecasting: Principles and Practice.

applied macroeconomic forecasting methods, Big Data, macroeconomic forecasting

Is the Economy Moving Toward Recession?

Generally, a recession occurs when real, or inflation-adjusted Gross Domestic Product (GDP) shows negative growth for at least two consecutive quarters. But GDP estimates are available only at a lag, so it’s possible for a recession to be underway without confirmation from the national statistics.

Bottom line – go to the US Bureau of Economics Analysis website, click on the “National” tab, and you can get the latest official GDP estimates. Today, (January 25, 2016) this box announces “3rd Quarter 2015 GDP,” and we must wait until January 29th for “advance numbers” on the fourth quarter 2015 – numbers to be revised perhaps twice in two later monthly releases.

This means higher frequency data must be deployed for real-time information about GDP growth. And while there are many places with whole bunches of charts, what we really want is systematic analysis, or nowcasting.

A couple of initiatives at nowcasting US real GDP show that, as of December 2015, a recession is not underway, although the indications are growth is below trend and may be slowing.

This information comes from research departments of the US Federal Reserve Bank – the Chicago Fed National Activity Index (CFNAI) and the Federal Reserve Bank of Atlanta GDPNow model.

CFNAI

The Chicago Fed National Activity Index (CFNAI) for December 2015, released January 22nd, shows an improvement over November. The CFNAI moved –0.22 in December, up from –0.36 in November, and, in the big picture (see below) this number does not signal recession.

FREDCFNAI

The index is a weighted average of 85 existing monthly indicators of national economic activity from four general categories – production and income; employment, unemployment, and hours; personal consumption and housing; and sales, orders, and inventories.

It’s built – with Big Data techniques, incidentally- to have an average value of zero and a standard deviation of one.

Since economic activity trends up over time, generally, the zero for the CFNAI actually indicates growth above trend, while a negative index indicates growth below trend.

Recession levels are lower than the December 2015 number – probably starting around -0.7.

GDPNow Model

The GDPNow Model is developed at the Federal Reserve bank of Atlanta.

On January 20, the GDPNow site announced,

The GDPNow model forecast for real GDP growth (seasonally adjusted annual rate) in the fourth quarter of 2015 is 0.7 percent on January 20, up from 0.6 percent on January 15. The forecasts for fourth quarter real consumer spending growth and real residential investment growth each increased slightly after this morning’s Consumer Price Index release from the U.S. Bureau of Labor Statistics and the report on new residential construction from the U.S. Census Bureau.

The chart accompanying this accouncement shows a somewhat less sanguine possibility – namely that consensus estimates and the output of the GDPNow model have been on a downward trend if you look at things back to September 2015.

GDPNow

accuracy of forecasts, geopolitical risk, global business forecasts

Superforecasting – The Art and Science of Prediction

Philip Tetlock’s recent Superforecasting says, basically, some people do better at forecasting than others and, furthermore, networking higher performing forecasters, providing access to pooled data, can produce impressive results.

This is a change from Tetlock’s first study – Expert Political Judgment – which lasted about twenty years, concluding, famously, ‘the average expert was roughly as accurate as a dart-throwing chimpanzee.”

Tetlock’s recent research comes out of a tournament sponsored by the Intelligence Advanced Research Projects Activity (IARPA). This forecasting competition fits with the mission of IARPA, which is to improve assessments by the “intelligence community,” or IC. The IC is a generic label, according to Tetlock, for “the Central Intelligence Agency, the National Security Agency, the Defense Intelligence Agency, and thirteen other agencies.”

It is relevant that the IC is surmised (exact figures are classified) to have “a budget of more than $50 billion .. [and employ] one hundred thousand people.”

Thus, “Think how shocking it would be to the intelligence professionals who have spent their lives forecasting geopolical events – to be beaten by a few hundred ordinary people and some simple algorithms.”

Of course, Tetlock reports, this actually happened – “Thanks to IARPA, we now know a few hundred ordinary people and some simple math can not only compete with professionals supported by multibillion-dollar apparatus but also beat them.”

IARPA’s motivation, apparently, traces back to the “weapons of mass destruction (WMD)” uproar surrounding the Iraq war –

“After invading in 2003, the United States turned Iraq upside down looking for WMD’s but found nothing. It was one of the worst – arguable the worst – intelligence failure in modern history. The IC was humiliated. There were condemnations in the media, official investigations, and the familiar ritual of intelligence officials sitting in hearings ..”

So the IC needs improved methods, including utilizing “the wisdom of crowds” and practices of Tetlock’s “superforecaster” teams.

Unlike the famous M-competitions, the IARPA tournament collates subjective assessments of geopolitical risk, such as “will there be a fatal confrontation between vessels in the South China Sea” or “Will either the French or Swiss inquiries find elevated levels of polonium in the remains of Yasser Arafat’s body?”

Tetlock’s book is entertaining and thought-provoking, but many in business will page directly to the Appendix – Ten Commandments for Aspiring Superforecasters.

    1. Triage – focus on questions which are in the “Goldilocks” zone where effort pays off the most.
    2. Break seemingly intractable problems into tractable sub-problems. Tetlock really explicates this recommendation with his discussion of “Fermi-izing” questions such as “how many piano tuners there are in Chicago?.” The reference here, of course, is to Enrico Fermi, the nuclear physicist.
    3. Strike the right balance between inside and outside views. The outside view, as I understand it, is essentially “the big picture.” If you are trying to understand the likelihood of a terrorist attack, how many terrorist attacks have occurred in similar locations in the past ten years? Then, the inside view includes facts about this particular time and place that help adjust quantitative risk estimates.
    4. Strike the right balance between under- and overreacting to evidence. The problem with a precept like this is that turning it around makes it definitely false. Nobody would suggest “do not strike the right balance between under- and overreacting to evidence.” I guess keep the weight of evidence in mind.
    5. Look for clashing causal forces at work in each problem. This reminds me of one of my models of predicting real world developments – tracing out “threads” or causal pathways. When several “threads” or chains of events and developments converge, possibility can develop into likelihood. You have to be a “fox” (rather than a hedgehog) to do this effectively – being open to diverse perspectives on what drives people and how things happen.
    6. Strive to distinguish as many degrees of doubt as the problem permits but no more. Another precept that could be cast as a truism, but the reference is to an interesting discussion in the book about how the IC now brings quantitative probability estimates to the table, when developments – such as where Osama bin Laden lives – come under discussion.
    7. Strike the right balance between under- and overconfidence, between prudence and decisiveness. I really don’t see the particular value of this guideline, except to focus on whether you are being overconfident or indecisive. Give it some thought?
    8. Look for the errors behind your mistakes but beware of rearview-mirror hindsight biases. I had an intellectual mentor who served in the Marines and who was fond of saying, “we are always fighting the last war.” In this regard, I’m fond of the saying, “the only certain thing about the future is that there will be surprises.”
    9. Bring out the best in others and let others bring out the best in you. Tetlock’s following sentence is more to the point – “master the fine art of team management.”
  • Master the error-balancing cycle. Good to think about managing this, too.

Puckishly, Tetlocks adds an 11th Commandment – don’t treat commandments as commandments.

Great topic – forecasting subjective geopolitical developments in teams. Superforecasting touches on some fairly subtle points, illustrated with examples. I think it is well worth having on the bookshelf.

There are some corkers, too, like when Tetlock’s highlights the recommendations of 2nd Century physician to Roman emperors Galen, the medical authority for more than 1000 years.

Galen once wrote, apparently,

“All who drink of this treatment recover in a short time, except those whom it does not help, who all die…It is obvious, therefore, that it fails only in incurable cases.”

anatomy of an asset bubble, explosive processes, noncausal autoregressive models

An Update on Bitcoin

Fairly hum-drum days of articles on testing for unit roots in time series led to discovery of an extraordinary new forecasting approach – using the future to predict the present.

Since virtually the only empirical application of the new technique is predicting bubbles in Bitcoin values, I include some of the recent news about Bitcoins at the end of the post.

Noncausal Autoregressive Models

I think you have to describe the forecasting approach recently considered by Lanne and Saikkonen, as well as Hencic, Gouriéroux and others, as “exciting,” even “sexy” in a Saturday Night Live sort of way.

Here is a brief description from a 2015 article in the Econometrics of Risk called Noncausal Autoregressive Model in Application to Bitcoin/USD Exchange Rates

noncausal

I’ve always been a little behind the curve on lag operators, but basically Ψ(L-1) is a function of the standard lagged operators, while Φ(L) is a second function of offsets to future time periods.

To give an example, consider,

yt = k1yt-1+s1yt+1 + et

where subscripts t indicate time period.

In other words, the current value of the variable y is related to its immediately past value, and also to its future value, with an error term e being included.

This is what I mean by the future being used to predict the present.

Ordinarily in forecasting, one would consider such models rather fruitless. After all, you are trying to forecast y for period t+1, so how can you include this variable in the drivers for the forecasting setup?

But the surprising thing is that it is possible to estimate a relationship like this on historic data, and then take the estimated parameters and develop simulations which lead to predictions at the event horizon, of, say, the next period’s value of y.

This is explained in the paragraph following the one cited above –

noncausal2

In other words, because et in equation (1) can have infinite variance, it is definitely not normally distributed, or distributed according to a Gaussian probability distribution.

This is fascinating, since many financial time series are associated with nonGaussian error generating processes – distributions with fat tails that often are platykurtotic.

I recommend the Hencic and Gouriéroux article as a good read, as well as interesting analytics.

The authors proposed that a stationary time series is overlaid by explosive speculative periods, and that something can be abstracted in common from the structure of these speculative excesses.

Mt. Gox, of course, mentioned in this article, was raided in 2013 by Japanese authorities, after losses of more than $465 million from Bitcoin holders.

Now, two years later, the financial industry is showing increasing interest in the underlying Bitcoin technology and Bitcoin prices are on the rise once again.

bitcoin

Anyway, the bottom line is that I really, really like a forecast methodology based on recognition that data come from nonGaussian processes, and am intrigued by the fact that the ability to forecast with noncausal AR models depends on the error process being nonGaussian.

Federal Reserve policies and impacts, financial forecasting, forecasting turning points, macroeconomic forecasting

Coming Attractions

Well, I have been doing a deep dive into financial modeling, but I want to get back to blogging more often. It gets in your blood, and really helps explore complex ideas.

So- one coming attraction here is going to be deeper discussion of the fractal market hypothesis.

Ladislav Kristoufek writes in a fascinating analysis (Fractal Markets Hypothesis and the Global Financial Crisis:Scaling, Investment Horizons and Liquidity) that,

“..it is known that capital markets comprise of various investors with very different investment horizons { from algorithmically-based market makers with the investment horizon of fractions of a second, through noise traders with the horizon of several minutes, technical traders with the horizons of days and weeks, and fundamental analysts with the monthly horizons to pension funds with the horizons of several years. For each of these groups, the information has different value and is treated variously. Moreover, each group has its own trading rules and strategies, while for one group the information can mean severe losses, for the other, it can be taken a profitable opportunity.”

The mathematician and discoverer of fractals Mandelbrot and investor Peters started the ball rolling, but the idea maybe seemed like a fad of the 1980’s and 1990s.

But, more and more,  new work in this area (as well as my personal research) points to the fact that the fractal market hypothesis is vitally important.

Forget chaos theory, but do notice the power laws.

The latest  fractal market research is rich in mathematics – especially wavelets, which figure in forecasting, but which I have not spent much time discussing here.

There is some beautiful stuff produced in connection with wavelet analysis.

For example, here is a construction from a wavelet analysis of the NASDAQ from another paper by Kristoufek

Wavlet1

The idea is that around 2008, for example, investing horizons collapsed, with long term traders exiting and trading becoming more and more short term. This is associated with problems of liquidity – a concept in the fractal market hypothesis, but almost completely absent from many versions of the so-called “efficient market hypothesis.”

Now, maybe like some physicists, I am open to the discovery of deep keys to phenomena which open doors of interpretation across broad areas of life.

Another coming attraction will be further discussion of forward information on turning points in markets and the business cycle generally.

The current economic expansion is growing long in tooth, pushing towards the upper historically observed lengths of business expansions in the United States.

The basic facts are there for anyone to notice, and almost sound like a litany of complaints about how the last crisis in 2008-2009 was mishandled. But China is decelerating, and the emerging economies do not seem positioned to make up the global growth gap, as in 2008-2009. Interest rates still bounce along the zero bound. With signs of deteriorating markets and employment conditions, the Fed may never find the right time to raise short term rates – or if they plunge ahead will garner virulent outcry. Financial institutions are even larger and more concentrated now than before 2008, so “too big to fail” can be a future theme again.

What is the best panel of financial and macroeconomic data to watch the developments in the business cycle now?

So those are a couple of topics to be discussed in posts here in the future.

And, of course, politics, including geopolitics will probably intervene at various points.

Initially, I started this blog to explore issues I encountered in real-time business forecasting.

But I have wide-ranging interests – being more of a fox than a hedgehog in terms of Nate Silver’s intellectual classification.

I’m a hybrid in terms of my skill set. I’m seriously interested in mathematics and things mathematical. I maybe have a knack for picking through long mathematical arguments to grab the key points. I had a moment of apparent prodigy late in my undergrad college career, when I took graduate math courses and got straight A’s and even A+ scores on final exams and the like.

Mathematics is time consuming, and I’ve broadened my interests into economics and global developments, working around 2002-2005 partly in China.

As a trivia note,  my parents were immigrants to the US from Great Britain , where their families were in some respects connected to the British Empire that more or less vanished after World War II and, in my father’s case, to the Bank of England. But I grew up in what is known as “the West” (Colorado, not California, interestingly), where I became a sort of British cowboy and subsequently, hopefully, have continued to mature in terms of attitudes and understanding.

accuracy of forecasts, financial forecasting, stock market forecasts

One Month Forecast of SPY ETF Price Level – Extreme Value Prediction Algorithm (EVPA)

Here is a chart showing backtests and a prediction for the midpoints of the monthly trading ranges of the SPY exchange traded fund.

MonthSPY

The orange line traces out the sequence of actual monthly midpoints of the price range – the average of the high and low price for the month.

The blue line charts the backtests going back to 2013 and a forecast for September 2015 – which is the right-most endpoint of the blue line. The predicted September midpoint is $190.43.

The predictions come from the EVPA, an algorithm backtested to 1994 for the SPY. Since 1994, the forecast accuracy, measured by the MAPE or mean absolute percent error, is 2.4 percent. This significantly improves on a No Change model, one of the alternative forecast benchmarks for this series, and the OOS R2 of the forecast of the midpoint of the monthly trading range is a solid 0.33.

Just from eyeballing the graph above, it seems like there are systematic errors to the downside in the forecasts.

However, the reverse appears to happen, when the SPY prices are falling over a longer period of time.

2008SPYMonth

I would suggest, therefore, that the prediction of about $190 for September is probably going to be higher than the actual number.

Now – disclaimer. This discussion is provided for scientific and entertainment purposes only. We assume no responsibility for any trading actions taken, based on this information. The stock market, particularly now, is volatile. There are lots of balls in the air – China, potential Fed actions on interest rates, more companies missing profit expectations, and so forth – so literally anything may happen.

The EVPA is purely probabilistic. It is right more often than wrong. Its metrics are better than those of alternative forecast models in the cases I have studied. But, withal, it still is a gamble to act on its predictions.

But the performance of the EVPA is remarkable, and I believe further improvements are within reach.

applied macroeconomic forecasting methods, macroeconomic forecasting

Economic Impact Modeling

I had a chance, recently, to watch computer simulations and interact with a regional economic impact model called REMI. This is a multi-equation model of some vintage (dating back the 1980’s) that has continued to evolve. It’s probably currently the leader in the field and has seen recent application to assessing proposals for increasing the minimum wage – in California, Vermont, San Francisco – and to evaluating  a carbon tax for the Citizen’s Climate Initiative  (see the video presentation at the end of this post).

One way to interact with REMI is to click on blocks in a computer screen based on the following schematic

REMIblocs

I watched Brian Lewandowski do this at Colorado University’s Leeds School of Business.

Brian set parameters for increases in labor productivity for professional services and changes in investment in primary and secondary educational by clicking on boxes or blocks. Brian, Richard Wobbekind (pictured below), and I discussed results, and how REMI is helpful in exploring “what-if’s” and might have applications to  optimizing tax policies at the state level..

RichWobbekind

Wobbekind is himself a leader in preparing and presenting State-level forecasts for Colorado, and is active in the International Institute of Forecasters (IIF) which sponsors the International Journal of Forecasting and Foresight – as well as being an Associate Dean of CU’s Leeds School of Business.

Key Point About Multi-Equation, Multivariate Economic Models

From the standpoint of forecasting, the best way I can understand where REMI should be placed in the tool-kit is to remember the distinction between conditional and unconditional forecasts.

REMI model documentation indicates that,

The REMI model consists of thousands of simultaneous equations with a structure that is relatively straightforward. The exact number of equations used varies depending on the extent of industry, demographic, demand, and other detail in the specific model being used. The overall structure of the model can be summarized in five major blocks: (1) Output, (2) Labor and Capital Demand, (3) Population and Labor Supply, (4) Wages, Prices, and Costs, and (5) Market Shares

So you might have equations such as,

X1t = a0 + a1Z1t +..+ akZkt

X2t = b0 + b1Z1t +..+ brZrt

In order to predict unconditionally what (X1t,X2t) will be at some specific future time T*, it is necessary to correctly derive the parameters (a0,a1,..,ak,b0,b1,,…,br).

And it also is necessary, for an unconditional forecast, to predict the future values of all the exogenous variables on the right-hand side of the equation – that is all the Z variables that are not in fact X variables.

This usually means that unconditional forecasts from multivariate forecast models have wide and rapidly diverging confidence intervals.

Thus, if you try to forecast future employment in, say, California with such models, they may underperform simpler, single equation models – such as those based on exponential smoothing, for example.

This does not invalidate general systems models such as REMI.

Assuming the flows and linkages of sectors and blocks are realistic and correctly modeled, such models can help think through the consequences of policy decisions, new legislation, and infrastructure investments.

This is essentially to say that these models may present good conditional forecasts – basically “what-if’s” without being the best forecasting tool available.

Here is a video presentation based on the Citizen’s Climate Initiative application of REMI to assessing a carbon tax – an interesting proposal.

accuracy of forecasts, asset bubbles, stock market forecasts

Today’s Stock Market

Shakespeare’s Hamlet says at one point, “There are more things Horatio, than are dreamt of in your philosophy.” This is also a worthy thought when it comes to stock market forecasts.

So here is a chart showing recent daily predictions of the midpoint (MP) of the daily price ranges for the SPY exchange traded fund (ETF), compared with the actual dollar value of the midpoint of the trading range for each day.

FlashCrash

The predictions are generated by the EVPA – extreme value prediction algorithm. This is a grown-up version of the idea that ratios, like the ratio between the opening price and the previous high or low, can give us a leg up in predicting the high or low prices (or midpoint) for a trading day.

Note the EVPA is quite accurate up to recent days, when forecast errors amplify at the end of the week of August 21. In fact, in backtests to 1994, EVPA forecasts the daily MP with a mean absolute percent error (MAPE) less that 0.4%.

Here is a chart of the forecast errors corresponding to the predicted and actuals in the first chart above.

FCerror

Note the apparently missing bars for some dates are just those forecasts which show such tiny errors they don’t even make it to the display, which is dominated by the forecast error August 24 at about 4 percent.

Note also, the EVPA adjusts quickly, so that by Monday August 28th – when the most dramatic drop in the midpoint (closing price, etc.) occurs – the forecasts (or backcasts) show much lower forecast error (-0.1%).

Where This Is All Going

The EVPA keys off the opening price, which this morning is listed as 198.11 – lower than Friday’s closing price of 199.24.

This should tell you that the predictions for the day are not particularly rosy, and, indeed, the EVPA forecast for the week’s MP of the trading range is almost flat, compared to the midpoint of the trading range for last week. My forecast of the MP for this week is up 0.13%.

What that means, I am not quite sure.

Here is a chart showing the performance of the weekly EVPA forecasts of the SPY MP for recent weeks.

WeeklySPYMP

This chart does not include the forecast for the current week.

What I find really interesting is that, according to this model, the slide in the SPY might have been spotted as early as the second week in August.

The EVPA is an amazing piece of data analytics, but it exists in an environment of enormous complexity. Studies showing that various international stock markets are cointegrated, and the sense of this clearly applies to Chinese and US stock markets. Also, there is talk of pulling the punchbowl of “free money” or zero interest rates, and that seems to have a dampening effect on the trading outlook.

Quite frankly, in recent weeks I have been so absorbed in R programming, testing various approaches, and, as noted in my previous post, weddings, that I have neglected these simple series. No longer. I plan to make these two prediction series automatic with my systems.

We will see.

accuracy of forecasts, financial forecasts, stock market forecasts, stock trading algorithms

Back to the Drawing Board

Well, not exactly, since I never left it.

But the US and other markets opened higher today, after round-the-clock negotiations on the Greek debt.

I notice that Jeff Miller of Dash of Insight frequently writes stuff like, We would all like to know the direction of the market in advance. Good luck with that! Second best is planning what to look for and how to react.

Running the EVPA with this morning’s pop up in the opening price of the SPY, I get a predicted high for the day of 210.17 and a predicted low of 207.5. The predicted low for the day will be spot-on, if the current actual low for the trading range holds.

I can think of any number of arguments to the point that the stock market is basically not predictable, because unanticipated events constantly make an impact on prices. I think it would even be possible to invoke Goedel’s Theorem – you know, the one that uses meta-mathematics to show that every axiomatic system of complexity greater than a group is essentially incomplete. There are always new truths.

On the other hand, backtesting the EVPA – extreme value prediction algorithm – is opening up new vistas. I’m appreciative of helpful comments of and discussions with professionals in the finance and stock market investing field.

I strain every resource to develop backtests which are out-of-sample (OOS), and recently have found a way to predict closing prices with resources from the EVPA.

MOnthlyROISPYevpa

Great chart. The wider gold lines are the actual monthly ROI for the SPY, based on monthly closing prices. The blue line shows the OOS prediction of these closing prices, based on EVPA metrics. As you can see, the blue line predictions flat out miss or under-predict some developments in the closing prices. At the same time, in other cases, the EVPA predictions show uncanny accuracy, particularly in some of the big dips down.

Recognize this is something new. Rather than, say, predicting developments likely over a range of trading days – the high and low of a month, the  chart above shows predictions for stock prices at specific times, at the closing bell of the market the last trading day of each month.

I calculate the OOS R2 at 0.63 for the above series, which I understand is better than can be achieved with an autoregressive model for the closing prices and associated ROI’s.

I’ve also developed spreadsheets showing profits, after broker fees and short term capital gains taxes, from trading based on forecasts of the EVPA.

But, in addition to guidance for my personal trading, I’m interested in following out the implications of how much the historic prices predict about the prices to come.