Category Archives: stock market forecasts

financial forecasting, stock market forecasts

Daily High and Low Stock Prices – Falling Knives

July 1, 2016 Clive Jones

The mathematics of random walks form the logical underpinning for the dynamics of prices in stock, currency, futures, and commodity markets.

Once you accept this, what is really interesting is to consider departures from random walk movements of prices. Such distortions can signal underlying biases brought to the table by investors and others with influence in these markets.

“Falling knives” may be an example.

A good discussion is presented in Falling Knives: Do Stocks Really Drop 3 Times Faster Than They Rise?

This article in Seeking Alpha is more or less organized around the following chart.

The authors argue this classic chart is really the result of a “Black Swan” event – namely the Great Recession of 2008-2009. Outside of unusual deviations, however, they try to show that “the rate of rallies and pullbacks are approximately equal.”

I’ve been exploring related issues and presently am confident that there are systematic differences in the volatility of high and low prices over a range of time periods.

This seems odd to say, since high and low prices exist within the continuum of prices, their only distinguishing feature being that they are extreme values over the relevant interval – a trading day or collection of trading days.

However, the variance or standard deviation of daily percent changes or rates of change of high and low prices are systematically different for high and low prices in many examples I have seen.

Consider, for example, rates of change of daily high and low prices for the SPY exchange traded fund (ETF) – the security charted in the preceding graph.

This chart shows the standard deviation of daily rates of change of high and low prices for the SPY over rolling annual time windows.

This evidence suggests higher volatility for daily growth or rates of change of low prices is more than something linked just with “Black Swan events.”

Thus, while the largest differences between standard deviations occur in late 2008 through 2009 – precisely the period of the financial crisis – in 2011 and 2012, as well as recently, we see the variance of daily rates of change of low prices significantly higher than those for high prices.

The following chart shows the distribution of these standard deviations of rates of change of daily high and low prices.

You can see the distribution of the daily growth rates for low prices – the blue line – is fatter in a certain sense, with more instances of somewhat greater standard deviations than the daily growth rates for high prices. As a consequence, too, the distribution of the daily growth rates of low prices shows less concentration near the modal value, which is sharply peaked for both curves.

These are not Gaussian or normal distributions, of course. And I find it interesting that the finance literature, despite decades of recognition of these shapes, does not appear to have a consensus on exactly what types of distributions these are. So I am not going to jump in with my two bits worth, although I’ve long thought that these resemble Laplace Distributions.

In any case, what we have here is quite peculiar, and can be replicated for most of the top 100 ETF’s by market capitalization. The standard deviation of rates of change of current low price to previous low prices generally exceeds the standard deviation of rates of change of high prices, similarly computed.

Some of this might be arithmetic, since by definition high prices are greater numerically than low prices, and we are computing rates of change.

However, it’s easy to dispel the idea that this could account for the types of effects seen with SPY and other securities. You can simulate a random walk, for example, and in thousands of replications with positive prices essentially lose any arithmetic effect of this type in noise.

I believe there is more to this, also.

For example, I find evidence that movements of low prices lead movements of high prices over some time frames.

Investor psychology is probably the most likely explanation, although today we have to take into account the “psychology” of robot trading algorithms. Presumeably, these reflect, in some measure, the predispositions of their human creators.

It’s kind of a puzzle.

Top image from SGS Swinger BlogSpot

fractal market hypothesis, long memory in time series, macroeconomic forecasting, stock market forecasts

Fractal Markets, Fractional Integration, and Long Memory in Financial Time Series – I

November 20, 2015 Clive Jones

The concepts – ‘fractal market hypothesis,’ ‘fractional integration of time series,’ and ‘long memory and persistence in time series’ – are related in terms of their proponents and history.

I’m going to put up ideas, videos, observations, and analysis relating to these concepts over the next several posts, since, more and more, I think they lead to really fundamental things, which, possibly, have not yet been fully explicated.

And there are all sorts of clear connections with practical business and financial forecasting – for example, if macroeconomic or financial time series have “long memory,” why isn’t this characteristic being exploited in applied forecasting contexts?

And, since it is Friday, here are a couple of relevant videos to start the ball rolling.

Benoit Mandelbrot, maverick mathematician and discoverer of ‘fractals,’ stands at the crossroads in the 1970’s, contributing or suggesting many of the concepts still being intensively researched.

In economics, business, and finance, the self-similarity at all scales idea is trimmed in various ways, since none of the relevant time series are infinitely divisible.

A lot of energy has gone into following Mandelbrot suggestions on the estimation of Hurst exponents for stock market returns.

This YouTube by a Parallax Financial in Redmond, WA gives you a good flavor of how Hurst exponents are being used in technical analysis. Later, I will put up materials on the econometrics involved.

Blog posts are a really good way to get into this material, by the way. There is a kind of formalism – such as all the stuff in time series about backward shift operators and conventional Box-Jenkins – which is necessary to get into the discussion. And the analytics are by no means standardized yet.

Chinese economy, futures markets, stock market forecasts

China Update – The Beatings Will Continue Until Morale Improves

September 14, 2015 Clive Jones

It’s essentially illegal to short a stock in China today.

This developed in response to the more than 40 percent drop in market value of most stocks– as in the chart for Shenzhen Index B shown below.

B shares (Chinese: B股, officially Domestically Listed Foreign Investment Shares) on the Shanghai and Shenzhen stock exchanges refers to those that are traded in foreign currencies. Shares that are traded on the two mainland Chinese stock exchanges in Renminbi, the currency in mainland China, are called A shares.

China Seen Delaying Shenzhen Stock Link Until 2016 Amid Turmoil

Chinese authorities have gone to extreme lengths to prop up the stock market as both the Shanghai Composite Index and the Shenzhen Composite Index lost more than 40 percent from their June highs. They’ve banned major shareholders from selling stakes, armed a state-run margin trader with billions of dollars to buy equities, allowed hundreds of companies to halt their shares and placed curbs on bearish bets in the futures market.

One casualty has been the World’s Biggest Stock-Index Futures Market

Volumes in the country’s CSI 300 Index and CSI 500 Index futures sank to record lows on Wednesday after falling 99 percent from their June highs. Ranked by the World Federation of Exchanges as the most active market for index futures as recently as July, liquidity in China has dried up as authorities raised margin requirements, tightened position limits and started a police probe into bearish wagers.

China’s Response to Stock Plunge Rattles Traders

A business journalist has been detained and shown apologizing on national television for writing an article that could have hurt the market.

Apparently, the beatings will continue until morale improves.

The spectacle of Wang Xiaolu, reporter at a Chinese business publication, enacting Monday what authorities called a confession of spreading “panic and disorder” in the stock market through his reporting should scare investors.

The announcement Sunday by China’s Ministry of Public Security that 197 people had been punished for spreading rumors about stocks and other issues should scare investors.

That the head of hedge fund manager Man Group Plc’s China unit has been taken into custody, as reported by Bloomberg on Monday, should scare global investors.

This is not to say that China isn’t capable of browbeating and arm-twisting and public money spending its way to a higher stock market. China asked brokerages to increase their support of a market rescue fund by 100 billion yuan last week, according to reports, a move perhaps tied to a desire for a rally, or at least stability, ahead of Thursday’s parade commemorating victory over Japan…….

They do know that a huge country is scared so badly of something that they will not, cannot, allow things to be said that cause stocks to fall.

Reflections

These strong-arm tactics go beyond anything seen in the West, in terms of damage control. Transparency, an key feature in financial analysis, has been thrown out the window. With Goons going around bullying people to hold stocks that have lost 40 percent of their value, pressing journalists to keep quiet about problems in the market, it is a strange world in Chinese finance and stocks. People “disappear” – no record of arrest being announced.

There is no question but that Chinese macroeconomic statistics must also be viewed in a more skeptical light.

This makes assessment of the Chinese slowdown more difficult.

accuracy of forecasts, financial forecasting, stock market forecasts

One Month Forecast of SPY ETF Price Level – Extreme Value Prediction Algorithm (EVPA)

September 2, 2015 Clive Jones

Here is a chart showing backtests and a prediction for the midpoints of the monthly trading ranges of the SPY exchange traded fund.

The orange line traces out the sequence of actual monthly midpoints of the price range – the average of the high and low price for the month.

The blue line charts the backtests going back to 2013 and a forecast for September 2015 – which is the right-most endpoint of the blue line. The predicted September midpoint is $190.43.

The predictions come from the EVPA, an algorithm backtested to 1994 for the SPY. Since 1994, the forecast accuracy, measured by the MAPE or mean absolute percent error, is 2.4 percent. This significantly improves on a No Change model, one of the alternative forecast benchmarks for this series, and the OOS R2 of the forecast of the midpoint of the monthly trading range is a solid 0.33.

Just from eyeballing the graph above, it seems like there are systematic errors to the downside in the forecasts.

However, the reverse appears to happen, when the SPY prices are falling over a longer period of time.

I would suggest, therefore, that the prediction of about $190 for September is probably going to be higher than the actual number.

Now – disclaimer. This discussion is provided for scientific and entertainment purposes only. We assume no responsibility for any trading actions taken, based on this information. The stock market, particularly now, is volatile. There are lots of balls in the air – China, potential Fed actions on interest rates, more companies missing profit expectations, and so forth – so literally anything may happen.

The EVPA is purely probabilistic. It is right more often than wrong. Its metrics are better than those of alternative forecast models in the cases I have studied. But, withal, it still is a gamble to act on its predictions.

But the performance of the EVPA is remarkable, and I believe further improvements are within reach.

accuracy of forecasts, asset bubbles, stock market forecasts

Today’s Stock Market

August 31, 2015 Clive Jones

Shakespeare’s Hamlet says at one point, “There are more things Horatio, than are dreamt of in your philosophy.” This is also a worthy thought when it comes to stock market forecasts.

So here is a chart showing recent daily predictions of the midpoint (MP) of the daily price ranges for the SPY exchange traded fund (ETF), compared with the actual dollar value of the midpoint of the trading range for each day.

The predictions are generated by the EVPA – extreme value prediction algorithm. This is a grown-up version of the idea that ratios, like the ratio between the opening price and the previous high or low, can give us a leg up in predicting the high or low prices (or midpoint) for a trading day.

Note the EVPA is quite accurate up to recent days, when forecast errors amplify at the end of the week of August 21. In fact, in backtests to 1994, EVPA forecasts the daily MP with a mean absolute percent error (MAPE) less that 0.4%.

Here is a chart of the forecast errors corresponding to the predicted and actuals in the first chart above.

Note the apparently missing bars for some dates are just those forecasts which show such tiny errors they don’t even make it to the display, which is dominated by the forecast error August 24 at about 4 percent.

Note also, the EVPA adjusts quickly, so that by Monday August 28^th – when the most dramatic drop in the midpoint (closing price, etc.) occurs – the forecasts (or backcasts) show much lower forecast error (-0.1%).

Where This Is All Going

The EVPA keys off the opening price, which this morning is listed as 198.11 – lower than Friday’s closing price of 199.24.

This should tell you that the predictions for the day are not particularly rosy, and, indeed, the EVPA forecast for the week’s MP of the trading range is almost flat, compared to the midpoint of the trading range for last week. My forecast of the MP for this week is up 0.13%.

What that means, I am not quite sure.

Here is a chart showing the performance of the weekly EVPA forecasts of the SPY MP for recent weeks.

This chart does not include the forecast for the current week.

What I find really interesting is that, according to this model, the slide in the SPY might have been spotted as early as the second week in August.

The EVPA is an amazing piece of data analytics, but it exists in an environment of enormous complexity. Studies showing that various international stock markets are cointegrated, and the sense of this clearly applies to Chinese and US stock markets. Also, there is talk of pulling the punchbowl of “free money” or zero interest rates, and that seems to have a dampening effect on the trading outlook.

Quite frankly, in recent weeks I have been so absorbed in R programming, testing various approaches, and, as noted in my previous post, weddings, that I have neglected these simple series. No longer. I plan to make these two prediction series automatic with my systems.

We will see.

Chinese economy, predicting turning points, stock market forecasts

Recent Events in the US Stock Market

August 26, 2015 Clive Jones

The recent drop in US stocks is dramatic, as the steep falloff of the SPY exchange traded fund (ETF) Monday, August 24^th– almost the most recent action in the chart – shows.

At the same time, this is by no means the steepest drop in closing prices, as the following chart of daily returns highlights.

TV commentators and others point to China and the prospective liftoff of US short term interest rates, with the Federal Reserve finally raising rates off the zero bound in – it was thought – September.

I have been impressed at the accuracy of Michael Pettis’ predictions in his China Financial Markets. Pettis has warned about a debt bubble in China for two years and consistently makes other correct calls. I have some first-hand experience doing business in China, and plan a longer post of the collapse of Chinese stock markets and the economic slowdown there.

You can imagine, if you will, a sort of global input-output table with a corresponding table of import/export flows. China has gotten a lot bigger since 2008-2009, absorbing significant amounts of the global output of iron and steel, oil, and other commodities.

Also, in 2008-2009 and in the earlier recession of 2001, China led the way to greater spending, buoying the global economy which, otherwise, was in sad shape. That’s not going to happen this time, if a real recession takes hold.

All very scary, but while the latest stuff took place, this is what I was doing.

In other words, I was the father of the groom at a splendid wedding for my younger son at the Pearl Buck estate just outside Philadelphia.

Well, that wonderful thing being done, I plan to return to more frequent posting on BusinessForecastblog.

I also apologize for having the tools to predict the current downturn, at least after developments later last week, and not signaling readers.

But frankly, I’m not sure the extreme value prediction algorithms (EVPA) reliably predict major turning points. In fact, there seem to be outside influences at key junctures. However, once a correction is underway, predictability returns. Thus, the algorithms do more than simply forecast the growth in stock prices. The EVPA also works to predict the extent of downturns.

Here’s a tip. Start watching ratios such as those between differences between the opening price in a trading day and the previous day’s high or low price, divided by the previous day’s high or low price, respectively. Very significant predictors of the change in daily highs and lows, and with significance for changes in closing prices, if you bring some data analytics to bear.

accuracy of forecasts, financial forecasts, stock market forecasts, stock trading algorithms

Back to the Drawing Board

July 13, 2015 Clive Jones

Well, not exactly, since I never left it.

But the US and other markets opened higher today, after round-the-clock negotiations on the Greek debt.

I notice that Jeff Miller of Dash of Insight frequently writes stuff like, We would all like to know the direction of the market in advance. Good luck with that! Second best is planning what to look for and how to react.

Running the EVPA with this morning’s pop up in the opening price of the SPY, I get a predicted high for the day of 210.17 and a predicted low of 207.5. The predicted low for the day will be spot-on, if the current actual low for the trading range holds.

I can think of any number of arguments to the point that the stock market is basically not predictable, because unanticipated events constantly make an impact on prices. I think it would even be possible to invoke Goedel’s Theorem – you know, the one that uses meta-mathematics to show that every axiomatic system of complexity greater than a group is essentially incomplete. There are always new truths.

On the other hand, backtesting the EVPA – extreme value prediction algorithm – is opening up new vistas. I’m appreciative of helpful comments of and discussions with professionals in the finance and stock market investing field.

I strain every resource to develop backtests which are out-of-sample (OOS), and recently have found a way to predict closing prices with resources from the EVPA.

Great chart. The wider gold lines are the actual monthly ROI for the SPY, based on monthly closing prices. The blue line shows the OOS prediction of these closing prices, based on EVPA metrics. As you can see, the blue line predictions flat out miss or under-predict some developments in the closing prices. At the same time, in other cases, the EVPA predictions show uncanny accuracy, particularly in some of the big dips down.

Recognize this is something new. Rather than, say, predicting developments likely over a range of trading days – the high and low of a month, the chart above shows predictions for stock prices at specific times, at the closing bell of the market the last trading day of each month.

I calculate the OOS R² at 0.63 for the above series, which I understand is better than can be achieved with an autoregressive model for the closing prices and associated ROI’s.

I’ve also developed spreadsheets showing profits, after broker fees and short term capital gains taxes, from trading based on forecasts of the EVPA.

But, in addition to guidance for my personal trading, I’m interested in following out the implications of how much the historic prices predict about the prices to come.

accuracy of forecasts, predictive analytics, stock market forecasts

Direction of the Market Next Week – July 13

July 10, 2015 Clive Jones

Last Friday, before July 4^th, I ran some numbers on the SPY exchange traded fund, looking at backcasts from the EVPA (extreme value prediction algorithm) for the Monday and Tuesday before, when Greece kept the banks closed and defaulted on its IMF payment. I also put up a ten day look forward on the EVPA predictions.

Of course, the SPY is an ETF which tracks the S&P 500.

The EVPA predicted the SPY high and low would drop at the beginning of the following week, beginning July 6, but seemed to suggest some rebound by the end of this week – that is today, July 10.

Here is a chart for today and next week with comments on interpreting the forecasts.

So the EVPA predicts the high and low over the current trading day, and aggregations of 2,3,4,.. trading days going forward.

The red diamonds in the chart map out forecasts for the high price of the SPY today, July 10, and for groups of trading days beginning today and ending Monday, July 13, and the rest of the days of next week.

Similarly, the blue crosses map out forecasts for the SPY low prices which are predicted to be reached over 1 day, the next two trading days, the next three trading days, and so forth.

Attentive readers will notice an apparent glitch in the forecasts for the high prices to come – namely that the predicted high of the next two trading days is lower than the predicted high for today – which is, of course, logically impossible.

But, hey, this is econometrics, not logic, and what we need to do is interpret the output of the models against what it is we are looking for.

In this case, a solid reduction in the predicted high of the coming two day period, compared with the prediction of today’s high signals that the high of the SPY is likely to be lower Monday than today.

This is consistent with predictions for the low today and for the next two trading days shown in blue – which indicates lower lows will be reached the second day.

Following that, the EVPA predictions for higher groupings of trading days are inconclusive, given statistical tolerances of the approach.

Note that the predictions of the high and low for today, Friday, July 10, are quite accurate, assuming these bounds have been reached by this point – two o’clock on Wall Street. In percentage error terms, the EVAP forecasts are over-forecasting 0.3% for the high and 0.2% for the low.

Again, the EVPA always keys off the opening price of the period being forecast.

I also have a version of the EVPA which forecasts ahead for the coming week, for two week periods, and so forth.

Leading up to the financial crisis of 2008 and then after the worst in October of that year, the EVPA weekly forecasts clearly highlight turning points.

Currently, weekly forecasts going up to monthly durations do not signal any clear trend in the market, but rather signal increasing volatility.

asset bubbles, Chinese economy, global business forecasts, stock market forecasts

Chinese Stock Market Collapse

July 8, 2015 Clive Jones

Chinese stocks are more volatile, in terms of percent swings, than stocks on other global markets, as this Bloomberg chart highlights.

So the implication is maybe that the current bursting of the Chinese stock bubble is not such a big deal for the global economy, or perhaps it can be contained – despite signs of correlations between the Global Stocks and Shanghai Composite series.

Facts and Figures

Panic selling hit the major Chinese exchanges in Shanghai and Shenzeng, spreading now to the Hong Kong exchange.

Trades on most companies are limited or frozen, and major indexes continue to drop, despite support from the Chinese government.

Chinese Trading Suspensions Freeze $1.4 Trillion of Shares Amid Rout

The rout in Chinese shares has erased at least $3.2 trillion in value, or twice the size of India’s entire stock market. The Shenzhen Composite Index has led declines with a 38 percent plunge since its June 12 peak, as margin traders unwound bullish bets.

China: The Stock Market Meltdown Continues

Briefly put, there are few alternatives for saving in China. The formal banking system provides negative returns (low deposit yields, lower than inflation typically). Housing is no longer returning positive capital gains — partly as a consequence of deliberate policy actions to moderate a perceived housing bubble. So, what’s left (given you can’t easily save in overseas assets)? Equities. We have a typical boom-bust phenomenon, amplified by underdeveloped financial markets, opacity in valuations, and uncertainty regarding the government’s intentions (and will-power).

Stock Sell-Off Is Unabated in China (New York Times)

Most of the trades on Chinese exchanges are made by “retail traders,” basically individuals speculating on the market. These individuals often are highly leveraged or operating with borrowed money.

The Chinese markets moved into bubble territory several months back, and when a correction hit and as it accelerated recently, the Chinese government has tried all sorts of stuff, some charted below.

Public/private funds to buy stocks and slow the fall in their prices have been created, also.

Risks of Contagion

It’s hard for foreign investors to gain access to the Chinese markets, where there are different classes of stocks for Chinese and foreign traders. So, by that light, only a few percent of Chinese stocks are held by foreign interests, and direct linkages between the sharp turn in values in China and elsewhere should be limited.

There may indirect linkages going from the Chinese stock market to the Chinese economy, and then to foreign supplies.

Here’s why the crash in Chinese stocks matters so much to Australia, i.e. Australian property markets and reduced Chinese demand for iron.

Iron ore demand by China and the drop in Chinese stocks actually seems more related to somewhat independent linkage with the longer term cascade down by Chinese GDP growth, illustrated here (See Ongoing Developments in China).

But maybe the most dangerous and unpredictable linkage is psychological.

Thus, the Financial Express of India reports Shanghai blues trigger panic selling on Dalal Street, metals feel the heat

accuracy of forecasts, European economy, stock market forecasts

How Did BusinessForecastBlog’s Stock Market Forecast Algorithm Perform June 20 and July 1?

July 3, 2015 Clive Jones

As a spinoff from blogging for the past several years, I’ve discovered a way to predict the high and low of stock prices over periods, like one or several days, a week, or other periods.

As a general rule, I can forecast the high and low of the SPY – the exchange traded fund (ETF) which tracks the S&P 500 – with average absolute errors around 1 percent.

Recently, friends asked me – “how did you do Monday?” – referring to June 29^th when Greece closed its banks, punting on a scheduled loan payment to the International Monetary Fund (IMF) the following day.

SPY closing prices tumbled more than 2 percent June 30th, the largest daily drop since June 20, 2013.

Performance of the EVPA

I’m now calling my approach the EVPA or extreme value prediction algorithm. I’ve codified procedures and moved from spreadsheets to programming languages, like Matlab and R.

The performance of the EVPA June 29th depends on whether you allow the programs the Monday morning opening price – something I typically build in to the information set. That is, if I am forecasting a week ahead, I trigger the forecast after the opening of that week’s trading, obtaining the opening price for that week.

Given the June 29 opening price for the SPY ($208.05 a share), the EVPA predicts a Monday high and low of 209.25 and 207.11, for percent forecast errors of -0.6% and -1% respectively.

Of course, Monday’s opening price was significantly down from the previous Friday (by -1.1%).

Without Monday’s opening price, the performance of the EVPA degrades somewhat in the face of the surprising incompetence of Eurozone negotiators. The following chart shows forecast errors for predictions of the daily low price, using only the information available at the close of the trading day Friday June 26.

	Actual	Forecast	% Error
29-Jun	205.33	208.71	1.6%
30-Jun	205.28	208.75	1.7%

Forecasts of the high price for one and two-trading day periods average 1 percent errors (over actuals), when generated only with closing information from the previous week.

Where the Market Is Going

So where is the market going?

The following chart shows the high and low for Monday through Wednesday of the week of June 30 to July 3, and forecasts for the high and low which will be reached in a nested series of periods from one to ten trading days, starting Wednesday.

What makes interpretation of these predictions tricky is the fact that they do not pertain to 1, 2, and so forth trading days forward, per se. Rather, they are forecasts for 1 day periods, 2 day periods, 3 day periods, and so forth.

One classic pattern is the highs level, but predictions for the lows drop over increasing groups of trading days. That is a signal for a drop in the averages for the security in question, since highs can be reached initially and still stand for these periods of increasing trading days.

These forecasts offer some grounds for increases in the SPY averages going forward, after an initial decrease through the beginning of the coming week.

Of course the Greek tragedy is by no means over, and there can be more surprises.

Still, I’m frankly amazed at how well the EVPA does, in the humming, buzzing and chaotic confusion of global events.

Business Forecasting