Links – end of March

US Economy and Social Issues

Reasons for Declining Labor Force Participation

LFchartVital Signs: Still No Momentum in Business Spending


Urban Institute Study – How big is the underground sex economy in eight cities employs an advanced statistical design. It’s sort of a model study, really.

Americans Can’t Retire When Bill Gross Sees Repression

Feeble returns on the safest investments such as bank deposits and fixed-income securities represent a “financial repression” transferring money from savers to borrowers, says Bill Gross, manager of the world’s biggest bond fund.

Robert Reich – The New Billionaire Political Bosses

American democracy used to depend on political parties that more or less represented most of us. Political scientists of the 1950s and 1960s marveled at American “pluralism,” by which they meant the capacities of parties and other membership groups to reflect the preferences of the vast majority of citizens.

Then around a quarter century ago, as income and wealth began concentrating at the top, the Republican and Democratic Parties started to morph into mechanisms for extracting money, mostly from wealthy people.

Finally, after the Supreme Court’s “Citizen’s United” decision in 2010, billionaires began creating their own political mechanisms, separate from the political parties. They started providing big money directly to political candidates of their choice, and creating their own media campaigns to sway public opinion toward their own views.

Global Economy

Top global risks you can’t ignore – good, short read

How Can Africa’s Water and Sanitation Shortfall be Solved? – interesting comments by experts on the scene, including –

Most African water utilities began experiencing a nose-dive in the late 1970s under World Bank and IMF policies. Many countries were suffering from serious trade deficits which had enormous implications for their budgets, incomes, and their abilities to honour loan obligations to, among others, bilateral and multilateral partners. These difficulties for African countries coincided around that period, with a major shift in global economic thought; a shift from heterodox economic thinking which favoured state intervention in critical sectors of the economy, to neoliberal economic thought which is more hostile to state intervention and prefers the deregulation of markets and their unfettered operation. This thought became dominant in the IMF and World Bank and influenced structural adjustment austerity packages that the two institutions prescribed to the struggling African economies at the time. This point is fundamental and cannot be divorced from any comprehensive analysis of the access deficit in African countries.

The austerity measures enforced by the Bank and IMF ensured a drastic reduction of state funding to the utilities, resulting in deterioration of facilities, poor conditions for staff and a mass exodus of expert staff. In the face of the resulting difficulties, the Bank and IMF held out only one option for the governments; the option of full cost recovery and of privatisation. This sealed the expectations of any funding for the sector as the private sector found the water sector highly risky to invest in. Following the common interventions set out by the World Bank, the countries achieved mostly poor results.

Contrary to much mainstream discourse, neither privatisation nor commercialisation constitute an adequate or sustainable way of managing urban water utilities to ensure access to people in Africa given the extreme poverty that confronts a significant portion of the population. The solution lies in a progressive tax-supported water delivery system that ensures access for all, supported by a management structure and a balanced set of incentives that ensure performance.


Machine Learning in 7 Pictures

Basic machine learning concepts of Bias vs Variance Tradeoff, Avoiding overfitting, Bayesian inference and Occam razor, Feature combination, Non-linear basis functions, and more – explained via pictures

The Universe

Great picture of the planet Mercury


Interest Rates – Forecasting and Hedging

A lot relating to forecasting interest rates is encoded in the original graph I put up, several posts ago, of two major interest rate series – the federal funds and the prime rates.


This chart illustrates key features of interest rate series and signals several important questions. Thus, there is relationship between a very short term rate and a longer term interest rates – a sort of two point yield curve. Almost always, the federal funds rate is below the prime rate. If for short periods this is not the case, it indicates a radical reversion of the typical slope of the yield curve.

Credit spreads are not illustrated in this figure, but have been shown to be significant in forecasting key macroeconomic variables.

The shape of the yield curve itself can be brought into play in forecasting future rates, as can typical spreads between interest rates.

But the bottom line is that interest rates cannot be forecast with much accuracy beyond about a two quarter forecast horizon.

There is quite a bit of research showing this to be true, including –

Professional Forecasts of Interest Rates and Exchange Rates: Evidence from the Wall Street Journal’s Panel of Economists

We use individual economists’ 6-month-ahead forecasts of interest rates and exchange rates from the Wall Street Journal’s survey to test for forecast unbiasedness, accuracy, and heterogeneity. We find that a majority of economists produced unbiased forecasts but that none predicted directions of changes more accurately than chance. We find that the forecast accuracy of most of the economists is statistically indistinguishable from that of the random walk model when forecasting the Treasury bill rate but that the forecast accuracy is significantly worse for many of the forecasters for predictions of the Treasury bond rate and the exchange rate. Regressions involving deviations in economists’ forecasts from forecast averages produced evidence of systematic heterogeneity across economists, including evidence that independent economists make more radical forecasts

Then, there is research from the London School of Economics Interest Rate Forecasts: A Pathology

In this paper we have demonstrated that, in the two countries and short data periods studied, the forecasts of interest rates had little or no informational value when the horizon exceeded two quarters (six months), though they were good in the next quarter and reasonable in the second quarter out. Moreover, all the forecasts were ex post and, systematically, inefficient, underestimating (overestimating) future outturns during up (down) cycle phases. The main reason for this is that forecasters cannot predict the timing of cyclical turning points, and hence predict future developments as a convex combination of autoregressive momentum and a reversion to equilibrium

Also, the Chapter in the Handbook of Forecasting Forecasting interest rates is relevant, although highly theoretical.

Hedging Interest Rate Risk

As if in validation of this basic finding – beyond about two quarters, interest rate forecasts generally do not beat a random walk forecast – interest rate swaps, are the largest category of interest rate contracts of derivatives, according to the Bank of International Settlements (BIS).


Not only that, but interest rate contracts generally are, by an order of magnitude, the largest category of OTC derivatives – totaling more than a half a quadrillion dollars as of the BIS survey in July 2013.

The gross value of these contracts was only somewhat less than the Gross Domestic Product (GDP) of the US.

A Bank of International Settlements background document defines “gross market values” as follows;

Gross positive and negative market values: Gross market values are defined as the sums of the absolute values of all open contracts with either positive or negative replacement values evaluated at market prices prevailing on the reporting date. Thus, the gross positive market value of a dealer’s outstanding contracts is the sum of the replacement values of all contracts that are in a current gain position to the reporter at current market prices (and therefore, if they were settled immediately, would represent claims on counterparties). The gross negative market value is the sum of the values of all contracts that have a negative value on the reporting date (ie those that are in a current loss position and therefore, if they were settled immediately, would represent liabilities of the dealer to its counterparties).  The term “gross” indicates that contracts with positive and negative replacement values with the same counterparty are not netted. Nor are the sums of positive and negative contract values within a market risk category such as foreign exchange contracts, interest rate contracts, equities and commodities set off against one another. As stated above, gross market values supply information about the potential scale of market risk in derivatives transactions. Furthermore, gross market value at current market prices provides a measure of economic significance that is readily comparable across markets and products.

Clearly, by any account, large sums of money and considerable exposure are tied up in interest rate contracts in the over the counter (OTC) market.

A Final Thought

This link between forecastability and financial derivatives is interesting. There is no question but that, in practical terms, business is putting eggs in the basket of managing interest rate risk, as opposed to refining forecasts – which may not be possible beyond a certain point, in any case.

What is going to happen when the quantitative easing maneuvers of central banks around the world cease, as they must, and long term interest rates rise in a consistent fashion? That’s probably where to put the forecasting money.

Credit Spreads As Predictors of Real-Time Economic Activity

Several distinguished macroeconomic researchers, including Ben Bernanke, highlight the predictive power of the “paper-bill” spread.

The following graphs, from a 1993 article by Benjamin M. Friedman and Kenneth N. Kuttner, show the promise of credit spreads in forecasting recessions – indicated by the shaded blocks in the charts.


Credit spreads, of course, are the differences in yields between various corporate debt instruments and government securities of comparable maturity.

The classic credit spread illustrated above is the difference between six-month commercial paper rates and 6 month Treasury bill rates.

Recent Research

More recent research underlines the importance of building up credit spreads from metrics relating to individual corporate bonds , rather than a mishmash of bonds with different duration, credit risk and other characteristics.

Credit Spreads as Predictors of Real-Time Economic Activity: A Bayesian Model-Averaging Approach is key research in this regard.

The authors first note that,

the “paper-bill” spread—the difference between yields on nonfinancial commercial paper and comparable-maturity Treasury bills—had substantial forecasting power for economic activity during the 1970s and the 1980s, but its predictive ability vanished in the subsequent decade

They then acknowledge that credit spreads based on indexes of speculative-grade or “junk” corporate bonds work fairly well for the 1990s, but their performance is uneven.

Accordingly, Faust, Gilchrist, Wright, and Zakrajsek (GYZ) write that

In part to address these problems, GYZ constructed 20 monthly credit spread indexes for different maturity and credit risk categories using secondary market prices of individual senior unsecured corporate bonds.. [measuring]..the underlying credit risk by the issuer’s expected default frequency (EDF™), a market-based default-risk indicator calculated by Moody’s/KMV that is more timely that the issuer’s credit rating]

Their findings indicate that these credit spread indexes have substantial predictive power, at both short- and longer-term horizons, for the growth of payroll employment and industrial production. Moreover, they significantly outperform the predictive ability of the standard default-risk indicators, a result that suggests that using “cleaner” measures of credit spreads may, indeed, lead to more accurate forecasts of economic activity.

Their research applies credit spreads constructed from the ground up, as it were, to out-of-sample forecasts of

…real economic activity, as measured by real GDP, real personal consumption expenditures (PCE), real business fixed investment, industrial production, private payroll employment, the civilian unemployment rate, real exports, and real imports over the period from 1986:Q1 to 2011:Q3. All of these series are in quarter-over-quarter growth rates (actually 400 times log first differences), except for the unemployment rate, which is simply in first differences

The results are forecasts which significantly beat univariate (autoregressive) model forecass, as shown in the following table.


Here BMA is an abbreviation for Bayesian Model Averaging, the author’s method of incorporating these calculated credit spreads in predictive relationships.

Additional research validates the usefulness of credit spreads so constructed for predicting macroeconomic dynamics in several European economies –

We find that credit spreads and excess bond premiums, when used alongside monetary policy tightness indicators and leading indicators of economic performance, are highly significant for predicting the growth in the index of industrial production, employment growth, the unemployment rate and real GDP growth at horizons ranging from one quarter to two years ahead. These results are confirmed for individual countries in the euroarea and for the United Kingdom, and are robust to different measures of the credit spread. It is the unpredictable part associated with the excess bond premium that has greater influence on real activity compared to the predictable part of the credit spread. The implications of our results are that careful selection of the bonds used to construct the credit spreads, excluding those with embedded options and or illiquid secondary markets, delivers a robust indicator of financial market tightness that is distinct from tightness due to monetary policy measures or leading indicators of economic activity.

The Situation Today

A Morgan Stanley Credit Report for fixed income, released March 21, 2014, notes that

Spreads in both IG and HY are at the lowest levels we have seen since 2007, roughly 110bp for IG and 415bp for HY. A question we are commonly asked is how much tighter can spreads go in this cycle

So this is definitely something to watch. 

Interest Rates – 3

Can interest rates be nonstationary?

This seems like a strange question, since interest rates are bounded, except in circumstances, perhaps, of total economic collapse.

“Standard” nonstationary processes, by contrast, can increase or decrease without limit, as can conventional random walks.

But, be careful. It’s mathematically possible to define and study random walks with reflecting barriers –which, when they reach a maximum or minimum, “bounce” back from the barrier.

This is more than esoteric, since the 30 year fixed mortgage rate monthly averages series discussed in the previous post has a curious property. It can be differenced many times, and yet display first order autocorrelation of the resulting series.

This contrasts with the 10 year fixed maturity Treasury bond rates (also monthly averages). After first differencing this Treasury bond series, the resulting residuals do not show statistically significant first order autocorrelation.

Here a stationary stochastic process is one in which the probability distribution of the outcomes does not shift with time, so the conditional mean and conditional variance are, in the strict case, constant. A classic example is white noise, where each element can be viewed as an independent draw from a Gaussian distribution with zero mean and constant variance.

30 Year Fixed Mortgage Monthly Averages – a Nonstationary Time Series?

Here are some autocorrelation functions (ACF’s) and partial autocorrelation functions (PACF’s) of the 30 year fixed mortgage monthly averages from April 1971 to January 2014, first differences of this series, and second differences of this series – altogether six charts produced by MATLAB’s plot routines.

Data for this and the following series are downloaded from the St. Louis Fed FRED site.


Here the PACF appears to cut off after 4 periods, but maybe not quite, since there are values for lags which touch the statistical significance boundary further out.


This seems more satisfactory, since there is only one major spike in the ACF and 2-3 initial spikes in the PACF. Again, however, values for lags far out on the horizontal axis appear to touch the boundary of statistical significance.


Here are the ACF and PACF’s of the “difference of the first difference” or the second difference, if you like. This spike at period 2 for the ACF and PACF is intriguing, and, for me, difficult to interpret.

The data series includes 514 values, so we are not dealing with a small sample in conventional terms.

I also checked for seasonal variation – either additive or multiplicative seasonal components or factors. After taking steps to remove this type of variation, if it exists, the same pattern of repeated significance of autocorrelations of differences and higher order differences persists.

Forecast Pro, a good business workhorse for automatic forecasting, selects ARIMA(0,1,1) as the optimal forecast model for this 30 year fixed interest mortgage monthly averages. In other words, Forecast Pro glosses over the fact that the residuals from an ARIMA(0,1,1) setup still contain significant autocorrelation.

Here is a sample of the output (click to enlarge)


10 Year Treasury Bonds Constant Maturity

The situation is quite different for 10 year Treasury Bonds monthly averages, where the downloaded series starts April 1953 and, again, ends January 2014.

Here is the ordinary least squares (OLS) regression of the first order autocorrelation.

10yrTreasregHere the R2 or coefficient of determination is much lower than for the 30 year fixed mortgage monthly averages, but the first order lagged rate is highly significant statistically.

On the other hand, the residuals of this regression do not exhibit a high degree of first order autocorrelation, falling below the 80 percent significance level.

What Does This Mean?

The closest I have come to formulating an explanation for this weird difference between these two “interest rates” is the discussion in a paper from 2002 –

On Mean Reversion in Real Interest Rates: An Application of Threshold Cointegration

The authors of this research paper from the Institute for Advanced Studies in Vienna acknowledge findings that some interests rates may be nonstationary, at least over some periods of time. Their solution is a nonlinear time series approach, but they highlight several of the more exotic statistical features of interest rates in passing – such as evidence of non-normal distributions, excess kurtosis, conditional heteroskedasticity, and long memory.

In any case, I wonder whether the 30 year fixed mortgage monthly averages might be suitable for some type of boosting model working on residuals and residuals of residuals.

I’m going to try that later on this Spring.

Interest Rates – 2

I’ve been looking at forecasting interest rates, the accuracy of interest rate forecasts, and teasing out predictive information from the yield curve.

This literature can be intensely theoretical and statistically demanding. But it might be quickly summarized by saying that, for horizons of more than a few months, most forecasts (such as from the Wall Street Journal’s Panel of Economists) do not beat a random walk forecast.

At the same time, there are hints that improvements on a random walk forecast might be possible under special circumstances, or for periods of time.

For example, suppose we attempt to forecast the 30 year fixed mortgage rate monthly averages, picking a six month forecast horizon.

The following chart compares a random walk forecast with an autoregressive (AR) model.


Let’s dwell for a moment on some of the underlying details of the data and forecast models.

The thick red line is the 30 year fixed mortgage rate for the prediction period which extends from 2007 to the most recent monthly average in 2014 in January 2014. These mortgage rates are downloaded from the St. Louis Fed data site FRED.

This is, incidentally, an out-of-sample period, as the autoregressive model is estimated over data beginning in April 1971 and ending September 2007. The autoregressive model is simple, employing a single explanatory variable, which is the 30 year fixed rate at a lag of six months. It has the following form,

rt = k + βrt-6

where the constant term k and the coefficient β of the lagged rate rt-6 are estimated by ordinary least squares (OLS).

The random walk model forecast, as always, is the most current value projected ahead however many periods there are in the forecast horizon. This works out to using the value of the 30 year fixed mortgage in any month as the best forecast of the rate that will obtain six months in the future.

Finally, the errors for the random walk and autoregressive models are calculated as the forecast minus the actual value.

When an Autoregressive Model Beats a Random Walk Forecast

The random walk errors are smaller in absolute value than the autoregressive model errors over most of this out-of-sample period, but there are times when this is not true, as shown in the graph below.


This chart itself suggests that further work could be done on optimizing the autoregressive model, perhaps by adding further corrections from the residuals, which themselves are autocorrelated.

However, just taking this at face value, it’s clear the AR model beats the random walk forecast when the direction of interest rates changes from a downward movement.

Does this mean that going forward, an AR model, probably considerably more sophisticated than developed for this exercise, could beat a random walk forecast over six month forecast horizons?

That’s an interesting and bankable question. It of course depends on the rate at which the Fed “withdraws the punch bowl” but it’s also clear the Fed is no longer in complete control in this situation. The markets themselves will develop a dynamic based on expectations and so forth.

In closing, for reference, I include a longer picture of the 30 year fixed mortgage rates, which as can be seen, resemble the whole spectrum of rates in having a peak in the early 1980’s and showing what amounts to trends before and after that.


Interest Rates – 1

Let’s focus on forecasting interest rates.

The first question, of course, is “which interest rate”?

So, there is a range of interest rates from short term rates to rates on longer term loans and bonds. The St. Louis Fed data service FRED lists 719 series under “interest rates.”

Interest rates, however, tend to move together over time, as this chart on the bank prime rate of interest and the federal funds rate shows.


There’s a lot in this chart.

There is the surge in interest rates at the beginning of the 1980’s. The prime rate rocketed to more than 20 percent, or, in the words of the German Chancellor at the time higher “than any year since the time of Jesus Christ.” This ramp-up in interest rates followed actions of the US Federal Reserve Bank under Paul Volcker – extreme and successful tactics to break the back of inflation running at a faster and faster pace in the 1970’s.

Recessions are indicated on this graph with shaded areas.

Also, almost every recession in this more than fifty year period is preceded by a spike in the federal funds rate – the rate under the control of or targeted by the central bank.

Another feature of this chart is the federal funds rate is almost always less than the prime rate, often by several percentages.

This makes sense because the federal funds rate is a very short term interest rate – on overnight loans by depository institutions in surplus at the Federal Reserve to banks in deficit at the end of the business day – surplus and deficit with respect to the reserve requirement.

The interest rate the borrowing bank pays the lending bank is negotiated, and the weighted average across all such transactions is the federal funds effective rate. This “effective rate” is subject to targets set by the Federal Reserve Open Market Committee. Fed open market operations influence the supply of money to bring the federal funds effective rate in line with the federal funds target rate.

The prime rate, on the other hand, is the underlying index for most credit cards, home equity loans and lines of credit, auto loans, and personal loans. Many small business loans are also indexed to the prime rate. The term of these loans is typically longer than “overnight,” i.e. the prime rate applies to longer term loans.

The Yield Curve

The relationship between interest rates on shorter term and longer term loans and bonds is a kind of predictive relationship. It is summarized in the yield curve.

The US Treasury maintains a page Daily Treasury Yield Curve Rates which documents the yield on a security to its time to maturity .. based on the closing market bid yields on actively traded Treasury securities in the over-the-counter market.

The current yield curve is shown by the blue line in the chart below, and can be contrasted with a yield curve seven years previously, prior to the financial crisis of 2008-09 shown by the red line.


Treasury notes on this curve report that –

These market yields are calculated from composites of quotations obtained by the Federal Reserve Bank of New York. The yield values are read from the yield curve at fixed maturities, currently 1, 3 and 6 months and 1, 2, 3, 5, 7, 10, 20, and 30 years. This method provides a yield for a 10 year maturity, for example, even if no outstanding security has exactly 10 years remaining to maturity.

Short term yields are typically less than longer term yields because there is an opportunity cost in tying up money for longer periods.

However, on occasion, there is an inversion of the yield curve, as shown for March 21, 2007 in the chart.

Inversion of the yield curve is often a sign of oncoming recession – although even the Fed authorities, who had some hand in causing the increase in the short term rates at the time, appeared clueless about what was coming in Spring 2007.

Current Prospects for Interest Rates

Globally, we have experienced an extraordinary period of low interest rates with short term rates hovering just at the zero bound. Clearly, this cannot go on forever, so the longer term outlook is for interest rates of all sorts to rise.

The Survey of Professional Forecasters develops consensus forecasts of key macroeconomic indicators, such as interest rates.

The latest survey, from the first quarter of 2014, includes the following consensus projections for the 3-month Treasury bill and the 10-year Treasury bond rates.

SPFforecast has short articles predicting mortgage rates, car loans, credit card rates, and bonds over the next year or two. Mortgage rates might rise to 5 percent by the end of 2014, but that is predicated on a strong recovery in the economy, according to this site.

As anyone participating in modern civilization knows, a great deal depends on the actions of the US Federal Reserve bank. Currently, the Fed influences both short and longer term interest rates. Short term rates are keyed closely to the federal funds rate. Longer term rates are influenced by Fed Quantitative Easing (QE) programs of bond-buying. The Fed’s bond buying is scheduled to be cut back step-by-step (“tapering”) about $10 billion per month.

Actions of the Bank of Japan and the European central bank in Frankfurt also bear on global prospects and impacts of higher interest rates.

Interest rates, however, are not wholly controlled by central banks. Capital markets have a dynamic all their own, which makes forecasting interest rates an increasingly relevant topic.

And Now – David Stockman

David Stockman, according to his new website Contra Corner,

is the ultimate Washington insider turned iconoclast. He began his career in Washington as a young man and quickly rose through the ranks of the Republican Party to become the Director of the Office of Management and Budget under President Ronald Reagan. After leaving the White House, Stockman had a 20-year career on Wall Street.

Currently, Stockman takes the contrarian view that the US Federal Reserve Bank is feeding a giant bubble which is bound to collapse

He states his opinions with humor and wit, as some of article titles on Contra Corner indicate –

Fed’s Taper Kabuki is Farce; Gong Show of Cacophony, Confusion and Calamity Coming


General John McCain Strikes Again!

Forecasting the Price of Gold – 3

Ukraine developments and other counter-currents, such as Janet Yellen’s recent comments, highlight my final topic on gold price forecasting – multivariate gold price forecasting models.

On the one hand, there has been increasing uncertainty as a result of Ukrainian turmoil, counterbalanced today by the reaction to the seemingly hawkish comments by Chairperson Janet Yellen of the US Federal Reserve Bank.


Traditionally, gold is considered a hedge against uncertainty. Indulge your imagination and it’s not hard to conjure up scary scenarios in the Ukraine. On the other hand, some interpret Yellen as signaling an earlier move to moving the Federal funds rate off zero, increasing interest rates, and, in the eyes of the market, making gold more expensive to hold.

Multivariate Forecasting Models of Gold Price – Some Considerations

It’s this zoo of factors and influences that you have to enter, if you want to try to forecast the price of gold in the short or longer term.

Variables to consider include inflation, exchange rates, gold lease rates, interest rates, stock market levels and volatility, and political uncertainty.

A lot of effort has been devoted to proving or attempting to question that gold is a hedge against inflation.

The bottom line appears to be that gold prices rise with inflation – over a matter of decades, but in shorter time periods, intervening factors can drive the real price of gold substantially away from a constant relationship to the overall price level.

Real (and possibly nominal) interest rates are a significant influence on gold prices in shorter time periods, but this relationship is complex. My reading of the literature suggests a better understanding of the supply side of the picture is probably necessary to bring all this into focus.

The Goldman Sachs Global Economics Paper 183 – Forecasting Gold as a Commodity – focuses on the supply side with charts such as the following –


The story here is that gold mine production responds to real interest rates, and thus the semi-periodic fluctuations in real interest rates are linked with a cycle of growth in gold production.

The Goldman Sachs Paper 183 suggests that higher real interest rates speed extraction, since the opportunity cost of leaving ore deposits in the ground increases. This is indeed the flip side of the negative impact of real interest rates on investment.

And, as noted in an earlier post,the Goldman Sachs forecast in 2010 proved prescient. Real interest rates have remained low since that time, and gold prices drifted down from higher levels at the end of the last decade.


Elasticities of response in a regression relationship show how percentage changes in the dependent variable – gold prices in this case – respond to percentage changes in, for example, the price level.

For gold to be an effective hedge against inflation, the elasticity of gold price with respect to changes in the price level should be approximately equal to 1.

This appears to be a credible elasticity for the United States, based on two studies conducted with different time spans of gold price data.

These studies are Gold as an Inflation Hedge? and the more recent Does Gold Act As An Inflation Hedge in the US and Japan. Also, a Gold Council report, Short-run and long-run determinants of the price of gold, develops a competent analysis.

These studies explore the cointegration of gold prices and inflation. Cointegration of unit root time series is an alternative to first differencing to reduce such time series to stationarity.

Thus, it’s not hard to show strong evidence that standard gold price series are one type or another of a random walk. Accordingly, straight-forward regression analysis of such series can easily lead to spurious correlation.

You might, for example, regress the price of gold onto some metric of the cumulative activity of an amoeba (characterized by Brownian motion) and come up with t-statistics that are, apparently, statistically significant. But that would, of course, be nonsense, and the relationship could evaporate with subsequent movements of either series.

So, the better research always gives consideration to the question of whether the variables in the models are, first of all, nonstationary OR whether there are cointegrated relationships.

While I am on the topic literature, I have to recommend looking at Theories of Gold Price Movements: Common Wisdom or Myths? This appears in the Wesleyan University Undergraduate Economic Review and makes for lively reading.

Thus, instead of viewing gold as a special asset, the authors suggest it is more reasonable to view gold as another currency, whose value is a reflection of the value of U.S. dollar.

The authors consider and reject a variety of hypotheses – such as the safe haven or consumer fear motivation to hold gold. They find a very significant relationship between the price movement of gold, real interest rates and the exchange rate, suggesting a close relationship between gold and the value of U.S. dollar. The multiple linear regressions verify these findings.

The Bottom Line

Over relatively long time periods – one to several decades – the price of gold moves more or less in concert with measures of the price level. In the shorter term, forecasting faces serious challenges, although there is a literature on the multivariate prediction of gold prices.

One prediction, however, seems reasonable on the basis of this review. Real interest rates should rise as the US Federal Reserve backs off from quantitative easing and other central banks around the world follow suit. Thus, increases in real interest rates seem likely at some point in the next few years. This seems to indicate that gold mining will strive to increase output, and perhaps that gold mining stocks might be a play.

Russia and Energy – Some Geopolitics

A couple of charts highlight the dominant position Russia holds with respect to energy, and, specifically, specifically, natural gas production.

First, there is this trade graphic from the BP Statistical Review of World Energy 2013.


Clearly, Russia has dominant global position in natural gas trades.

The Europeans are primary consumers for Russian natural gas, and there are some significant dependencies, as this graphic shows.


So Russia’s position as a major energy supplier no doubt is operating as a constraint on sanctions for the annexation of Crimea.

On the other  hand, this is a mutual dependency. The US Energy Information Agency, for example, reports that oil and gas revenues accounted for 52% of federal budget revenues and over 70% of total exports in 2012.

Forecasting the Price of Gold – 2

Searching “forecasting gold prices” on Google lands on a number of ARIMA (autoregressive integrated moving average) models of gold prices. Ideally, researchers focus on shorter term forecast horizons with this type of time series model.

I take a look at this approach here, moving onto multivariate approaches in subsequent posts.

Stylized Facts

These ARIMA models support stylized facts about gold prices such as: (1) gold prices constitute a nonstationary time series, (2) first differencing can reduce gold price time series to a stationary process, and, usually, (3) gold prices are random walks.

For example, consider daily gold prices from 1978 to the present.


This chart, based World Gold Council data and the London PM fix, shows gold prices do not fluctuate about a fixed level, but can move in patterns with a marked trend over several years.

The trick is to reduce such series to a mean stationary series through appropriate differencing and, perhaps, other data transformations, such as detrending and taking out seasonal variation. Guidance in this is provided by tools such as the autocorrelation function (ACF) and partial autocorrelation function (PACF) of the time series, as well as tests for unit roots.

Some Terminology

I want to talk about specific ARIMA models, such as ARIMA(0,1,1) or ARIMA(p,d,q), so it might be a good idea to review what this means.

Quickly, ARIMA models are described by three parameters: (1) the autoregressive parameter p, (2) the number of times d the time series needs to be differenced to reduce it to a mean stationary series, and (3) the moving average parameter q.

ARIMA(0,1,1) indicates a model where the original time series yt is differenced once (d=1), and which has one lagged moving average term.

If the original time series is yt, t=1,2,..n, the first differenced series is zt=yt-yt-1, and an ARIMA(0,1,1) model looks like,

zt = θ1εt-1

or converting back into the original series yt,

yt = μ + yt-1 + θ1εt-1

This is a random walk process with a drift term μ, incidentally.

As a note in the general case, the p and q parameters describe the span of the lags and moving average terms in the model.  This is often done with backshift operators Lk (click to enlarge)  


So you could have a sum of these backshift operators of different orders operating against yt or zt to generate a series of lags of order p. Similarly a sum of backshift operators of order q can operate against the error terms at various times. This supposedly provides a compact way of representing the general model with p lags and q moving average terms.

Similar terminology can indicate the nature of seasonality, when that is operative in a time series.

These parameters are determined by considering the autocorrelation function ACF and partial autocorrelation function PACF, as well as tests for unit roots.

I’ve seen this referred to as “reading the tea leaves.”

Gold Price ARIMA models

I’ve looked over several papers on ARIMA models for gold prices, and conducted my own analysis.

My research confirms that the ACF and PACF indicates gold prices (of course, always defined as from some data source and for some trading frequency) are, in fact, random walks.

So this means that we can take, for example, the recent research of Dr. M. Massarrat Ali Khan of College of Computer Science and Information System, Institute of Business Management, Korangi Creek, Karachi as representative in developing an ARIMA model to forecast gold prices.

Dr. Massarrat’s analysis uses daily London PM fix data from January 02, 2003 to March 1, 2012, concluding that an ARIMA(0,1,1) has the best forecasting performance. This research also applies unit root tests to verify that the daily gold price series is stationary, after first differencing. Significantly, an ARIMA(1,1,0) model produced roughly similar, but somewhat inferior forecasts.

I think some of the other attempts at ARIMA analysis of gold price time series illustrate various modeling problems.

For example there is the classic over-reach of research by Australian researchers in An overview of global gold market and gold price forecasting. These academics identify the nonstationarity of gold prices, but attempt a ten year forecast, based on a modeling approach that incorporates jumps as well as standard ARIMA structure.

A new model proposed a trend stationary process to solve the nonstationary problems in previous models. The advantage of this model is that it includes the jump and dip components into the model as parameters. The behaviour of historical commodities prices includes three differ- ent components: long-term reversion, diffusion and jump/dip diffusion. The proposed model was validated with historical gold prices. The model was then applied to forecast the gold price for the next 10 years. The results indicated that, assuming the current price jump initiated in 2007 behaves in the same manner as that experienced in 1978, the gold price would stay abnormally high up to the end of 2014. After that, the price would revert to the long-term trend until 2018.

As the introductory graph shows, this forecast issued in 2009 or 2010 was massively wrong, since gold prices slumped significantly after about 2012.

So much for long-term forecasts based on univariate time series.

Summing Up

I have not referenced many ARIMA forecasting papers relating to gold price I have seen, but focused on a couple – one which “gets it right” and another which makes a heroically wrong but interesting ten year forecast.

Gold prices appear to be random walks in many frequencies – daily, monthly average, and so forth.

Attempts at superimposing long term trends or even jump patterns seem destined to failure.

However, multivariate modeling approaches, when carefully implemented, may offer some hope of disentangling longer term trends and changes in volatility. I’m working on that post now.