Category Archives: technology forecasting

Video Friday – Fracking

Here is Brian Ellis from Michigan University Engineering with a look at the technology of hydraulic fracturing (fracking) and horizontal drilling – the innovations that recently pushed US oil production near the 10 million barrel per day mark.

I’m putting this up, rather than other, often excellent film clips showing people lighting water from their kitchen taps because the scale of shale oil and gas production has become so large. There really is a huge tradeoff between current employment and business activity and long term environmental effects.

Price, rather than environmental concerns, are likely to be the crucial factor in any scaleback.

At the same time, there is the possibility of further technical advance in the US shale oil and gas technologies, advances which may push extraction prices lower, giving the industry a longer lease during what may be a year or more of lower oil prices.

Fracking and its possible dynamics are critical to a lot of business activity and, thus, forecasting in the US.

Video Friday on Steroids

Here is a list of the URL’s for all the YouTube and other videos shown on this blog from January 2014 through May of this year. I encourage you to shop this list, clicking on the links. There’s a lot of good stuff, including several  instructional videos on machine learning and other technical topics, a series on robotics, and several videos on climate and climate change.

January 2014

The Polar Vortex Explained in Two Minutes

https://www.youtube.com/watch?v=5eDTzV6a9F4

NASA – Six Decades of a Warming Earth

https://www.youtube.com/watch?v=gaJJtS_WDmI

“CHASING ICE” captures largest video calving of glacier

https://www.youtube.com/watch?v=hC3VTgIPoGU

Machine Learning and Econometrics

https://www.youtube.com/watch?v=EraG-2p9VuE

Can Crime Prediction Software Stop Criminals?

https://www.youtube.com/watch?v=s1-pbJKA3H8

Analytics 2013 – Day 1

https://www.youtube.com/watch?v=LsyOLBroVx4

The birth of a salesman

https://www.youtube.com/watch?v=pWM1dR_V7uw

Economies Improve

https://www.youtube.com/watch?v=5_DeCMIig_M

Kaggle – Energy Applications for Machine Learning

https://www.youtube.com/watch?v=mZZFXTUz-nI

2014 Outlook with Jan Hatzius

https://www.youtube.com/watch?v=Ggv0oC8L3Tk

Nassim Taleb Lectures at the NSF

https://www.youtube.com/watch?v=omsYJBMoIJU

Vernon Smith – Experimental Markets

https://www.youtube.com/watch?v=Uncl-wRfoK8

 

 

Forecast Pro – Quick Tour

https://www.youtube.com/watch?v=s8jMp5qS8v4

February 2014

Stephen Wolfram’s Introduction to the Wolfram Language

https://www.youtube.com/watch?v=_P9HqHVPeik

Tornados

https://www.youtube.com/watch?v=TEGhgsiNFJ4

Econometrics – Quantile Regression

https://www.youtube.com/watch?v=P9lMmEkXuBw

Quantile Regression Example

https://www.youtube.com/watch?v=qrriFC_WGj8

Brooklyn Grange – A New York Growing Season

http://vimeo.com/86266334

Getting in Shape for the Sport of Data Science

https://www.youtube.com/watch?v=kwt6XEh7U3g

Machine Learning – Decision Trees

https://www.youtube.com/watch?v=-dCtJjlEEgM

Machine Learning – Random Forests

https://www.youtube.com/watch?v=3kYujfDgmNk

Machine Learning – Random Forecasts Applications

https://www.youtube.com/watch?v=zFGPjRPwyFw

Malcolm Gladwell on the 10,000 Hour Rule

https://www.youtube.com/watch?v=XS5EsTc_-2Q

Sornette Talk

https://www.youtube.com/watch?v=Eomb_vbgvpk

Head of India Central Bank Interview

https://www.youtube.com/watch?v=BrVzema7pWE

March 2014

David Stockman

https://www.youtube.com/watch?v=DI718wFmReo

Partial Least Squares Regression

https://www.youtube.com/watch?v=WKEGhyFx0Dg

April 2014

Thomas Piketty on Economic Inequality

https://www.youtube.com/watch?v=qp3AaI5bWPQ

Bonobo builds a fire and tastes marshmellows

https://www.youtube.com/watch?v=GQcN7lHSD5Y

Future Technology

https://www.youtube.com/watch?v=JbQeABIoO6A

May 2014

Ray Kurzweil: The Coming Singularity

https://www.youtube.com/watch?v=1uIzS1uCOcE

Paul Root Wolpe: Kurzweil Critique

https://www.youtube.com/watch?v=qRgMTjTMovc

The Future of Robotics and Artificial Intelligence

https://www.youtube.com/watch?v=AY4ajbu_G3k

Car Factory – KIA Sportage Assembly Line

https://www.youtube.com/watch?v=sjAZGUcjrP8

10 Most Popular Applications for Robots

https://www.youtube.com/watch?v=fH4VwTgfyrQ

Predator Drones

https://www.youtube.com/watch?v=nMh8Cjnzen8

The Future of Robotic Warfare

https://www.youtube.com/watch?v=_atffUtxXtk

Bionic Kangaroo

https://www.youtube.com/watch?v=HUxQM0O7LpQ

Ping Pong Playing Robot

https://www.youtube.com/watch?v=tIIJME8-au8

Baxter, the Industrial Robot

https://www.youtube.com/watch?v=ukehzvP9lqg

Bootstrapping

https://www.youtube.com/watch?v=1OC9ul-1PVg

Links – Labor Day Weekend

Tech

Amazon’s Cloud Is So Pervasive, Even Apple Uses It

Your iCloud storage is apparently on Amazon.

Amazon’s Cloud Is The Fastest Growing Software Business In History

AWS

AWS is Amazon Web Services. The author discounts Google growth, since it is primarily a result of selling advertising. 

How Microsoft and Apple’s Ads Define Their Strategy

Microsoft approaches the market from the top down, while Apple goes after the market from the bottom up.

Mathematical Predictions for the iPhone 6

Can you predict features of the iPhone6 scheduled to be released September 6?

iphoneplot

Predictive Analytics

Comparison of statistical software

Good links for R, Matlab, SAS, Stata, and SPSS.

Types and Uses of Predictive Analytics, What they are and Where You Can Put Them to Work

Gartner says that predictive analytics is a mature technology yet only one company in eight is currently utilizing this ability to predict the future of sales, finance, production, and virtually every other area of the enterprise. What is the promise of predictive analytics and what exactly are they [types and uses of predictive analytics]? Good highlighting of main uses of predictive analytics in companies.

The Four Traps of Predictive Analytics

Magical thinking/ Starting at the Top/ Building Cottages, not Factories/ Seeking Purified Data. Good discussion. This short article in the Sloan Management Review is spot on, in my opinion. The way to develop good predictive analytics is to pick an area, indeed, pick the “low-handing fruit.” Develop workable applications, use them, improve them, broaden the scope. The “throw everything including the kitchen sink” approach of some early Big Data deployments is almost bound to fail. Flashy, trendy, but, in the final analysis, using “exhaust data” to come up with obscure customer metrics probably will not cut in the longer run.

Economic Issues

The Secular Stagnation Controversy

– discusses the e-book Secular Stagnation: Facts, Causes and Cures. The blogger Timothy Taylor points out that “secular” here has no relationship to lacking a religious context, but refers to the idea that market economies, or, if you like, capitalist economies, can experience long periods (decade or more) of desultory economic growth. Check the e-book for Larry Summer’s latest take on the secular stagnation hypothesis.

Here’s how much aid the US wants to send foreign countries in 2015, and why (INFOGRAPHIC

foreignaid

2020 and 2030 – Forecasts and Projections

I’d like to establish a context for discussing longer term forecasts, in this case to 2020 and 2030.

So, just below, I give you my take on 1990-2005. A lot happened that was unanticipated at the beginning of this period. One should expect, I think, the same to be true for 2015-2030.

Along those lines, I also suggest Big Picture factors that may come into play over the next fifteen or so years.

In coming posts, I want to summarize forecasts and projections I have seen for this period.

And I’m a little unusual in the technical forecasting community, since I’m equipped to do matrix programming, discuss boosting and bagging and so forth, and, on the other side of the aisle, weave together these stories and scenarios about process, causes, and factors. The quantitative is usually where I get paid, but, at the same time, I think it is easy to underestimate the benefit of trying to keep track of the Big Picture, the global dynamics, the political economy, and so forth.

1990-2005

The 1990’s rolled out with a nasty little recession in 1991 and voters throwing the first George Bush out of office, in favor of a clarinet-playing former Governor of Arkansas with a penchant for the ladies. Then, the United States experienced the longest period of economic prosperity since the 1960’s, fueled by the tech revolution and rise of the Internet. The breakup of the Soviet Union became official with democratic forms struggling to take root in Russia and former Soviet Republics. The US defense budget was cut about 40 percent from 1980 levels. Deregulation became a theme, and deregulation of telecoms led to burgeoning investments in telecom systems. The end of the decade saw the absurd Y2K problem, where details of computer clocks were supposed to stop everything at midnight, the turn of the century.

The New Millennium saw another recession in 2001, which was particularly sharp for the tech industry. Another Bush took the Presidency, after the Supreme Court intervened in the disputed General Election. Then there was 9/11 – September 11, 2001, with the destruction of the World Trade Center by large airliners being flown into the upper stories. This was a pivotal event. There was immediate surge in the military budget and in US military action in Afghanistan and then the invasion of Iraq, putatively because Saddam Hussein possessed “weapons of mass destruction.”

The US economy pretty much languished after the 2001-2002 recession, being stimulated to an extent by the rise in the defense budget, then by housing activity triggered by continued lowering of interest rates by the US Federal Reserve Bank under the redoubtable Alan Greenspan.

Another development that became especially noticeable after 2000 was the rise of China as a manufacturing and export power. The construction of the Shanghai skyline from the late 1990’s to the middle of the last decade was nothing less than stupendous.

The Importance of Technical Change

So what is important over a span of time? Are there underlying determinants?

I’ve got to believe technical change is an important element in historical process. If we take the fifteen year period sketched above, for example, a lot of the story is driven, at some level, by technical developments, especially in information technology (IT).

My favorite explanation of the collapse of the Soviet Union, for example, includes Silicon Valley as a key driver. The Soviet planned economy was a huge lumbering machine, compared to the nimble, change-oriented shops in the Valley, innovating new computer setups every few months. One immediate consequence was the US fighter aircraft came to totally dominate the old MIG planes, with their electronically guided missiles and tracking systems.

And to go on in this vein, focusing on the rise of US tech and then the movement of production to China is a strategic process for understanding the past couple of decades.

Big Picture Factors

Suffice it to say – new technology will be as much a driver of change in the next fifteen years, as it has been over the past fifteen.

Indeed, according to the futurist Ray Kurzweil, something called The Singularity stalks the human future. Perhaps around 2045, somewhat outside our forecast horizon in this discussion, technology will converge to completely outperform human intelligence. Commentators ranging from Stanislaus Ulam to Kurzweil believe that it is impossible to project human history beyond this point – hence the name.

Conventionally, this will involve biotechnology, computer technology, and robotics – but also could involve nanotechnology.

In any case, hefty doses of new technology may be necessary just to keep on a level course. I’m thinking, for example, of the diminishing effectiveness of antibiotics. So we have the evolution of “superbugs,” as well as the emergence of new epidemics through mutation or disease vectors jumping species lines. Ebola is a particularly gruesome example.

And while on technology, it is fair to observe that complex technologies just at or beyond the boundary of human control present deep challenges. Deep-sea oil drilling and the Gulf of Mexico oil spill, under British Petroleum, and the Fukishima nuclear disaster, still leaking radioactivity into the Pacific, are two examples.

Population or more generally demography is another Big Picture factor. Populations are aging in the United States, Europe, and Japan, but also in China. And global population continues to grow, possibly by another billion by 2030.

Climate change is another Big Picture factor.

The global climate is a complex, dynamic system. There is lots of noise in the discussion and uncertainties, such as whether there may be a cooling interval, as carbon dioxide and methane concentrations continue to rise globally. A number of studies commissioned by US and other intelligence agencies, though, highlight the potential for massive impacts from, say, basic changes in monsoon patterns in South Asia.

In terms of geopolitics, I suspect the shift in the economic center of gravity to somewhere along the Asian rim is another Big Picture development.

There are many relevant metrics. The proportion of global output produced by the United States, according to the World Economic Outlook (WEO) of the International Monetary Fund (IMF), will continue to diminuish, as Chinese growth in the worst case is projected to exceed levels of economic growth in the US and, certainly, in Europe.

Then, there is the issue of the US being the policeman of the world. At some point, the cost of maintaining a global span of military bases and force readiness for multiple theatres of action will weigh heavily on the US – as one could argue is already happening to some degree.

Challenges to the global dominance of the US dollar can be predicted, also, in the next fifteen years.

Sustainability

Whether any of the above “Big Picture” factors actually come into play by 2020 or 2030 is, of course, a speculation. But I think the basic technique of long term forecasting is to inventory possible influences like these. Then, you construct scenarios.

One thing appears certain. And that is there will be surprises.

In looking at forecasts for the next five to fifteen years, I also want to give thought to sustainability. Are there institutions and arrangements which could offer a backup to the various types of instabilities which could emerge?

And there is apparently an increasing chance of an increase in the general level of warfare, perhaps with linking of action in various theatres. I have to say, too, that I am poorly equipped to comment on these conflicts, although, as they ramp up, I attempt to learn more about the players and underlying dynamics.

I’ll be using this venue as a scratch-pad to record the projections of others and some thoughts I might have in response vis a vis 2020 and 2030.

Video Friday – Ecommerce Trends

Trends for 2014

Matt made this at the end of 2013, but it hits the mark for what we are seeing this year. It’s only two minutes! Part of a series called ’Two Minute Tuesdays’, but of course we are showing it on a Friday.

But a lot of what you find on ecommerce is US-centric. This leads to the question –

Should We Be Afraid of Alibaba?

Alibaba is bigger than Amazon and eBay combined, leading to an alarmist Bloomberg article earlier this month Alibaba’s IPO May Herald the End of U.S. E-Commerce Dominance

Ecommerce Trends in China

This YouTube video is a test run of a talk given May 2014 in China, and contains some material at the beginning which I consider to be superfluous – biography of the presenter, etc. However, if you get beyond that, there are a lot of key stats presented in the slides and presentation. Valuable.

Will Online Retail Cannibalize Brick-and-mortar Sales?

Online retail or ecommerce is growing at three times the rate of retails sales generally (15 percent compared with 5 percent). And within online sales, mobile ecommerce is rocketing ahead by growth rates on the order of 25 percent per year in the US. Are these faster growing elements complementary to or cannibalizing conventional retail sales?

First, some stores – such as Blockbuster, Movie Gallery, Borders, and stores selling records and CD’s – are clearly casualties of Internet competition.

Other brick-and-mortar operations are following a multi-channel strategy, opening up online sales divisions parallel and in addition to their stores with goods on the shelves.

But the handwriting may be on the wall.

For one thing, in the 2013 holiday season, U.S. retailers saw approximately half the holiday foot traffic they experienced just three years ago.

And some of the foot traffic in brick-and-mortar stores is “showrooming” with practices highlighted in this infographic from Adweek (click to enlarge).

data-bargain-hunting-01-2013

And it’s significant a pure-play ecommerce provider like Amazon has risen to one of the ten largest retailers in the United States, with 2013 sales of $44 billion.

While Amazon is still back in the pack (see Table below), its annual growth rate is unsurpassed.

ranking

Bottom line – the “fulfillment center” may become a growing trend.

People like to see the product, especially if it is a larger ticket item.

Interestingly, Amazon is now opening fulfillment centers in key urban markets. Other formerly brick-and-mortar stores may repurpose some of their floor area to warehousing and fulfillment of customer orders.

Recognize, however, that we’re talking about $3-4 trillion in retail sales in the US, and the game on the ground is likely to change relatively slowly – over five or ten years.

Mobile e-commerce

Mobile ecommerce is no longer just another means consumers use to buy products online. It’s now the predominant way buyers visit ecommerce sites.

And mobile applications are radically changing the nature of shopping. The Emeritus founder of comScore, for example, highlights the two aspects of mobile ecommerce,

comScoreslide1

Examples of m-Shopping, according to Internet Retailer, include –

Online consumers use their smartphones and tablets for many shopping-related activities. In Q2 2013, 57% of smartphone users while in a retailer’s store visited that retailer’s site or app compared with 43% who consulted another company’s site or app, comScore says. The top reason consumers consulted retailers’ sites or apps was to compare prices. Among those smartphone users who went to the same retailer’s site, 59% wanted to see if there was an online discount available, the report says. Similarly, among those who checked a different retailer’s site, 92% wanted to see if they could get a better deal on price.

Smartphone owners also used their devices while in stores to take a picture of a product (23%), text or call family or friends about a product (17%), and send a picture of a product to family and friends (17%).

According to Gian Fulgoni, “m-Buying” is the predominant way shoppers now engage with retail brands online in the US.

comScore2

Growth, Adoption, and Use of Mobile E-Commerce explores patterns of mobile ecommerce with extensive data on eBay transactions.

One of the more interesting findings is that,

..adoption of the mobile shopping application is associated with both an immediate and sustained increase in total platform purchasing. The data also do not suggest that mobile application purchases are simply purchases that would have been made otherwise on the regular Internet platform.

The following chart illustrates this effect.

mobileplus

Finally. responsive web design seems to be a key to optimizing for mobile ecommerce.

Responsive web design is a process of making your website content adaptable to the size of the screen you are viewing it on. By doing so, you can optimise your site for mobile and tablet traffic, without the need to manage multiple templates, or separate content.

e-commerce and Forecasting

The Census Bureau announced numbers from its latest e-commerce survey August 15.

The basic pattern continues. US retail e-commerce sales increased about 16 percent on a year-over-year basis from the second quarter of 2013. By comparison, total retail sales for the second quarter 2014 increased just short of 5 percent on a year-over-year basis.

 ecommercepercent

As with other government statistics relating to IT (information technology), one can quarrel with the numbers (they may, for example, be low), but there is impressive growth no matter how you cut it.

Some of the top e-retailers from the standpoint of clicks and sales numbers are listed in Panagiotelis et al. Note these are sample data, from comScore with the totals for each company or site representing a small fraction of their actual 2007 online sales.

eretailers

Forecasting Issues

Forecasting issues related to e-commerce run the gamut.

Website optimization and target marketing raise questions such as the profitability of “stickiness” to e-commerce retailers. There are advanced methods to tease out nonlinear, nonnormal multivariate relationships between, say, duration and page views and the decision to purchase – such as copulas previously applied in financial risk assessment and health studies.

Mobile e-commerce is a rapidly growing area with special platform and communications characteristics all its own.

Then, there are the pros and cons of expanding tax collection for online sales.

All in all, Darrell Rigby’s article in the Harvard Business Review – The Future of Shopping – is hard to beat. Traditional retailers generally have to move to a multi-channel model, supplementing brick-and-mortar stores with online services.

I plan several posts on these questions and issues, and am open for your questions.

Top graphic by DIGISECRETS

First peek at “Revolutions” exhibit at Computer History Museum with Woz

I think Steve Wozniack is a kind of hero – from what I understand still connected with helping young people and in this video, giving some “straight from the horses mouth” commentary on the history of computing. 

And I am making plans to return to pattern on this blog.

That is, I will be focusing on issues tagged a couple of posts ago – namely geopolitical risks (ebola, unfolding warfare at several locations), the emerging financial bubble, and 21st century data analysis and forecasting techniques.

But, I think perhaps a little like Woz, I am a technological utopian at heart. If we could develop technologies which would allow younger people around the globe some type of “hands on” potential – maybe a little like the old computer systems which these technical leaders, now mostly all billionaires, had access to – if we could find these new technologies, I think we could knit the world together once again. Of course, this idea devolves when the “hands on” potential is occasioned by weapons – and the image of the child soldiers in Africa comes to mind.

I like the part in the video where Woz describes using a nonstandard card punch machine to get his card deck in order at Berkeley – the part where he draws a lesson about learning to do what works, not what the symbols indicate.