Photo courtesy of iStockphoto.
The Byzantine stronghold of Constantinople, the Rome of the East, fell to the Ottoman Turks in 1453 after a millennium of rule from behind its massive walls. Sultan Mehmet II, the Conqueror, laid siege to the city using a brand new innovation of war, massive cannons called bombards. The Turks had already pushed the Byzantines out of Anatolia over the previous few hundred years, and had recently encroached into Thrace and the Balkans by defeating the Serbians at the Battle of Kosovo in 1389. Constantinople was mostly depopulated by this time, and surrounded by Turks on all sides. Besides the use of cannons, Mehmet used his Navy to attack Constantinople from the Bosphorus Strait, intending to hit the city’s weakest walls from the Golden Horn inlet. But the Byzantines placed a massive chain at the entrance to the Golden Horn to prevent entry by boat. To deal with this Mehmet had his men and oxen physically pull massive galleons over land on log rollers, from the Bosphorus to the Golden Horn. The horrified Byzantines were able to hear the spectacle and watch it from the city walls. When Constantinople fell it was renamed Istanbul and became the seat of the Ottoman Empire until Mustafa Kemal Atatürk created the modern Republic of Turkey from the wreckage of World War I.
In May of 2013 I traveled to the historical city of Istanbul, Turkey. I learned a great deal on my trip, including some clumsy Turkish in advance of my travels, and I found that even a short four day visit can cause one to fall in love with a place and its people. I learned about the Byzantine legacy. If you didn’t already know, the small outpost of Byzantium was chosen as the new capital of the Roman Empire by Constantine in 330 AD, becoming Constantinople, right about the time that the iconoclastic Emperor was institutionalizing that new religion Christianity across his domain. I learned about the Ottoman Empire and its great lineage of Sultans, like Mehmet the Conqueror of Constantinople, and the longest reining Sultan Süleyman the Magnificent, also known as Kanuni, “the law giver,” who expanded the Ottoman Empire Northwest across Hungary to the walls of Vienna and Southeast to Bagdad, instituted significant law reforms, and commissioned many grand and beautiful works from master architect Mimar Sinan. One such work is the Süleymaniye Mosque, which I had a chance to visit (with my shoes respectfully off of course).
Süleymaniye Mosque, photo taken by Jared Endicott, May 2013.
My impetus for traveling to Istanbul was not primarily for pleasure, or even a historical lesson, but to attend a two day workshop on demand forecasting. Facilitated by Hans Levenbach of Delphus, Inc., these forecasting workshops are part of a program that trains Certified Professional Demand Forecasters (CPDF) . The certification has three levels, Basic, Master, and Professional, and having attended the Basic workshop in Chicago back in 2010 I was now on to learning the Master level. The two day workshop does not fulfill the certification requirement all the way, so for that I have also been completing self-paced forecasting projects at home. I thought it would be fun to use some of the new things I learned while in Istanbul in my attempt to get better at forecasting Thanksgiving Dinner and related variables. This will be the fourth installment in my annual Thanksgiving Prices Forecasting series, a follow up to 2010’s Inflation and the Cost of Thanksgiving Dinner, 2011’s Thanksgiving Dinner and Inflation Forecasts, and 2012’s Thanksgiving Dinner Inflation, Droughts, and the Dust Bowl. So this article is literally about what I learned in Turkey applied to forecasting turkey dinner.
Both the workshops and the follow-up projects have been great learning experiences, helping me to understand in a practical way what are often tricky forecasting concepts and techniques. The names of these concepts alone are extremely daunting for those not initiated in statistical jargon and notation. Holt-Winters exponential smoothing, autoregressive integrated moving averages (ARIMA), median absolute deviation (MdAD), and the merits of using the robust correlation coefficient of standardized sums and differences [r(SSD)] over the Pearson’s product moment correlation coefficient (r) are prime examples of how intimidating these forecasting tools sound. I can comfortably say that the CPDF workshops and certification exercises, along with the mentoring from Hans, has helped me to grasp the above concepts by showing me how to put them to use in practice.
The core framework for the CPDF training is the PEER forecasting process. This is a structured systematic cyclical process to maintain forecasting excellence with four phases: preparation, execution, evaluation, and reconciliation. In the preparation phase the forecaster must gauge the needs and requirements, get a grasp on the business environment, seek our reliable data sources, locate predictive factors and use a feedback process to review and prioritize these factors. In the execution phase the forecaster applies forecasting methodologies by choosing quantitative and qualitative approaches, building and selecting models, and producing forecasts with these models. In the evaluation phase the forecaster measures the relative accuracy of the various models, backtests with hold-out sample simulations, and observes performance through ongoing examination of model accuracy in an iterative cyclical process of structured judgment. In the reconciliation phase the forecaster must bring together the findings into a final model and forecast selection with an articulate presentation to decision makers and stakeholders.
I began learning the PEER method back in November 2010 and it has been the number one most successful tool in my forecasting career since. My annual posts about forecasting Thanksgiving Dinner prices began the same month as that first workshop, so I have already been incorporating what I have learned about PEER and the forecasting methodologies in the first three articles of the series. On an annual cycle I have done research to gather information of the relevant factors and theory that pertains to Thanksgiving Dinner prices (evaluation), developed and utilized forecasting models (execution), monitored and analyzed the accuracy and errors of the previous year’s forecasts (evaluation), and provided presentations and explanations of my findings in an annual series of blog articles (reconciliation). This is basically the PEER process, although a little less structured than my day job.
For now we will skip ahead to the evaluation stage in order to consider last year’s forecast of the 2013 cost for Thanksgiving dinner. The price survey I have been attempting to forecast comes from the American Farm Bureau Federation (AFBF), and it is the average cost of Thanksgiving dinner for 10 people. I predicted an increase of 3.6%, but average prices actually fell by 0.9%, so where I forecast $51.26 the cost went down to $49.04. My miss was favorable in terms of the consumer, but falling prices can also be a troubling sign of deflation and a weakening economy. The AFBF analysis does not indicate weakness in the economy though, with most factors simply unchanged, although the biggest factor–price of turkey–is down. Deputy chief economist for the AFBF John Anderson says, “Slightly higher turkey production for much of the year coupled with an increase in birds in cold storage may be responsible for the moderate price decrease our shoppers reported.” (Grondine, Sirekis)
Upon further evaluation I find that the actual number fell outside my 95% prediction interval. This seemed odd to me, since my forecast was only 4.5% off, which is a better performance than my last two attempts, and those higher errors were within my 68% prediction interval. It turns out I may have set the interval much too narrow, especially in terms of the possibility of price declines, which have occurred eight times of the last twenty-seven years. I will make sure to give the prediction intervals more focus in my next forecast for 2014.
Last year I calculated absolute percent errors (APEs) and mean absolute percent error (MAPE), but I learned this year in Istanbul that it is good to consider other evaluation measure as well, such as median absolute percent error (MdAPE). My APEs were 9.9%, 4.8%, and 4.5% for 2011, 2012, and 2013 respectively. This works out to a MAPE of 6.4% and a MdAPE of 4.8%. It is good to look at multiple measures of performance, with the MdAPE being particularly helpful because it is more robust and less sensitive to outliers (Levenbach, Cleary 78). I will cover what I learned about robust methods in more detail in a later section.
Learning ARIMA
Understanding how to use ARIMA models for forecasting has always stumped me. That was until I completed the Master level CPDF certification in Turkey. A recent tutorial on ARIMA in the Summer 2013 edition of Foresight: The International Journal of Applied Forecasting was also helpful for me. While not an expert by any means, I feel much more comfortable with the theory, conception, and appropriate use of ARIMA models now. This is a time series method, so the history of the data being forecasted is used to build the model. The major benefit of ARIMA models over exponential smoothing or seasonal decomposition models is the way that this alternative approach takes into account autocorrelations. Autocorrelation is the momentum in the data that correlates the current observation to past observations. The standard strategy to ARIMA modeling is called Box-Jenkins (named after the developers of the method), and it involves a three part iterative process of identification, estimation, and diagnostic checking.
It is not the purpose of this article to teach these methods, for that I recommend taking the CPDF workshops, but rather to demonstrate some of what I have learned on my Thanksgiving Dinner forecasts. Given that Thanksgiving Dinner forecasts are annual, and I really want to use a seasonal ARIMA model for the demonstration, I thought it would be interesting and relevant to forecast the monthly price changes for poultry. First thing to do is visualize and analyze the historical data to determine if there is a season and trend pattern that can be modeled. For this I might just look at the data. Or, to have a more reliable indication of pattern, I can use a method I learned from Hans back in 2012 at the Basic CPDF workshop in Chicago, constructing an analysis of variance (ANOVA) that measures how much of the differences between the data points belong to season, trend, or irregular factors.
For ARIMA, the identification phase is typically carried out with autocorrelation functions (ACFs)and partial autocorrelation functions (PACFs). Next with estimation, the model parameters are inferred from the data, and once preliminary models are chosen the forecasts produced from them can be checked diagnostically with an analysis of the residuals between the model data and the actual data. The model building process of identification, estimation, and diagnostic checking is iterated until the residuals from the model are randomly distributed, like white noise. For the first iteration of my poultry price forecasts I used a common benchmark, ARIMA(0,1,1)(0,1,1)12, the “Airline Model” used by Box and Jenkins to demonstrate their seasonal method with airline passenger mile data. It turns out this is the best performer compared to three other seasonal ARIMA models I tried, in terms of testing against a hold-out sample of actual data, with monthly simulations evaluated using a waterfall table. An illustration of the poultry price data and forecasts are displayed in the chart below.
In the following chart there is a distinct seasonal pattern in the month over month change in poultry prices, with a dramatic drop in price every November. This seems a little counter to economic assumptions at first glance. The law of demand suggests that if there is an increase in demand of something, say turkey for Thanksgiving dinner, then the price will be driven up, all other things the same. During Thanksgiving turkeys sell at prices well below cost, with the mark down being a strategy by competitive retailers to get customers in the door in order to capture the sales on other items (Sullivan). The laws of supply and demand are still contingent, and the competitive custom of displaying the holiday bird as an enticing loss leader becomes an extenuating circumstance that defies normal price behavior. The turkey price reverses course in December, going back up, and over time the trend is generally a higher price for poultry each year, as illustrated by the chart below.
Learning Robust Methods
Another technique I became familiar with during Chicago’s workshop was using transformations to handle nonlinear data. In 2013, at the Master CPDF Workshop in Istanbul, I enhanced this knowledge by embracing nonconventional methods of analysis, techniques that are robust and more suited for data that does not conform to the conventional assumptions. In terms of statistical models, it is conventional to use measures of central tendency and dispersion that are derived from the mean and the standard deviation. In a linear and symmetric world, where all data distributions conform to the normal bell curve, these methods are optimal, but in the real world where we are often confronted with data that is nonlinear these methods can be ill suited. It is these nonlinear situations that call for nonconventional solutions.
The most conventional and familiar statistical methods, in terms of data analysis and forecasting, derive from the mean and the standard deviation. This is the method of calculating a simple average from a set of numbers and then using that average to calculate the spread of the data around the average. After the mean and the standard deviation are analyzed, one or more data sets can be related using ordinary correlation and linear regression. If the underlying data are linear and normally distributed, meaning that the data conform to a bell curve model, then the ordinary conventional methods are optimal. Unfortunately, much business and economic data are not linear and do not conform to standard assumptions. Conventional methods of analysis may provide misleading results when applied to nonlinear data, because these methods are sensitive to outliers and distortions. When these problematic situations arise it is typical to adjust or transform the data into a linear perspective, by handling outliers, calculating differences, or converting to logarithms. Massaged data can then be input into the conventional models to better effect.
Rather than locate and handle outliers another approach is to forego the ordinary methods and apply robust methods that are less sensitive to non-normal data. Instead of using the mean as the measure of central tendency, the median will not be affected as much by outliers. Rather than use standard deviation to estimate the scale, or spread, of the data, use median absolute deviation (MdAD). In lieu of the standard product-moment correlation coefficient, try relating two or more variables using a robust correlation method such as standardized sums and differences [r*(SSD)]. Below are two correlation matrices that display the correlation coefficients of the prices changes of Thanksgiving dinner against related variables, one produced by the ordinary calculation and the other with a robust method. After that scatter plots of these factors are illustrated, with linear regression lines to exhibit the ordinary correlation.
The changing price of Thanksgiving dinner is most closely associated with the price of food more generally, and is also the only variable that showed an improved view of the correlation with the robust method. The association with changes in consumer prices is much weaker with the robust correlation, and poultry prices in November show a different direction of the correlation altogether, going from positive in the ordinary method to negative in the robust method. The reversal of signs between the two measures of correlation between Thanksgiving Dinner and November poultry prices is curious and suggests a closer analysis may be in order.
Forecasting Thanksgiving Dinner Inflation for 2014
For my primary forecasting task of predicting the price of Thanksgiving Dinner next year I tested out a few different methods and models. Times series methods are one possible approach, like the ARIMA example discussed earlier, or exponential smoothing, which both use the historical values of the same data one wants to forecast, allowing for seasonal and trend effects to indicate a likely future course. On the other hand, regression gives the predictioneer a tool by which to leverage correlations and causes, using independent variables to estimate future values of a forecast variable, whether using linear regression or robust regression. There are pros and cons to each approach, contingent upon the forecasting situation, such as whether leading indicators are available or patterns can be found in the historical data. With this in mind I blended a few approaches. Combining forecasts from diverse models has merits, including robustness, which I discuss in more depth in the article Macroeconomic Forecasting with Diverse Predictions.
My forecast for the cost of Thanksgiving dinner in 2014, as measured by the American Farm Bureau Federation, is $50.63. This is a 3.2% lift from the 2013 cost of $49.04. I am 68% confident that prices will come in between $48.10 and $53.43. Alternatively, I am 95% confident that prices will land between $45.58 and $56.23. My prediction intervals are wider than I set them for the 2013 forecast, and this should allow for greater contingency in the price movements. I am expecting consumer prices to grow by 2.1% and food prices to grow by 3.0%, and I expect the change in price for turkey dinner to move in the same direction with a modest magnitude.
Conclusion
The largest lesson I learned while I was in Turkey though is not one that I can apply to forecasting Thanksgiving Dinner, but it is one that gives me great hope about this magnificent nation’s future. I fell in love with Istanbul in the short time I was there, which was a relatively serene week the month before the Taksim Square protests over Gezi Park broke out, a situation I covered in the article, Taksim: Tranquility Before The Tumult. I marveled in the history of the Byzantine and Ottoman Empires with visits to Galata Tower, Blue Mosque, Aya Sofya, Topkapı Palace, Basilica Cistern, and a relaxing cruise down the Bosphorus Strait with my friend Sinan. I even visited the Grand Bazaar where I picked up a necklace for my wife, and spent some time negotiating over carpets. While enjoying cay, tasty kebabs, and Turkish coffee, I encountered a culture that has preserved and honored its rich heritage while also embracing modernity and a dynamic future. In an interesting coincidence for my forecasting trip, I also had a chance to see a relic from the Oracle of Delphi, a piece of the Pythia’s tripod located on the site of Constantinople’s old Hippodrome which is now in front of the Blue Mosque.
Relic of the Oracle of Delphi, photo taken by Jared Endicott, May 2013.
You may not always see what I saw in the way the American media portrays the region, but the Turks I met were youthful and intelligent professionals from the energy, electricity, airline, banking, and hospitality industries. They were all eager and excited to learn and achieve, with an ambition that any American should admire. Not only were they learning and communicating with me in English, we all spoke fluent Excel. It is no surprise to me that there are seven more CPDF workshops scheduled for Istanbul in 2014 while so far the US only has one to look forward to next year (at the time of this writing). After all, I traveled all the way from Seattle to Istanbul for the valuable insight into foresight that Hans Levenbach brings as a mentor. The forecasting skills I went to Turley to learn have not just added to my set of tools for predicting turkey dinner, but have also added challenging and essential expertise to my professional repertoire. For me this is invaluable human capital and well worth the trip.
Blue Mosque, photo taken by Jared Endicott, May 2013.
Aya Sofya, photo taken by Jared Endicott, May 2013.
Aya Sofya, photo taken by Jared Endicott, May 2013.
Topkapı Palace, photo taken by Jared Endicott, May 2013.
Basilica Cistern, photo taken by Jared Endicott, May 2013.
Jared Roy Endicott
Works Cited
Levenbach, Hans, and James P. Cleary. Forecasting: Practice and Process for Demand Management. Belmont, CA: Duxbury, Thomson Brooks/Cole, 2006. Print.
SWIFT - Structured Workshop in Forecaster Training. CPDF - Certified Professional in Demand Forecasting, Delphus, Inc., 2010.
CPDF_Master - Demand Forecasting Methodology and Performance Measurement. CPDF - Certified Professional in Demand Forecasting, Delphus, Inc., 2010-2013.
“Cost of Classic Thanksgiving Dinner Down for 2013”. American Farm Bureau Federation . Contacts: Tracy Taylor Grondine and Cyndie Sirekis. 14 Nov. 2013. Web. 14 Nov. 2013.
Sullivan, Paul. “In the Labyrinth of Turkey Pricing, a Reason Under Every Giblet.” The New York Times. 8 Nov 2011. Web. 15 Sep 2013.
“ARIMA: The Models of Box and Jenkins.”Foresight: The International Journal of Applied Forecasting