Earthquake damage in L’Aquila, Italy. Photo courtesy of iStockphoto.
In April 2009 a devastating earthquake struck the central Italian town of L’Aquila, killing over 300 people. This was a human tragedy that has today morphed into a travesty against science and forecasting. Seven Italian geologists and seismologists have been convicted of manslaughter and sentenced each to six years in prison. All because they did not properly warn of the quake ahead of time. A flurry of tremors preceded the bigger 6.8 earthquake, and according to the court the experts gave “inexact, incomplete and contradictory information” which amounted to “monumental negligence” and criminal liability. The problem with this legal finding is that predicting earthquakes is not something scientists really know how to do very well, and flurries of tremors happen all the time without leading to major quakes. That this sort of liability for prediction error might be prosecuted so is stunning scientists around the world, and the charges have been condemned by the American Association for the Advancement of Science. (Camilli, D’Emelio) I am a practitioner in the art of forecasting myself, and I am pursuing further education in future studies, so I know some stuff about prediction, particularly the limits, and I felt a strong need to explain why this conviction troubles me so much.
Given the current understanding of the laws of physics, predicting the future of large open systems with certainty is impossible. Even if we assume that the universe is completely deterministic and predictable in principle, given perfect information about the positions and velocities of all the particles in the universe and the requisite laws of physics to extrapolate their trajectories, we could still not predict the future for any length of time. An infinite omniscience, such as the Newtonian mathematician Pierre-Simon Laplace’s thought experiment about the demon who had the aforementioned predictive powers, would need infinite precession in the data, a condition that requires temperatures of absolute zero. According the third law of thermodynamics it is impossible to obtain absolute zero, and without infinite precision the laws of chaos will prevent perfect prediction, except for short intervals. Gathering 100% more data obtains about 10% more predictive power, so error in forecasting is a given and should be expected (Schumacher).
I am not saying that forecasters can’t make good, useful, and accurate predictions, they certainly can, but there are fundamental limits to human forecasting potential, at least under current scientific paradigms, and predicting earthquakes is still especially difficult for today’s seismologists. In a recent Great Courses lecture series on futuristic science, Professor Jeffrey C. Grossman in the Department of Materials Science and Engineering at MIT explaind the incredible challenges with predicting earthquakes:
“[C]onsider the need to predict earthquakes and when relevant their effects on tsunamis, to allow for evacuations and emergency plans. When the earthquake hit Japan in 2011 and its tsunami took place damaging a nuclear reactor at Fukishima it would have helped enormously to have had a way to predict the effects of the earthquake on the production of the tsunami. This example is particularly interesting to highlight because while we can to some extent predict the evolution of the tsunami once the earthquake strikes we still cannot predict the earthquake itself.
Will we be able to do so in the future? Currently many different methods for forecasting earthquakes have been suggested. All so far look at possible precursors to earthquakes, not causes. Essentially they all look at early warnings that we humans cannot detect. For example, some models look for increased levels of radon, which is a common underground radioactive gas. Once in the air it has a half time decay of a few days. This means that any amount that is measured in significant quantities has to have emerged very recently, and this release can be associated with underground changes due to rock cracking. In principle this approach seems like it should be able to give a warning about earthquakes, but in practice many other factors affect the level of radon fluctuations that can be measured, like soil humidity. So, while long hoped as a way to detect earthquakes this method has failed so far.
Other methods have involved measuring changes in electromagnetic signals due to static charging that derives from rocks slipping against one another. This has been associated, although with little data to support it, with sudden changes in animal behavior before a major earthquake. Recently, underwater temperature increases due to slipping plates have been linked to the formation of plankton that emerging at the surface could signal a developing earthquake. Again, the data supporting these precursor based techniques are limited and not always directly correlated to an actual earthquake. In other words, these methods are too indirect and likely will never have great predictive power.
One of the problems in developing better ways to predict earthquakes is due to the fact that we know very little about the conditions and structures of underground faults, and of the plates themselves, and in order to understand quakes and therefore predict them, it’s crucial to know more about the dynamics of the soil.”
As a forecaster I am painfully aware of a problem in the assessment of forecasts called hindsight bias, “the tendency to exaggerate in hindsight what one was able to predict in foresight-or would be able to predict had one been asked.” (Fischhoff) In my professional environment I can control for hindsight bias by getting decision makers to agree on assumptions and uncertainties upfront, in a formal context, reducing the ambiguity in the forecasting process. Of course, my forecasts don’t normally involve life or death implications if I make an error, but that does not mean that a large financial loss as a result of a bad call on my part could not put my job at risk. A wise forecasting mentor once told me that in most organizations the forecaster is a second-class citizen, the proverbial messenger who is sometimes shot for an incorrect message. Unfortunately it is much easier to suggest that a forecaster should have seen something coming that they didn’t after it has already happened, than it is to actually forecast the thing before hand. I suspect this bias had a role in the conviction of the scapegoated of Italian scientists.
A couple of years ago I wrote about another problem in the evaluation of forecasts that I dubbed favorability bias. This is the acknowledgment that it is worse for a weather forecaster to predict sun and get rained out than it is for them to predict rain and get a sunny surprise. The issue with this sort of bias in forecast judgment is that it puts pressure on the forecaster to systematically bias their forecasts so that misses will be favorable. In the context of earthquake prediction in Italy, favorability bias means that predicting a devastating earthquake which does not happen is not harshly punished, but failing to predict an earthquake that does happen is negligent and severely punished.
If an unfavorable error in a forecast is likely to result in jail time or other punitive measures for the forecaster, then it should be easy to see what the effect might be on future forecasts. It could cause a systematic bias toward predictions for horrible earthquakes whenever there is the slightest indication, and there could be many more false alarms, resulting in forecasters who consistently cry wolf. Eventually there could be a massive Cassandra effect in which dire predictions become a noise that no one pays any attention to after so many earthquake drills. Or maybe this puts a damper on the motivation to work as a seismologist or geologist in Italy, resulting in even less predictive power. I am not guaranteeing these implications (remember that there are fundamental limits for the forecaster), but I am suggesting that this is a more likely outcome than a condition in which forecasters are scared straight into accurate prophecy.
Convictions in Italy have an automatic appeals process built in, so these seven scientists have another shot at acquittal before they might face jail time. I hope that their convictions are ultimately overturned, that the Italians pull another Amanda Knox, because if not this will set a troubling precedent in regards to the liability of forecasters for their incorrect predictions. We can already lose our jobs, but the idea that we could be imprisoned for imperfect prophecy harkens back to a time when oracles and soothsayers payed dearly for their misses, and the response was to dial back on specifics and provide vague and ambiguous fortunes. Accuracy, credibility, and utility are essential goals of forecasting, and the reason why we get employment, but prediction is not such a perfect science that making an error should be punished by imprisonment. I experience the Loma Prieta World Series earthquake while living in Salinas, CA in 1989. I did not get advance warning, and neither did the people who died when the Bay Bridge collapsed, but I remember anyone blaming the seismologists these deaths. Sometimes an act of God is an act of God.
Jared Roy Endicott
Works Cited
Camilli, Annalisa, and Frances D’Emelio. ”Conviction of Italian Experts who Failed to Predict Earthquake Rattles Scientists.” The Globe and Mail, Associated Press. 22 Oct 2012. Web. 22 Oct 2012.
Endicott, Jared Roy. ”Favorability Bias in Forecasting.” Realizing Resonance - Philosophy Blog. 8 Nov. 2010. Web.
Fischhoff, Baruch. “Learning from Experience: Coping with Hindsight Bias and Ambiguity.” Principles of Forecasting: A Handbook for Researchers and Practitioners. Ed. J Scott Armstrong. Philadelphia: Springer, 2001. 543-554. Print.
Grossman, Jeffrey C.. “Prediction—From Storms to Stocks.” Understanding the Science for Tomorrow: Myth and Reality. Chantilly, Virginia: The Great Courses, The Teaching Company, 2011. Video.
Schumacher, Benjamin. “Predicting the Future.” Impossible: Physics Beyond the Edge. Chantilly, Virginia: The Great Courses, The Teaching Company, 2010. Video.
