The New York Times (here) recently featured historian Allan Lichtman’s system of thirteen Keys that when turned (TRUE or FALSE) predict the outcome of the presidential election. It is an entertaining and thoughtful piece but looking at history through a data science lens provides an even simpler perspective.
In Data Science there is a principle called Occam’s Razor, which basically states that simpler is better: fewer assumptions and fewer variables leads to solutions that are more likely to be accurate when presented with new data. With thirteen TRUE or FALSE keys, there are 8,192 possible combinations. There have only been forty elections since 1860, which is the first election in professor Lichtman’s data base. Most of the possible key combinations have not been realized. How will the prediction system react to a new combination? Is there a useful smaller subset?
Fortunately, professor Lichtman provides the answer on page 16 of his book, PREDICTING THE NEXT PRESIDENT, THE KEYS TO THE WHITE HOUSE. “The contest key and the short-term economy key together call all but one election (1880).” How might this insight be applied to more recent elections?
Answer just two questions, and you have a high confidence understanding and prediction of the coming election:
- Does the incumbent party candidate receive strong support at the party nominating convention?
- Is the unemployment rate going down or at least not going up too much at the time of the election?
If the answer to both questions is YES, then the incumbent is highly likely to win. A NO answer to either question means the incumbent is highly likely to lose. Let me explain.
The twenty-two presidential elections since 1932 are modelled in the Figure. The horizontal axis is the percentage of first ballot votes1 received by the incumbent party’s ultimate nominee. This is the contest key and measures whether or not the incumbent party is unified ahead of the election. The vertical axis is the year-over-year change in unemployment rate2 just ahead of the election. This is inspired by Lichtman’s short-term economy key, and it is a measure of whether or not the economy is improving.
The model clusters the incumbent wins in the lower right quadrant. The decision boundaries are a natural fall-out of the data: the incumbent needs to receive 75% or more of the first ballot votes at the nominating convention, and the year-over-year change in unemployment rate cannot be more than 2.6%. With one exception, the challenger wins everywhere else.
The exception was the 2000 election. The Democratic party was unified behind Vice President Gore, and the unemployment rate was declining, but the election came down to which candidate won Florida. After various recount efforts driven by the state’s law and judicial rulings, the US Supreme Court in a 5-to-4 decision stopped the recounts which effectively awarded the Presidency to the challenging Republican party and George Bush. The incumbent Democrats and Gore won the popular vote by 500,000 votes, but the challenging Republicans and Bush won the electoral college by five votes, 271 to 266. This was the closest outcome since the “stolen election5” of 1876.
In 1976 the statistician George Box3 wrote that “Essentially, all models are wrong, but some models are useful.” The usefulness of this model is that it gets twenty-one of the twenty-two (95%) elections correct. What does the model say about the 2020 election?
There is little doubt that the Republican party strongly supports Trump. He received all of the fitst ballot votes at this year’s convention, so the cross mark that plots the 2020 election is on the far right of the graph.
The October-to-October change in unemployment rate is Trump’s Achilles heel. The October 2019 unemployment rate was 3.6%. The August unemployment rate this year was 8.4%, and the Congressional Budget Office4 forecasts the rate will decline to 7.6% in 2021. Optimistically applying this figure to October of this year yields a year-over-year change of 111%, placing the cross mark off the chart. The upper extreme of the chart was set by the 48% year-over-year increase in unemployment rate before the 1932 election between the Republican incumbent Hoover and the Democratic challenger Roosevelt. Hoover received strong first ballot support at his convention that year, but his re-election was doomed by the unemployment rate and the horrible state of the economy that the rate reflected.
Could the Republicans and Trump prevail in 2020? Perhaps. With only twenty-two elections in this data base, a statistical analysis suggests the model’s prediction accuracy is between 72.7% and 99.9%. Maybe an improving but still high unemployment rate at the time of the election will be enough. The statistical door is open for the Republicans and Trump to prevail.
However, in his book “Predicting the Next President”, Lichtman made the following observation5, “study of history shows that a pragmatic American electorate chooses a president according to the performance of the party holding the White House … If the nation fares well during the term of the incumbent party, that party wins another four years in office; otherwise, the challenging party prevails.” The year-over-year change in unemployment rate is a quantitative way of answering the question of whether or not the nation is “faring well.” This simple model and Lichtman’s deep historical insight yield the same result: the answer is NO, and the Republicans and Trump are highly likely to lose the 2020 presidential election.
References:
1Multiple (can be provided on request)
2The Bureau of Labor Statistics has monthly unemployment rate data going back to 1948 (https://www.bls.gov/charts/employment-situation/civilian-unemployment-rate.htm) and yearly data going back to 1929 (https://www.bls.gov/opub/mlr/1948/article/pdf/labor-force-employment-and-unemployment-1929-39-estimating-methods.pdf).
3https://www.kdnuggets.com/2019/06/all-models-are-wrong.html
4https://www.cbo.gov/system/files/2020-07/56442-CBO-update-economic-outlook.pdf
5PREDICTING THE NEXT PRESIDENT, THE KEYS TO THE WHITE HOUSE 2016 by Allan J. Lichtman, page ix.