Scott Alexander wrote two reviews of my work on the structural-demographic theory: Book Review: Secular Cycles and Book Review: Ages of Discord. The first one, on Secular Cycles, is quite positive, but the tone of the second review is a bit uneven–generally positive, but with some notes of critique and suspicion. His suspicions of the data and theory in Ages of Discord (AoD), however, are misplaced. Still, I would be the first person to admit that AoD is a highly technical and, thus, difficult book — full of models, formulas, tables, and graphics. I am constantly thinking about a popular version of AoD, but I haven’t yet figured out how to lay it out for the general public (I am also currently completely occupied with the analysis of Seshat data). Here I offer a few thoughts in response to issues raised by Alexander.

While what follows may sound critical of his review, I emphasize (and reiterate at the end) that I greatly appreciate the amount of time and effort he invested into reading and digesting my books.

First, Alexander starts the review of Ages of Discord with on overview of empirical patterns and only much later gets to theory. This is not how AoD is organized and, as a result, he ends up confusing himself and, I think, readers. AoD is only the latest installment in my work on structural-demographic theory (see Chapter 7 and 8 in Historical Dynamics published in 2003). By the time I wrote AoD, structural-demographic theory has matured to the point where one could (and I did) make predictions about novel cases. AoD, thus, is mainly an empirical test of predictions for the USA between 1780 and 2010. It is true that the US has gone through only 1.5 secular cycles in its history, but my identification of these cycles is not based on just these 1.5 “points”. The case of the US is an “out of sample prediction”, as it is known among the analysts.

Predictions were listed in Table 1.1 of Secular Cycles, published in 2009. I devoted two chapters at the beginning of AoD to extending the structural-demographic theory to industrializing societies and refining the predictions for the US. These two chapters, however, have very significant mathematical component. By his own admission, mathematics is not one of Alexander’s strong suits, which is probably why he jumps into data right away. But by doing this, his review fails to give justice to the logical structure of AoD.

Second, Alexander does some “spot-checks to see whether the data are any good”. This is somewhat strange — all data sources are listed in AoD, does he think that I have falsified them? Naturally, he concluded that “Turchin’s data all seem basically accurate.”

Next, in an attempt to check whether I “cherry-picked the data series that worked”, he looks at a variety of random indicators, for example, treasure bonds. Here again, by missing the theoretical part he doesn’t do justice to AoD. The variables I focus on all follow from general theory. There are fundamental variables in the theory that drive the dynamics (immiseration, elite overproduction, state strength, and socio-political instability) and there are “proxies” — variables closely correlated with the fundamental drivers. Then there are variables about which the theory is silent. Treasure bonds in no way are part of the theory, and I have no idea whether they would correlate with anything important, so I never looked at them.

There are also variables that are affected by fundamental ones, but are not part of the feedback web (they don’t affect the main drivers). For example, homicide rates. We expect that popular immiseration and elite overproduction would result in social pressures for instability, and one surface indicator of that could be growing homicide rates. I discuss homicide data in both Secular Cycles and AoD. However, one should keep in mind that there are many other factors, apart from structural-demographic ones, acting on this class of variables. Thus, we do not necessarily expect a perfect correlation. In fact, while structural-demographic pressures continued to grow during the last four decades, homicide rates actually declined during the 1990s. One possible explanation is that incarceration rates have quadrupled over this period of time. But the more important point here is that homicide rates are not a fundamental driver in the theory.

A particularly strange indicator that Alexander looked at is the USD/GBP exchange rate. I have no idea why he did that. Once again, in my research on structural-demographic dynamics I do not trawl through the thousands of time-series available on the web to look for correlations. Such trawling inevitably will yield correlations, but with high probability such correlations will be spurious.

Third, one has to be careful with data on meaningful indicators, and examine its provenance. In his first review (of Secular Cycles), Alexander starts by showing “Chinese population over time.” He doesn’t specify the source, but I recognize it — it’s McEvedy and Jones. 1978. Atlas of World Population History. Unfortunately, this resource is quite dated. Furthermore, it smooths out a lot of cycles in the data. Compare it with this chart (for provenance, see Historical Dynamics):

Quite a difference! Not everything in this graphic is solid, but the main point I am making is that one simply can’t grab the first available chart; it’s important to give thought to data sources and to understand their limitations.

Fourth, you cannot take on faith various opinions — myths is not too strong a word — propounded by social scientists; they have to be evaluated critically. This is particularly true of economics, because economists have an enormous vested interest in propounding theories that would please various powers-that-be. I’ve written about it in several of my blog posts, e.g.

The post Why Theory Is Important: Responding to two reviews by Scott Alexander appeared first on Peter Turchin.