Tuesday, February 18, 2020

The Herfindahl-Hirschman Index: Story, Primer, Alternatives

It seems clear that the concept of what is now is called the Herfindahl-Hirschman Index was originated in 1945 by Albert O. Hirschman, who may be best-remembered today for his 1970 book Exit, Voice, Loyalty, discussing the options available to a dissatisfied group member.  However, the concept was then attributed to Orrin Herfindahl, who wrote five years later in 1950, and further confusion arose when it was sometimes referred to as a Gini index. Here's a primer and the the story.

The HHI, as it is often abbreviated, is a way of measuring industry concentration that is taught in every intro econ textbook. Assume that you have an industry where one big company has 50% of sales, three companies have 10% each, and 20 companies have 1% each. How can a researcher sum up the degree of concentration in this industry in a single number? 

One common approach is to use a "concentration ratio." Pick a number of firms, like the top 4 or the top 8 in an industry. Add up their market share. Thus, the 4-firm concentration ratio in this example would be 80% and the 8-firm concentration ration would be 84%. 

But this concentration ratio approach has an obvious problem. An industry where the four top firms each had 20% of the market would have the same 4-firm concentration ratio of 80% as the example above. An industry where the top eight firms each had 10.5% of the market would have the same 8-firm concentration ratio of 84%. It would seem odd to say that these counterexamples, with a number of firms of roughly equivalent size, have the same concentration as the original example, where the largest firm has a full half of the market. 

Thus, the HHI uses a different calculation. First you square the market shares of existing firms; then you add them up. Thus, in the original example the HHI would be (50)2 + 3(10)2 + 20(1)2 = 2,820. The maximum value for an HHI would be 10,000, for a single firm with 100% of the market. An industry with a large number of very small firm that each have less than 1% of the market could have an HHI lower than 100. (In some cases, the HHI is described on a scale from 0 to 1, instead of 0  to 10,000, which is the numbers you get if the market shares are expressed with decimal points before being squared--thus, a 50% market share squared would be .25, not 2500.).

The idea of measuring industry concentration in this way originated with Albert O. Hirschman in his 1945 book National Power and the Structure of Foreign Trade. As he points out, there were already ways of measuring concentration, with the Lorenz curve and the Gini coefficient for measuring inequality of income especially well-known. But as Hirschman points out (p. 158):
In various instances, however, the number of elements in a series the concentration of which is being measured is an important consideration. This is so whenever concentration means "control by the few," i.e., particularly in connection with market phenomena. Control of an industry by few producers can be brought about by an inequality of distribution of the individual output shares when there are many producers or by the fact that only few producers exist. One of the well-known conditions of perfect competition is that no individual seller should command an important share of the total market supply; this condition implies the presence of both relative equality of distribution and of large numbers. 
To put this point a little differently, imagine a market with all of equal-sized producers--maybe a small number like two or three or four, or a large number like 100 or 1,000.  A measure of equality like the measures used for income  would point out that all firms firms are of equal size. In contrast, a measure of concentration would emphasize that the number of firms matters, and that four firms means more competition than two, and 100 firms means more competition than four.  By squaring the market shares, Hirschman's measure gave greater weight to larger firms, thus emphasizing the idea that when it comes to concentration of an industry, large firms matter more.

In these ways, Hirschman's proposed measure of industry concentration was fundamentally different than the common measures of income equality. In fact, it was such a good idea that five years later, the idea was reinvented by Orris C. Herfindahl in his 1950 PhD dissertation, Concentration in the U.S. Steel Industry. Herfindahl mentions Hirschman's earlier work in a footnote.

There were surface differences between the Hirschman and Herfindahl measures. Hirschman's study was looking at concentrations of exports and imports of countries, both according to sources and destinations of international trade, while Herfindahl was applying the measure to the US steel industry. In addition, Herfindahl used (essentially) the measure described briefly above, while Hirschman took the square root of that measure.

But that's not how it evolved. Gideon Rosenbluth wrote  chapter called "Measures of Concentration," which appeared in a 1955 NBER conference volume called Business Concentration and Price Policy (pp. 57-95).  Rosenbluth wrote in 1955:
But summary measures can be devised to measure concentration, just as they have been developed for other characteristics of size distributions. An ingenious measure of this type has been employed by O. C. Herfindahl in an investigation of concentration in the steel industry. It consists of the sum of squares of firm sizes, all measured as percentages of total industry size. This index is equal to the reciprocal of the number of firms if all firms are of the same size, and reaches its maximum value of unity when there is only one firm in the industry.
But a few years later in a 1961 essay, " Remarks" in Die Konzentration in der Wirtschaft, Schriften des Vereins fuir Sozialpolitik (New Series, Vol. 22, pp. 391-92), Rosenbluth wrote:
The first point I want to make causes me some embarrassment. There is a good deal of discussion int the background material about "Herfindahl's Index." Actually, it is a mistake to ascribe this index to Herfindahl, and I believe my paper on measures of concentration, published in 1955, is the source of this mistake. I discovered later that the man who first proposed this index was Albert O.  Hirschman in his book "National Power and the Structure of Foreign Trade," published by the University of California Press in 1945. Hirschman actually proposed the square root of what I call Herfindahl's Index, since this gives a more even distribution of values. 
Hirschman made an attempt to lay out this chronology in a short note appearing in the American Economic Review in 1964 (54: 5, September, p. 761). He points out that in a number of recent papers, the index was being referred to as a "Gini index," although he had made some effort back in 1945, along with Herfindahl and Rosenbluth in later work, to be clear that it was not an index of equality. Hirschman writes: "Upon devising the index I went carefully through the relevant literature because I strongly suspected that so simple a measure might already have occurred to someone. But no prior inventor was to be found." He also points out that Rosenbluth had originally attributed the index to Herfindahl.  Hirschman concludes on a wry note: "The net result is that my index is named either after Gini who did not invent it at all or after Herfindahl who reinvented it. Well, it's a cruel world."

In the world of economics, this problem of attribution is sometimes called Stigler's law: "No scientific discovery is named after its original discoverer." Of course, Steve Stigler was quick to point out in his 1980 article that he didn't discover his own law, either! In this case, it does not seem to me a grievous miscarriage of justice to have the names of both Hirschman and Herfindahl on the index, although Hirschman should probably come first.

Those who have read this far are probably the kind of people who would be interested in knowing that justification and analysis of concentration indexes is an ongoing task.  For getting up to speed, a useful starting point is the NBER working paper by  Paolo M. Adajar, Ernst R. Berndt, and Rena M. Conti, "The Surprising Hybrid Pedigree of Measures of Diversity and Economic Concentration" (November 2019, #26512).
The characterization of industry structure and industry concentration has long been a task facing empirical economic researchers, for it is widely believed that market structure, market behavior and various market performance outcomes are important interrelated phenomena. Although a number of alternative measures of market concentration are commonly used, such as the k‐firm concentration measure and the Herfindahl‐Hirschman index (HHI), their foundations in economic theory and statistics are limited and have not been developed extensively, leaving their unqualified use as measures of market power potentially vulnerable to the criticism of “measurement without theory”.
For example, perhaps it makes sense at some intuitive level to give greater weight to the market share of large firms when measuring concentration. But why square the market shares? Why not adjust them in some other way? Indeed, there is a set of alternative concentration measures using different weights going back to the work of Gideon Rosenbluth, and known as Rosenbluth/Hall‐Tideman (RHT) metrics. Adajar, Berndt, and Conti offer an analytical basis for the idea that squaring the market shares makes sense, based on a conceptually similar diversity measure from ecology. They write:
In this paper, we have traced the pedigree of the much‐used Herfindahl‐Hirschman (HHI) economic concentration index to the Simpson Index of diversity originally developed in ecology, where an identical calculation to the HHI is interpreted as the probability of two organisms randomly selected from a sample habitat belonging to the same species (analogous in economics to the probability a pair of randomly and independently selected products are being marketed by the same manufacturer). This probabilistic foundation of the HHI to some extent shields it from the allegation that the sum of squared shares calculation is arbitrary and unscientific, even as its links to market power and antitrust competition analysis remain ambiguous. 
For those wanting to dig deeper into alternative indexes of concentration, Adajar, Berndt, and Conti write:
We have also considered alternative proposed measures of concentrations, some of them mathematical generalizations of the HHI, others such as entropy originating from information theory in engineering and physics, another set that is developed axiomatically, and still others incorporating related concepts such as inequality and absolute population size. We have considered computational and interpretability aspects of the various concentration measures, and noted the extent to which they incorporate considerations not only of relative inequality such as the Gini coefficient and Lorenz curve, but also of absolute population size. 
Other things equal, markets with a large number of competitors suggest barriers to entry are limited, and therefore such markets could plausibly be expected to be competitive, other things equal. Therefore, to economists concentration metrics incorporating both variability/relative inequality and absolute population size considerations are preferable, for if one believes that economic performance outcomes depend not only on relative sizes but also on the  absolute number of competitors in a market, then one prefers a concentration measure that incorporates both features. The existing economic literature comparing the various concentration metrics on a priori statistical and axiomatic criteria appears to view the HHI and the closely related Rosenbluth/Hall‐Tideman (RHT) metrics most favorably. Choice between these two measures on a priori grounds is indeterminate, since the choice involves selection of weights and is therefore similar to choice among alternative index number formula in economic index number theory.