Much of the history of the United States for its first century is a story of expansion from its original foothold on the eastern seaboard across the North American continent. By 1890 or so, that expansion was over. The historian Frederick J. Turner wrote about this shift in a famous 1893 essay, "The Significance of the Frontier in American History." He began:
In a recent bulletin of the Superintendent of the Census for 1890 appear these significant words: “Up to and including 1880 the country had a frontier of settlement, but at present the unsettled area has been so broken into by isolated bodies of settlement that there can hardly be said to be a frontier line. In the discussion of its extent, its westward movement, etc., it can not, therefore, any longer have a place in the census reports.” This brief official statement marks the closing of a great historic movement. Up to our own day American history has been in a large degree the history of the colonization of the Great West. The existence of an area of free land, its continuous recession, and the advance of American settlement westward, explain American development.America's narrative of opportunity and mobility shifted at the tail end of the 19th century. In the 20th century, the narrative focused much less on a rural frontier, and instead was a story of moving to the city, where the opportunities to raise one's economic status and social status. But over the last 50 years or so, Edward L. Glaeser argues, we have in effect seen "The Closing of America’s Urban Frontier" (Cityscape: US Department of Housing and Urban Development, 2020, 22:2, pp. 5-21).
As Glaeser points out that the end of the western frontier was an important cultural event. But even back in the decades before Turner was writing, a relatively small share of the US population was actually made up of settlers. The land that mattered most for US economic growth and was not plots of ranchland in wide-open western states, but instead the areas surrounding the urban cores of that time, what Glaeser calls the "open urban frontier." The overwhelming population movement of the late 19th and early 20th century was from rural to urban areas, often in search of economic opportunity. Glaeser writes (citations omitted):
In a sense, America’s urban frontier became more open during Turner’s lifetime [1861-1932] because the traditional downsides of urban crowding, such as contagious disease, became less problematic. The urban frontier remained largely open during the dynamic 25 years that followed World War II. African-Americans migrated north by the millions to flee the Jim Crow South and take advantage of urban industrial jobs. Americans built new car-oriented cities in Sun Belt states like Arizona and Texas. The movement of people and firms diminished the vast income differences that once existed between locations.
Sometime around 1970, the urban frontier began to close. Community groups mobilized and opposed new housing and infrastructure. Highway revolts slowed urban expansion in car-oriented suburbs. Historic preservation made it more difficult to add new density in older cities. Suburbs crafted land-use restrictions that stopped new construction. While some productive Sun Belt cities still permitted significant amounts of new housing, even those one-time refuges of affordable urbanism had begun to be more restrictive. ...
Migration has fallen dramatically over the past 20 years, and poorer migrants no longer move disproportionately to richer places. Housing costs have risen sharply in more productive places, which has generated a wealth shift from the young to the old. Income convergence across regions has stalled. ... America’s growing geographic sclerosis makes it increasingly difficult for out-migration to solve the problems of local joblessness. ...
The closing of America’s urban frontier seems to be a far more significant event in American economic history than Turner’s motivating fact that “the unsettled area has been so broken into by isolated bodies of settlement that there can hardly be said to be a frontier line.” Vast amounts of the American West were unpopulated in Turner’s time and remain so today. Cheap land could still be had for homesteading in 1893, and there remains plenty of inexpensive ranchland today for anyone who wants the rugged life of a mid-19th century frontiersman. By contrast, the high price of accessing America’s most productive urban areas today is an important fact of life for tens of millions of Americans.Glaeser discusses these shifts in some detail, but ultimately, his focus is on housing costs. In the last few decades, metro areas with exceptionally high productivity and a large share of high-skilled workers have experienced a virtuous circle, where they become magnets for other high-skilled workers and still greater productivity. However, when this shift is combined with limits on housing and infrastructure, these urban areas become unaffordably expensive for those who are not already high-skill workers. The young up-and-coming go-getter, especially if the person is thinking about starting a family, would have to hesitate before moving to these areas unless there is already a high-paying job lined up.
At least in theory, at some point the cost of living in these high-cost, high-skilled, high-productivity areas should rise so high that economic activity is displaced to other areas. But at least so far, any such process of displacement has been very slow. What gets displaced to other metro areas is often a fairly standard factory or office, not the companies (or parts of companies) with the highest economic value-added.
There are essentially two non-exclusive approaches here. One is to make it easier for people of average and lower skill levels to move to the skill-magnet cities. The major step here is make it more affordable to live in those metro areas, which involves thinking about a large surge of housing construction along with patterns of transportation. The other approach is to figure out how to increase the number of cities with concentrations of high-skilled and highly productive workers, and in particular how to have such cities more spread across the country. I've written about some proposals along these lines in "Re-seeding America's Economic and Technological Future" (January 31, 2020). Both of these approaches have numerous costs and tradeoffs, and actual policies to achieve these goals are largely unproven.
Although I don't have a proven policy to propose, I'll note that this separation across US metro areas isn't healthy in economic terms. When people are hindered by high housing costs from moving to whether they could be more productive and earn higher wages, economic growth suffers. It's also not healthy in political terms. When looking at a political map of the United States, this separation between the cities that have become magnets for high-skilled, knowledge-based growth and the rest of the country is very stark.