We Will Need All Those Data Centers (Part 1)

A massive overbuild of compute facilities, mostly to serve the mania for large language models (LLMs) and other AI applications, is propping up weak world economies and fueling near-universal warnings that a bursting of the bubble could bring those economies down.

In this two-part series, I argue that building all those data centers is a good activity (although it should be done more ecologically) and that we’ll eventually use them even if there’s a temporary downturn—one that might sweep away investors and even the economy that was kept afloat by their investments.

Closer to the concerns of Linux Professional Institute, data centers also have an impact on the employment and education of computer administrators and programmers.

To illustrate my thinking, I’ll turn to the history of another high-tech bubble.

Investments That Pay Off For Societies, Not Investors

The 1990s were the heady era of the dot-com boom. The internet was clearly altering the way we worked. Much more than that, in fact—it was creating entirely new ways to carry out planning and to coordinate people (cross-reference: open source development models).

Internet fever raced through the public with the release of the Mosaic browser in 1993 (and even more a year later with the Netscape Navigator browser), which made it clear that most of us would start getting most of our information and do most of our remote communication using those weird, deceptively simple standards that comprise the World Wide Web.

Investors would throw a million dollars at anyone who could scribble a business plan on the back of an envelope and grab an attractive .com domain name (Pets.com, anyone)?

All these millions of home pages and new online visitors required infrastructure. One survey of optical fiber growth in the United States in 1998 found that « Traffic and capacity of the public internet grew at rates of about 100% per year in the early 1990s. There was then a brief period of explosive growth in 1995 and 1996. During those two years, traffic grew by a factor of about 100, which is about 1,000% per year. »

I don’t believe that companies investing in fiber really believed they would do 10 times more business each year. Instead, they were racing against each other. Each thought that by outspending its competitors it could take new business and leave them to fail. An article titled « The Late 1990s Telecom Bubble » states, « Companies like Global Crossing, WorldCom, Qwest, and Level 3 Communications borrowed tens of billions to build international fiber networks that spanned continents and oceans….Some estimates suggested that less than 5 percent of the fiber installed during the boom years was ever lit, meaning most of the cables remained unused. »

The unused capacity was called « dark fiber, » the expectation being that it would be lit in the future with another explosion in internet traffic.

It’s no coincidence that the chair of the U.S. Federal Reserve Bank, Alan Greenspan, described the economic atmosphere in 1996 as « irrational exuberance. » The venture capital community had overreacted to the internet boom, and telecom companies were no exception. Then a market correction in 2000 joined with the September 11, 2001 attack that destroyed the World Trade Center and brought all kinds of new fears and uncertainties into the world. In the ensuing dot-com bust, according to a 2002 article, « Twenty-three telecom companies have gone bankrupt » over a two-year period.

There’s a principle behind this catastrophe. Technologies just take a while to find practical uses—that is, to be deployed in ways that people pay money for. Consider:

• E-commerce was about 1% of retail sales in the United States in 2000. If the percentage had been 16%, as it is today, customers and retailers would both have paid a lot more for bandwidth.
• If Software as a Service (SaaS), which scarcely existed in the 1990s, had been an almost 400 billion dollar industry as it is today, the fiber companies could certainly have earned a lot more on their investments.

For those who remember the dot-com bust, it’s easy to compare the data center boom to the fiber overbuild. As I’ll explain later, AI faces even greater hurdles to practical adoption than fiber did.

But now we get to the point of my article. What happened to all that « dark fiber »? It gradually came into use. One observer finds that surplus fiber drove all kinds of innovative uses, including the opportunistic siting of data centers in remote places. (Thus, the two high-tech booms connect.)

We can look across tech history for more examples of booms that ruined the investors but ended up benefitting the public. The most famous example, explored by business guru Peter Drucker, is that of railroads in 19th-century America. One author writes, « In the late 19th century, nearly one-fifth of the railway mileage in the US suffered from defaults and was put under receivership. The railway network was at risk of being liquidated and dismantled. »

But of course, railroads ultimately modernized the world, supercharging the Industrial Revolution, bringing food to fast-growing cities everywhere, and creating filthy rich Gilded Age families such as the Vanderbilts and the Goulds.

An article I cited earlier says that telephone and telegraph companies also went through a market correction in the U.S. during the early 20th century, although it took place in a more orderly manner by having AT&T absorb them.

These stories of early tech overbuilding—besides showing that rich people aren’t always as smart as they think they are—remind us that the famous Gartner hype cycle is not just a journalistic excess: it has economic consequences. Companies overinvest, often driven by competition and « fear of missing out, » then often disappear after the bubble bursts.

Having finished with our history, I’ll turn to the future for the next article in this series, discussing both the uses for data centers and the skills they demand.