Internet History: backbone network

Internet History: backbone network

Other articles in the series:

Introduction

In the early 1970s, Larry Roberts came to AT&T, a massive telecommunications monopoly in the U.S., Larry Roberts with an interesting proposal. At that time, he was the director of the computing division of the Advanced Research Projects Agency (ARPA), a relatively young organization operating under the Department of Defense and engaged in long-term research detached from reality. In the five years prior, Roberts had overseen the creation of ARPANET, the first significant computer network connecting computers located in 25 different sites across the country.

The network turned out to be successful, but its long-term existence and all the associated bureaucracy fell outside ARPA's authority. Roberts was looking for a way to offload this responsibility onto someone else. So he reached out to the directors of AT&T to offer them the 'keys' to this system. After careful consideration, AT&T ultimately turned down the offer. Senior engineers and managers believed that the fundamental technology of ARPANET was impractical and unstable, and that it had no place in a system designed to provide reliable and universal service.

ARPANET, of course, became the seed around which the internet crystallized; a prototype of a huge information system covering the whole world, whose kaleidoscopic possibilities are impossible to quantify. How could AT&T not see such potential, being so stuck in the past? Bob Taylor, who hired Roberts as the project manager for ARPANET in 1966, later stated bluntly: 'Working with AT&T would be like working with Cro-Magnons.' However, before we confront such ignorant ignorance from unknown corporate bureaucrats, let's take a step back. The theme of our story will be the history of the internet, so it would be good to first get a more general understanding of what this is all about.

Among all the technological systems developed in the latter half of the 20th century, the internet has undoubtedly had the greatest impact on modern society, culture, and economy. Its closest competitor in this regard might be the advent of jet travel. With the internet, people can instantly share photos, videos, and thoughts—both desirable and undesirable—with friends and family all over the world. Young people living thousands of kilometers apart are now frequently falling in love and even getting married within the virtual realm. An endless shopping mall is accessible at any hour of the day or night from millions of comfortable homes.

For the most part, this is all familiar and indeed how it is. However, as the author can confirm, the internet has also proven to be perhaps the greatest distraction, time-waster, and source of mental corruption in human history, surpassing television—which was no easy feat. It has allowed various maladjusted individuals, fanatics, and conspiracy theorists to spread their nonsense around the globe at the speed of light—some of which can be deemed harmless, while other parts are not. It has enabled countless organizations, both private and publicly traded, to slowly accumulate, and in some cases quickly and disgracefully lose, vast amounts of data. Overall, it has become an amplifier of human wisdom and folly, with the latter being quite frightening.

But what exactly does the subject we are discussing represent, its physical structure, all this machinery that has facilitated these social and cultural changes? What is the internet? If we could somehow filter this substance and place it in a glass container, we would see it separate into three layers. At the bottom would settle the global communication network. This layer predates the internet by about a hundred years and initially consisted of copper or iron wires, but since then has been replaced by coaxial cables, microwave relays, optical fibers, and cellular radio communication.

The next layer consists of computers communicating with each other through this system using common languages, or protocols. Among the most fundamental are the Internet Protocol (IP), Transmission Control Protocol (TCP), and Border Gateway Protocol (BGP). This forms the core of the internet, with its specific expression emerging as a network of special computers called routers, responsible for finding the route for a message to travel from the source computer to the destination.

Finally, at the top layer are various applications that people and machines use for work and play on the internet, many of which employ specialized languages: web browsers, communication apps, video games, trading applications, and so on. For an application to utilize the internet, it simply needs to encapsulate the message in a format that routers can understand. The message could be a chess move, a tiny snippet of a movie, or a request to transfer money from one bank account to another—routers do not differentiate, and they will treat it all the same.

Our narrative will weave these three threads together to tell the story of the internet. First, the global communications network. In the end, all the splendor of various programs that allow computer users to entertain themselves or do something useful online. Together, they are connected by technologies and protocols that enable different computers to communicate with each other. The creators of these technologies and protocols built upon the achievements of the past (the network) while having a vague conception of the future they were groping toward (future programs).

In addition to these creators, the government will be one of the main characters in our story. This will particularly relate to the level of telecommunications networks, which were either managed by the government or closely regulated by it. This brings us back to AT&T. No matter how unpleasant it is for them to realize, the fate of Taylor, Roberts, and their colleagues at ARPA was hopelessly tied to telecommunications operators, the very foundation of the future internet. Their network operations were entirely dependent on such services. How can we explain their hostility, their belief that ARPANET represents a new world that fundamentally opposes the retrograde officials overseeing telecommunications?

In reality, these two groups were divided not by temporary but by philosophical differences. The directors and engineers at AT&T viewed themselves as overseers of a vast and complex machine that provided reliable and universal communication services from one person to another. All the equipment was under the responsibility of the Bell System. In contrast, ARPANET architects regarded the system as a conduit for arbitrary bits of data, believing that its operators should not intervene in how this data was created and used from both ends of the wire.

Therefore, we must begin with the story of how the authority of the U.S. government resolved this deadlock concerning the nature of American telecommunications.

Internet History: backbone network

One system, universal service?

The internet was born in a unique environment of American telecommunications — in the U.S., telephone and telegraph providers were treated quite differently than in the rest of the world — and there is every reason to believe that this environment played a formative role in the development and shaping of the spirit of the future internet. So, let us carefully examine how all of this happened. To do this, we will go back to the time of the birth of the American telegraph.

The American anomaly

In 1843 Samuel Morse and his allies convinced Congress to spend $30,000 on creating a telegraph line between Washington, D.C. and Baltimore. They believed that this would be the first link in the government-funded network of telegraph lines that would spread across the continent. In a letter to the House of Representatives, Morse proposed that the government buy out all rights to his telegraph patents and then commission private companies to build segments of the network, reserving certain lines for official communication. In such a case, Morse wrote, "it won't be long before the entire surface of this country is stitched with these nerves, which will spread knowledge of everything happening on earth at the speed of thought, turning the whole country into one large settlement."

He felt that such a vital communication system naturally served the public interest, thus falling under the government's responsibilities. Ensuring communication between several states through postal services was one of the few tasks of the federal government specifically mentioned in the U.S. Constitution. However, his motives were not solely defined by public service. Government control gave Morse and his supporters the opportunity to successfully complete their venture – to receive one significant payout from public funds. In 1845, Cave Johnson, the U.S. Postmaster General under the 11th President, James Polk, expressed his support for the public telegraph system proposed by Morse: "The use of such a powerful tool, for good or ill, for the safety of people cannot be left in private hands," he wrote. But that was where it ended. Other members of Polk's democratic administration were unwilling to have anything to do with the public telegraph, just as the democratic Congress was. The party disliked the plans of the Whigs, which forced the government to spend money on "internal improvements" – they viewed these plans as encouraging favoritism, bribery, and corruption.

Due to the government's reluctance to act, one of Morse's team members, Amos Kendall, began developing a telegraph network scheme with the support of private sponsors. However, Morse's patent was insufficient to ensure a monopoly on telegraphic communication. In ten years, dozens of competitors emerged, either purchasing licenses for alternative telegraph technologies (primarily based on Royal House's printing telegraph) or simply engaging in semi-legal activities on shaky legal grounds. Lawsuits piled up, fortunes rose and fell, bankrupt companies collapsed or sold to competitors after artificially inflating stock prices. Out of all this chaos, by the end of the 1860s, one major player emerged: Western Union.

A frightened rumor began to spread about a 'monopoly.' The telegraph had become essential for several aspects of American life: finance, railroads, and newspapers. Never before had a private organization grown to such proportions. The proposal for government control of the telegraph received new life. In the decade following the Civil War, Congressional postal committees devised various plans to bring the telegraph under the orbit of the postal service. Three basic options emerged: 1) The postal service sponsors another competitor to Western Union, giving it special access to post offices and routes in exchange for imposing rate restrictions. 2) The postal service launches its own telegraph to compete with WU and other private firms. 3) The government nationalizes the entire telegraph, placing it under the management of the postal service.

The plans to create a postal telegraph gained several loyal supporters in Congress, including Alexander Ramsey, chairman of the postal committee in the Senate. However, much of the campaign's energy was supplied by external lobbyists, particularly Gardiner Hubbard, who had experience in public service as an organizer of urban water supply and gas lighting systems in Cambridge (he later became a significant early sponsor of Alexander Bell and the founder of the National Geographic Society). Hubbard and his supporters argued that a public system would provide the same useful dissemination of information that paper mail did, keeping rates low. They claimed that this approach would surely serve society better than the WU system, which targeted the business elite. WU, of course, contended that the cost of telegrams was determined by their production costs, and that a public system, artificially lowering rates, would face issues and benefit no one.

In any case, the postal telegraph never received enough support to become a battleground in Congress. All proposed legislation quietly suffocated. The scale of the monopoly did not reach levels that would overcome fears of government abuse. The Democrats regained control of Congress in 1874, the spirit of national reconstruction in the period immediately following the Civil War was muted, and the initially weak attempts to establish a postal telegraph faded away. The idea of placing the telegraph (and later the telephone) under government control periodically emerged in subsequent years, but aside from brief periods of nominal government control of the telephone during wartime in 1918, nothing came of it.

Such disregard by the government for the telegraph and telephone was an anomaly on a global scale. In France, the telegraph was nationalized even before its electrification. In 1837, when a private company attempted to establish an optical telegraph (using signaling towers) alongside the existing government-controlled system, the French Parliament passed a law prohibiting the development of telegraphs not authorized by the government. In Britain, private telegraphy was allowed to develop for several decades. However, public discontent with the resulting duopoly led to government control of the situation in 1868. Across Europe, governments placed telegraphy and telephony under the control of the state postal service, as proposed by Hubbard and his supporters. [In Russia, the state enterprise ‘Central Telegraph’ was founded on October 1, 1852 / translator's note.]

Outside of Europe and North America, much of the world was controlled by colonial powers, and therefore had no voice in the development and regulation of telegraphy. Where independent governments existed, they typically established state telegraph systems based on the European model. These systems often lacked the funds for expansion at the rate observed in the US and European countries. For example, the Brazilian state telegraph company, operating under the Ministry of Agriculture, Commerce, and Labor, had only 2,100 km of telegraph lines by 1869, while the US, in a similarly sized territory where four times as many people lived, had already extended 130,000 km by 1866.

New Deal

Why did the USA take such a unique path? This can be attributed to the local system of distributing government positions among supporters of the winning party, which existed until the late 19th century. Government bureaucracy, up to postmasters, consisted of political appointments that rewarded loyal allies. Both parties were reluctant to create new major sources of patronage for their opponents, which would have inevitably happened if the telegraph had come under federal control. However, the simplest explanation lies in the traditional American distrust of a powerful central government – for the same reason, the structures of American healthcare, education, and other public institutions differ significantly from those in other countries.

Considering the increasing importance of electric communication for state life and security, the USA could not completely detach from the development of communications. In the early decades of the 20th century, a hybrid system emerged in which private communication systems were checked by two forces: on one hand, bureaucracy constantly monitored the rates of communication companies to ensure they did not take a monopolistic position or extract excessive profits; on the other hand, there was the threat of being broken up under antitrust laws in the case of improper behavior. As we will see, these two forces could sometimes conflict: the theory of rate regulation considered monopoly a natural phenomenon under certain circumstances, while duplicating services would be an unnecessary waste of resources. Regulators generally tried to minimize the downsides of monopoly by controlling prices. Meanwhile, antitrust legislation aimed to eradicate monopolies outright by forcibly organizing a competitive market.

The concept of rate regulation originated on the railroads and was implemented at the federal level through the Interstate Commerce Commission (ICC), established by Congress in 1887. The main driving force behind the law was small business and independent farmers, who often had no choice but to use the railroads to transport their products to market. They claimed that railroad companies exploited this by squeezing every last penny from them while offering luxurious terms to large corporations. A five-member commission was granted the power to oversee the services and rates of the railroads and prevent abuse of monopoly power, in particular by prohibiting railroads from offering special rates to select companies (a precursor to the concept we now call 'net neutrality'). The Mann-Elkins Act of 1910 expanded the ICC's authority to include telegraph and telephone services. However, the ICC, focusing on transportation, never showed much interest in these new areas of responsibility, largely ignoring them.

At the same time, the federal government developed a completely new tool to combat monopolies. Sherman Act of 1890 empowered Attorneys General to challenge in court any commercial 'combination' suspected of 'restraining trade' — that is, suppressing competition through monopoly power. This law was used in the following two decades to dismantle several of the largest corporations, including the Supreme Court's 1911 ruling to break up Standard Oil into 34 parts.

Internet History: backbone network
The Standard Oil octopus from a 1904 cartoon, before the breakup.

By that time, telephony and its main provider, AT&T, had overshadowed telegraphy and WU in importance and capabilities to such an extent that in 1909, AT&T was able to acquire a controlling stake in WU. Theodore Vail became the president of the merged companies and began the process of unifying them. Vail firmly believed that a benevolent telecommunications monopoly would better serve the interests of society, promoting the company's new slogan: "One policy, one system, universal service." Ultimately, Vail became a target for antitrust opponents.

Internet History: backbone network
Theodore Vail, c. 1918

The inauguration of the Woodrow Wilson administration in 1913 provided members of his Progressive Party a timely opportunity to threaten with their antitrust club. Postmaster General Sidney Burleson leaned toward the complete postalization of the telephone along European lines, but this idea, as usual, did not gain support. Instead, Attorney General George Wickersham expressed the opinion that the continuous acquisition of independent telephone companies by AT&T violates the Sherman Act. Rather than going to court, Vail and his deputy Nathan Kingsbury entered into what became known as the "Kingsbury Agreement," in which AT&T committed to:

  1. Cease purchasing independent companies.
  2. Sell its stake in WU.
  3. Allow independent telephone companies to connect to the long-distance network.

But after this critical moment for monopolies, decades of calm followed. A steady star of tariff regulation rose, assuming the presence of natural monopolies in communications. By the early 1920s, relaxations were made, and AT&T resumed the process of absorbing small independent telephone companies. This approach was established in the 1934 Act, which founded the Federal Communications Commission (FCC) of the United States, replacing the ICC as the regulator of wired communications tariffs. By that time, the Bell System controlled at least 90% of the telephone business in America by any measure: 135 out of 140 million kilometers of wire, 2.1 out of 2.3 billion monthly calls, and 990 million out of a billion dollars in annual profit. However, the primary goal of the FCC was not to restore competition but to “make available, as far as possible, to all residents of the United States, fast, efficient, governmental, and worldwide communication via wire and radio waves with adequate convenience and at a reasonable price.” If such a service could be provided by one organization, then so be it.

In the mid-20th century, local and state telecommunications regulators in the United States developed a multi-tiered system of cross-subsidization to accelerate the development of universal service. Regulatory commissions set rates based on the perceived value of the network for each customer, rather than on the cost of providing the service to that customer. As a result, business users, who relied on telephony for conducting business, paid more than individuals (for whom this service provided social conveniences). Clients in large urban markets, who had easy access to many other users, paid more than residents of small towns, despite the greater efficiency of large telephone exchanges. Long-distance users paid excessively, even though technology was constantly reducing the cost of long-distance calls, while the profits of local switches increased. This complex system of capital redistribution worked quite well as long as there was one monolithic provider within which all of this could function.

New technology

We tend to view monopoly as a decelerating force that creates inertia and lethargy. We expect that a monopoly will fiercely guard its position and the status quo, rather than serve as a driver of technological, economic, and cultural transformation. However, it's difficult to apply this perspective to AT&T at the height of its prominence, as it produced innovation after innovation, anticipating and accelerating the emergence of each new breakthrough in communications.

For example, in 1922 AT&T established a commercial broadcasting station in its building on Manhattan, just a year and a half after the first major station, KDKA by Westinghouse, opened. The following year, it used its long-distance network to relay a speech by President Warren Harding to numerous local radio stations across the country. A few years later, AT&T also made inroads into the film industry after engineers from Bell Labs developed a machine that combined video with recorded sound. The Warner Brothers studio used this "VitaPhone" to release the first Hollywood film with synchronized music, " Don Juan", followed by the first feature film in history using synchronized dialogue, "The Jazz Singer«.

Internet History: backbone network
VitaPhone

" Walter Gifford, who became president of AT&T in 1925, decided to divest the company of such ancillary enterprises as broadcasting and film, particularly to avoid scrutiny from antitrust regulators. Although the U.S. Justice Department had not threatened the company since the Kingsbury Commitment, it was wise not to attract excessive attention with actions that could be interpreted as an attempt to exploit its monopoly position in telephony to unfairly promote itself in other markets. So instead of organizing its own broadcasting, AT&T became the primary provider for transmitting signals for the American Radio Corporation RCA and other radio networks, relaying programs from their New York studios and other major cities to the affiliates of radio stations across the country.

Meanwhile, in 1927, a radiotelephony service spanned the Atlantic, initiated by a simple question posed by Gifford to his counterpart from the British postal service: "How's the weather in London?". This is certainly not "This is what God is doing!" [the first phrase officially transmitted in Morse code over the telegraph / note from the translator], but it marked an important milestone: the emergence of intercontinental conversations decades before the installation of an underwater telephone cable, albeit at a high cost and with low quality.

However, the most significant events for our story involved the transmission of large amounts of data over long distances. AT&T always aimed to increase the traffic on its long-distance networks, which served as a key competitive advantage over several still-living independent companies and led to greater profits. The easiest way to attract customers was through the development of new technology that reduced transmission costs—typically, this meant the ability to fit more conversations into the same wires or cables. But, as we have already seen, the demands for long-distance communication went beyond traditional telegraph and telephone messages from one person to another. Radio networks needed their own channels, and television was already on the horizon, with significantly larger demands for bandwidth.

The most promising way to meet new demands was the installation of coaxial cables made from concentric metallic cylinders [коаксиальный, co-axial – с общей осью / прим. перев.]. The properties of such conductors were studied as early as the 19th century by giants of classical physics: Maxwell, Heaviside, Rayleigh, Kelvin, and Thomson. It had significant theoretical advantages as a transmission line, capable of transmitting broadband signals while its structure completely shielded it from cross-interaction and interference from external signals. With the development of television in the 1920s, none of the existing technologies could provide the megahertz (or higher) bandwidth required for high-quality broadcasting. Therefore, engineers at Bell Labs set out to turn the theoretical advantages of the cable into a practical long-distance and broadband transmission line, including the creation of all necessary auxiliary equipment for generating, amplifying, receiving, and otherwise processing signals. In 1936, AT&T conducted field tests of a cable longer than 160 km, running from Manhattan to Philadelphia, with permission from the FCC. After the first system check with 27 voice circuits, engineers successfully learned to transmit video by the end of 1937.

At that time, another demand for long-distance communications with high bandwidth began to emerge: radio relay communication. Radiotelephony, used in the transatlantic communications of 1927, utilized a pair of broadcasting radio signals to create a two-way voice channel on short waves. Connecting two radio transmitters and receivers, using the entire frequency range for a single phone call, was economically unfeasible from a landline perspective. If it were possible to pack multiple conversations into one radio beam, that would be a different matter entirely. While each individual radio station would be quite expensive, hundreds of such stations would be sufficient to transmit signals across the entire United States.

Two frequency bands competed for the right to be used in such a system: ultra high frequencies (decimeter waves) UHF and microwaves (centimeter waves). The higher frequency of microwaves promised greater bandwidth but also posed greater technological challenges. In the 1930s, AT&T's responsible opinion leaned towards the safer option with UHF.

However, microwave technology made a significant leap forward during World War II due to its active use in radar systems. Bell Laboratories demonstrated the viability of microwave radio with the AN/TRC-69, a mobile system capable of transmitting eight telephone lines to another antenna in direct line of sight. This allowed military headquarters to quickly restore voice communication after relocation without waiting for cable installation (and without the risk of losing communication after a cable was cut, whether accidentally or during enemy action).

Internet History: backbone network
The deployed microwave relay station AN/TRC-6

After the war, Harold T. Friis, a Danish officer from Bell Labs, led the development of microwave relay communication. A test line stretching 350 km from New York to Boston was opened in late 1945. Waves jumped 50 km sections between ground towers—using a principle essentially similar to that of the optical telegraph, or even a chain of signal lights. Up the river to the Hudson Highlands, across the hills of Connecticut, to Mount Ashnebumskit in western Massachusetts, and then down to Boston Harbor.

AT&T was not the only company interested in microwave communication nor the only one with military experience managing microwave signals. Philco, General Electric, Raytheon, and television broadcasting companies built or planned their own experimental systems in the post-war years. Philco outpaced AT&T by constructing a communication line between Washington and Philadelphia in the spring of 1945.

Internet History: backbone network
The AT&T microwave relay station in Creston (Wyoming), part of the first transcontinental line, 1951.

For over 30 years, AT&T avoided problems with antitrust committees and other government regulators. For the most part, it was shielded by the notion of a natural monopoly – that it would be terribly inefficient to create multiple competing and unconnected systems laying their wires across the country. Microwave communication became the first serious dent in this armor, allowing many companies to provide long-distance communication without excessive costs.

Microwave transmissions significantly lowered the barrier to entry for potential competitors. Since the technology required just a chain of stations spaced 50 km apart, creating a viable system didn't necessitate purchasing thousands of kilometers of land and maintaining thousands of kilometers of cable. Moreover, the bandwidth of microwaves greatly exceeded that of traditional pair cables, as each relay station could transmit thousands of phone calls or multiple television broadcasts. The competitive advantage of the existing wired long-distance system of AT&T waned.

However, the FCC protected AT&T from the consequences of such competition for many years, making two decisions in the 1940s and 1950s. Initially, the commission refused to grant licenses, except temporary and experimental ones, to new communication providers that did not offer their services to the entire population (or, for example, only provided communication within a single enterprise). Therefore, entering this market risked losing one's license. Commission members were concerned about the emergence of a problem similar to the one that threatened broadcasting twenty years earlier and led to the creation of the FCC itself: the cacophony of interference from many different transmitters contaminating the limited radio spectrum.

The second solution concerned interconnection. Let’s remember that the Kingsbury agreement required AT&T to allow local telephone companies to connect to its long-distance network. Were these requirements applicable to microwave relay communication? The FCC ruled that they were applicable only in places where there was no adequate public telecommunications coverage. Thus, any competitor creating a regional or local network risked suddenly being cut off from the rest of the country when AT&T decided to enter their area. The only alternative to maintain connectivity was to create a new, independent national network, which was daunting to do under an experimental license.

By the end of the 1950s, there was only one major player in the long-distance telecommunications market – AT&T. Its microwave network transmitted 6,000 telephone lines on each route and reached every continental state.

Internet History: backbone network
AT&T's microwave relay network in 1960

However, the first significant obstacle to AT&T's full and comprehensive control over the telecommunications network came from an entirely different direction.

Further reading

  • Gerald W. Brock, The Telecommunications Industry (1981) The telecommunications industry: the dynamics of market structure / Gerald W. Brock
  • John Brooks, Telephone: The First Hundred Years (1976)
  • M. D. Fagen, ed., History of Engineering and Science in the Bell System: Transmission Technology (1985)
  • Joshua D. Wolff, Western Union and the Creation of the American Corporate Order (2013)

Source: habr.com

Buy reliable website hosting with DDoS protection, VPS VDS servers 🔥 Buy reliable website hosting with DDoS protection, VPS VDS servers | ProHoster