If you've spent some time thinking about complex systems, you surely understand the importance of networks. Networks govern our world. From chemical reactions within a cell to the network of relationships in an ecosystem, to the trade and political networks that shape the course of history.

Or consider this article you’re reading. You probably found it in a social network, downloaded from the computer network and are currently deciphering the meaning with your neural network..

But no matter how much I've pondered networks over the years, until recently, I didn't grasp the significance of simple diffusion..

That's our topic for today: how chaotic everything moves and spreads. Here are some examples to whet your appetite:

• Infectious diseases spreading from host to host within a population.
• Memes spreading across a follower graph on social media.
• A forest fire.
• Ideas and practices penetrating culture.
• A cascade of neutrons in enriched uranium.

A brief note on form.

Unlike all my previous works, this essay is interactive [in the original article interactive examples with sliders and buttons that control objects on the screen are provided—ed. note].

So let’s get started. The first task is to develop a visual vocabulary for spreading through networks.

A simple model.

I’m sure you all know the basics of networks, that is, nodes + edges. To explore diffusion, we simply need to mark some nodes as active.Or, as epidemiologists like to say, infected.:

This activation or infection spreads through the network from node to node according to the rules we will develop below.

Real networks are generally much larger than this simple network of seven nodes. They are also far more complex. But for simplicity, we'll build a toy model here to study the lattice, that is, the lattice network.

(What the grid lacks in realism is compensated by the fact that it's easy to draw 😉

Unless otherwise specified, each network node has four neighbors, for example:

And you need to imagine that these grids stretch infinitely in all directions. In other words, we're not interested in behavior that occurs only at the edges of the network or in small populations.

Given that grids are so orderly, they can be simplified to pixels. For example, these two images represent the same network:

In one of the behavior variations, the active node always transmits the infection to its (uninfected) neighbors. But that's boring. Much more interesting things happen when transmission is probabilistic.

SIR and SIS

In models SIR (Susceptible-Infected-Removed) a node can be in three states:

• Susceptible
• Infected
• Removed

Here’s how the interactive simulation works [in the original article you can choose the infection transmission speed from 0 to 1, watch the process step by step or all at once — ed. note]:

• Nodes start as susceptible, except for a few nodes that begin as infected.
• At each time step, infected nodes have a chance to transmit the infection to each of their susceptible neighbors with a probability equal to the transmission speed.
• Infected nodes then move to the 'removed' state, meaning they are no longer able to infect others or become infected themselves.

In the context of disease, removal can mean that a person has died or that they have developed immunity to the pathogen. We say they are 'removed' from the simulation because nothing else happens to them.

Depending on what we are trying to model, another model may be needed that differs from SIR.

If we are simulating the spread of measles or a wildfire outbreak, SIR is perfectly suitable. But let's assume we are simulating the spread of a new cultural practice, such as meditation. Initially, the node (person) is susceptible because they have never done it before. Then, if they start meditating (perhaps hearing about it from a friend), we will model them as infected. But if they stop the practice, they will not die or fall out of the simulation, because in the future, they can easily take up the habit again. Thus, they revert back to the susceptible state.

This model SIS (Susceptible–Infected–Susceptible). In the classic model, there are two parameters: transmission rate and recovery rate. However, in the simulations for this article, I decided to simplify by omitting the recovery rate parameter. Instead, an infected node automatically returns to a susceptible state at the next time step unless it is infected by one of its neighbors. Additionally, we allow a node infected at step n to infect itself at step n+1 with a probability equal to the transmission rate.

Discussion

As you can see, this differs significantly from the SIR model.

Since nodes are never removed, even a very small and limited grid can sustain SIS infection for a long time. The infection simply jumps from node to node and then back again.

Despite the differences, SIR and SIS turn out to be surprisingly interchangeable for our purposes. Therefore, for the remainder of the article, we will focus on SIS – mainly because it is more resilient and thus more interesting to work with.

Critical Level

By playing around with the SIR and SIS models, you may have noticed something about the longevity of infection. At very low transmission rates, such as 10%, the infection tends to die out. Meanwhile, at higher values, such as 50%, the infection remains alive and occupies a larger portion of the network. If the network were infinite, we might imagine it continuing to spread indefinitely.

This kind of boundless diffusion has many names: 'viral', 'nuclear', or (in the title of this article) critical.

It turns out that there is a specific tipping point that separates subcritical networks (doomed to extinction) from supercritical networks (capable of infinite growth). This tipping point is called the critical threshold, and it is quite a common feature of diffusion processes in ordinary networks.

The exact value of the critical threshold varies between networks. What is common is the presence of that such a value.

[In the interactive demonstration from the original article , you can manually try to find the critical threshold of the network by adjusting the transmission rate value. It lies somewhere between 22% and 23% – editor's note.]

At 22% (and below), the infection eventually dies out. At 23% (and above), the initial infection sometimes dies out, but in most cases, it manages to survive and spread long enough to ensure its eternal existence.

(By the way, there is an entire scientific field dedicated to finding these critical thresholds for different network topologies. For a quick introduction, I recommend scrolling through the Wikipedia article on the percolation threshold).

In general, here's how it works: below the critical threshold, any terminal infection in the network is guaranteed to die out eventually (with a probability of 1). But above the critical threshold, there is a probability (p > 0) that the infection will continue indefinitely and spread arbitrarily far from the original location.

However, note that a supercritical network does not guarantee, that the infection will continue forever. In fact, it often dies out, especially in the early stages of the modeling. Let's see how this happens.

Suppose we started with one infected node and four neighbors. At the first step of the simulation, the infection has 5 independent chances to spread (including the chance to 'spread' to itself in the next step):

Now let's assume the transmission probability is 50%. In this case, at the first step, we flip a coin five times. If we get five heads, the infection will be eradicated. This happens in about 3% of cases—and that's only at the first step. For the infection that survived the first step, there is some (usually lower) probability of dying out at the second step, some (even lower) probability of dying out at the third step, and so on.

Thus, even when the network is supercritical—if the transmission probability is 99%—there is a chance that the infection will disappear.

But importantly, it will not recommendations only diminish. If we sum the probability of dying out across all steps to infinity, the result is less than 1. In other words, with a non-zero probability, the infection will continue indefinitely. That's what it means for a network to be supercritical.

SISa: spontaneous activation

Up to this point, all our simulations started with a small piece of pre-infected nodes at the center.

But what if we start from scratch? Then we will model spontaneous activation — the process by which a susceptible node becomes infected randomly (not from one of its neighbors).

This the name model of SISa. The letter “a” stands for ‘automatic’.

In the SISa simulation, a new parameter appears — the speed of spontaneous activation, which changes the frequency of spontaneous infection (the transmission speed parameter we saw earlier is also present).

What is needed for the infection to spread throughout the entire network?

Discussion

You might have noticed in the simulation that increasing the speed of spontaneous activation does not change whether the infection spreads throughout the network or not. Only the transmission speed determines whether the network is subcritical or supercritical. And when the network is subcritical (the transmission speed is less than or equal to 22%), no infection can spread throughout the grid, regardless of how often it starts.

It’s like trying to start a fire on a wet field. You can light a few dry leaves, but the flames will quickly go out because the rest of the landscape is not easily ignitable (subcritical). Whereas on a very dry field (supercritical), just one spark can ignite a raging fire.

Similar things are observed in the sphere of ideas and inventions. Often, the world is not ready for an idea, and in such cases, it can be invented over and over again, but it does not capture the masses. On the other hand, the world may be fully ready for an invention (a large hidden demand), and once it is born, it is accepted by all. In between are ideas that are invented in several places and spread locally, but not enough for any single version to cover the entire network at once. In this last category, we find, for example, agriculture and writing, which were independently invented by different human civilizations about ten and three times, respectively.

Immunity

Suppose we make some nodes completely invulnerable, that is, impervious to activation. It’s as if they are initially in a remote state, while the SIS(a) model runs on the remaining nodes.

The "immunity" slider controls the percentage of removed nodes. Try changing its value (while the model is running!) and observe how it affects the state of the network, whether it becomes supercritical or not.

Discussion

Changing the number of immune nodes completely alters the picture, determining whether the network is subcritical or supercritical. It is easy to see why. With a large number of immune nodes, there are fewer opportunities for the infection to spread to new hosts.

It turns out that this has a number of very important practical consequences.

One of them is preventing the spread of wildfires. At the local level, everyone should take their own precautions (for example, never leave an open flame unattended). But on a larger scale, individual outbreaks are inevitable. Thus, another method of protection is ensuring there are enough "gaps" (in a network of flammable materials) so that an outbreak does not engulf the entire network. This function is performed by firebreaks:

Another outbreak that is crucial to stop is infectious disease. Here, the concept of herd immunityis introduced. This is the idea that some people cannot be vaccinated (for example, their immune system is compromised), but if enough people are immune to the infection, the disease cannot spread indefinitely. In other words, it is necessary to vaccinate enough of the population to shift it from a supercritical to a subcritical state. When this happens, one patient can still become infected (for example, after traveling to another region), but without a supercritical network to grow in, the disease will infect only a small handful of people.

Finally, the concept of immune nodes explains what happens in a nuclear reactor. In a chain reaction, a decaying uranium-235 atom releases about three neutrons, which (on average) cause the fission of more than one U-235 atom. The new neutrons then cause further fission of atoms, and so on exponentially:

When creating a bomb, the essence is to ensure the unimpeded continuation of exponential growth. However, in a power plant, the goal is to produce energy without harming everyone around. For this purpose, we use control rods, made from materials capable of absorbing neutrons (such as silver or boron). Since they absorb rather than release neutrons, in our simulation, they act as insensitive nodes, thereby preventing the radioactive core from reaching a critical state.

Thus, the trick of a nuclear reactor lies in keeping the reaction close to the critical threshold by moving the control rods back and forth, ensuring that whenever something goes wrong, the rods descend into the core and stop it.

The degree of

The degree of a node is the number of its neighbors. Up to this point, we have considered networks of degree 4. But what happens if we change this parameter?

For example, we could connect each node not only to four immediate neighbors but also to four diagonally. In such a network, the degree would be 8.

Grids with degrees 4 and 8 are quite symmetrical. However, with degree 5 (for example), a problem arises: which five neighbors to choose? In this case, we choose the four nearest neighbors (N, E, S, W), and then randomly select one neighbor from the set {NE, SE, SW, NW}. The selection is made independently for each node at each time step.

Discussion

Again, it is not difficult to understand what is happening here. When each node has more neighbors, the chances of infection spreading increase—and thus the network is more likely to become critical.

However, the consequences can be unexpected, as we will see below.

Cities and network density

Until now, our networks have been completely homogeneous. Each node looks like any other. But what if we change the conditions and allow for different states of nodes throughout the network?

For instance, let's try to model cities. For this, we will increase the density in some parts of the network (higher node degree). We base this on the data that city dwellers have a wider social circle and more social interactions, than people outside of cities.

In our model, the perceptive nodes are colored based on their degree. Nodes in the 'countryside' have a degree of 4 (and are colored light gray), while nodes in 'cities' have higher degrees (and are colored darker), ranging from degree 5 at the outskirts to 8 in the city center.

Try to find a propagation speed that allows the activation to cover the cities and then not extend beyond their borders.

I find this simulation both obvious and surprising. Of course, cities better support the cultural level than rural areas — everyone knows this. What surprises me is that part of this cultural diversity arises simply based on the topology of the social network.

This is an interesting point; I will try to explain in more detail.

Here we are dealing with forms of culture that are transmitted simply and directly from person to person. For example, manners, parlor games, fashion trends, linguistic trends, small group rituals, and products that spread from mouth to mouth, plus entire packages of information that we call ideas.

(Note: the spread of information between people is greatly complicated by the media. It is easier to envision a technologically primitive environment, such as Ancient Greece, where nearly every spark of culture was transmitted through interaction in physical space).

From the simulation, I learned that there are ideas and cultural practices that can take root and spread in the city, but they simply cannot spread in the countryside (mathematically cannot). These are the same ideas and the same people. It's not that rural residents are somehow 'less bright': when interacting with the same idea, they have exactly the same chances of picking it up, as city dwellers. The issue is that the idea cannot go viral in the countryside because there aren't enough connections through which it can spread.

This is perhaps easiest to see in the fashion realm — clothing, hairstyles, etc. In the fashion network, we can pinpoint the grid edge when two individuals notice each other's outfits. In urban centers, each person can see over 1000 others daily — on the street, in the subway, in a packed restaurant, etc. In rural areas, on the other hand, each person may only see a couple of dozen others. Based on this difference alone, cities can support more fashion trends. And only the most compelling trends — with the highest transmission speed — will manage to establish themselves beyond the city.

We tend to think that if an idea is good, it will eventually reach everyone, and if an idea is bad, it will disappear. Certainly, this holds true in extreme cases, but in between there are countless ideas and practices that can only become viral in specific networks. This is truly fascinating.

Not only cities

Here we are considering the influence of network density. It is defined for a given set of nodes as the number of actual edges, divided by the number of potential edges. That is, the percentage of possible connections that actually exist.

So, we have seen that network density in urban centers is higher than in rural areas. But cities are not the only places where we find dense networks.

An interesting example is high schools. For instance, within a specific district, let's compare the network that exists among students with the network that exists among their parents. The same geographical area and similar population size, but one network is many times denser than the other. Therefore, it is not surprising that fashion and linguistic trends spread much faster among teenagers.

Similarly, elite networks tend to be much denser than non-elite networks - in my opinion, this fact is underestimated (people who are popular or influential spend more time building networks, so they have more 'neighbors' than ordinary people). Based on the simulations above, we expect elite networks to sustain some cultural forms that cannot be supported by the mainstream, simply by mathematical laws concerning average network density. I'll leave you to ponder what those cultural forms might be.

Finally, we can apply this idea to the internet, modeling it as a huge and very dense city. It’s no surprise that many new cultural forms flourish online that simply cannot be sustained in purely spatial networks: niche hobbies, higher design standards, greater awareness of injustice, etc. And it’s not just pleasant things. Just as the first cities were breeding grounds for diseases that could not spread at low population densities, the internet is a fertile environment for malignant cultural forms like clickbait, fake news, and instigating artificial outrage.

Knowledge

‘Attracting the right expert at the right moment is often the most valuable resource for creatively solving a problem.’ — Michael Nielsen, ‘Inventing Discoveries’

We often think of discovery or invention as a process that occurs in the mind of a single genius. They are struck by a flash of inspiration and — eureka! — suddenly we get a new way of measuring volume. Or the equation of gravity. Or the light bulb.

But if we adopt the perspective of the solitary inventor at the moment of discovery, we look at the phenomenon from the perspective of a node. It would be more accurate to interpret invention as a networked phenomenon.

The network is important in at least two respects. First, existing ideas must penetrate the consciousness of the inventor. These are quotes from a new article, the bibliography of a new book - the giants on whose shoulders Newton stood. Second, the network is crucial to bringing a new idea back. In the world; an invention that has not spread can hardly be considered an 'invention' at all. Thus, for both these reasons, it makes sense to model invention — or, broadly speaking, the growth of knowledge — as a diffusion process.

In a moment, I will present a rough simulation of how knowledge may spread and grow within a network. But first, I need to explain.

At the start of the simulation, each quadrant of the grid contains four experts arranged as follows:

Expert 1 comes up with the first version of the idea — let’s call it Idea 1.0. Expert 2 is the person who knows how to transform Idea 1.0 into Idea 2.0. Expert 3 knows how to convert Idea 2.0 into Idea 3.0. And, finally, the fourth expert knows how to make the final touches to create Idea 4.0.

It's similar to a technique like origami, where methods are developed and combined with other techniques to create more interesting structures. Or it could be a field of knowledge, like physics, where later work builds on the foundational work of predecessors.

The essence of this simulation is that we need all four experts to contribute to the final version of the idea. And at each stage, the idea needs to be advanced to the appropriate expert.

A few caveats. The simulation encodes a number of unrealistic assumptions. Here are just a few of them:

1. It is assumed that ideas cannot be preserved and transmitted otherwise than from person to person (i.e., there are no books or media).
2. It is assumed that there are constant experts in the population who can generate ideas, although in reality, many random factors influence the emergence of a discovery or invention.
3. The same set of SIS parameters (transmission rate, percentage of immunity, etc.) is used for all four versions of the idea, although it would likely be more realistic to use different parameters for each version (1.0, 2.0, etc.)
4. It is assumed that Idea N+1 always completely displaces Idea N, although in practice, both the old and new versions often circulate simultaneously, without a definitive winner.

... and many others.

Discussion

This is a laughably simplified model of how knowledge actually grows. Many important details remain outside the model (see above). Nonetheless, it reflects the essential nature of the process. Therefore, we can, with qualifications, discuss the growth of knowledge using our understanding of diffusion.

In particular, the diffusion model provides insight into how accelerate the process: we need to ease the exchange of ideas between expert nodes. This may mean clearing the network of dead nodes that hinder diffusion. Or it could mean placing all the experts in a city or cluster with high network density, where ideas spread quickly. Or simply gathering them in one room:

So... that's all I can tell you about diffusion.

But I have one last thought, and it is very important. It's about the growth (and stagnation) of knowledge in scientific communities. This idea differs in tone and content from everything said above, but I hope you will forgive me.

About scientific networks

The illustration shows one of the most important positive feedback cycles in the world (and it has been this way for quite some time):

The upward part of the cycle (K ⟶ T) is quite simple: we use new knowledge to develop new tools. For example, understanding semiconductor physics allows us to create computers.

However, the downward part requires some explanation. How does technological development lead to knowledge growth?

One way—perhaps the most direct—is when new technologies give us new ways to perceive the world. For instance, better microscopes allow us to look deeper inside cells, generating ideas for molecular biology. GPS trackers show how animals move. Sonar allows us to explore the oceans. And so on.

Undoubtedly, this is a vital mechanism, but there are at least two other pathways from technology to knowledge. They may not be as straightforward, but I believe they are just as important:

First. Technology leads to economic abundance (that is, wealth), enabling more people to engage in knowledge production.

If 90% of your country's population is engaged in agriculture, and the remaining 10% is involved in some form of trade (or warfare), then people have very little free time to reflect on the laws of nature. Perhaps this is why, in the past, science was primarily promoted by children from wealthy families.

The US graduates more than 50,000 PhDs each year. Instead of going to work in a factory at 18 (or earlier), a graduate student may need to be financed until they are 30 or possibly even 40— and even then, it's unclear whether their work will yield any real economic impact. Yet, this is necessary for a person to reach advanced frontiers in their discipline, especially in complex fields like physics or biology.

The fact is that from a systems perspective, specialists are costly. And the ultimate source of public wealth that funds these specialists is new technology: the plow subsidizes the pen.

SecondNew technologies, especially in the fields of travel and communications, are changing the structure of social networks in which knowledge grows. In particular, this allows experts and specialists to interact more closely with one another.

Notable inventions here include the printing press, steamships, and railroads (which facilitate travel and/or the sending of mail over long distances), telephones, airplanes, and the internet. All of these technologies contribute to the increasing density of networks, especially within specialized communities (where most knowledge growth occurs). For example, correspondence networks that emerged among European scholars at the end of the Middle Ages, or how modern physicists utilize arXiv.

Ultimately, both paths are similar. Both increase the density of specialist networks, which in turn leads to a growth in knowledge:

For many years, I had quite a disdainful attitude towards higher education. A brief stint in graduate school left a bad taste in my mouth. But now, looking back and reflecting (while trying to abstract from all personal issues), I must conclude that higher education is still extremely important.

Academic social networks (for example, research communities) are among the most sophisticated and valuable structures created by our civilization. Nowhere else have we amassed such a concentration of specialists focused on the production of knowledge. Nowhere have people developed a greater ability to understand and critique each other's ideas. This is the beating heart of progress. It is within these networks that the fire of enlightenment burns the brightest.

However, we cannot take progress for granted. If the crisis of reproducibility in experiments has taught us anything, it is that science can encounter systemic issues. This represents a kind of degradation of the network.

Let us assume we distinguish between two ways of engaging in science: true science and careerism. True science involves practices that reliably generate knowledge. It is driven by curiosity and characterized by honesty (Feynman: 'You see, I just need to understand the world'). Careerism, on the other hand, is motivated by professional ambitions and characterized by political maneuvering and scientific labels. It may look and act like science, but do not produces unreliable knowledge.

(Yes, this is an exaggerated dichotomy. Just a thought experiment. Please forgive me).

The fact is that when careerists take up space in the actual research community, they disrupt the work. They seek to promote themselves while the rest of the community strives to gain and share new knowledge. Instead of aiming for clarity, careerists complicate and confuse matters to sound more impressive. They engage in (as Harry Frankfurt would say) scientific nonsense. Therefore, we could model them as dead nodes, unresponsive to the good-faith exchange of information necessary for the growth of knowledge:

Perhaps the better model is one in which careerist nodes are not just unresponsive to knowledge, but actively disseminate false knowledge. False knowledge may include insignificant results, the importance of which is artificially inflated, or genuinely false results that arise due to manipulation or fabricated data.

Regardless of how we model them, careerists can certainly stifle our academic communities.

It's like a chain nuclear reaction that we desperately need — we need an explosion of knowledge — only our enriched U-235 has too much of the non-reactive isotope U-238, which suppresses the chain reaction.

Of course, there is no clear distinction between careerists and true scientists. Each of us harbors a bit of careerism. The question is, how long will the network endure before the dissemination of knowledge fades?

Oh, you've read to the end. Thank you for reading.

License

CC0 all rights not reserved. You can use this work however you wish :).

Acknowledgments

• Kevin Kwok and Niki Case for thoughtful comments and suggestions on various drafts.
• Nick Barr — for moral support throughout the process and for the most useful feedback on my work.
• Kit A. for pointing out the phenomenon of percolation and the flow threshold.
• Jeff Lonsdale for the reference to this essay, which (despite its many shortcomings) became the main impetus for working on this post.

Samples of interactive essays

• All of Niki Case's work, especially “The Polygon Parable” (with Vi Hart) and “Designing a Better Voting Bulletin”. This sets a high standard for what an interactive essay can be.
• Distill.pub: exceptionally high-quality interactive descriptions of machine learning.
• Bret Victor's classic work “Up and Down the Ladder of Abstraction”. I didn't do very well at climbing the ladder, but there's always one more attempt left.

Source: habr.com