At the time of this writing, a search on a popular job site for the phrase "Network Engineer" gave out about three hundred vacancies throughout Russia. For comparison, a search for the phrase "system administrator" returns almost 2.5 thousand vacancies, and "DevOps engineer" - almost 800.
Does this mean that networkers are no longer needed in the days of the winning clouds, docker, kubernetis and the ubiquitous public Wi-Fi?
Let's figure it out (s)
Let's get acquainted. My name is Alexey, and I'm a networker.
I've been doing networking for over 10 years and have been working with various *nix systems for over 15 years (I had a chance to pick both Linux and FreeBSD). I worked in telecom operators, large companies, which are considered to be “enterprises”, and recently I have been working in “young and daring” fintech, where clouds, devops, kubernetes and other scary words that will definitely make me and my colleagues unnecessary. Some day. May be.
disclaimer: “In our life, not everything, always and everywhere, but something, sometimes in places” (c) Maxim Dorofeev.
Everything written below can and should be considered the personal opinion of the author, not claiming to be the ultimate truth, and even a full-fledged study. All characters are fictitious, all coincidences are random.
Welcome to my world.
Where can you find networkers?
1. Telecom operators, service companies and other integrators. Everything is simple: the network for them is a business. They either directly sell connectivity (operators) or provide services for launching/maintaining their customers' networks.
There is a lot of experience here, but not much money (unless you are a director or a successful sales manager). And yet, if you like networks, and you are just at the beginning of your journey, a career in support of some not very large operator will be, even now, an ideal starting point (everything is very scripted in federal ones, and there is little room for creativity). Well, stories about the fact that it is possible to grow from an engineer on duty in a few years to a C-level manager are also quite real, although rare, for obvious reasons. There is always a need for personnel, because there is still a turnover. This is both good and bad at the same time - there are always vacancies, on the other hand - often the most active/smart ones quickly enough either get promoted or go to other, "warmer" places.
2. Conditional "enterprise". It does not matter whether his main activity is related to IT or not. The main thing is that it has its own IT department, which is engaged in ensuring the operation of the company's internal systems, including networks in offices, communication channels to branches, etc. The functions of a network engineer in such companies can be "part-time" performed by a system administrator (if the network infrastructure is small, or an external contractor is engaged in it), and the network manager, if he still exists, can look after telephony and SAN at the same time (not nuacho). They pay in different ways - it strongly depends on the margin of the business, the size of the company and the structure. I worked both with companies where ciscos were regularly “loaded in barrels”, and with companies where the network was built from feces, sticks and blue electrical tape, and the servers were not updated, approximately, ever (it is necessary to say that there were no reserves either) . There is much less experience here, and it will almost certainly be in the area of a hard vendor-lock, or “how to make something out of nothing”. Personally, it seemed to me wildly boring there, although many people like it - everything is quite measured and predictable (if we are talking about large companies), “dorah-bajato”, etc. At least once a year, some major vendor says that they have come up with another mega-super-duper system that automates everything now and all system administrators and networkers can be overclocked, leaving a couple to press buttons in a beautiful interface. The reality is that, even if we ignore the cost of the solution, networkers will not go anywhere from there. Yes, it is possible that instead of the console there will again be a web interface (but not a specific piece of iron, but a large system that manages tens and hundreds of such pieces of iron), but knowledge of “how everything works inside” will still be needed.
3. Product companies, whose profit comes from the development (and, often, operation) of some software or platform - that very product. Usually they are small and nimble, they are still far from the scale of enterprises and their bureaucratization. It is here that those same devops, cubers, dockers and other terrible words are found in large numbers that will surely make the network and network engineers an unnecessary rudiment.
How is a network manager different from a sysadmin?
In the understanding of people not from IT - nothing. Both of them look at the black screen and write some spells, sometimes swearing softly.
In the understanding of programmers - perhaps the subject area. Sysadmins administer servers, networkers administer switches and routers. Sometimes the admin is bad, and everything falls down for everyone. Well, in case of any strange, networkers are also to blame. Just because fuck you, that's why.
In fact, the main difference is the approach to work. Perhaps, it is among networkers that most of all there are supporters of the approach “It works - don’t touch it!”. You can usually do something (within one vendor) in only one way, the entire configuration of the box - here it is, in the palm of your hand. The cost of an error is high, and sometimes very high (for example, you have to travel several hundred kilometers to reboot the router, and at this time several thousand people will be without communication - a situation that is quite common for a telecom operator).
In my opinion, this is why network engineers, on the one hand, are extremely strongly motivated for network stability (and changes are the main enemy of stability), and secondly, their knowledge goes more in depth than in breadth (you do not need to be able to configure dozens of different demons , you need to know the technologies and their implementation from a particular equipment manufacturer). That is why the system administrator, who googled how to register a vlan on a tsiska, is not yet a network manager. And it is unlikely that he will be able to effectively support (as well as troubleshoot) a more or less complex network.
But why do you need a network manager if you have a hoster?
For an extra money (and if you are a very large and beloved client, maybe even for free, “as a friend”), data center engineers will configure your switches to suit your needs, and, perhaps, even help raise a BGP interface with providers (if you have your own subnet of ip addresses to announce).
The main problem is that the data center is not your IT department, it is a separate company whose goal is to make a profit. Including at the expense of you as a client. The data center provides racks, provides them with electricity and cold, and also gives some "default" connectivity to the Internet. Based on this infrastructure, the data center can co-locate your equipment (colocation), rent you a server (dedicated server), or provide a managed service (for example, OpenStack or K8s). But the business of the data center (usually) is not the administration of customer infrastructure, because this process is rather labor-intensive, poorly automated (and in a normal data center everything is automated), unified even worse (each client is individual) and generally fraught with claims (“you tell me the server was set up, and now it has crashed, it's all your fault!!!111"). Therefore, if the hoster will help you with something, then he will try to make it as simple and “condo” as possible. For it is difficult to do - unprofitable, at least from the point of view of the labor costs of the engineers of this very hoster (but the situations are different, see the disclaimer). This does not mean that the host will necessarily do everything badly. But it’s not at all a fact that he will do exactly what you really needed.
It would seem that the thing is quite obvious, but several times in my practice I came across the fact that companies began to rely on their hosting provider a little more than they should, and this did not lead to anything good. It took a long time to explain in detail that no SLA will cover downtime losses (there are exceptions, but usually it is very, VERY expensive for the client) and that the hoster is not at all aware of what is happening in the customer's infrastructure (except for very general indicators). And the hoster does not make backups for you either. The situation is even worse if you have more than one hoster. In case of any problems between them, they certainly will not find out for you what went wrong.
Actually, the motives here are exactly the same as when choosing “own team of admins vs outsourcing”. If the risks are calculated, the quality suits, and the business does not mind - why not try it. On the other hand, the network is one of the most basic layers of infrastructure, and it is hardly worth giving it away to outsiders if you already support everything else yourself.
When do you need a networker?
Further, we will focus on modern product companies. With operators and enterprises, everything is plus or minus clear - little has changed there in recent years, and networkers were needed there before, they are needed now. But with those very “young and daring” things are not so simple. Often they place their infrastructure entirely in the clouds, so they don’t even need admins, except for the admins of those same clouds, of course. The infrastructure, on the one hand, is quite simple in its design, on the other hand, it is well automated (ansible / puppet, terraform, ci / cd ... well, you know). But even here there are situations when you cannot do without a network engineer.
Example 1, classic
Suppose a company starts with one server with a public ip-address, which is located in the data center. Then there are two servers. Then more ... Sooner or later, there is a need for a private network between servers. Because “external” traffic is limited both by bandwidth (no more than 100Mbps, for example) and by the amount of downloaded / uploaded per month (different hosters have different tariffs, but the bandwidth to the outside world, as a rule, is much more expensive than a private network).
The hoster adds additional network cards to the servers and includes them in their switches in a separate vlan. A “flat” LAN appears between servers. Comfortable!
The number of servers is growing, the traffic in the private network is also growing - backups, replications, etc. The hoster offers to move you to separate switches so that you do not interfere with other clients, and they do not interfere with you. The hoster puts some kind of switches and somehow configures them - most likely, leaving one flat network between all your servers. Everything works well, but at a certain point, problems begin: delays between hosts periodically grow, logs swear at too many arp packets per second, and the pentester raped your entire local area during the audit, breaking only one server.
What should I do?
Divide the network into segments - vlans. Set up your own addressing in each vlan, select a gateway that will transfer traffic between networks. On the gateway, configure acl to restrict access between segments, or even put a separate firewall next to it.
Example 1, continued
Servers are connected to the local area with one cord. The switches in the racks are somehow interconnected, but in the event of an accident in one rack, three more neighboring ones fall off. Schemes exist, but there are doubts about their relevance. Each server has its own public address, which is issued by the host and tied to the rack. Those. when moving the server, the address has to be changed.
What should I do?
Connect servers using LAG (Link Aggregation Group) with two cords to switches in the rack (they also need to be redundant). Reserve the connections between the racks, remake them with a “star” (or the now fashionable CLOS) so that the loss of one rack does not affect the others. Select "central" racks where the network core will be located and where other racks will be included. At the same time, put public addressing in order, take from the hoster (or from the RIR, if possible) a subnet, which you yourself (or through the hoster) announce to the world.
Can a “regular” system administrator who does not have deep knowledge of networks do all this? Not sure. Will the host do it? Maybe it will, but you will need a rather detailed TOR, which will also need to be compiled by someone. and then check that everything is done correctly.
Example 2: Cloudy
Suppose you have a VPC in some public cloud. To get access from the office or on-prem part of the infrastructure to the local network inside the VPC, you need to set up a connection via IPSec or a dedicated channel. On the one hand, IPSec is cheaper. no need to buy additional hardware, you can set up a tunnel between your server with a public address and the cloud. But - delays, limited performance (since the channel needs to be encrypted), plus non-guaranteed connectivity (since access goes through the regular Internet).
What should I do?
Raise the connection through a dedicated channel (for example, AWS calls it Direct Connect). To do this, find a partner operator who will connect you, decide on the connection point closest to you (both you into the operator and the operator into the cloud), and, finally, set everything up. Can all this be done without a network engineer? For sure, yes. But how to troubleshoot without it in case of problems is no longer so clear.
And there may also be problems with availability between clouds (if you have a multicloud) or problems with delays between different regions, etc. Of course, now there are many tools that increase the transparency of what is happening in the cloud (the same Thousand Eyes), but these are all network engineer tools, and not a replacement.
I could sketch out a dozen more such examples from my practice, but I think it’s clear that in a team, starting from a certain level of infrastructure development, there should be a person (or better, more than one) who knows how the network works, can configure network equipment and deal with problems if they arise. Believe me, he will have something to do
What should a networker know?
It is not at all necessary (and even sometimes harmful) for a network engineer to deal only with the network and nothing else. Even if we do not consider the option with an infrastructure that lives almost entirely in a public cloud (and, whatever one may say, it is becoming more and more popular), and take, for example, on premise or private clouds, where only “knowledge at the CCNP level "You won't leave.
In addition to, in fact, networks - although there is simply an endless field for study, even if you concentrate only on one direction (provider networks, enterprises, data centers, Wi-Fi ...)
Of course, many of you will now remember Python and other "network automation", but this is only a necessary, but not sufficient condition. In order for a network engineer to “successfully join the team”, he must be able to speak the same language with both developers and fellow admins / devops. What does it mean?
- be able to not only work in Linux as a user, but also administer it, at least at the level of a sysadmin-junior: install the necessary software, restart a fallen service, write a simple systemd-unit.
- Understand (at least in general terms) how the network stack works in Linux, how the network works in hypervisors and containers (lxc / docker / kubernetes).
- Of course, be able to work with ansible/chef/puppet or another SCM system.
- A separate line should be written about SDN and networks for private clouds (for example, TungstenFabric or OpenvSwitch). This is another huge piece of knowledge.
In short, I described a typical T-shape specialist (as it is now fashionable to say). It seems to be nothing new, however, according to the experience of interviews, not all network engineers can boast of knowledge of at least two topics from the list above. In practice, the lack of knowledge "in related areas" makes it very difficult not only to communicate with colleagues, but also to understand the requirements that the business imposes on the network as the lowest-level infrastructure of the project. And without this understanding, it becomes more difficult to reasonably defend your point of view and “sell” it to business.
On the other hand, the very habit of “understanding how the system works” gives networkers a very good advantage over various “generalists” who know about technologies from articles on Habré/medium and telegram chats, but have absolutely no idea what principles this or that software works. And the knowledge of some regularities, as you know, successfully replaces the knowledge of many facts.
Conclusions, or simply TL;DR
- A network administrator (like a DBA or a VoIP engineer) is a specialist of a rather narrow profile (unlike sysadmins / devops / SRE), the need for which does not arise immediately (and may not arise for a long time, in fact). But if it does arise, then it is unlikely to be replaced by outside expertise (outsourcing or ordinary generalist administrators, “who also look after the network”). What is somewhat sadder is that the need for such specialists is small, and, conditionally, in a company with 800 programmers and 30 devops / admins, there can only be two networkers who do their job perfectly. Those. the market was and is very, very small, and even less with a good salary.
- On the other hand, a good networker in the modern world should know not only the networks themselves (and how to automate their configuration), but also how operating systems and software interact with them, which run over these networks. Without this, it will be extremely difficult to understand what colleagues are asking from you and convey (reasonably) your wishes / requirements to them.
- There is no cloud, it's just someone else's computer. You need to understand that the use of public / private clouds or the services of hosting providers "who do everything for you" does not negate the fact that your application is still using the network, and problems with it will affect the operation of your application. Your choice is where the competence center will be located, which will be responsible for the network of your project.
Source: habr.com