The National Environmental Satellite Data Information Service (NESDIS) reduced its Red Hat Enterprise Linux (RHEL) configuration management costs by 35% by moving from Puppet Enterprise to Ansible Tower. In this "how we did it" video, Systems Engineer Michael Rau explains the rationale for doing this migration, shares useful tips and lessons learned from migrating from one SCM to another.
In this video you will learn:
- how to justify to management the feasibility of switching from Puppet Enterprise to Ansible Tower;
- what strategies to use for the smoothest possible transition;
- tips for transcoding PE manifests in Ansible Playbook;
- recommendations for the optimal installation of Ansible Tower.
Hello everyone, my name is Michael Rau, I am a senior systems engineer at ActioNet, which works for the National Oceanic and Atmospheric Administration (NOAA) NESDIS service. Today we will talk about string trimming - my own experience of migrating from Puppet Enterprise to Ansible Tower. The theme of this presentation is to "take a look at my scars" after I made this transition earlier in the year. I want to share what I learned during this process. So when you take on this, using my experience, you can make the transition without too much work.
You see slides like this one at the start of every Ansible Fest presentation. This slide outlines the history of my company's automation. I'm not new to this because I've been using Puppet/Puppet Enterprise since 2007. I started working with Ansible in 2016, and like many other users of this product, I was attracted by the possibility of "tricks" using the command line and simple scripts (playbooks). At the end of 2017, I reached out to my management about good reasons to move to Ansible Tower. In a minute, I will talk about the reasons that prompted me to take this step. It took a few more months after receiving the approval of management to complete the plan, and I made the transition in January-February of this year. So, we have completely abandoned Puppet in favor of Ansible, and this is a great thing.
What attracts me most about Ansible is the ability to write and use roles and scripts (playbooks). Roles are great for creating different but related tasks (tasks) and putting all the data related to these tasks in one place. Playbook is a YAML syntax, a script file that describes actions for one or more hosts. I talk about these features to users, primarily software developers. Ansible Tower gives you the ability to say "no, you don't have shell access, but I give you the ability to start all Tower processes and restart the service when you need to." I will tell you about the working environment and the equipment we use.
These are a federal LAN, 7 physical sites connected via cloud MPLS, 140 RHEL servers, 99% of which are virtual (vSphere), SuperMicro hardware, NexentaStore network storage, a set of Cisco, Arista and Cumulus switches and Fortinet UTM unified threat management tools on each site .
The federal network means that I must use all the means of information protection provided for by legislative acts. You should be aware that Puppet Enterprise does not support most of the hardware we use. We are forced to use budgetary hardware, since government agencies are experiencing problems with financing this item of expenditure. So we buy SuperMicro grade hardware and assemble our hardware from parts that are guaranteed to be maintained by government contracts. We use Linux, and this is one of the important reasons for switching to Ansible.
Our history with Puppet is as follows.
In 2007 we had a small network of 20-25 nodes where we deployed Puppet. Basically, these nodes were just RedHat "boxes". In 2010, we started using the Puppet Dashboard web interface for 45 nodes. As the network continued to expand, we migrated to PE 2014 in 3.3, making a complete migration with a manifest rewrite for 75 nodes. This had to be done because Puppet likes to change the rules of the game, and in this case they completely changed the language. A year later, when support for version 3 of Puppet Enterprise ended, we were forced to migrate to PE 2015.2. I had to rewrite the manifest again for new servers and purchase a license with a margin of 100 nodes, although at that time we had only 85 nodes.
Only 2 years have passed, and again we had to do a lot of work on the transition to the new version of PE 2016.4. We bought a license for 300 nodes, with only 130. We again had to make major changes to the manifest because the new language version had a different syntax than the language of the 2015 version. As a result, our SCM moved from the SVN version control system to Bitbucket (Git). That was our "relationship" with Puppet.
So, I had to explain to management why we need to migrate to another SCM using the following arguments. The first is the high price of the service. I talked to the guys at RedHat, and they said that the cost of maintaining a network of 300 nodes with Ansible Tower is half the cost of Puppet Enterprise. If you also buy Ansible Engine, the cost will be about the same, but you will get many more features than PE. Since we are a state-owned company financed from the federal budget, this is a pretty weighty argument.
The second argument is universality. Puppet only supports hardware that has a Puppet agent. This means that an agent must be installed on all switches, and it must be the latest version. And if some of your switches support one version, and some of them support another, you will need to install a new version of the PE agent on them so that they can all work on the same SCM system.
The Ansible Tower system works differently because it does not have any agents, but instead there are modules that support Cisco switches and all other switches. This SCM supports Qubes OS, Linux and 4.NET UTM. Ansible Tower also supports NexentaStore network storage controllers based on the Illumos kernel, an open-source Unix-based operating system. This is very little support, but Ansible Tower provides it anyway.
The third argument, which is very important both for me and for our administration, is ease of use. I learned Puppet modules and manifest code for 10 years, but learned Ansible in a week because this SCM is so much easier to work with. If you run executables, of course, if you do not do this unnecessarily, then reasonable and responsive handlers work with them. Playbooks based on YAML are easy to learn and fast to use. Those who have never heard of YAML before can simply read the scripts and easily understand how it works.
To be honest, Puppet makes your job as a developer much more difficult because it is based on the use of Puppet Master. This is the only machine authorized to communicate with Puppet agents. If you have made any changes to the manifest and want to test your code, you must rewrite the code for the Puppet Master, that is, configure the /etc/hosts Puppet master file to connect all clients and start the Puppet Server service. Only after that you will be able to test the operation of network equipment on one host. This is a rather painful procedure.
In Ansible, everything is much simpler. All you have to do is develop code for a machine that has the ability to SSH into the host under test. This is much easier to work with.
The next big plus of Ansible Tower is the ability to use the support system you already have and keep the existing hardware configuration. This SCM uses all available information about your infrastructure and equipment, virtual machines, servers, etc. without any additional actions. It can talk to your RH Satellite servers, if any, and gives you an integration that you never get with Puppet.
Another important thing is detailed control. You know that Puppet is a modular system, it's a client/server application, so you should define the existing aspects of how all your machines work in one long manifest. At the same time, the state of each individual element of the system must be tested every half an hour - this is the default period. Here's how Puppet works.
Tower gets rid of that. You can run a variety of processes on a wide variety of hardware without restrictions, you can do the main work, run other important processes, set up a security system, work with databases. You can do everything that is difficult in Puppet Enterprise. So, if you have configured on one host, it will take time for the changes to take effect on other hosts. In Ansible, all changes take effect at the same time.
Finally, consider the security module. In Ansible Tower, it is simply amazing, with great precision and care. You can grant users access to specific services or to specific hosts. I do this with my employees who are used to working on Windows by restricting their access to the Linux shell. I provide them with access to the Tower so that they can only do the work and run only those services that are within their competence.
Let's look at things that need to be done beforehand to ease the transition to Ansible Tower. First of all, you need to prepare your equipment. If some elements of your infrastructure are not yet in the database, they need to be added there. There are systems that do not change their characteristics and therefore are not in the Puppet database, but if you do not add them there before moving to Tower, you will lose a number of advantages. This may be a "dirty", preliminary database, but it should contain information about all the equipment you have. Therefore, you should write a dynamic hardware script that will automatically make all infrastructure changes to the database, then Ansible will know which hosts should be on the new system. You won't need to tell this SCM which hosts you have added and which hosts no longer exist, because it will know all this automatically. The more data there is in the database, the more useful and flexible Ansible will be. It works as if it were just reading a hardware status barcode from a database.
Spend some time getting familiar with how Ansible's command line works. Run some custom commands to test the hardware script, write and run some simple but useful playbook scripts, use Jinja2 templates where appropriate. Try writing a role and script for a complex, multi-step process using a standard, common hardware configuration. Play with these things, test how it works. In this way, you will learn how to work with the library creation tools used in Tower. I have already said that it took me about 3 months to prepare for the transition. I think that based on my experience, you will be able to do it faster. Do not consider this time wasted, because later you will experience all the benefits of the work done.
Next, you need to decide what you expect from Ansible Tower, what exactly this system should do for you.
Do you need to deploy the system on empty hardware, on empty virtual machines? Or do you want to keep the original operating conditions and settings of existing equipment? This is a very important aspect for the operation of public companies, so you must be sure that you can migrate and deploy Ansible on an existing configuration. Identify routine administrative processes that you want to automate. Find out if you need to deploy specific applications and services on the new system. Make a list of what you want to do and prioritize.
Then, start writing the scripts and roles that will ensure that the tasks you have planned will be completed. Combine them into Projects, a logical collection of relevant playbooks. Each Project will refer to a separate Git repository or another repository depending on which code manager you are using. You can manage playbook scripts and playbook directories by manually placing them in the Project Base Path on the Tower server, or by placing the playbook in any source code management (SCM) system supported by Tower, including Git, Subversion, Mercurial, and Red Hat Insights. Within one Project, you can place as many scripts as you want. For example, I created one base Project, in which I placed the script for the main elements of RedHat, the script for the Linux base, the scripts for the rest of the basics. Thus, in one project there were a variety of roles and scripts that were managed from one Git repository.
Run all of these things from the command line, it's a good way to check if they work. This will prepare you for the Tower installation.
Let's talk a little about transcoding the Puppet manifest, because I spent a lot of time on this until I figured out what really needs to be done.
As I said before, Puppet stores all the settings and hardware parameters in one long manifest, and everything that this SCM should do is stored in this manifest. When you migrate, you don't need to cram all your tasks into one list, instead think about the structure of the new system: roles, scripts, tags, groups, and what should go there. Some of the autonomous network elements should be organized into groups that can be scripted. More complex infrastructure elements that use a lot of resources, including including self-contained classes, can be combined into roles. Before migrating, you need to decide on this. If you are creating large roles or scripts that do not fit on one screen, you should use tags to be able to capture specific parts of the infrastructure.
18:00
Some ads π
Thank you for staying with us. Do you like our articles? Want to see more interesting content? Support us by placing an order or recommending to friends, , a unique analogue of entry-level servers, which was invented by us for you: (available with RAID1 and RAID10, up to 24 cores and up to 40GB DDR4).
Dell R730xd 2 times cheaper in Equinix Tier IV data center in Amsterdam? Only here in the Netherlands! Dell R420 - 2x E5-2430 2.2Ghz 6C 128GB DDR3 2x960GB SSD 1Gbps 100TB - from $99! Read about
Source: habr.com