The National Environmental Satellite, Data, and Information Service (NESDIS) reduced its costs for managing Red Hat Enterprise Linux (RHEL) configuration by 35% by transitioning from Puppet Enterprise to Ansible Tower. In this 'how we did it' video, Systems Engineer Michael Rau explains the rationale behind this migration, shares useful tips, and lessons learned from moving from one SCM to another.
In this video you will learn:
- how to justify the transition from Puppet Enterprise to Ansible Tower to management;
- which strategies to use for a smooth transition;
- tips for transcoding PE manifests into Ansible Playbooks;
- recommendations for the optimal installation of Ansible Tower.
Hello everyone, my name is Michael Rau, and I am a Senior Systems Engineer at ActioNet, which is working with the National Oceanic and Atmospheric Administration (NOAA) NESDIS service. Today we will discuss string trimming – my own experience of migrating from Puppet Enterprise to Ansible Tower. The focus of this presentation is to 'look at my scars' left after I made this transition earlier this year. I want to share what I learned during this process. So when you undertake something similar, using my experience, you will be able to make the transition with ease.
You see slides like this at the beginning of every presentation at Ansible Fest. This slide outlines the automation story of my company. I'm not new to this as I have been using Puppet/Puppet Enterprise since 2007. I started working with Ansible in 2016, and like many other users of this product, I was attracted by the 'tricks' possible with the command line and simple playbooks. At the end of 2017, I approached my management with serious reasons to transition to Ansible Tower. In a moment, I will share the reasons that drove me to this step. After receiving management's approval, it took a few more months to execute the plan, and I completed the transition in January-February of this year. So, we have completely moved away from Puppet in favor of Ansible, and it is a great thing.
What attracts me most about Ansible is the ability to write and use roles and playbooks. Roles are great for creating various yet interconnected tasks and consolidating all the data related to those tasks in one place. A playbook is a YAML syntax file-script that describes actions for one or more hosts. I introduce these capabilities to users, primarily software developers. Ansible Tower allows me to say, 'No, you do not have shell access, but I provide you the ability to start all Tower processes and restart the service whenever you need.' I will tell you about the working environment and the equipment we use.
This is a federal LAN with seven physical sites connected via cloud MPLS, comprising 140 RHEL servers, 99% of which are virtual (vSphere), SuperMicro hardware, NexentaStore network storage, a set of Cisco, Arista, and Cumulus switches, along with unified threat management tools like Fortinet UTM at each site.
The federal network means I must employ all the security measures stipulated by legislation. You should be aware that Puppet Enterprise does not support much of the hardware we use. We are forced to use budget hardware because government entities face funding issues for this expense. Thus, we purchase SuperMicro-class 'iron' and build our equipment from individual parts, the maintenance of which is guaranteed by government contracts. We use Linux, and this is one of the key reasons for our transition to Ansible.
Our history of working with Puppet is as follows.
In 2007, we had a small network of 20-25 nodes, where we deployed Puppet. These nodes mainly consisted of just RedHat "boxes". By 2010, we started using the Puppet Dashboard web interface for 45 nodes. As the network continued to grow, in 2014 we transitioned to PE 3.3, completely rewriting the manifest for 75 nodes. This was necessary because Puppet likes to change the rules, and in this case, they completely changed the language. A year later, when support for Puppet Enterprise version 3 was discontinued, we had to migrate to PE 2015.2. We had to rewrite the manifest for the new servers again and purchase a license sufficient for 100 nodes, even though we only had 85 nodes at that time.
Only two years passed, and we had to undertake significant work again to transition to the new version PE 2016.4. We purchased a license for 300 nodes while only having 130. We again had to make serious changes to the manifest because the new version of the language had syntax different from the 2015 version. As a result, our SCM transitioned from using SVN for version control to Bitbucket (Git). Such were our "relationships" with Puppet.
So, I had to explain to management why we needed to switch to another SCM, using the following arguments. First – the high cost of the service. I spoke with the guys from RedHat, and they said that maintaining a network of 300 nodes with Ansible Tower costs half as much as Puppet Enterprise. If we also purchase Ansible Engine, the cost will be roughly the same, but you will get many more features than with PE. Since we are a government company funded by federal budget, this is a pretty weighty argument.
The second argument is versatility. Puppet only supports the equipment on which the Puppet agent is installed. This means that an agent must be installed on all switches, and it must be the latest version. If some of your switches support one version and others support another, you will need to install the new version of the PE agent on them so that they can all operate within the same SCM system.
Ansible Tower operates differently because it has no agents but does have modules that support Cisco switches and all other switches. This SCM supports Qubes OS, Linux, and .NET UTM. Ansible Tower also supports NexentaStore storage controllers based on the Illumos kernel – an open-source operating system based on Unix. This is a very limited support, but Ansible Tower still provides it.
The third argument, which is very important for both me and our administration, is the ease of learning. I spent 10 years mastering Puppet modules and manifest code, but I learned Ansible in a week because this SCM is much easier to work with. If you're running executables, of course, and not doing so unnecessarily, then sensible and responsive handlers work with them. Playbook scripts based on YAML are characterized by ease of learning and speed of use. Those who have never heard of YAML before can simply read the scripts and easily understand how they work.
Honestly, Puppet significantly complicates your work as a developer because it relies on the use of Puppet Master. This is the only machine authorized to communicate with Puppet agents. If you've made changes to the manifest and want to test your code, you must rewrite the code for Puppet Master, that is, configure the Puppet master file /etc/hosts to connect all clients and start the Puppet Server service. Only after that can you test the network equipment on a single host. This is quite a painful procedure.
In Ansible, everything is much simpler. All you need to do is write code for a machine that can connect to the test host via SSH. It's much easier to work with this.
The next big advantage of Ansible Tower is the ability to leverage your existing support system and maintain your current hardware configuration. This SCM seamlessly utilizes all existing information about your infrastructure and hardware, virtual machines, servers, etc. It can communicate with your RH Satellite servers, if you have any, and provides you with integrations that you will never achieve while working with Puppet.
Another important aspect is detailed control. You know that Puppet is a modular system; it is a client-server application, so you need to define the existing operational aspects of all your machines in one long manifest. Moreover, the state of each individual element in the system should be tested every half hour – this is the default period. This is how Puppet works.
Tower frees you from this. You can execute a wide variety of processes on diverse hardware without limitations, perform core tasks, run other critical processes, configure security systems, and work with databases. You can achieve everything that poses certain challenges in Puppet Enterprise. For instance, if you configure one host, it will take time for the changes to take effect on the other hosts. In Ansible, all changes take effect simultaneously.
Finally, let’s consider the security module. In Ansible Tower, it is implemented brilliantly, with great accuracy and thoroughness. You can grant users access to specific services or specific hosts. I do this with my employees who are used to working on Windows, restricting their access to the Linux shell. I ensure they have access to Tower so they can perform only the work and run only the services that pertain to their competencies.
Let's consider the things that need to be done in advance to facilitate the transition to Ansible Tower. First and foremost, you need to prepare your hardware. If any elements of your infrastructure are not yet present in the database, they need to be added. There are systems that do not change their characteristics and therefore are absent from the Puppet database, but if you don't add them before transitioning to Tower, you will miss out on several benefits. It may be a 'dirty' preliminary database, but it must contain information about all the hardware you have. Therefore, you should write a dynamic hardware script that automatically records all changes in the infrastructure into the database, so Ansible will know which hosts should exist in the new system. You will not need to inform this SCM about which hosts you have added and which hosts no longer exist, as it will learn all this automatically. The more data there is in the database, the more useful and flexible Ansible will be. It operates as if it simply reads the state of the hardware from the database.
Spend some time getting familiar with the command line in Ansible. Run some special commands to test the hardware script, write and execute simple but useful playbook scenarios, and use Jinja2 templates where appropriate. Try writing a role and a script for a complex multi-stage process using a standard, commonly encountered hardware configuration. Play around with these things, test how they work. This way, you will learn to work with the tools to create libraries used in Tower. I've already mentioned that my preparation for the transition took about 3 months. Based on my experience, I think you will be able to accomplish this more quickly. Don't consider this time wasted, as you will later feel all the benefits of the work done.
Next, you need to decide what you expect from Ansible Tower, and what exactly this system should do for you.
Do you need to deploy the system on bare metal, on empty virtual machines? Or do you want to maintain the original working conditions and settings of the existing hardware? This is a very important aspect for governmental organizations, so you must ensure that you can perform the migration and deploy Ansible on the existing configuration. Identify the routine administrative processes you want to automate. Determine whether you need to deploy specific applications and services on the new system. Compile a list of what you want to achieve and prioritize your tasks.
Next, start writing the scripts and roles that will ensure the execution of your planned tasks. Organize them into Projects, a logical collection of related playbook scripts. Each Project will correspond to a separate Git repository or another repository depending on which code manager you are using. You can manage playbook scripts and directories by manually placing them in the Project Base Path on the Tower server or by storing the playbook in any source control system (SCM) supported by Tower, including Git, Subversion, Mercurial, and Red Hat Insights. Within a single Project, you can include as many scripts as you want. For example, I created one core Project where I stored a script for core RedHat elements, a script for the Linux basics, and scripts for other key metrics. Thus, a single project contained a diverse set of roles and scripts managed from one Git repository.
Run all these items through the command line; it's a good way to verify their functionality. This way, you will prepare for the Tower installation.
Let's discuss transcoding the Puppet manifest a bit, as I spent a lot of time on this before realizing what really needed to be done.
As I mentioned earlier, Puppet stores all configurations and hardware parameters in one long manifest, and this manifest contains all the tasks that this SCM should perform. When transitioning, you don't need to cram all your tasks into a single list; instead, think about the structure of the new system: roles, playbooks, tags, groups, and what should be included. Some of the standalone network elements should be grouped for which playbooks can be created. More complex infrastructure elements that involve a significant amount of resources, including standalone classes, can be incorporated into roles. You need to determine this before migration. If you are creating expansive roles or playbooks that do not fit on one screen, you should use tags to capture specific parts of the infrastructure.
18:00
A little advertisement 🙂
Thank you for staying with us. Do you enjoy our articles? Want to see more interesting content? Support us by placing an order or recommending us to your friends, , a unique entry-level server alternative that we have created for you: (options available with RAID1 and RAID10, up to 24 cores and up to 40GB DDR4).
Dell R730xd at half the price in the Equinix Tier IV data center in Amsterdam? Only with us in the Netherlands! Dell R420 — 2x E5-2430 2.2GHz 6C 128GB DDR3 2x960GB SSD 1Gbps 100TB — from $99! Read about how
Source: habr.com
