It may seem that Terraform developers offer quite convenient best practices for working with AWS infrastructure. However, there's a catch. Over time, the number of environments increases, each with its own peculiarities. Almost a copy of the application stack appears in a neighboring region. The Terraform code needs to be carefully copied and edited according to the new requirements or made into a snowflake.
My presentation will be about Terraform patterns to combat chaos and manual routines in large, long-term projects.
Video:
I am 40 years old and have 20 years of experience in IT. I've worked at Ixtens for 12 years. We specialize in ecommerce-driven development. And I've been practicing DevOps for 5 years.
My talk will be about my experience in a project at a company whose name I wonāt disclose due to a non-disclosure agreement.
The numbers on the slide are there to illustrate the scale of the project. Everything I will discuss further is related to Amazon.
I joined this project 4 years ago. At that time, we were in the midst of refactoring the infrastructure because the project had grown. The patterns we used were no longer suitable. Given the planned growth of the project, something new had to be devised.
Thanks to Matvey, who shared yesterday what was happening at Dodo Pizza. This was similar to our situation 4 years ago.
Developers came in and started creating infrastructure code.
The most obvious reasons why this was needed were time to market. We needed to ensure that the DevOps team was not a bottleneck during deployment. Additionally, Terraform and Puppet were used at the very first level.
Terraform is an open-source project by HashiCorp. For those who are unfamiliar with it, the next few slides will explain.
Infrastructure as code means we can describe our infrastructure and instruct some robots to create the resources we have defined.
For example, we need the virtual machine. We will define it and add several required parameters.
After that, we will configure access to Amazon in the console and request Terraform plan. Terraform plan will say: 'Okay, for your resource, we can do the following things.' And at least one resource will be added with no changes anticipated.
Once everything is satisfactory, you can ask Terraform to apply, and Terraform will create an instance for you, giving you a virtual machine in your cloud.
Next, our project evolves. We are implementing some changes there. We request more instances, and we add 53 records.
And we repeat. We request a plan. We see what changes are planned. We apply them. Thus, our infrastructure grows.
Terraform uses something called state files. That is, all changes sent to Amazon are saved in a file where, for each resource you've described, there are corresponding resources created in Amazon. This way, when changing the description of a resource, Terraform knows exactly what needs to be changed in Amazon.
These state files were initially just files. We stored them in Git, which was extremely inconvenient. Someone was constantly forgetting to commit their changes, leading to many conflicts.
Now there is an option to use a backend, meaning Terraform specifies which bucket and key to save the state file to. Terraform will take care of fetching this state file, performing all the magic, and placing the final result back.
Our infrastructure is growing. Here is our code. Now we don't just want to create a virtual machine; we want to have a testing environment.
Terraform allows you to create something like a module, meaning describing the same thing in a specific folder.
For example, in testing, you can call this module and get the same result as if you had executed Terraform apply within the module. Here is the code for testing.
For production, we can send some changes there because large instances are not needed in testing; in production, large instances will be useful.
Then I will return to the project. It was a complex task; the infrastructure was planned to be very large. It was necessary to arrange all the code in a way that would be convenient for everyone: both for those who maintain this code and for those who make changes. It was planned that any developer could go and adjust the infrastructure as needed for their part of the platform.
This is the directory tree recommended by HashiCorp itself, especially if you have a large project and it's worth splitting the entire infrastructure into smaller pieces, with each piece described in a separate folder.
With an extensive library of resources, you can call essentially the same thing in both testing and production.
In our case, this was not quite suitable because the test stack for developers or testing needed to be obtained more simply. We didnāt want to go through folders and apply them in the necessary order while worrying about the database starting up, and then the instance that uses this database. Therefore, all testing was initiated from a single folder. The same modules were called, but everything was executed in one run.
Terraform takes care of all dependencies. It always creates resources in the order necessary to obtain, for example, an IP address from a newly created instance and to get that IP address in a route53 record.
Moreover, the platform is quite large. Launching a test stack, even for an hour, let alone for 8 hours, is quite an expensive matter.
We automated this process, and the Jenkins job allowed us to launch the stack. It required running a pull request with the changes that the developer wanted to test, specifying all the necessary options, components, and sizes. If performance testing was desired, more instances could be used. If it was just necessary to check that a certain form opens, it could be started with minimal resources. It was also possible to specify whether a cluster was needed or not, etc.
Then Jenkins triggered a shell script that slightly modified the code in the Terraform folder. It removed unnecessary files and added the required ones. Subsequently, with one run of Terraform apply, the stack was launched.
And then there were other steps, which I donāt want to delve into.
Because we required a bit more options for testing than in production, we had to create module copies so that features only needed for testing could be added to these copies.
Thus, in testing, we sort of wanted to test the changes that would eventually go to production. But in reality, one thing was tested while something slightly different was applied in production. There was a small disconnect in that all changes in production were implemented by the operations team. Sometimes, the changes that should have transitioned from testing to production remained in a different version.
Additionally, there was an issue where a new service was added that slightly differed from an existing one. Instead of modifying the current module, a copy had to be made and the necessary changes added.
Essentially, Terraform is not a true language. It is a declaration. When we need to declare something, we do so. And it all works.
At one point, when one of my pull requests was being discussed, a colleague mentioned that we shouldn't create snowflakes. I was curious about what he meant. Thereās a scientific fact that no two snowflakes in the world are identical; they all have slight differences. As soon as I heard this, I realized the weight of Terraform code. Because when it was necessary to transition from one version to another, Terraform required breaking changes, meaning the code was no longer compatible with the next version. This necessitated a pull request covering almost half of the files in the infrastructure to bring it to the next version of Terraform.
And after such a snowflake appeared, all the Terraform code we had turned into a huge pile of snow.
For an external developer who is outside of operations, this doesn't carry much significance because they made a pull request, their resource started up, and that's it; it's no longer their concern. However, for the DevOps team, who ensure everything is fine, all these changes need to be implemented. The cost of these changes increased significantly with each additional snowflake.
There's a story about a student in a seminar who draws two perfect circles on the board with chalk. The teacher is amazed at how he managed to draw such perfect circles without a compass. The student replies: "It's very simple; I spent two years in the army cranking a meat grinder."
Out of the four years I've been involved in this project, I've been working with Terraform for about two years. Of course, I have some tips and tricks on how to simplify Terraform code, work with it as a programming language, and reduce the burden on developers who must keep this code up to date.
The first thing I would like to start with is Symlinks. Terraform has a lot of repetitive code. For example, the provider call is almost the same in every place where we create a piece of infrastructure. It makes sense to move it to a separate folder. And wherever the provider is needed, create Symlinks to this file.
For instance, you have an assume role in production that allows you to gain access rights to some external Amazon account. By changing one file, all the remaining ones in the resource tree will have the required permissions so that Terraform knows which Amazon segment to address.
Where do Symlinks not work? As mentioned, Terraform has state files. And they are really, really great. However, the issue is that Terraform initializes the backend first and cannot use any variables in its parameters; they always have to be written as text.
As a result, when someone creates a new resource, they copy part of the code from other folders. They may make a mistake with the key or the bucket. For example, they bring a sandbox scenario into production, and that can lead to the bucket in production being used from the sandbox. Of course, this will be quickly discovered. It can be fixed somehow, but nonetheless, it's a waste of time and resources to some extent.
What can we do next? Before working with Terraform, it needs to be initialized. During initialization, Terraform fetches all the plugins. At some point, they broke away from a monolithic structure into a more microservices architecture. You always need to run Terraform init to pull in all the modules and plugins.
You can use a shell script that, firstly, can fetch all the variables. A shell script is not limited in any way. Secondly, paths. If we always use the path in the repository as the key to the state file, then the error here will be excluded.
Where to get the data? JSON file. Terraform allows you to write infrastructure not only in HCL (HashiCorp Configuration Language) but also in JSON.
JSON is easily readable from a shell script. Accordingly, you can store a configuration file with the bucket somewhere and use that bucket in both Terraform code and the shell script for initialization.
Why is it important to have a bucket for Terraform? Because there is something called remote state files. That is, when I launch a resource, I need to specify a lot of mandatory parameters to tell Amazon, "Please launch an instance."
These identifiers are stored in another folder. I can say, "Terraform, please go to the state file of that resource and retrieve these identifiers for me." This creates a certain unification among different regions or environments.
It is not always possible to use a remote state file. For example, you manually created a VPC. The Terraform code that creates a VPC might create such a different VPC that you would have to spend a lot of time adjusting one to match the other. Therefore, you can use the following trick.
That is, create a module that, in a sense, makes a VPC and gives you identifiers, while in reality, there is just a file with hardcoded values that can be used to create the same instance.
It is not always necessary to store the state file in the cloud. For example, when testing modules, you can initialize the backend so that the file will simply be saved on disk during the testing period.
Now a bit about testing. What can be tested in Terraform? Probably a lot can be tested, but I will talk about these four items.
HashiCorp has guidelines on how to format Terraform code. Terraform fmt allows you to format the code you are editing according to these guidelines. Accordingly, the tests must check whether the formatting complies with what HashiCorp intended, so you don't have to change the placement of brackets, etc.
Next is Terraform validate. It does a bit more than just syntax checking, like ensuring all brackets are matched. What is important here? Our infrastructure is quite sprawling. There are many different folders, and in each, you need to run Terraform validate.
To speed up testing, we run several processes in parallel using concurrency.
Concurrency is a very cool feature, make sure to use it.
But every time Terraform initializes, it contacts HashiCorp and asks, "What are the latest versions of the plugins? And the plugin I have cached ā is it the right one or not?" This adds a delay at every step.
If Terraform is told where the plugins are located, it will say, "Okay, probably this is the latest version available. I won't go anywhere; I'll start validating your Terraform code right away."
To populate the folder with the necessary plugins, we have a very simple Terraform code that just needs to be initialized. Here, of course, all providers participating in your code must be specified; otherwise, Terraform will say, "I don't know this provider because it's not in the cache."
Next is the Terraform plan. As mentioned, development is cyclical. We create code with changes, and then we need to know what changes are planned for the infrastructure.
When the infrastructure is very, very large, changing one module, fixing a test environment, or a specific region can break something adjacent. Therefore, the Terraform plan should cover the entire infrastructure and show what changes are planned.
This can be done smartly. For example, we wrote a Python script that resolves dependencies. Depending on whether a Terraform module or just a specific component has changed, it generates plans for all dependent folders.
Terraform plans should be generated on demand. At least, that's what we do.
Tests are certainly good to run for every change and every commit, but plans are quite expensive. In the pull request, we say, "Please provide me with the plans." A bot starts running and sends comments or attaches all the plans that are anticipated from your changes.
The plan is a fairly expensive process. It takes time because Terraform queries Amazon, asking, "Does this instance still exist? Are these autoscale parameters exactly correct?" To speed this up, you can use a parameter like refresh=false. This means Terraform will pull the state from S3 and will trust that the state matches what is actually in Amazon.
This Terraform plan executes much faster, but the state must align with your infrastructure, meaning that at some point, you need to run Terraform refresh. Terraform refresh does exactly that, ensuring the state matches what exists in the actual infrastructure.
And we need to address security. This should have been the starting point. Where you run Terraform and it interacts with your infrastructure, there is a vulnerability. Essentially, you are executing code. If a pull request contains malicious code, it could run on infrastructure that has too much access. So be cautious about where you execute the Terraform plan.
Next, I want to discuss testing user-data.
What is user-data? In Amazon, when we create an instance, we can send some metadata from the instance. When the instance starts, cloud-init is typically always present on these instances. Cloud-init reads this message and says, 'Okay, today I'm a load balancer.' Accordingly, it performs certain actions.
However, unfortunately, when we run Terraform plan and Terraform apply, user-data appears as a jumble of numbers. It simply sends you a hash. All you can see in the plan is whether there are any changes or if the hash remains the same.
And if you ignore this, then a corrupted text file could be sent to Amazon, affecting the actual infrastructure.
As an option, when executing, you can specify not the entire infrastructure, but just the template. In the code, you can say, 'Please display this template for me.' This way, you can get a printout of what your data will look like on Amazon.
Another option is to use a module to generate user-data. You apply this module, obtain a file on disk, and compare it with a reference. This way, if any junior developer decides to make some minor adjustments to user-data, your tests will indicate, 'Okay, there are some changes here and here ā that's fine.'
Next, I want to talk about automating Terraform apply.
Of course, itās quite daunting to execute Terraform apply automatically because who knows what changes might be introduced and how detrimental they could be to the live infrastructure.
For a test environment, this is perfectly fine. That is, the job that creates the test environment is essential for all developers. And expressions like 'everything worked for me' are not just funny memes, but proof that someone took the effort to set up the stack, ran some tests on that stack, and verified that everything is fine, saying, 'Okay, the code I'm releasing has been tested.'
In production, sandbox, and other environments that are more critical for the business, you can partially apply some resources quite safely, as this does not lead to someone dying. These include: auto-scaling groups, security groups, roles, Route 53, and the list can be quite extensive. But keep an eye on whatās happening, and read the reports on automated applications.
Where itās dangerous or scary to apply changes, such as with resources that have persistence or databases, get reports on any unapplied changes in certain parts of the infrastructure. An engineer, under supervision, will then run jobs to apply them or do it from their console.
Amazon has something called Terminate Protection. It can protect against unwanted changes in some cases. That is, Terraform goes to Amazon and says, 'I need to terminate this instance to create another one.' And Amazon replies, 'Sorry, not today. We have Terminate Protection in place.'
And the cherry on top is code optimization. When we work with Terraform code, we need to pass a large number of parameters into the module. These are the parameters necessary to create a resource. The code turns into large lists of parameters that need to be passed from module to module, especially if the modules are nested.
And this makes it very hard to read. It's very difficult to do a review. Often, some parameters pass the review and they arenāt exactly whatās needed. And it costs time and money to fix later.
Therefore, I suggest using a complex parameter that includes a tree of values. That is, you need a folder where all the values you would like to have for a certain environment are specified.
By invoking this module, you can obtain a tree that is generated in a single common module, i.e., in a common module that works uniformly for the entire infrastructure.
In this module, you can perform certain calculations using a new feature in Terraform known as locals. Then, you can output a complex parameter that may include hashes, arrays, etc.
That brings us to the end of my best findings. Iād like to share a story about Columbus. When he was seeking funding for his expedition to discover India (as he thought), no one believed him and deemed it impossible. He then said, 'Make it so the egg doesnāt fall.' All the bankers, who were presumably wealthy and smart, tried to somehow balance the egg, but it kept falling. Columbus then took the egg, applied a little pressure, and the shell crumbled, keeping the egg upright. They said, 'Oh, thatās too simple!' To which he replied, 'Yes, it is too simple. And when I discover India, everyone will use this trade route.'
What Iāve shared with you may seem relatively straightforward and trivial. Once you learn and start using them, it becomes second nature. So, take advantage of it. And if these are fairly normal concepts for you, at least now you know how to balance the egg without it falling.
In summary:
- Try to avoid snowflakes. The fewer snowflakes you have, the fewer resources you will need to make changes across your large infrastructure.
- Continuous changes. That is, when there are modifications in the code, your infrastructure must be updated as quickly as possible to reflect these changes. There should never be a situation where someone returns after two or three months to check Elasticsearch, runs a Terraform plan, and finds a bunch of changes they werenāt expecting. It wastes a lot of time to restore everything back to order.
- Testing and automation. The more your code is covered by tests and features, the more confidence you will have that youāre doing everything correctly. Automated delivery will significantly boost your confidence.
- The code for testing and production environments should be nearly identical. Practically speaking, production is slightly different, and there will still be certain nuances that exceed the testing environment. Nevertheless, it can generally be ensured.
- If you have a lot of Terraform code and keeping it up-to-date takes a significant amount of time, it's never too late to refactor and bring it into good shape.
- Immutable infrastructure. Scheduled delivery of AMI.
- Structure for Route53 when you have many records and you want them to be in an organized order.
- Dealing with API rate limits. This is when Amazon says, 'That's it, I can't accept any more requests, please wait.' And half the company is waiting until they can start their infrastructure.
- Spot Instances. Amazon is not cheap, and spot instances can save quite a bit. There's enough to talk about for an entire presentation on this topic.
- Security and IAM roles.
- Finding lost resources when you have instances in Amazon of unclear origin that are racking up costs. Even if an instance costs $100-150 a month, thatās over $1,000 a year. Identifying such resources is a profitable endeavor.
- And reserved instances.
Thatās all from me. Terraform is really cool; make use of it. Thank you!
Questions
Thank you for the presentation! Your state file is stored in S3, but how do you handle the problem of multiple people accessing the state file and trying to deploy?
First, we don't rush. Second, there are flags that indicate we're working on a certain piece of code. That is, despite the infrastructure being very large, it doesn't mean that someone is constantly applying changes. When there was an active phase, this was a problem; we stored state files in Git. It was important, otherwise someone would create a state file, and we had to manually aggregate them to continue further. Now, such a problem doesn't exist. In general, Terraform has resolved this issue. If changes are happening all the time, locks can be used to prevent what you mentioned.
Are you using the open version or the enterprise version?
No enterprise at all; everything can be downloaded for free.
My name is Stanislav. I wanted to make a small addition. You talked about the Amazon feature that allows you to make an instance unkillable. This is also in Terraform; in the Life Cycle block, you can specify a prevention of modification or destruction.
I was limited in time. Good point.
I also wanted to ask two things. First, you mentioned testing. Have you used any tools for testing? I've heard about the Test Kitchen plugin. Is there anything else? I would also like to ask about Local Values. How do they fundamentally differ from Input Variables? And why can't I parameterize something only through Local Values? I tried to understand this topic, but I couldn't figure it out myself.
We can talk in more detail over there. The testing tools we have are homemade. Thereās nothing proper for testing. In general, there are options where automated tests spin up infrastructure, check that everything is OK, and then destroy it with a report stating that your infrastructure is still in good shape. We donāt have that because testing stacks run every day. Thatās enough. If something starts breaking, it will start breaking without us needing to check it elsewhere.
Regarding Local Values, letās continue our discussion over there.
Hi! Thanks for the presentation! It was very informative. You mentioned that you have a lot of repetitive code for describing infrastructure. Have you considered generating this code?
Great question, thank you! The thing is, when we use infrastructure as code, we assume that we look at the code and understand what infrastructure lies behind it. If the code is generated, then we need to envision what code will be generated to understand what infrastructure will be there. Whether we generate code, commit it, and, in essence, it turns out to be the same. Thatās why we went down the path we wrote, and that's what we got. Plus, generators appeared a bit later when we started doing it, and it was too late to change.
Have you heard anything about jsonnet?
No.
Look, this is a really cool tool. I see a specific case where it can be applied to generate data structures.
Generators are great, like in the joke about the razor. The first time your face is different, but then everyone ends up with the same face. Generators are really cool, but unfortunately, our faces are a bit different. That's a problem.
Just take a look. Thank you!
My name is Maxim, I'm from Sberbank. You mentioned that you were trying to bring Terraform closer to a programming language. Isn't it easier to use Ansible?
These are very different things. You can create resources with Ansible and Puppet in Amazon as well. But Terraform is specifically designed for that.
Do you only use Amazon?
It's not just that we only use Amazon. We almost exclusively use Amazon. But the key feature is that Terraform remembers. In Ansible, if you say, 'Spin up 5 instances,' it will do that, and then you say, 'Now I need 3.' Terraform will say, 'Okay, I'll kill 2,' while Ansible will say, 'Okay, here are your 3.' So in total, thatās 8.
Hello! Thank you for your presentation! It was very interesting to hear about Terraform. I want to make a quick comment regarding the fact that Terraform doesn't have a stable release, so approach it with caution.
A spoon is good at dinner, meaning if you need a solution, you sometimes push aside what is unstable, etc., but it works and has helped us.
I have a question. Youāre using a Remote backend; youāre using S3. Why aren't you using the official backend?
Official?
Terraform Cloud.
When did it appear?
About 4 months ago.
If it had appeared 4 years ago, I would have probably answered your question.
It already has built-in functions and locks, and you can store the state file. Give it a try. But I haven't tested it either.
Weāre on a large train moving at high speed. You canāt just throw out a few cars.
You talked about snowflakes; why didn't you use branches? Why couldn't it be done that way?
We have an approach where the entire infrastructure is in one repository. Terraform, Puppet, all scripts related to it are all in one repository. This way, we can guarantee that incremental changes are tested one after another. If there were a bunch of branches, maintaining such a project would be nearly impossible. Six months pass, and they diverge so much that it's just a punishment of sorts. Thatās what we wanted to escape before refactoring.
So it doesn't work?
This doesn't work at all.
In the branch, I removed the slide of the folder. That is, if we create a separate folder for each test stack, for example, Team A has its own folder, Team B has its own folder, this also doesn't work. We developed a unified code for the testing environment, which was flexible enough to suit everyone. In other words, we maintained one code.
Hello! My name is Yura! Thank you for the presentation! I have a question about modules. You mentioned that you use modules. How do you handle situations when changes made in one module are incompatible with changes made by someone else? Do you version the modules or try to make a workaround to meet two requirements?
This is the problem of a big snowball. This is something we suffer from when some harmless change can break a part of the infrastructure. And it will only become noticeable after a long time.
So, it is currently unresolved?
You create universal modules. Avoid snowflakes. And everything will work out. The second half of the presentation is about how to avoid this.
Hello! Thank you for the presentation! I would like to clarify something. A big pile remains offstage, which is why I came. How are Puppet and role distribution integrated?
User-data.
So, you just spit out a file and execute based on it somehow?
User-data is a note, that is, when we clone an image, a daemon is started and tries to figure out what it is by reading a note that it is a load balancer.
So, is this some separate process that gets allocated?
We didn't invent it. We use it.
Hello! I have a question about User-data. You mentioned that there are issues where someone might pass the wrong thing. Is there a way to store user-data in the same Git so that it's always clear what User-data refers to?
We generate User-data from the template. That is, some number of variables come in there. And Terraform generates the final result. So you can't just look at the template and say what it will produce because all the problems are related to the fact that the developer thinks they are passing a string in this variable, but instead, an array comes in. And suddenly, there's this and that, the next line, and everything is broken. If this is a new resource and someone is bringing it up, they see that something isn't working, and it can be quickly resolved. But if it's an autoscale group that has been updated, at some point, the instances in the autoscale group start to change. And bam, something isn't working. That's frustrating.
So the only solution is to test?
Yes, you see the problem, you add testing steps. That is, you can also test the output. It might not be as convenient, but you can also set some markers ā check that User-data is firmly nailed down here.
My name is Timur. It's great that there are talks on how to properly organize Terraform.
I haven't even started.
I think there might be one at the next conference. I have a simple question. Why do you hardcode a value in a separate module instead of using tfvars? How is a module with values better than tfvars?
That is, I need to write here (slide: Production/environment/settings.tf): domain = variable, domain vpcnetwork, variable vpcnetwork, and retrieve the same thing from stvars?
We do exactly that. We refer to the module setting source, for example.
Essentially, this is just like tfvars. Tfvars is very convenient in testing environments. I have tfvars for large instances, for small ones. And I just dropped one file into the folder. And I got what I wanted. When weāre building infrastructure, we want to be able to look at it and understand everything immediately. But now it turns out that you need to look here, then look in tfvars.
So, it's to have everything in one place?
Yes, tfvars is when you have one code base. And it's used in several different places with different nuances. Then you would use tfvars and get your nuances. But we treat infrastructure as code in its purest form. You look at it and understand.
Hello! Have you encountered situations where the cloud provider interferes with what you've done using Terraform? Let's say we modify the metadata. There are SSH keys. And Google constantly injects its own metadata and keys. Terraform always reports that there are changes. After every run, even if nothing changes, it still says it will update that field.
With the keys, yes, part of the infrastructure is affected by this, i.e., Terraform cannot change anything. We can't change anything manually either. For now, we're living with it.
So you've encountered this, but came up with no solution; it just keeps doing its thing?
Unfortunately, yes.
Hello! My name is Stanislav Starkov. Mail.ru Group. How do you solve the problem with generating tags for ā¦, how do you pass them internally? I understand it's through user data to specify the host name, directing Puppet? And the second part of the question. How do you address this in SG, i.e., when generating SG and a hundred identical instances, how do you properly name them?
The instances that are very important to us, we name them nicely. Those that are not needed, we add a suffix indicating it's an autoscale group. And theoretically, this can be terminated, and a new one can be obtained.
Regarding the issue with tags, there isnāt a problem per se; thereās a task. Tags are used extensively because the infrastructure is large and expensive. We need to monitor where the money goes, which is why tags help clarify what spent what. Accordingly, it's about finding where there's excessive spending.
What was the other question?
When SG creates a hundred instances, do they need to be distinguished somehow?
No, they donāt. Each instance has an agent that reports if there's a problem. If the agent reports, it knows about it, and at the very least, the IP address exists. You can go check that. Secondly, we use Consul for discovery, where thereās no Kubernetes. Consul also shows the IP address of the instance.
So you rely on IP, not the host name?
It's impossible to rely on the host name; there are just too many. There are instance identifiers ā AE, etc. You can find it somewhere, you can look it up.
Hi! I understand that Terraform is a great tool designed for clouds.
Not just that.
This is exactly the question I'm interested in. If you decide to move, let's say, to Bare Metal en masse with all your instances, will there be any issues? Or will you still have to use other products, like Ansible, which was mentioned here?
Ansible is somewhat different. That is, Ansible operates once the instance has started. Terraform functions before the instance starts. Transitioning to Bare Metal is not applicable.
Not right now, but the business may come and say: 'Let's do it.'
Switching to another cloud is possible, but there is a bit of a different aspect. You need to write Terraform code in such a way that transitioning to another cloud is less challenging.
Initially, the task was set so that our entire infrastructure would be agnostic, meaning any cloud should fit, but at some point, the business conceded and said: 'Okay, for the next N years we wonāt go anywhere, we can use services from Amazon.'
Terraform allows for creating jobs in Front-End, configuring PagerDuty, data docs, etc. It has a lot of capabilities. It can practically manage the entire world.
Thank you for the presentation! I've also been working with Terraform for 4 years. During the smooth transition to Terraform and the move to declarative descriptions, we encountered situations where someone did something manually, and you tried to execute a plan. And you'd get errors. How do you troubleshoot such issues? How do you find lost resources that were specified?
Mostly by hand and visually. If we see something strange in the report, we analyze whatās going on or simply remove it. Generally, pull requests are standard practice.
If there's an error, do you perform a rollback? Have you tried doing that?
No, that's a decision made by the person in the moment when they see the problem.
Source: habr.com
