Good afternoon friends. In anticipation of the start of a new stream at the rate We share with you a new translation. Go.
Using Pulumi and general purpose programming languages for infrastructure code (Infrastructure as Code) brings many benefits: having the skills and knowledge, eliminating the boilerplate in the code through abstraction, tools familiar to your team, such as IDEs and linters. All these software engineering tools not only make us more productive, but also improve the quality of our code. Therefore, it is quite natural that the use of general-purpose programming languages allows the introduction of another important software development practice − testing.
In this article, we'll look at how Pulumi helps us test our "infrastructure as code".
Why test infrastructure?
Before going into details, it is worth asking the question: “Why test infrastructure at all?” There are many reasons for this and here are some of them:
- Unit testing individual functions or pieces of logic in your program
- Checking the desired state of the infrastructure against certain constraints.
- Detection of common errors such as lack of storage bucket encryption or insecure, public access from the Internet to virtual machines.
- Checking the implementation of infrastructure provisioning.
- Performing runtime testing of the logic of an application running inside your "programmed" infrastructure to check if it works after provisioning.
- As we can see, there is a wide range of infrastructure testing options. Polumi has mechanisms for testing at every point on this spectrum. Let's get started and see how it works.
Unit testing
Pulumi programs are written in general purpose programming languages such as JavaScript, Python, TypeScript, or Go. Therefore, the full power of these languages is available to them, including their toolkit and libraries, including test frameworks. Pulumi is multi-cloud, which means you can use any cloud provider for testing.
(In this article, despite the multilingual and multicloud, we use JavaScript and Mocha and focus on AWS. You can use Python unittest, the Go test framework, or whatever test framework you like. And of course Pulumi works great with Azure, Google Cloud, Kubernetes.)
As we have seen, there are several reasons why you might want to test your infrastructure code. One of them is the usual unit testing. Since your code may have functions - for example, to calculate CIDR, dynamically calculate names, tags, etc. - you probably want to test them. This is the same as writing regular unit tests for applications in your favorite programming language.
To make it a little more complicated, you can check how your program allocates resources. To illustrate, let's imagine that we need to create a simple EC2 server and we want to be sure of the following:
- Instances have a tag
Name. - Instances must not use inline script
userData- we have to use AMI (image). - There should not be SSH open to the internet.
This example is based on :
index.js:
"use strict";
let aws = require("@pulumi/aws");
let group = new aws.ec2.SecurityGroup("web-secgrp", {
ingress: [
{ protocol: "tcp", fromPort: 22, toPort: 22, cidrBlocks: ["0.0.0.0/0"] },
{ protocol: "tcp", fromPort: 80, toPort: 80, cidrBlocks: ["0.0.0.0/0"] },
],
});
let userData =
`#!/bin/bash
echo "Hello, World!" > index.html
nohup python -m SimpleHTTPServer 80 &`;
let server = new aws.ec2.Instance("web-server-www", {
instanceType: "t2.micro",
securityGroups: [ group.name ], // reference the group object above
ami: "ami-c55673a0" // AMI for us-east-2 (Ohio),
userData: userData // start a simple web server
});
exports.group = group;
exports.server = server;
exports.publicIp = server.publicIp;
exports.publicHostName = server.publicDns;
This is the basic Pulumi program: it just allocates an EC2 security group and an instance. However, it should be noted that here we violate all three rules outlined above. Let's write tests!
We write tests
The general structure of our tests will look like regular Mocha tests:
ec2tests.js
test.js:
let assert = require("assert");
let mocha = require("mocha");
let pulumi = require("@pulumi/pulumi");
let infra = require("./index");
describe("Infrastructure", function() {
let server = infra.server;
describe("#server", function() {
// TODO(check 1): Должен быть тэг Name.
// TODO(check 2): Не должно быть inline-скрипта userData.
});
let group = infra.group;
describe("#group", function() {
// TODO(check 3): Не должно быть SSH, открытого в Интернет.
});
});
Now let's write our first test: make sure instances have a tag Name. To test this, we simply get the EC2 instance object and check the corresponding property tags:
// check 1: Должен быть тэг Name.
it("must have a name tag", function(done) {
pulumi.all([server.urn, server.tags]).apply(([urn, tags]) => {
if (!tags || !tags["Name"]) {
done(new Error(`Missing a name tag on server ${urn}`));
} else {
done();
}
});
});It looks like a regular test, but with a few features worth noting:
- Because we query the state of the resource before deploying, our tests are always run in 'plan' (or 'preview') mode. Thus, there are many properties whose values simply will not be retrieved or will be undefined. This includes all output properties computed by your cloud provider. For our tests, this is normal - we only check the input data. We will return to this issue later, when it comes to integration tests.
- Since all Pulumi resource properties are "outputs" and many of them are evaluated asynchronously, we need to use the apply method to access the values. This is very similar to promises and afunctions.
then. - Since we are using several properties to show the URN of the resource in the error message, we need to use the function
pulumi.allto combine them. - Finally, since these values are calculated asynchronously, we need to use Mocha's built-in asynchronous callback capability
doneor the return of a promise.
Once we've got everything set up, we'll have access to the inputs as simple JavaScript values. Property tags is a map (an associative array), so we'll just make sure it's (1) not false, and (2) there's a key for Name. It's very simple and now we can check anything!
Now let's write our second check. It's even easier:
// check 2: Не должно быть inline-скрипта userData.
it("must not use userData (use an AMI instead)", function(done) {
pulumi.all([server.urn, server.userData]).apply(([urn, userData]) => {
if (userData) {
done(new Error(`Illegal use of userData on server ${urn}`));
} else {
done();
}
});
});And finally, let's write the third test. This is going to be a bit tricky because we're looking for entry rules associated with a security group, which can be many, and the CIDR ranges in those rules, which can also be many. But we did it:
// check 3: Не должно быть SSH, открытого в Интернет.
it("must not open port 22 (SSH) to the Internet", function(done) {
pulumi.all([ group.urn, group.ingress ]).apply(([ urn, ingress ]) => {
if (ingress.find(rule =>
rule.fromPort == 22 && rule.cidrBlocks.find(block =>
block === "0.0.0.0/0"))) {
done(new Error(`Illegal SSH port 22 open to the Internet (CIDR 0.0.0.0/0) on group ${urn}`));
} else {
done();
}
});
});That's all. Now let's run the tests!
Running Tests
In most cases, you can run tests in the usual way using the test framework of your choice. But there is one feature of Pulumi that is worth paying attention to.
Usually, to launch Pulumi programs, the pulimi CLI (Command Line interface, command line interface) is used, which configures the runtime of the language, controls the launches of the Pulumi engine so that resource operations can be captured and included in the plan, etc. However, there is one problem. When run under the control of your test framework, there will be no connection between the CLI and the Pulumi engine.
To get around this problem, we just need to specify the following:
- The name of the project, which is contained in the environment variable
PULUMI_NODEJS_PROJECT(or, more generally,PULUMI__PROJECT для других языков).
The name of the stack as specified in the environment variablePULUMI_NODEJS_STACK(or, more generally,PULUMI__ STACK).
Your stack configuration variables. They can be obtained using the environment variablePULUMI_CONFIGand their format is a JSON map with key/value pairs.The program will issue warnings saying that the connection to the CLI/engine is not available at run time. This is important because your program won't actually deploy anything and it might come as a surprise if that's not what you wanted to do! To tell Pulumi that this is exactly what you need, you can install
PULUMI_TEST_MODEвtrue.Imagine that we need to specify the name of the project in
my-ws, stack namedev, and AWS Regionus-west-2. The command line to run Mocha tests would look like this:$ PULUMI_TEST_MODE=true PULUMI_NODEJS_STACK="my-ws" PULUMI_NODEJS_PROJECT="dev" PULUMI_CONFIG='{ "aws:region": "us-west-2" }' mocha tests.jsRunning this as expected will show us that we have three failed tests!
Infrastructure #server 1) must have a name tag 2) must not use userData (use an AMI instead) #group 3) must not open port 22 (SSH) to the Internet 0 passing (17ms) 3 failing 1) Infrastructure #server must have a name tag: Error: Missing a name tag on server urn:pulumi:my-ws::my-dev::aws:ec2/instance:Instance::web-server-www 2) Infrastructure #server must not use userData (use an AMI instead): Error: Illegal use of userData on server urn:pulumi:my-ws::my-dev::aws:ec2/instance:Instance::web-server-www 3) Infrastructure #group must not open port 22 (SSH) to the Internet: Error: Illegal SSH port 22 open to the Internet (CIDR 0.0.0.0/0) on groupLet's fix our program:
"use strict"; let aws = require("@pulumi/aws"); let group = new aws.ec2.SecurityGroup("web-secgrp", { ingress: [ { protocol: "tcp", fromPort: 80, toPort: 80, cidrBlocks: ["0.0.0.0/0"] }, ], }); let server = new aws.ec2.Instance("web-server-www", { tags: { "Name": "web-server-www" }, instanceType: "t2.micro", securityGroups: [ group.name ], // reference the group object above ami: "ami-c55673a0" // AMI for us-east-2 (Ohio), }); exports.group = group; exports.server = server; exports.publicIp = server.publicIp; exports.publicHostName = server.publicDns;And then re-run the tests:
Infrastructure #server ✓ must have a name tag ✓ must not use userData (use an AMI instead) #group ✓ must not open port 22 (SSH) to the Internet 3 passing (16ms)Everything went well… Hurrah! ✓✓✓
That's all for today, but we'll talk about deployment testing in the second part of the translation 😉
Source: habr.com