Terraform์„ ์‚ฌ์šฉํ•œ ๋น ๋ฅธ ๋ฐฐํฌ vm ESXi

์•ˆ๋…•ํ•˜์„ธ์š” ์—ฌ๋Ÿฌ๋ถ„, ์ œ ์ด๋ฆ„์€ Ivan์ด๊ณ  ์•Œ์ฝ”์˜ฌ ์ค‘๋… ์‹œ์Šคํ…œ ๊ด€๋ฆฌ์ž(OPS)์ž…๋‹ˆ๋‹ค.

Terraform์„ ์‚ฌ์šฉํ•˜์—ฌ vCenter ์—†์ด ESXi์— ๊ฐ€์ƒ ๋จธ์‹ ์„ ๋ฐฐํฌํ•˜๋Š” ๋ฐฉ๋ฒ•์„ ์•Œ๋ ค๋“œ๋ฆฌ๊ณ ์ž ํ•ฉ๋‹ˆ๋‹ค.

์ด ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜ ๋˜๋Š” ์ € ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜์„ ํ…Œ์ŠคํŠธํ•˜๊ธฐ ์œ„ํ•ด ๊ฐ€์ƒ ๋จธ์‹ ์„ ๋ฐฐํฌ/์žฌ์ž‘์„ฑํ•ด์•ผ ํ•˜๋Š” ๊ฒฝ์šฐ๊ฐ€ ๋งŽ์Šต๋‹ˆ๋‹ค. ๊ฒŒ์œผ๋ฆ„ ๋•Œ๋ฌธ์— ํ”„๋กœ์„ธ์Šค ์ž๋™ํ™”์— ๋Œ€ํ•ด ์ƒ๊ฐํ–ˆ์Šต๋‹ˆ๋‹ค. ๋‚ด ๊ฒ€์ƒ‰์€ ํšŒ์‚ฌ์˜ ๋ฉ‹์ง„ ์ œํ’ˆ์œผ๋กœ ๋‚˜๋ฅผ ์ด๋Œ์—ˆ์Šต๋‹ˆ๋‹ค. ํ•ด์‹œ์ฝ”ํ”„, ํ…Œ๋ผ ํฌ๋ฆ„.

๋งŽ์€ ์‚ฌ๋žŒ๋“ค์ด Terraform์ด ๋ฌด์—‡์ธ์ง€ ์•Œ๊ณ  ์žˆ๊ณ  ๋ชจ๋ฅด๋Š” ์‚ฌ๋žŒ์€ IasC ๊ฐœ๋…์„ ์‚ฌ์šฉํ•˜์—ฌ ํด๋ผ์šฐ๋“œ, ์ธํ”„๋ผ ๋˜๋Š” ์„œ๋น„์Šค๋ฅผ ๊ด€๋ฆฌํ•˜๊ธฐ ์œ„ํ•œ ์‘์šฉ ํ”„๋กœ๊ทธ๋žจ์ž…๋‹ˆ๋‹ค.์ฝ”๋“œํ˜• ์ธํ”„๋ผ).

๊ฐ€์ƒํ™” ํ™˜๊ฒฝ์œผ๋กœ ESXi๋ฅผ ์‚ฌ์šฉํ•ฉ๋‹ˆ๋‹ค. ๋งค์šฐ ๊ฐ„๋‹จํ•˜๊ณ  ํŽธ๋ฆฌํ•˜๋ฉฐ ์‹ ๋ขฐํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
๋‚˜๋Š” ์งˆ๋ฌธ์„ ์˜ˆ์ƒํ•œ๋‹ค.

vCenter Server๋ฅผ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ๋Š”๋ฐ ์™œ Terraform์„ ์‚ฌ์šฉํ•ฉ๋‹ˆ๊นŒ?

๋ฌผ๋ก  ํ•  ์ˆ˜ ์žˆ์ง€๋งŒ. ์ฒซ์งธ, ์ด๊ฒƒ์€ ์ถ”๊ฐ€ ๋ผ์ด์„ผ์Šค์ด๊ณ , ๋‘˜์งธ, ์ด ์ œํ’ˆ์€ ๋ฆฌ์†Œ์Šค ์ง‘์•ฝ์ ์ด๋ฉฐ ๋‚ด ํ™ˆ ์„œ๋ฒ„์— ์ ํ•ฉํ•˜์ง€ ์•Š์œผ๋ฉฐ, ์…‹์งธ, ๊ธฐ์ˆ  ์—…๊ทธ๋ ˆ์ด๋“œ ๊ธฐ๋Šฅ์ž…๋‹ˆ๋‹ค.

์ธํ…” NUC ํ”Œ๋žซํผ์€ ์„œ๋ฒ„ ์—ญํ• ์„ ํ•ฉ๋‹ˆ๋‹ค.

CPU: 2 CPUs x Intel(R) Core(TM) i3-4010U CPU @ 1.70GHz
RAM: 8Gb
HDD: 500Gb
ESXi version: ESXi-6.5.0-4564106-standard (VMware, Inc.)

๊ทธ๋ž˜์„œ ๊ฐ€์žฅ ๋จผ์ € ํ•ด์•ผ ํ•  ์ผ์ด ์žˆ์Šต๋‹ˆ๋‹ค.

์ง€๊ธˆ์€ ๋ฐฉํ™”๋ฒฝ ์„ค์ •์—์„œ VNC ํฌํŠธ๋ฅผ ์—ด์–ด esxi๋ฅผ ์„ค์ •ํ•ด ๋ณด๊ฒ ์Šต๋‹ˆ๋‹ค.

๊ธฐ๋ณธ์ ์œผ๋กœ ํŒŒ์ผ์€ ์“ฐ๊ธฐ ๊ธˆ์ง€๋˜์–ด ์žˆ์Šต๋‹ˆ๋‹ค. ๋‹ค์Œ ์กฐ์ž‘์„ ์ˆ˜ํ–‰ํ•ฉ๋‹ˆ๋‹ค.

chmod 644 /etc/vmware/firewall/service.xml
chmod +t /etc/vmware/firewall/service.xml
vi /etc/vmware/firewall/service.xml

ํŒŒ์ผ ๋์— ๋‹ค์Œ ๋ธ”๋ก์„ ์ถ”๊ฐ€ํ•ฉ๋‹ˆ๋‹ค.

<service id="1000">
  <id>packer-vnc</id>
  <rule id="0000">
    <direction>inbound</direction>
    <protocol>tcp</protocol>
    <porttype>dst</porttype>
    <port>
      <begin>5900</begin>
      <end>6000</end>
    </port>
  </rule>
  <enabled>true</enabled>
  <required>true</required>
</service>

์ข…๋ฃŒํ•˜๊ณ  ์ €์žฅํ•˜์‹ญ์‹œ์˜ค. ๊ถŒํ•œ์„ ๋‹ค์‹œ ๋ณ€๊ฒฝํ•˜๊ณ  ์„œ๋น„์Šค๋ฅผ ๋‹ค์‹œ ์‹œ์ž‘ํ•˜์‹ญ์‹œ์˜ค.

chmod 444 /etc/vmware/firewall/service.xml
esxcli network firewall refresh

ํ˜ธ์ŠคํŠธ๊ฐ€ ์žฌ๋ถ€ํŒ…๋  ๋•Œ๊นŒ์ง€ ์‹ค์ œ์ž…๋‹ˆ๋‹ค. ๊ทธ ํ›„์—๋Š” ์ด ์กฐ์ž‘์„ ๋ฐ˜๋ณตํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.

๋˜ํ•œ ๋™์ผํ•œ ์„œ๋ฒ„์˜ ๊ฐ€์ƒ ๋จธ์‹ ์—์„œ ๋ชจ๋“  ์ž‘์—…์„ ์ˆ˜ํ–‰ํ•ฉ๋‹ˆ๋‹ค.

ํŠน์ง• :

OS: Centos 7 x86_64 minimal
RAM: 1GB
HDD: 20GB
Selinux: disable
firewalld: disable

๋‹ค์Œ์œผ๋กœ, ์šฐ๋ฆฌ๋Š” ํ•„์š”ํ•ฉ๋‹ˆ๋‹ค ํฌ์žฅ ๊ธฐ๊ณ„, ๋˜ํ•œ HashiCorp์˜ ์ œํ’ˆ์ž…๋‹ˆ๋‹ค.

"๊ณจ๋“ " ์ด๋ฏธ์ง€๋ฅผ ์ž๋™์œผ๋กœ ์กฐ๋ฆฝํ•˜๋Š” ๋ฐ ํ•„์š”ํ•ฉ๋‹ˆ๋‹ค. ์šฐ๋ฆฌ๊ฐ€ ๋ฏธ๋ž˜์— ์‚ฌ์šฉํ•  ๊ฒƒ์ž…๋‹ˆ๋‹ค.

yum install unzip git -y
curl -O https://releases.hashicorp.com/packer/1.5.5/packer_1.5.5_linux_amd64.zip
unzip packer_1.5.5_linux_amd64.zip -d /usr/bin && rm -rf packer_1.5.5_linux_amd64.zip
packer version
Packer v1.5.5

์ด๋™ ์ค‘ ํŒจ์ปค ๋ฒ„์ „ RedHat ๊ธฐ๋ฐ˜์—๋Š” ๋™์ผํ•œ ์ด๋ฆ„์˜ ํŒจํ‚ค์ง€๊ฐ€ ํฌํ•จ๋  ์ˆ˜ ์žˆ์œผ๋ฏ€๋กœ ์˜ค๋ฅ˜๊ฐ€ ๋ฐœ์ƒํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

which -a packer
/usr/sbin/packer

์†”๋ฃจ์…˜์˜ ๊ฒฝ์šฐ symlink๋ฅผ ๋งŒ๋“ค๊ฑฐ๋‚˜ ์ ˆ๋Œ€ ๊ฒฝ๋กœ๋ฅผ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. /usr/๋นˆ/ํŒจ์ปค.

์ด์ œ ์šฐ๋ฆฌ๋Š” ovftool์ด ํ•„์š”ํ•ฉ๋‹ˆ๋‹ค ๋‹ค์šด๋กœ๋“œ ๋งํฌ. ๋‹ค์šด๋กœ๋“œํ•˜์—ฌ ์„œ๋ฒ„์— ๋†“๊ณ  ์„ค์น˜ํ•˜์‹ญ์‹œ์˜ค.

chmod +x VMware-ovftool-4.4.0-15722219-lin.x86_64.bundle
./VMware-ovftool-4.4.0-15722219-lin.x86_64.bundle
Extracting VMware Installer...done.
You must accept the VMware OVF Tool component for Linux End User
License Agreement to continue.  Press Enter to proceed.
VMWARE END USER LICENSE AGREEMENT
Do you agree? [yes/no]:yes
The product is ready to be installed.  Press Enter to begin
installation or Ctrl-C to cancel. 
Installing VMware OVF Tool component for Linux 4.4.0
    Configuring...
[######################################################################] 100%
Installation was successful.

๊ณ„์† ์ง„ํ–‰ํ•ฉ๋‹ˆ๋‹ค.

๋‚˜๋Š” ๋‹น์‹ ์ด ํ•„์š”ํ•œ ๋ชจ๋“  ๊ฒƒ์„ ์ค€๋น„ํ–ˆ์Šต๋‹ˆ๋‹ค.

git clone https://github.com/letnab/create-and-deploy-esxi.git && cd create-and-deploy-esxi

ํด๋”๋กœ ์ด์†Œ ์šด์˜ ์ฒด์ œ์˜ ๋ฐฐํฌ ํ‚คํŠธ๋ฅผ ๋„ฃ์–ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. ์ œ ๊ฒฝ์šฐ์—๋Š” centos 7์ž…๋‹ˆ๋‹ค.

๋˜ํ•œ ํŒŒ์ผ์„ ํŽธ์ง‘ํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. centos-7-base.json:

variables: ัƒะบะฐะทะฐั‚ัŒ ัะฒะพะธ ะดะฐะฝะฝั‹ะต ะดะปั ะฟะพะดะบะปัŽั‡ะตะฝะธั
iso_urls: ัƒะบะฐะทะฐั‚ัŒ ะฐะบั‚ัƒะฐะปัŒะฝั‹ะน
iso_checksum: ั‡ะตะบััƒะผะผะฐ ะฒะฐัˆะตะณะพ ะพะฑั€ะฐะทะฐ 

๋ชจ๋“  ๋ณ€๊ฒฝ ํ›„ ์–ด์…ˆ๋ธ”๋ฆฌ๋ฅผ ์‹คํ–‰ํ•ฉ๋‹ˆ๋‹ค.

/usr/bin/packer build centos-7-base.json

๋ชจ๋“  ๊ฒƒ์ด ์˜ฌ๋ฐ”๋ฅด๊ฒŒ ๊ตฌ์„ฑ๋˜๊ณ  ์ง€์ •๋˜๋ฉด ์šด์˜ ์ฒด์ œ ์ž๋™ ์„ค์น˜ ๊ทธ๋ฆผ์ด ํ‘œ์‹œ๋ฉ๋‹ˆ๋‹ค. ์ด ํ”„๋กœ์„ธ์Šค๋Š” 7-8๋ถ„ ์ •๋„ ๊ฑธ๋ฆฝ๋‹ˆ๋‹ค.

ํด๋”์—์„œ ์„ฑ๊ณต์ ์œผ๋กœ ์™„๋ฃŒํ•œ ํ›„ ์ถœ๋ ฅ ํŒจ์ปค-centos7-x86_64 ova ํŒŒ์ผ์„ ์ฐพ์Šต๋‹ˆ๋‹ค.

Terraform ์„ค์น˜:

curl -O https://releases.hashicorp.com/terraform/0.12.24/terraform_0.12.24_linux_amd64.zip
unzip terraform_0.12.24_linux_amd64.zip -d /usr/bin/ && rm -rf terraform_0.12.24_linux_amd64.zip
terraform version
Terraform v0.12.24

Terraform์—๋Š” ESXi์šฉ ๊ณต๊ธ‰์ž๊ฐ€ ์—†์œผ๋ฏ€๋กœ ํ•˜๋‚˜๋ฅผ ๊ตฌ์ถ•ํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.

์šฐ๋ฆฌ๋Š” ์ด๋™:

cd /tmp
curl -O https://dl.google.com/go/go1.14.2.linux-amd64.tar.gz
tar -C /usr/local -xzf go1.14.2.linux-amd64.tar.gz && rm -rf go1.14.2.linux-amd64.tar.gz
export PATH=$PATH:/usr/local/go/bin
go version
go version go1.14.2 linux/amd64

๋‹ค์Œ์œผ๋กœ ๊ณต๊ธ‰์ž๋ฅผ ์ˆ˜์ง‘ํ•ฉ๋‹ˆ๋‹ค.

go get -u -v golang.org/x/crypto/ssh
go get -u -v github.com/hashicorp/terraform
go get -u -v github.com/josenk/terraform-provider-esxi
export GOPATH="$HOME/go"
cd $GOPATH/src/github.com/josenk/terraform-provider-esxi
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -ldflags '-w -extldflags "-static"' -o terraform-provider-esxi_`cat version`
cp terraform-provider-esxi_`cat version` /usr/bin

์šฐ๋ฆฌ๋Š” ๊ฒฐ์Šน์„ ์— ์žˆ์Šต๋‹ˆ๋‹ค. ์ด๋ฏธ์ง€๋ฅผ ๋กค์•„์›ƒํ•ด ๋ณด๊ฒ ์Šต๋‹ˆ๋‹ค.

๋‹ค์Œ ํด๋”๋กœ ์ด๋™ํ•ฉ๋‹ˆ๋‹ค.

cd /root/create-and-deploy-esxi/centos7

๋จผ์ € ํŒŒ์ผ์„ ์ˆ˜์ •ํ•ฉ๋‹ˆ๋‹ค. ๋ณ€์ˆ˜.tf. ESXi ์„œ๋ฒ„์— ๋Œ€ํ•œ ์—ฐ๊ฒฐ์„ ์ง€์ •ํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.

ํŒŒ์ผ์—์„œ network_config.cfg ๋ฏธ๋ž˜ ๊ฐ€์ƒ ๋จธ์‹ ์˜ ๋„คํŠธ์›Œํฌ ์„ค์ •์„ ํฌํ•จํ•ฉ๋‹ˆ๋‹ค. ํ•„์š”์— ๋”ฐ๋ผ ๋ณ€๊ฒฝํ•˜๊ณ  ํ•œ ์ค„์งœ๋ฆฌ๋ฅผ ์‹คํ–‰ํ•ฉ๋‹ˆ๋‹ค.

sed -i -e '2d' -e '3i "network": "'$(gzip < network_config.cfg| base64 | tr -d 'n')'",' metadata.json

์ž, ํŒŒ์ผ์—์„œ main.tf ova ํŒŒ์ผ์˜ ๊ฒฝ๋กœ๊ฐ€ ๋‹ค๋ฅธ ๊ฒฝ์šฐ ์ž์‹ ์˜ ๊ฒƒ์œผ๋กœ ๋ณ€๊ฒฝํ•˜์‹ญ์‹œ์˜ค.

์ง„์‹ค์˜ ์ˆœ๊ฐ„.

terraform init
Initializing the backend...

Initializing provider plugins...

The following providers do not have any version constraints in configuration,
so the latest version was installed.

To prevent automatic upgrades to new major versions that may contain breaking
changes, it is recommended to add version = "..." constraints to the
corresponding provider blocks in configuration, with the constraint strings
suggested below.

* provider.esxi: version = "~> 1.6"
* provider.template: version = "~> 2.1"

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

data.template_file.Default: Refreshing state...
data.template_file.network_config: Refreshing state...

------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # esxi_guest.Default will be created
  + resource "esxi_guest" "Default" {
      + boot_disk_size         = (known after apply)
      + disk_store             = "datastore1"
      + guest_name             = "centos7-test"
      + guest_shutdown_timeout = (known after apply)
      + guest_startup_timeout  = (known after apply)
      + guestinfo              = {
          + "metadata"          = "base64text"
          + "metadata.encoding" = "gzip+base64"
          + "userdata"          = "base64text"
          + "userdata.encoding" = "gzip+base64"
        }
      + guestos                = (known after apply)
      + id                     = (known after apply)
      + ip_address             = (known after apply)
      + memsize                = "1024"
      + notes                  = (known after apply)
      + numvcpus               = (known after apply)
      + ovf_properties_timer   = (known after apply)
      + ovf_source             = "/root/create-and-deploy-esxi/output-packer-centos7-x86_64/packer-centos7-x86_64.ova"
      + power                  = "on"
      + resource_pool_name     = (known after apply)
      + virthwver              = (known after apply)

      + network_interfaces {
          + mac_address     = (known after apply)
          + nic_type        = (known after apply)
          + virtual_network = "VM Network"
        }
    }

Plan: 1 to add, 0 to change, 0 to destroy.

------------------------------------------------------------------------

Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.

๋งˆ์น˜๋‹ค:

terraform apply

๋ชจ๋“  ๊ฒƒ์ด ์˜ฌ๋ฐ”๋ฅด๊ฒŒ ์™„๋ฃŒ๋˜๋ฉด 2-3๋ถ„ ์•ˆ์— ์ด์ „์— ๋งŒ๋“  ์ด๋ฏธ์ง€์—์„œ ์ƒˆ ๊ฐ€์ƒ ๋จธ์‹ ์ด ๋ฐฐํฌ๋ฉ๋‹ˆ๋‹ค.

์ด ๋ชจ๋“  ๊ฒƒ์˜ ์šฉ๋„๋Š” ๋‹น์‹ ์˜ ์ƒ์ƒ๋ ฅ์— ์˜ํ•ด์„œ๋งŒ ์ œํ•œ๋ฉ๋‹ˆ๋‹ค.

๋ชจ๋ฒ” ์‚ฌ๋ก€๋ฅผ ๊ณต์œ ํ•˜๊ณ  ์ด๋Ÿฌํ•œ ์ œํ’ˆ์œผ๋กœ ์ž‘์—…ํ•  ๋•Œ ์ฃผ์š” ์‚ฌํ•ญ์„ ๋ณด์—ฌ์ฃผ๊ณ  ์‹ถ์—ˆ์Šต๋‹ˆ๋‹ค.

๊ท€ํ•˜์˜ ๊ด€์‹ฌ์„ ๊ฐ€์ ธ ์ฃผ์…”์„œ ๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค!

์ถ”์‹ : ๊ฑด์„ค์ ์ธ ๋น„ํŒ์— ๊ธฐ๊บผ์ด ์‘ํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค.

์ถœ์ฒ˜ : habr.com

์ฝ”๋ฉ˜ํŠธ๋ฅผ ์ถ”๊ฐ€