{"id":74905,"date":"2020-03-21T20:42:25","date_gmt":"2020-03-21T17:42:25","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/bare-metal-provisioning-svoimi-rukami-ili-avtomaticheskaya-podgotovka-serverov-s-nulya"},"modified":"2020-03-21T20:42:25","modified_gmt":"2020-03-21T17:42:25","slug":"bare-metal-provisioning-svoimi-rukami-ili-avtomaticheskaya-podgotovka-serverov-s-nulya","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/bare-metal-provisioning-svoimi-rukami-ili-avtomaticheskaya-podgotovka-serverov-s-nulya","title":{"rendered":"Bare-Metal Provisioning Yourself, or Automatic Server Preparation from Scratch","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p>Hello, I'm Denis, and one of my areas of expertise is developing infrastructure solutions at X5. Today, I would like to share how to deploy an automatic server preparation system using widely available tools. In my opinion, this is an interesting, simple, and flexible solution.<\/p>\n<p><img decoding=\"async\" alt=\"Bare-Metal Provisioning Yourself, or Automatic Server Preparation from Scratch\" src=\"\/wp-content\/uploads\/2020\/03\/b49cca7e782c0875a95feee497f141b0.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><br \/>\nPreparation means: turning a new out-of-the-box server into a fully configured server with a Linux operating system or with the ESXi hypervisor (Windows is not discussed in this article). <a class=\"wpil_keyword_link\" href=\"https:\/\/prohoster.info\/en\/server\/\"   title=\"servers\" data-wpil-keyword-link=\"linked\"  data-wpil-monitor-id=\"1469\">servers<\/a> Windows is not discussed in this article.<\/p>\n<p><b>Terms<\/b>:<\/p>\n<ul>\n<li>Servers \u2013 servers that need to be configured.<\/li>\n<li>Install server \u2013 the main server that facilitates the entire preparation process over the network.<\/li>\n<\/ul>\n<p><\/p>\n<h3>Why is automation needed?<\/h3>\n<p>\nLet's say there is a task: to prepare servers in bulk from scratch, with a peak of 30 per day. These are servers from various manufacturers and models, they can have different operating systems installed, and there may or may not be a hypervisor.<\/p>\n<p>What operations are included in the setup process (without automation):<\/p>\n<ul>\n<li>connect a keyboard, mouse, and monitor to the server;<\/li>\n<li>configure the BIOS, RAID, IPMI;<\/li>\n<li>update component firmware;<\/li>\n<li>deploy a filesystem image (or install a hypervisor and copy virtual machines);<\/li>\n<\/ul>\n<p>\nNote. As an option, OS deployment can be done via an installation with an answer file. However, this will not be discussed in the article. Although below you will see that adding this functionality is not complicated.<\/p>\n<ul>\n<li>configure the OS parameters (hostname, IP, etc.).<\/li>\n<\/ul>\n<p>\nWith this approach, the same settings are applied sequentially on each server. The efficiency of such work is very low.<\/p>\n<p>The essence of automation is to eliminate human involvement in the server preparation process as much as possible. <\/p>\n<p>Thanks to automation, downtime between operations is reduced, allowing for the preparation of several servers simultaneously. It also significantly decreases the likelihood of errors due to human factors.<\/p>\n<p><img decoding=\"async\" alt=\"Bare-Metal Provisioning Yourself, or Automatic Server Preparation from Scratch\" src=\"\/wp-content\/uploads\/2020\/03\/8c196058891a7f41c5b93e7f6721480b.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<h3>How does automatic server configuration take place?<\/h3>\n<p>\nLet's break down each stage in detail.<\/p>\n<p>You have a Linux server that you use as a PXE install server. It has DHCP and TFTP services installed and configured.<\/p>\n<p>So, we boot the server (that needs to be configured) via PXE. Let's recall how this works:<\/p>\n<ul>\n<li>The server is set to boot from the network.<\/li>\n<li>The server loads the PXE-ROM of the network card and contacts the installation server via DHCP to obtain a network address.<\/li>\n<li>The DHCP installation server issues an address as well as instructions for further loading via PXE.<\/li>\n<li>The server downloads the network loader from the installation server via PXE; further loading occurs according to the PXE configuration file.<\/li>\n<li>Loading occurs based on the received parameters (kernel, initramfs, mount points, squashfs image, etc.).<\/li>\n<\/ul>\n<p>\nNote: This article describes PXE booting via BIOS mode. Currently, manufacturers are actively implementing UEFI boot mode. For PXE, the difference will be in the configuration of the DHCP server and the presence of an additional loader.<\/p>\n<p>Let's consider an example of PXE server configuration (pxelinux menu).<\/p>\n<p>File pxelinux.cfg\/default:<\/p>\n<pre><code class=\"plaintext\">default menu.c32\nprompt 0\ntimeout 100\nmenu title X5 PXE Boot Menu\nLABEL InstallServer Menu\n\tMENU LABEL InstallServer\n\tKERNEL menu.c32\n\tAPPEND pxelinux.cfg\/installserver\nLABEL VMware Menu\n\tMENU LABEL VMware ESXi Install\n\tKERNEL menu.c32\n\tAPPEND pxelinux.cfg\/vmware\nLABEL toolkit \/\/ default menu\n\tMENU LABEL Linux Scripting Toolkits\n\tMENU default\n\tKERNEL menu.c32\n\tAPPEND pxelinux.cfg\/toolkit \/\/ switch to next menu<\/code><\/pre>\n<p>\nFile pxelinux.cfg\/toolkit:<\/p>\n<pre><code class=\"plaintext\">prompt 0\ntimeout 100\nmenu title X5 PXE Boot Menu\nlabel mainmenu\n    menu label ^Return to Main Menu\n    kernel menu.c32\n    append pxelinux.cfg\/default\nlabel x5toolkit-auto \/\/ default \u2014 automatic mode\n        menu label x5 toolkit autoinstall\n        menu default\n        kernel toolkit\/tkcustom-kernel\n        append initrd=toolkit\/tk-initramfs.gz quiet net.ifnames=0 biosdevname=0 nfs_toolkit_ip=192.168.200.1 nfs_toolkit_path=tftpboot\/toolkit nfs_toolkit_script=scripts\/mount.sh script_cmd=master-install.sh CMDIS2=\"\u2026\"\nlabel x5toolkit-shell \/\/ for debugging - console\n        menu label x5 toolkit shell\n        kernel toolkit\/tkcustom-kernel\n        append initrd=toolkit\/tkcustom-initramfs.gz quiet net.ifnames=0 biosdevname=0 nfs_toolkit_ip=192.168.200.1 nfs_toolkit_path=tftpboot\/toolkit nfs_toolkit_script=scripts\/mount.sh script_cmd=\/bin\/bash CMDIS2=\"\u2026\"<\/code><\/pre>\n<p>\nAt this stage, the kernel and initramfs serve as an intermediate Linux image, which will be used for the main preparation and configuration of the server.<\/p>\n<p>As you can see, the bootloader passes many parameters to the kernel. Some of these parameters are used by the kernel itself, while others can be used for our purposes. This will be discussed later, but for now, just remember that all passed parameters will be available in the intermediate Linux image through \/proc\/cmdline.<\/p>\n<p>Where can we obtain the kernel and initramfs?<br \/>\nYou can choose any Linux distribution as a basis. What to pay attention to when selecting:<\/p>\n<ul>\n<li>the boot image should be universal (availability of drivers, ability to install additional utilities);<\/li>\n<li>It will most likely be necessary to customize initramfs.<\/li>\n<\/ul>\n<p>\nHow is this done in our solution for X5? We have chosen CentOS 7 as the basis. We will perform the following trick: prepare the future image structure, pack it into an archive, and create initramfs, within which our filesystem archive will be stored. When booting the image, the archive will be unpacked into the created tmpfs partition. This way, we will have a minimal yet fully functional live Linux image with all the necessary utilities, consisting of just two files: vmkernel and initramfs. <\/p>\n<pre><code class=\"plaintext\">#\u0441\u043e\u0437\u0434\u0430\u0435\u043c \u0434\u0438\u0440\u0435\u043a\u0442\u043e\u0440\u0438\u0438: \n\nmkdir -p \/tftpboot\/toolkit\/CustomTK\/rootfs \/tftpboot\/toolkit\/CustomTK\/initramfs\/bin\n\n#\u043f\u043e\u0434\u0433\u043e\u0442\u0430\u0432\u043b\u0438\u0432\u0430\u0435\u043c \u0441\u0442\u0440\u0443\u043a\u0442\u0443\u0440\u0443:\n\nyum groups -y install &quot;Minimal Install&quot; --installroot=\/tftpboot\/toolkit\/CustomTK\/rootfs\/\nyum -y install nfs-utils mariadb ntpdate mtools syslinux mdadm tbb libgomp efibootmgr dosfstools net-tools pciutils openssl make ipmitool OpenIPMI-modalias rng-tools --installroot=\/tftpboot\/toolkit\/CustomTK\/rootfs\/\nyum -y remove biosdevname --installroot=\/tftpboot\/toolkit\/CustomTK\/rootfs\/\n\n# \u043f\u043e\u0434\u0433\u043e\u0442\u0430\u0432\u043b\u0438\u0432\u0430\u0435\u043c initramfs:\n\nwget https:\/\/busybox.net\/downloads\/binaries\/1.31.0-defconfig-multiarch-musl\/busybox-x86_64 -O \/tftpboot\/toolkit\/CustomTK\/initramfs\/bin\/busybox\nchmod a+x \/tftpboot\/toolkit\/CustomTK\/initramfs\/bin\/busybox\ncp \/tftpboot\/toolkit\/CustomTK\/rootfs\/boot\/vmlinuz-3.10.0-957.el7.x86_64 \/tftpboot\/toolkit\/tkcustom-kernel\n\n# \u0441\u043e\u0437\u0434\u0430\u0435\u043c \/tftpboot\/toolkit\/CustomTK\/initramfs\/init (\u043d\u0438\u0436\u0435 \u0441\u043e\u0434\u0435\u0440\u0436\u0430\u043d\u0438\u0435 \u0441\u043a\u0440\u0438\u043f\u0442\u0430):\n\n#!\/bin\/busybox sh\n\/bin\/busybox --install \/bin\nmkdir -p \/dev \/proc \/sys \/var\/run \/newroot\nmount -t proc proc \/proc\nmount -o mode=0755 -t devtmpfs devtmpfs \/dev\nmkdir -p \/dev\/pts \/dev\/shm \/dev\/mapper \/dev\/vc\nmount -t devpts -o gid=5,mode=620 devpts \/dev\/pts\nmount -t sysfs sysfs \/sys\nmount -t tmpfs -o size=4000m tmpfs \/newroot\necho -n &quot;Extracting rootfs... &quot;\nxz -d -c -f rootfs.tar.xz | tar -x -f - -C \/newroot\necho &quot;done&quot;\nmkdir -p \/newroot\/dev \/newroot\/proc \/newroot\/sys\nmount --move \/sys  \/newroot\/sys\nmount --move \/proc \/newroot\/proc\nmount --move \/dev  \/newroot\/dev\nexec switch_root \/newroot \/sbin\/init\n\n# \u0443\u043f\u0430\u043a\u043e\u0432\u044b\u0432\u0430\u0435\u043c rootfs \u0438 initramfs:\n\ncd \/tftpboot\/toolkit\/CustomTK\/rootfs\ntar cJf \/tftpboot\/toolkit\/CustomTK\/initramfs\/rootfs.tar.xz --exclude .\/proc --exclude .\/sys --exclude .\/dev .\ncd \/tftpboot\/toolkit\/CustomTK\/initramfs\nfind . -print0 | cpio --null -ov --format=newc | gzip -9 &gt; \/tftpboot\/toolkit\/tkcustom-initramfs-new.gz<\/code><\/pre>\n<p>\nSo, we have specified the kernel and initramfs that should be loaded. As a result, at this stage, loading the intermediate Linux image via PXE will give us the OS console.<\/p>\n<p>Great, but now we need to transfer control to our automation.<\/p>\n<p>This can be done like this.<\/p>\n<p>Let's assume that after loading the image we plan to transfer control to the script mount.sh.<br \/>\nWe will include the script mount.sh in the autostart. To do this, it will be necessary to modify initramfs:<\/p>\n<ul>\n<li>unpack initramfs (if using the above version of initramfs, this is not required)<\/li>\n<li>include in the autostart code that will analyze the parameters passed through \/proc\/cmdline and transfer control further;<\/li>\n<li>pack initramfs.<\/li>\n<\/ul>\n<p>\nNote: In the case of the X5 toolkit, control is passed to the script. <code>\/opt\/x5\/toolkit\/bin\/hook.sh \u0441 \u043f\u043e\u043c\u043e\u0449\u044c\u044e override.conf \u0432 getty tty1 (ExecStart=\u2026) <\/code><\/p>\n<p>So, an image is booted, in which the script mount.sh starts in the autostart. Then, during its execution, the mount.sh script analyzes the passed parameters (script_cmd=) and launches the required program\/script.<\/p>\n<p>label toolkit-<b>auto<\/b><br \/>\nkernel \u2026<br \/>\nappend \u2026 nfs_toolkit_script=scripts\/mount.sh <b>script_cmd=master-install.sh<br \/>\n<\/b><br \/>\nlabel toolkit-<b>shell<\/b><br \/>\nkernel \u2026<br \/>\nappend \u2026 nfs_toolkit_script=scripts\/mount.sh <b>script_cmd=\/bin\/bash<\/b><\/p>\n<p><img decoding=\"async\" alt=\"Bare-Metal Provisioning Yourself, or Automatic Server Preparation from Scratch\" src=\"\/wp-content\/uploads\/2020\/03\/943eeb64fbf746feab5119ead5b1ca66.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nHere on the left is the PXE menu, and on the right is the control transfer diagram.<\/p>\n<p>We have figured out control transfer. Depending on the choice in the PXE menu, either the auto-configuration script or the console for debugging starts.<\/p>\n<p>In the case of automatic configuration, the necessary directories from the install server are mounted, which contain:<\/p>\n<ul>\n<li>scripts;<\/li>\n<li>saved BIOS\/UEFI templates for various servers;<\/li>\n<li>firmware;<\/li>\n<li>utilities for servers;<\/li>\n<li>logs.<\/li>\n<\/ul>\n<p>\nNext, the mount.sh script transfers control to the master-install.sh script from the scripts directory.<\/p>\n<p>The script tree (the order of their execution) looks something like this:<\/p>\n<ul>\n<li>master-install<\/li>\n<li>sharefunctions (common functions)<\/li>\n<li>info (information output)<\/li>\n<li>models (installation parameters setup based on the server model)<\/li>\n<li>prepare_utils (installation of necessary utilities)<\/li>\n<li>fwupdate (firmware update)<\/li>\n<li>diag (basic diagnostics)<\/li>\n<li>biosconf (BIOS\/UEFI configuration)<\/li>\n<li>clockfix (setting time on the motherboard)<\/li>\n<li>srmconf (configuring remote interface interface)<\/li>\n<li>raidconf (configuring logical volumes)<\/li>\n<\/ul>\n<p>\none of:<\/p>\n<ul>\n<li>preinstall (handing over control to the OS or hypervisor installer, e.g., ESXi)<\/li>\n<li>merged-install (directly starting the unpacking of the image)<\/li>\n<\/ul>\n<p>\n<b>Now we know:<\/p>\n<ul>\n<li>how to boot the server via PXE;<\/li>\n<li>how to hand over control to our own script.<\/li>\n<\/ul>\n<p><\/b><br \/>\nLet's continue. The following questions have become relevant:<\/p>\n<ul>\n<li>How to identify the server we are preparing?<\/li>\n<li>Which utilities and how to configure the server?<\/li>\n<li>How to get settings for a specific server?<\/li>\n<\/ul>\n<p><\/p>\n<h3>How to identify the server we are preparing?<\/h3>\n<p>\nIt's simple \u2013 DMI:<\/p>\n<pre><code class=\"plaintext\">dmidecode \u2013s system-product-name\ndmidecode \u2013s system-manufacturer\ndmidecode \u2013s system-serial-number<\/code><\/pre>\n<p>\nHere is everything you need: vendor, model, serial number. If you are not sure that this information is available on all servers, you can identify them by MAC address. Or use both methods simultaneously if vendors are different and some models simply do not have serial number information.<\/p>\n<p>Based on the received information, network folders are mounted from the install server and everything necessary (utilities, firmware, and more) is loaded.<\/p>\n<h3>Which utilities and how to configure the server?<\/h3>\n<p>\nI will provide utilities for Linux for some manufacturers. All utilities are available on the vendors' official websites.<\/p>\n<p><img decoding=\"async\" alt=\"Bare-Metal Provisioning Yourself, or Automatic Server Preparation from Scratch\" src=\"\/wp-content\/uploads\/2020\/03\/fab53d1fa6e9a222ec69248439551ad7.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nWith firmware, I think everything is clear. They are usually provided as packaged executable files. The executable file controls the firmware update process and reports the return code.<\/p>\n<p>BIOS and IPMI are usually configured via templates. If necessary, the template can be edited just before booting.<\/p>\n<p>Some vendor RAID utilities can also be configured via templates. If not, a configuration script will need to be written.<\/p>\n<p>The RAID setup order is usually as follows:<\/p>\n<ul>\n<li>Request the current configuration.<\/li>\n<li>If logical arrays already exist \u2013 delete them.<\/li>\n<li>Check what physical disks are present and how many there are.<\/li>\n<li>Create a new logical array. Interrupt the process in case of an error.<\/li>\n<\/ul>\n<p><\/p>\n<h3>How to get settings for a specific server?<\/h3>\n<p>\nAssuming all server settings will be stored on the install server. In this case, to answer our question, we first need to decide how to transfer settings to the install server.<\/p>\n<p>Initially, it is quite possible to rely on text files. (In the future, a text file can be used as a backup method for transferring settings). <\/p>\n<p>You can 'share' a text file on the install server and add its mounting to the mount.sh script.<\/p>\n<p>The lines will appear like this:<\/p>\n<p><\/p>\n<p>These lines will be sent to a file by the engineer from their work machine. Then, during server setup, parameters for the specific server will be read from the file.<\/p>\n<p>However, in the long run, it's better to use a database for storing settings, states, and installation logs of the servers.<\/p>\n<p>Of course, one database alone won\u2019t suffice, and a client-side component will need to be created to transmit the settings to the database. Implementing this is more complex compared to a text file, but in reality, it\u2019s not as difficult as it seems. A minimal version of the client, which simply transmits data to the database, can definitely be developed independently. Moreover, enhancing the client program later on can be done freely (reports, printing labels, sending notifications, and anything else that comes to mind).<\/p>\n<p>By making a specific request to the database and specifying the server's serial number, we will get the necessary parameters for configuring the server.<\/p>\n<p>Additionally, we won't need to devise locks for simultaneous access, as is the case with a text file.<\/p>\n<p>We can log the configuration at all stages into the database and control the installation process through events and stage flags.<\/p>\n<p><b>Now we know how to:<\/p>\n<ul>\n<li>boot the server via PXE;<\/li>\n<li>pass control to our script;<\/li>\n<li>identify the server that needs to be prepared by its serial number;<\/li>\n<li>configure the server with the appropriate utilities;<\/li>\n<li>pass settings to the install-server's database using the client component.<\/li>\n<\/ul>\n<p>\nWe figured out how:<\/p>\n<ul>\n<li>the installing server gets the necessary settings from the database;<\/li>\n<li>the entire preparation progress is recorded in the database (logs, events, stage flags).<\/li>\n<\/ul>\n<p><\/b><\/p>\n<h3>What about different types of installable software? How to install a hypervisor, copy a VM, and set all this up?<\/h3>\n<p>\nIn the case of deploying a file system image (Linux) to hardware, it\u2019s quite straightforward:<\/p>\n<ul>\n<li>After configuring all the server components, we deploy the image.<\/li>\n<li>We install the grub bootloader.<\/li>\n<li>We create a chroot and configure everything necessary.<\/li>\n<\/ul>\n<p>\n<b>How to hand over control to the OS installer (using ESXi as an example).<\/b><\/p>\n<ul>\n<li>We organize the transfer of control from our script to the hypervisor installer via a response file (kickstart):<\/li>\n<li>We delete the current partitions on the disk.<\/li>\n<li>We create a partition of size 500MB.<\/li>\n<li>We mark it as bootable.<\/li>\n<li>We format it to FAT32.<\/li>\n<li>We copy the installation files for ESXi to the root of it.<\/li>\n<li>We install syslinux.<\/li>\n<li>We copy syslinux.cfg to \/syslinux\/<\/li>\n<\/ul>\n<p><\/p>\n<pre><code class=\"plaintext\">default esxi\nprompt 1\ntimeout 50\nlabel esxi\nkernel mboot.c32\nappend -c boot.cfg<\/code><\/pre>\n<p><\/p>\n<ul>\n<li>We copy mboot.c32 to \/syslinux.<\/li>\n<li>In boot.cfg, it should be kernelopt=ks=ftp:\/\/\/ks_esxi.cfg<\/li>\n<li>Reboot the server.<\/li>\n<\/ul>\n<p>\nAfter rebooting the server from its hard drive, the ESXi installer will load. All necessary installer files will be loaded into memory, and the installation of ESXi will begin according to the specified response file.<\/p>\n<p>Here are a few lines from the response file ks_esxi.cfg:<\/p>\n<pre><code class=\"plaintext\">%firstboot --interpreter=busybox\n\u2026\n# Get the serial number\n\nSYSSN=$(esxcli hardware platform get | grep Serial | awk -F \" \" '{print $3}')\n\n# Get the IP\n\nIPADDRT=$(esxcli network ip interface ipv4 get | grep vmk0 | awk -F \" \" '{print $2}')\nLAST_OCTET=$(echo $IPADDRT | awk -F'.' '{print $4}')\n\n# Connect to the NFS install server\n\nesxcli storage nfs add -H is -s \/srv\/nfs_share -v nfsshare1\n\n# Copy temporary SSH settings for SSH client usage\n\nmv \/etc\/ssh \/etc\/ssh.tmp\ncp -R \/vmfs\/volumes\/nfsshare1\/ssh \/etc\/\nchmod go-r \/etc\/ssh\/ssh_host_rsa_key\n\n# Copy the ovftool for VM deployment now, may be useful later\n\ncp -R \/vmfs\/volumes\/nfsshare1\/ovftool \/vmfs\/volumes\/datastore1\/\n\n# Deploy the VMs\n\n\/vmfs\/volumes\/datastore1\/ovftool\/tools\/ovftool --acceptAllEulas --noSSLVerify --datastore=datastore1 --name=VM1 \/vmfs\/volumes\/nfsshare1\/VM_T\/VM1.ova vi:\/\/root:esxi_password@127.0.0.1\n\/vmfs\/volumes\/datastore1\/ovftool\/tools\/ovftool --acceptAllEulas --noSSLVerify --datastore=datastore1 --name=VM2 \/vmfs\/volumes\/nfsshare1\/VM_T\/VM2.ova vi:\/\/root:esxi_password@127.0.0.1\n\n# Get a line with our server settings\n\nssh root@is \"mysql -h'192.168.0.1' -D'servers' -u'user' -p'secretpassword' -e \"SELECT ... WHERE servers.serial='$SYSSN'\"\" | grep -v ^$ | sed 's\/NULL\/\/g' &gt; \/tmp\/servers\n...\n# Generate network configuration script\n\necho '#!\/bin\/sh' &gt; \/vmfs\/volumes\/datastore1\/netconf.sh\necho \"esxcli network ip interface ipv4 set -i=vmk0 -t=static --ipv4=$IPADDR --netmask=$S_SUB || exit 1\" &gt;&gt; \/vmfs\/volumes\/datastore1\/netconf.sh\necho \"esxcli network ip route ipv4 add -g=$S_GW -n=default || exit 1\" &gt;&gt; \/vmfs\/volumes\/datastore1\/netconf.sh\nchmod a+x \/vmfs\/volumes\/datastore1\/netconf.sh\n\n# Set the guestinfo.esxihost.id parameter with the serial number\n\necho \"guestinfo.esxihost.id = \"$SYSSN\"\" &gt;&gt; \/vmfs\/volumes\/datastore1\/VM1\/VM1.vmx\necho \"guestinfo.esxihost.id = \"$SYSSN\"\" &gt;&gt; \/vmfs\/volumes\/datastore1\/VM2\/VM2.vmx\n...\n# Update information in the database\n\nSYSNAME=$(esxcli hardware platform get | grep Product | sed 's\/Product Name:\/\/;s\/^ *\/\/')\nUUID=$(vim-cmd hostsvc\/hostsummary | grep uuid | sed 's\/ \/\/g;s\/,$\/\/;s\/^uuid=\"\/\/;s\/\"$\/\/')\nssh root@is \"mysql -D'servers' -u'user' -p'secretpassword' -e \"UPDATE servers ... SET ... WHERE servers.serial='$SYSSN'\"\"\nssh root@is \"mysql -D'servers' -u'user' -p'secretpassword' -e \"INSERT INTO events ...\"\"\n\n# Restore SSH settings\n\nrm -rf \/etc\/ssh\nmv \/etc\/ssh.tmp \/etc\/ssh\n\n# Configure the network and reboot\n\nesxcli system hostname set --fqdn=esx-${G_NICK}.x5.ru\n\/vmfs\/volumes\/datastore1\/netconf.sh\nreboot\n<\/code><\/pre>\n<p>\nAt this stage, the hypervisor is installed and configured, and the virtual machines have been copied.<\/p>\n<p>How to configure the virtual machines now?<\/p>\n<p>We played a little trick: during installation we set the parameter guestinfo.esxihost.id = \"$SYSSN\" in the VM1.vmx file, specifying the serial number of the physical server.<\/p>\n<p>Now, after starting, the virtual machine (with the VMware Tools package installed) can access this parameter:<\/p>\n<pre><code class=\"plaintext\">ESXI_SN=$(vmtoolsd --cmd \"info-get guestinfo.esxihost.id\")<\/code><\/pre>\n<p>\nIn other words, the VM will be able to identify itself (it knows the serial number of the physical host), make a request to the installation server database, and retrieve the parameters that need to be configured. This is all formalized in a script that should be automatically executed upon the startup of the guest OS VM (but only once: RunOnce).<\/p>\n<p><b>Now we know how to:<\/p>\n<ul>\n<li>boot the server via PXE;<\/li>\n<li>pass control to our script;<\/li>\n<li>identify the server that needs to be prepared by its serial number;<\/li>\n<li>configure the server using the appropriate utilities;<\/li>\n<li>transfer settings to the installation server database using the client-side;<\/li>\n<li>configure various types of software, including deploying the ESXi hypervisor and setting up virtual machines (and all automatically).<\/li>\n<\/ul>\n<p>\nWe figured out how:<\/p>\n<ul>\n<li>the installing server gets the necessary settings from the database;<\/li>\n<li>the entire preparation progress is recorded in the database (logs, events, stage flags).<\/li>\n<\/ul>\n<p><\/b><br \/>\n<u><b>Conclusion:<\/b><\/u><\/p>\n<p>I believe the uniqueness of this solution lies in its flexibility, simplicity, capabilities, and versatility.<\/p>\n<p>Please write in the comments what you think.<br \/>\n<br \/>Source: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/X5RetailGroup\/blog\/493124\/\">habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u041f\u0440\u0438\u0432\u0435\u0442, \u044f \u0414\u0435\u043d\u0438\u0441 \u0438 \u043e\u0434\u043d\u043e \u0438\u0437 \u043c\u043e\u0438\u0445 \u043d\u0430\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u0438\u0439 \u0434\u0435\u044f\u0442\u0435\u043b\u044c\u043d\u043e\u0441\u0442\u0438 \u2013 \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u043a\u0430 \u0438\u043d\u0444\u0440\u0430\u0441\u0442\u0440\u0443\u043a\u0442\u0443\u0440\u043d\u044b\u0445 \u0440\u0435\u0448\u0435\u043d\u0438\u0439 \u0432 X5. \u0421\u0435\u0433\u043e\u0434\u043d\u044f \u0445\u043e\u0442\u0435\u043b \u0431\u044b \u043f\u043e\u0434\u0435\u043b\u0438\u0442\u044c\u0441\u044f \u0441 \u0432\u0430\u043c\u0438 \u043e \u0442\u043e\u043c, \u043a\u0430\u043a \u043c\u043e\u0436\u043d\u043e \u043d\u0430 \u0431\u0430\u0437\u0435 \u043e\u0431\u0449\u0435\u0434\u043e\u0441\u0442\u0443\u043f\u043d\u044b\u0445 \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u043e\u0432 \u0440\u0430\u0437\u0432\u0435\u0440\u043d\u0443\u0442\u044c \u0430\u0432\u0442\u043e\u043c\u0430\u0442\u0438\u0447\u0435\u0441\u043a\u0443\u044e \u0441\u0438\u0441\u0442\u0435\u043c\u0443 \u043f\u043e\u0434\u0433\u043e\u0442\u043e\u0432\u043a\u0438 \u0441\u0435\u0440\u0432\u0435\u0440\u043e\u0432. \u041d\u0430 \u043c\u043e\u0439 \u0432\u0437\u0433\u043b\u044f\u0434, \u044d\u0442\u043e \u0438\u043d\u0442\u0435\u0440\u0435\u0441\u043d\u043e\u0435, \u043f\u0440\u043e\u0441\u0442\u043e\u0435 \u0438 \u0433\u0438\u0431\u043a\u043e\u0435 \u0440\u0435\u0448\u0435\u043d\u0438\u0435. \u041f\u043e\u0434 \u043f\u043e\u0434\u0433\u043e\u0442\u043e\u0432\u043a\u043e\u0439 \u043f\u043e\u0434\u0440\u0430\u0437\u0443\u043c\u0435\u0432\u0430\u0435\u0442\u0441\u044f: \u0441\u0434\u0435\u043b\u0430\u0442\u044c \u0438\u0437 \u043d\u043e\u0432\u043e\u0433\u043e \u0441\u0435\u0440\u0432\u0435\u0440\u0430 \u0438\u0437 \u043a\u043e\u0440\u043e\u0431\u043a\u0438, \u043f\u043e\u043b\u043d\u043e\u0441\u0442\u044c\u044e \u043d\u0430\u0441\u0442\u0440\u043e\u0435\u043d\u043d\u044b\u0439 \u0441\u0435\u0440\u0432\u0435\u0440 [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":74906,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-74905","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.2 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u041f\u0440\u0438\u0432\u0435\u0442, \u044f \u0414\u0435\u043d\u0438\u0441 \u0438 \u043e\u0434\u043d\u043e \u0438\u0437 \u043c\u043e\u0438\u0445 \u043d\u0430\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u0438\u0439 \u0434\u0435\u044f\u0442\u0435\u043b\u044c\u043d\u043e\u0441\u0442\u0438 \u2013 \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u043a\u0430 \u0438\u043d\u0444\u0440\u0430\u0441\u0442\u0440\u0443\u043a\u0442\u0443\u0440\u043d\u044b\u0445 \u0440\u0435\u0448\u0435\u043d\u0438\u0439 \u0432 X5.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/bare-metal-provisioning-svoimi-rukami-ili-avtomaticheskaya-podgotovka-serverov-s-nulya\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.2\" \/>\n\t\t<meta property=\"og:locale\" content=\"en_US\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47Bare-Metal Provisioning \u0441\u0432\u043e\u0438\u043c\u0438 \u0440\u0443\u043a\u0430\u043c\u0438, \u0438\u043b\u0438 \u0410\u0432\u0442\u043e\u043c\u0430\u0442\u0438\u0447\u0435\u0441\u043a\u0430\u044f \u043f\u043e\u0434\u0433\u043e\u0442\u043e\u0432\u043a\u0430 \u0441\u0435\u0440\u0432\u0435\u0440\u043e\u0432 \u0441 \u043d\u0443\u043b\u044f | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u041f\u0440\u0438\u0432\u0435\u0442, \u044f \u0414\u0435\u043d\u0438\u0441 \u0438 \u043e\u0434\u043d\u043e \u0438\u0437 \u043c\u043e\u0438\u0445 \u043d\u0430\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u0438\u0439 \u0434\u0435\u044f\u0442\u0435\u043b\u044c\u043d\u043e\u0441\u0442\u0438 \u2013 \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u043a\u0430 \u0438\u043d\u0444\u0440\u0430\u0441\u0442\u0440\u0443\u043a\u0442\u0443\u0440\u043d\u044b\u0445 \u0440\u0435\u0448\u0435\u043d\u0438\u0439 \u0432 X5.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/bare-metal-provisioning-svoimi-rukami-ili-avtomaticheskaya-podgotovka-serverov-s-nulya\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-03-21T17:42:25+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-03-21T17:42:25+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Bare-Metal Provisioning on your own, or Automatic Server Setup from Scratch | ProHoster","description":"Hi, I'm Denis, and one of my areas of expertise is developing infrastructure solutions at X5.","canonical_url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/bare-metal-provisioning-svoimi-rukami-ili-avtomaticheskaya-podgotovka-serverov-s-nulya","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"en_US","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47Bare-Metal Provisioning \u0441\u0432\u043e\u0438\u043c\u0438 \u0440\u0443\u043a\u0430\u043c\u0438, \u0438\u043b\u0438 \u0410\u0432\u0442\u043e\u043c\u0430\u0442\u0438\u0447\u0435\u0441\u043a\u0430\u044f \u043f\u043e\u0434\u0433\u043e\u0442\u043e\u0432\u043a\u0430 \u0441\u0435\u0440\u0432\u0435\u0440\u043e\u0432 \u0441 \u043d\u0443\u043b\u044f | ProHoster","og:description":"\u041f\u0440\u0438\u0432\u0435\u0442, \u044f \u0414\u0435\u043d\u0438\u0441 \u0438 \u043e\u0434\u043d\u043e \u0438\u0437 \u043c\u043e\u0438\u0445 \u043d\u0430\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u0438\u0439 \u0434\u0435\u044f\u0442\u0435\u043b\u044c\u043d\u043e\u0441\u0442\u0438 \u2013 \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u043a\u0430 \u0438\u043d\u0444\u0440\u0430\u0441\u0442\u0440\u0443\u043a\u0442\u0443\u0440\u043d\u044b\u0445 \u0440\u0435\u0448\u0435\u043d\u0438\u0439 \u0432 X5.","og:url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/bare-metal-provisioning-svoimi-rukami-ili-avtomaticheskaya-podgotovka-serverov-s-nulya","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-03-21T17:42:25+00:00","article:modified_time":"2020-03-21T17:42:25+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"74905","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 18:05:46","updated":"2026-02-09 16:50:18","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/74905","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/comments?post=74905"}],"version-history":[{"count":1,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/74905\/revisions"}],"predecessor-version":[{"id":158713,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/74905\/revisions\/158713"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media\/74906"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media?parent=74905"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/categories?post=74905"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/tags?post=74905"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}