To provide IaaS (Virtual Data Center) service, we are in using a commercial orchestrator (FCO). This solution has a rather unique architecture, which distinguishes it from the well-known Openstack and CloudStack.
KVM, VmWare, Xen, Virtuozzo6/7, as well as containers from the same Virtuozzo are supported as compute node hypervisors. Supported storages include local, NFS, Ceph and Virtuozzo Storage.
FCO supports the creation and management of multiple clusters from a single interface. That is, you can manage a Virtuozzo cluster and a KVM + Ceph cluster by switching between them with a mouse click.
In essence, FCO is a comprehensive solution for cloud providers, which, in addition to orchestration, also includes billing, with all settings, payment plugins, accounts, notifications, resellers, tariffs, and so on. However, the billing part is not able to cover all the Russian nuances, so we abandoned its use in favor of another solution.
I am very pleased with the flexible system of distribution of rights to all cloud resources: images, disks, products, servers, firewalls - all this can be "rummaged around" and issue rights between users, and even between users of different clients. Each client can create several independent data centers in his cloud and manage them from a single control panel.
Architecturally, FCO consists of several parts, each of which has its own independent code, and some of them have their own database.
skyline – admin and user interface
Jade – business logic, billing, task management
Tigerlily – service coordinator, manages and coordinates the exchange of information between business logic and clusters.
XVPManager – management of cluster elements: nodes, storage, network and virtual machines.
XVPAgent – agent installed on nodes to interact with XVPManager
We plan to include a detailed story about the architecture of each component in a series of articles, if, of course, the topic is of interest.
The main advantage of FCO comes from its "boxed" nature. Simplicity and minimalism are at your service. For the control node, one virtual machine on Ubuntu is allocated, into which all the necessary packages are installed. All settings are transferred to the configuration files in the form of a variable-value:
# cat /etc/extility/config/vars
…
export LIMIT_MAX_LIST_ADMIN_DEFAULT="30000"
export LIMIT_MAX_LIST_USER_DEFAULT="200"
export LOGDIR="/var/log/extility"
export LOG_FILE="misc.log"
export LOG_FILE_LOG4JHOSTBILLMODULE="hostbillmodule.log"
export LOG_FILE_LOG4JJADE="jade.log"
export LOG_FILE_LOG4JTL="tigerlily.log"
export LOG_FILE_LOG4JXVP="xvpmanager.log"
export LOG_FILE_VARS="misc.log"
…
The entire configuration is corrected initially in the templates, then the generator starts
#build-config which will generate a vars file and instruct services to reread the config. The user interface is nice and can be easily branded.
As you can see, the interface consists of widgets that can be controlled by the user. He can easily add / remove widgets from the page, thereby forming the dashboard he needs.
Despite its closeness, FCO is a highly customizable system. It has a huge number of settings and entry points for changing the workflow:
- Custom plugins are supported, for example, you can write your own billing method or your own external resource to provide the user
- Custom triggers for certain events are supported, for example, adding the first virtual machine to a client when it is created
- Custom widgets are supported in the interface, for example, embed a youtube video directly into the user interface.
All customization is written in the FDL language, which is based on Lua. If you know Lua, there will be no problem with FDL.
Here is an example of one of the simplest triggers we use. This trigger prevents users from sharing their own images with other clients. We do this so that one user cannot create a malicious image for other users.
function register()
return {"pre_user_api_publish"}
end
function pre_user_api_publish(p)
if(p==nil) then
return{
ref = "cancelPublishImage",
name = "Cancel publishing",
description = "Cancel all user’s images publishing",
triggerType = "PRE_USER_API_CALL",
triggerOptions = {"publishResource", "publishImage"},
api = "TRIGGER",
version = 1,
}
end
-- Turn publishing off
return {exitState = "CANCEL"}
end
The register function will be called by the FCO core. It will return the name of the function to be called. The “p” parameter of this function stores the call context, and the first time it is called it will be empty (nil). Which will allow us to register our trigger. In triggerType, we indicate that the trigger is called BEFORE the publish operation, and only applies to users. System administrators, of course, we allow to publish everything. In triggerOptions we detail the operations for which the trigger will fire.
And most importantly - return {exitState = "CANCEL"}, that's what the trigger was designed for. It will return failure when the user tries to share their image in the control panel.
In the FCO architecture, any object (disk, server, image, network, network adapter, etc.) is represented as a Resource entity, which has common parameters:
- Resource UUID
- resource name
- resource type
- resource owner UUID
- resource status (active, inactive)
- resource metadata
- resource keys
- The UUID of the product that owns the resource
- Resource VDC
This is very convenient when working with the API, when all resources are handled according to the same principle. Products are configured by the provider and ordered by the customer. Since our billing is aside, the client can freely order any product from the panel for free. It will be calculated later in billing. The product can be - an ip address per hour, an additional GB of disk per hour, or just a server.
You can mark certain resources with keys to change the logic of working with them. For example, we can mark three physical nodes with the Weight key, and mark some clients with the same key, thereby highlighting these nodes personally to these clients. We use this mechanism for VIP clients who do not like neighbors next to their VMs. The functionality itself can be applied much more widely.
The licensing model implies payment for each processor core of a physical node. The number of types of clusters also affects the cost. If you plan to use together, for example, KVM and VMware, then the cost of the license will increase.
FCO is a full-fledged product, its functionality is very rich, so we plan to prepare several articles at once with a detailed description of the functioning of the network part.
Having worked with this orchestrator for several years, we can mark it as very suitable. Alas, the product is not without flaws:
- we had to optimize the database, since queries began to slow down when the amount of data in them increased;
- after one accident, the recovery mechanism did not work due to a bug, and I had to raise the machines of unfortunate clients with my own set of scripts;
- The node unavailability detection mechanism is hardwired into the code and cannot be customized. That is, we cannot create our own policies for determining the unavailability of a node.
- logging is not always detailed. Sometimes, when you need to go down to a very low level to analyze a certain problem, there is not enough source code for some components to understand the reasons;
TOTAL: in general, the impressions of the product are good. We are in constant contact with the developers of the orchestrator. Guys are located to constructive cooperation.
Despite its simplicity, FCO has a wide range of functionality. In future articles, we plan to delve into the following topics:
- networking in FCO
- providing live-recovery and FQP protocol
- writing your own plugins and widgets
- connection of additional services, such as Load Balancer and Acronis
- backup
- unified mechanism for configuring and configuring nodes
- virtual machine metadata processing
ZY Write in the comments if you are interested in other aspects. Stay tuned!
Source: habr.com