Hello! We are "Hosting Technologies" and launched 5 years ago — the first vds hosting created specifically for developers. We strive to make it convenient, like DigitalOcean, but with Russian support, payment methods and servers in Russia. But DigitalOcean is not only reliability and price, it is also a service.
Software from ISPsystem turned out to be a rope that tied our hands on the way to a cool service. Three years ago, we used Billmanager billing and the VMmanager server control panel and quickly realized that it was almost impossible to provide a good service without our own control panel.
How ISPsystem Killed Convenience
Bugs
We could not fix the bug ourselves - each time we had to write to someone else's support and wait. The solution to any problem required the response of a third-party company.
ISPsystem support responded normally, but fixes came only after a few releases, and then not always and not all. Sometimes critical bugs were corrected for several weeks. We had to reassure customers, apologize and wait for ISPsystem to fix the bug.
Downtime Threat
Updates could generate unpredictable downtimes that provoked new errors.
Each update was a lottery: I had to cover up billing and make sacrifices to the gods of updates - a couple of times the update caused downtime for 10-15 minutes. Our admins at this time were sitting on their eyes - we never knew how long the downtime would last and could not predict when ISPsystem would decide to release a new update.
On the fifth generation, Billmanager got better, but in order to get access to the necessary features, I had to install a beta, which was already updated every week. If something broke, I had to give access to other developers so that they could fix something.
Inconvenient panel interface
Everything was divided into different panels and controlled from different places. For example, customers paid through Billmanager, and they had to reboot or reinstall VDS in VMManager. Our staff also had to switch between windows to help a client, check the load on his server, or see what OS he was using.
Such an interface takes time - both ours and our clients'. There is no question of any convenience, like that of DigitalOcean, in such a situation.
Short lifecycles with frequent API updates
We wrote our own plugins - for example, a plugin with additional payment methods that are not in VMManager.
In recent years, VMManager had a relatively short life cycle, and in new versions, the names of variables or functions in the API could change arbitrarily - this broke our plugins. Support for older versions was quickly phased out and had to be updated.
Can't be modified
More precisely, it is possible, but extremely inefficient. License restrictions do not allow you to make changes to the source code, you can only write plugins. Maximum plugins - some menu items, a step-by-step wizard. ISPsystem are designed for versatility, but we needed specialized solutions.
So the decision was ripe to write my own panel. We have set goals:
- Respond quickly to errors, bugs and be able to fix them yourself without making the client wait.
- Freely modify the interface for workflows and client needs.
- Increase usability with a clean and understandable design.
And we started development.
New Panel Architecture
We have a self-sufficient development team, so we wrote the panel ourselves.
The main work was done by three engineers - technical director Sergey came up with the architecture and wrote the server agent, Alexey did the billing, and the front-end was assembled by our front-ender Artysh.
Step 1: Server Agent
The server agent is a python web server that manages the library , which in turn governs .
The agent manages all services on the server: creating, stopping, deleting vds, installing operating systems, changing parameters, and so on through the libvirt library. At the time of publication of the article, these are more than forty different functions, which we supplement depending on the task and the needs of the client.
In theory, libvirt could be controlled directly from billing, but this required too much additional code and we decided to separate these functions between the agent and billing - billing simply makes requests to the agent via the JSON API.
The agent is the first thing we did, since it did not require any interface and it was possible to test it directly from the server console.
What the server agent gave us: a layer has appeared that simplifies life for everyone - billing does not need to send a whole bunch of commands, but only make a request. And the agent will do everything that is needed: for example, it will allocate disk space and RAM.
Step 2. Billing
For our developer Alex, this was not the first control panel - Alex has been in hosting for a long time, so he generally understood what the client needed and what the hoster needed.
We call billing among ourselves a “control panel”: it contains not only money and services, but also their management, customer support and much more.
To switch from ISPSystem software, it was necessary to fully preserve the previous functionality for customers, transfer all financial actions of users from the old billing to the new one, as well as all services and connections between them. We studied what is in the current product, then the solutions of competitors, mainly DO and Vultr. We looked at the disadvantages and advantages, collected feedback from people who worked with old products from ISPsystem.
The new billing used two stacks: classic PHP, MySQL (and in the future it is planned to switch to PostgreSQL), Yii2 as a framework on the backend and VueJS on the front. Stacks work independently of each other, are developed by different people, and communicate using the JSON API. For development then and now we use и from JetBrains and love them dearly (hey guys!)
The panel is designed on a modular basis: payment system modules, domain registrar module or, for example, an SSL certificate module. You can easily add a new feature or remove an old one. The groundwork for the expansion is laid architecturally, including in the opposite direction, “towards the hardware”.
What we got: a control panel over which we have full control. Now bugs are fixed in hours, not weeks, and new features are implemented at the request of customers, and not at the request of ISPSystem.
Step 3 Interface
The interface is our team brainchild.
First, we looked at what would happen if we made an add-on over the ISPsystem API without fundamentally changing anything in the interface. It turned out so-so and we decided to do everything from scratch.
We believed that the main thing is to make the interface logical, with a clean and minimalistic design, and then we will get a beautiful panel. The location of the elements was discussed in Megaplan and the interface that users see in the control panel now will gradually be born.
The design of the billing page was the first to appear, because we have already made payment plugins for ISPsystem.
Frontend
They decided to make the panel a SPA application - undemanding to resources and with fast data loading. Our front-ender Artysh decided to write it on Vue — at that time Vue had just appeared. We assumed that the framework would develop dynamically, like React, after some time the Vue community would grow and a sea of libraries would appear. We bet on Vue and did not regret it - now it takes little time to add new functions to the front that have already been programmed on the backend. We will tell you more about the front-end panel in a separate article.
Connecting the frontend to the backend
The frontend was connected to the backend via push notifications. I had to work hard and write my own handler, but now the information on the page is updated almost instantly.
What happened: The panel interface has become simpler. We made it adaptive, and fast loading allows you to use it even from mobile phones in the last minutes before takeoff, without installing a separate application to work with the panel.
Step 4. Testing and migration scheme
When everything started up and the first tests passed, the question of migration arose. First of all, we installed billing and started testing its operation with the server agent.
Then we wrote a simple script that transfers the database from the old billing to the new one.
I had to test and recheck literally everything, since the data was merged into one new database from three old ones: Billmanager, VMmanager and the manager's IPmanager. Perhaps the test migrations are the most difficult thing we encountered in the process of developing a new panel.
After rechecking, we closed the old billing. The final data migration was a very troubling moment, but, thank God, it was completed in a few minutes and without noticeable problems. There were minor bugs that we fixed during the week. Most of the time was spent testing what happened.
Then we sent letters to clients with the address of the new panel and billing and made a redirect.
In summary: IT'S ALIVE!
Happy end
From the first hours of work of our software, we felt all the delights of the transition. The code was completely ours and with a convenient architecture, and the interface was clean and logical.
First review after the launch of the new panel
We launched the transition process in December, on the eve of New Year 2017, when the load was the least, to make the transition easier for customers - almost no one works on the eve of the holidays.
The main thing that we got when switching to our system (apart from general reliability and convenience) is the ability to quickly add functionality for key customers - to be their face, not their ass.
What's next?
We are growing, the amount of data, customers, customer data is growing. I had to add a Memcached server and two queue managers with different tasks to the backend. The frontend has caching and its own queues.
Of course, we still had adventures as the product developed and became more complex, for example when we added HighLoad.
In the next article, we will tell you how the Hi-CPU tariff was launched: about hardware, software, what tasks we solved and what we did.
Source: habr.com