Easily manage microservice configurations with microconfig.io

One of the main problems in the development and subsequent operation of microservices is the competent and accurate configuration of their instances. In this, in my opinion, a new framework can help microconfig.io. It allows you to quite elegantly solve some of the routine tasks of setting up applications.

If you have a lot of microservices, and each of them comes with its own configuration file/files, then there is a high probability of making a mistake in one of them, which without proper dexterity and a logging system can be very difficult to catch. The main task that the framework sets for itself is to minimize duplicate instance settings, thereby reducing the likelihood of adding an error.

Let's look at an example. Let's say we have a simple application with a config file yaml. It can be any microservice in any language. Let's see how the framework can be applied to this service.

But first, for greater convenience, let's create an empty project in the Idea IDE, after installing the microconfig.io plugin in it:

Set up the plugin launch configuration, you can use the default configuration, as in the screenshot above.

Our service is called order, then in a new project we will create a similar structure:

In the folder with the name of the service we place the configuration file - application.yaml. All microservices are launched in some kind of environment, so, in addition to creating the config of the service itself, it is necessary to describe the environment itself: for this, we will create a folder envs and add a file to it with the name of our working environment. In this way, the framework will create configuration files for services in the environment giant, since this parameter is set in the plugin settings.

File structure dev.yaml would be pretty simple:

mainorder:
    components:
         - order

The framework works with configurations that are combined into groups. For our service, let's choose a name for the group main order. The framework finds each such group of applications in the environment file and creates configurations for all of them, which it finds in their respective folders.

In the service settings file itself order Let's specify only one parameter for now:

spring.application.name: order

Now let's run the plugin, and it will generate the necessary configuration of our service for us at the path specified in the properties:

One can get along and without installing a plugin, just download the framework distribution and run it from the command line.
This solution is suitable for use on the build server.

It should be noted that the framework perfectly understands property syntax, i.e. regular property files that can be used together in yaml configurations.

Let's add another service payment and complicate the existing one at the same time.
В order:

eureka:
 instance.preferIpAddress: true
 client:
   serviceUrl:
     defaultZone: http://192.89.89.111:6782/eureka/
server.port: 9999
spring.application.name: order
db.url: 192.168.0.100

В payment:

eureka:
 instance.preferIpAddress: true
 client:
   serviceUrl:
     defaultZone: http://192.89.89.111:6782/eureka/
server.port: 9998
spring.application.name: payments
db.url: 192.168.0.100

The main problem with these configurations is the presence of a large amount of copy-paste in the service settings. Let's see how the framework will help get rid of it. Let's start with the most obvious - the presence of a configuration eureka in the description of each microservice. Let's create a new directory with the settings file and add a new configuration to it:

And in each of our projects we will now add a line #include eureka.

The framework will automatically find the eureka configuration itself and copy it to the services configuration files, while a separate eureka configuration will not be created, since we will not specify it in the environment file dev.yaml... Service order:

#include eureka
server.port: 9999
spring.application.name: order
db.url: 192.168.0.100

We can also move the database settings to a separate configuration by changing the import line to #include eureka, oracle.

It is worth noting that the framework tracks every change when regenerating configuration files and places it in a special file next to the main configuration file. The entry in his log looks like this: “Stored 1 property changes to order/diff-application.yaml". This allows you to quickly detect changes in large configuration files.

Removing the common parts of the configuration allows you to get rid of a lot of unnecessary copy-paste, but does not allow you to flexibly create a configuration for different environments - the endpoints of our services are single and hardcoded, which is bad. Let's try to remove it.

A good solution would be to keep all endpoints in one configuration that the rest could refer to. To do this, support for placeholders has been introduced into the framework. Here is how the config file will change eureka:

 client:
   serviceUrl:
     defaultZone: http://${endpoints@eurekaip}:6782/eureka/

Now let's see how this placeholder works. The system finds a component named endpoints and look for the value in it eurekaip, after which it substitutes into our configuration. But what about different environments? To do this, create a settings file in endpoints the following kind application.dev.yaml. The framework independently, by file extension, decides which environment this configuration belongs to and loads it:

dev file content:

eurekaip: 192.89.89.111
dbip: 192.168.0.100

We can create the same configuration for the ports of our services:

server.port: ${ports@order}.

All important settings are in one place, thereby reducing the likelihood of errors due to scattered settings across configuration files.

The framework provides many ready-made placeholders, for example, you can get the name of the directory in which the configuration file is located and assign it:

#include eureka, oracle
server.port: ${ports@order}
spring.application.name: ${this@name}

Due to this, there is no need to additionally specify the application name in the configuration and it can also be moved to a common module, for example, to the same eureka:

client:
   serviceUrl:
     defaultZone: http://${endpoints@eurekaip}:6782/eureka/
 spring.application.name: ${this@name}

The configuration file order reduced to one line:

#include eureka, oracle
server.port: ${ports@order}

If we do not need any setting from the parent configuration, we can specify it in our configuration and it will be applied during generation. That is, if for some reason we need a unique name for the order service, we just leave the parameter spring.application.name.

Suppose you need to add customized logging settings to the service, which are stored in a separate file, for example, logback.xml. Let's create a separate group of settings for it:

In the basic configuration, we will tell the framework where to place the logging settings file we need using the placeholder @ConfigDir:

microconfig.template.logback.fromFile: ${logback@configDir}/logback.xml

In file logback.xml we set up standard appenders, which in turn can also contain placeholders that the framework will change during config generation, for example:

<file>logs/${this@name}.log</file>

By adding imports to service configurations logback, we automatically get configured logging for each service:

#include eureka, oracle, logback
server.port: ${ports@order}

It's time to take a closer look at all the available placeholders of the framework:

${this@env} - returns the name of the current environment.
${…@name} - returns the name of the component.
${…@configDir} returns the full path to the component's config directory.
${…@resultDir} — returns the full path to the component's destination directory (the resulting files will be placed in this directory).
${this@configRoot} - returns the full path to the root directory of the configuration repository.

The system also allows you to get environment variables, for example, the path to java:
${env@JAVA_HOME}
Or, since the framework is written in JAVA, we can get system variables similar to calling System::getProperty with a structure like this:
${[email protected]}
It is worth mentioning the support for language extensions Spring EL. In a configuration, expressions like this apply:

connection.timeoutInMs: #{5 * 60 * 1000}
datasource.maximum-pool-size: #{${[email protected]} + 10} 

and you can use local variables in configuration files with the expression #where:

#var feedRoot: ${[email protected]}/feed
folder:
 root: ${this@feedRoot}
 success: ${this@feedRoot}/archive
 error: ${this@feedRoot}/error

Thus, the framework is a fairly powerful tool for fine-tuning and flexible configuration of microservices. The framework perfectly fulfills its main task - eliminating copy-paste in settings, consolidating settings and, as a result, minimizing possible errors, while allowing you to easily combine configurations and change them for different environments.

If you are interested in this framework, then I recommend visiting its official page and getting acquainted with the full documentationor dig into the sources here.

Source: habr.com