nxs-data-anonymizer 1.4.0 has been released — a tool for anonymizing PostgreSQL and MySQL/MariaDB/Percona database dumps. The utility supports data anonymization based on templates and functions from the Sprig library. Among other things, values from other columns in the same row can be used for filling. The tool can be used through unnamed pipes in the command line and redirect dumps from the source database directly to the target database with the necessary transformations. The tool is written in Go and released under the Apache License 2.0.
Less than a year after the first release of version 1.0.0, the tool has gained the following features:
- A filter function has been added to work with null values.
- The -l/—log-format command has been implemented, allowing the selection of the logging format (json or plain).
- Indication of the anonymization process has been added — progress information is outputted at specified intervals during the execution of the operation.
- In version 1.4, the ability to set field values using external commands was introduced, by adding 'type: command' to the column value. For example: filters: some_table_name: columns: some_column_name: type: command value: /path/to/command/or/script.sh
If 'type: command' is specified for a column, the value field is treated as a file path to the command that will be executed each time for that field. During the execution of the command, additional environment variables will be available:
- ENVVARTABLE={TABLE_NAME}: contains the name of the filtered table
- ENVVARCOLUMN_{COLUMN_NAME}={COLUMN_VALUE}: contains all columns and their values (before replacement) for the current filtered row. The stdout of the command is considered as the new value for the filtered field.
Source: opennet.ru
