The release of the OpenNMT-tf 2.30.0 (Open Neural Machine Translation) translation system has been published, utilizing machine learning techniques. The code for the modules developed by the OpenNMT-tf project is written in Python, uses the TensorFlow library, and is distributed under the MIT license.
A variant of OpenNMT based on the PyTorch library is also being developed, differing in terms of supported features. Additionally, the PyTorch version of OpenNMT is presented as simpler to use and multimodal, while the TensorFlow version is noted for being modular, stable, and capable of leveraging GPU capabilities to accelerate the neural network training process. To facilitate product distribution, the project is also developing a self-contained version of the translator in C++ — CTranslate2, which uses pretrained models without relying on additional dependencies.
Models are prepared for English, German, and Catalan languages; for other languages, one can independently create a model based on the dataset from the OPUS project (the training system requires two files — one with sentences in the source language and a second with high-quality translations of these sentences into the target language).
The project is developed with the involvement of SYSTRAN, a company specializing in machine translation tools, and a group of researchers from Harvard, developing human language models for machine learning systems. The user interface is maximally simplified and requires only specifying the input file with the text and a file for saving the translation result. The extension system allows for additional functionality to be implemented based on OpenNMT, such as auto-referencing, text classification, and subtitle generation.
In the new version:
- Support for TensorFlow 2.11 has been added, but the new Keras optimizers are not yet supported (the use of tf.keras.optimizers.legacy mode is required).
- Support for the new branch of the CTranslate2 3.x engine has been added, designed for efficient execution of transformer architecture models.
- A training parameter for the models, pad_to_bucket_boundary has been added to include additional padding, aligning the block size to values that are multiples of length_bucket_width.
- Integrated support for chrf and chrf++ metrics from the SacreBLEU project, comparing machine translation with reference human translation.
- The ctranslate2_spec model attribute has been removed as it is no longer used in CTranslate2.
Source: opennet.ru
