release , a free lexical analyzer generator for C and C++ languages. Recall that re2c was written in 1993 by Peter Babulis as an experimental generator of very fast lexical analyzers, distinguished from other generators by the speed of the generated code and an unusually flexible user interface that allows easy and effective integration of analyzers into existing codebases. Since then, the project has been developed by the community and continues to serve as a platform for experimentation and research in formal grammars and finite automata.
The preparation of the release took almost a whole year. Most of the time, as always, was spent on developing the theoretical foundation and writing
the article “«.
The algorithms described in the article are implemented in the experimental library libre2c
(building the library and performance tests is disabled by default and can be enabled with the configure option “—enable-libs”). The library is designed not as a competitor to existing projects like RE2, but as a research platform for developing new
algorithms (which can then be used in re2c or other projects). It is also convenient for testing, measuring performance, and creating bindings for other languages.
The main innovations in re2c version 1.2:
- A new simplified way to check for the end of input data (“EOF rule”) has been added. For this, the configuration “re2c:eof” was introduced,
allowing the selection of a terminal symbol,
and a special rule “$” that triggers if the lexer
successfully reaches the end of the input data.
Historically, re2c offers several methods for checking for
end of input, varying in limitations, efficiency, and simplicity
of use. The new method is designed to simplify coding while
remaining efficient and widely applicable. The old methods
still work and may be preferable in certain cases. - The ability to include external files using the directive has been added,
“/*!include:re2c “file.re” */”, where “file.re” is the name of the included file. Re2c looks for files in the directory of the including file,
as well as in the list of paths specified with the “-I” option.
Included files may include other files.
Re2c provides “standard” files in the “include/” directory
of regular expressions will accumulate there, something akin to a standard library.
So far, at the request of users, one file with definitions of Unicode categories has been added.
The ability to generate header files with arbitrary - content using the -t —type-header options (or corresponding
with content using the options “-t —type-header” (or corresponding
configurations) and the new directives “/*!header:re2c:on*/” and
“/*!header:re2c:off*/”. This can be useful in cases where
used in other translation units.
Re2c now understands UTF8 literals and character classes in regular expressions. - By default, re2c parses expressions like "∀x ∃y" as.
By default, re2c parses expressions like “∀x ∃y” as
a sequence of 1-bit ASCII characters “e2 88 80 78 20 e2 88 83 79”
(hex codes), and users have to manually escape Unicode characters:
“\\u2200x \\u2203y”. This is very inconvenient and unexpected for many.
users (as evidenced by the constant bug reports). Therefore, now
Re2c provides the option “—input-encoding {ascii | utf8}”,
which allows changing the behavior and parsing “∀x ∃y” as
“2200 78 20 2203 79”. - Re2c now allows the use of regular re2c blocks in “-r —reuse” mode.
This is convenient if the input file contains many blocks, and only some of them
need to be reused. - A format for warnings and error messages can now be set
with the new option “—location-format {gnu | msvc}”. The GNU format is displayed
like "filename:line:column:", while the MSVC format is in the form of "filename(line,column)".
This feature may be useful for IDE enthusiasts.
An option "—verbose" has also been added, which outputs a brief success message if the operation is successful. - The compatibility mode has been enhanced with flex — some parsing issues have been fixed.
incorrect operator precedence in rare cases have been fixed.
Historically, the option "-F --flex-support" allows you to write code.
in a mix of flex style and re2c style, which complicates parsing somewhat.
The compatibility mode with flex is rarely used in new code,
but re2c continues to support it for backward compatibility. - The subtraction operator for character classes "\/" is now applied.
before expanding the encoding, which allows it to be applied in more cases,
if a variable-length character encoding (such as UTF8) is used. - The output file is now created atomically: re2c first creates a temporary file
and writes the result to it, then renames the temporary file to the output
in one operation. - The documentation has been completed and rewritten; in particular, new
chapters on
and .
The new documentation is compiled as
with examples (the same sources are rendered in manpage and online documentation).
Weak attempts have been made to improve the site's readability on phones. - From a developer's perspective, re2c has gained a more complete debugging subsystem.
Debugging code is now disabled in release builds and
can be enabled using the configure option "—enable-debug".
Source: opennet.ru
