A release of the Arkime 3.1 network packet capture, storage and indexing system has been prepared, which provides tools for visually assessing traffic flows and searching for information related to network activity. The project was originally developed by AOL with the goal of creating an open and hosted replacement for commercial network packet processing platforms that can scale to process traffic at speeds of tens of gigabits per second. The code of the traffic capturing component is written in C, and the interface is implemented in Node.js/JavaScript. The source code is distributed under the Apache 2.0 license. Work in Linux and FreeBSD is supported. Ready packages are prepared for Arch, CentOS and Ubuntu.
Arkime includes tools for capturing and indexing traffic in native PCAP format, and also provides tools for quick access to indexed data. Using the PCAP format greatly simplifies integration with existing traffic analyzers such as Wireshark. The amount of stored data is limited only by the size of the available disk array. Session metadata is indexed in a cluster based on the Elasticsearch engine.
To analyze the accumulated information, a web-interface is proposed that allows you to navigate, search and export samples. The web interface provides several viewing modes - from general statistics, connection maps and visual graphs with data on changes in network activity to tools for studying individual sessions, analyzing activity in the context of the protocols used, and parsing data from PCAP dumps. An API is also provided that allows you to pass captured packets in PCAP format and parsed sessions in JSON format to third-party applications.
Arkime consists of three basic components:
- The traffic capture system is a multi-threaded C application for monitoring traffic, writing PCAP dumps to disk, parsing captured packets, and sending stateful packet inspection (SPI) and protocol metadata to an Elasticsearch cluster. It is possible to store PCAP files in encrypted form.
- A web interface based on the Node.js platform that runs on each traffic capture server and processes requests related to accessing indexed data and transferring PCAP files through the API.
- Metadata storage based on Elasticsearch.
In the new release:
- Added support for IETF QUIC, GENEVE, VXLAN-GPE protocols.
- Added support for the Q-in-Q (Double VLAN) type, which allows encapsulating VLAN tags in second-level tags to expand the number of VLANs up to 16 million.
- Added support for the "float" field type.
- The Amazon Elastic Compute Cloud writer has been migrated to use the IMDSv2 (Instance Metadata Service) protocol.
- Code refactored to add UDP tunnels.
- Added support for elasticsearchAPIKey and elasticsearchBasicAuth.
Source: opennet.ru