{"id":81199,"date":"2020-05-11T19:42:01","date_gmt":"2020-05-11T17:42:01","guid":{"rendered":"https:\/\/prohoster.info\/blog\/novosti-interneta\/aol-opublikoval-sistemu-indeksaczii-setevogo-trafika-moloch-2-3"},"modified":"2020-05-11T19:42:01","modified_gmt":"2020-05-11T17:42:01","slug":"aol-opublikoval-sistemu-indeksaczii-setevogo-trafika-moloch-2-3","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/news\/aol-opublikoval-sistemu-indeksaczii-setevogo-trafika-moloch-2-3","title":{"rendered":"AOL has released the Moloch 2.3 network traffic indexing system","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p>AOL Inc. <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/aol\/moloch\/releases\">released<\/a><\/noindex> a system for capturing, storing, and indexing network packets <noindex><a rel=\"nofollow\" href=\"https:\/\/molo.ch\/\">Moloch 2.3<\/a><\/noindex>, providing tools for visually assessing traffic streams and finding information related to network activity. The code is written in C (with an interface in Node.js\/JavaScript) and <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/aol\/moloch\">is distributed<\/a><\/noindex> is licensed under Apache 2.0. It supports operation in Linux and FreeBSD. Pre-packaged <noindex><a rel=\"nofollow\" href=\"https:\/\/molo.ch\/downloads\">packages<\/a><\/noindex> are prepared for various versions of CentOS and Ubuntu.<\/p>\n<p>The project was initiated in 2012 with the aim of creating an open-source alternative to commercial network packet processing platforms, capable of scaling to handle traffic levels similar to those of AOL. Implementing the new system at AOL allowed for complete control over the infrastructure by deploying it on their own servers and significantly reducing costs \u2014 using Moloch for full traffic capture across all AOL networks cost about the same as using <noindex><a rel=\"nofollow\" href=\"https:\/\/molo.ch\/otherfpc\">on a commercial solution<\/a><\/noindex> that was exclusively capturing traffic in a single network. The system can scale to handle traffic at speeds of tens of gigabits per second. The volume of stored data is limited only by the size of the available storage array.<br \/>\nSession metadata is indexed in a cluster based on the engine <noindex><a rel=\"nofollow\" href=\"https:\/\/www.opennet.ru\/opennews\/art.shtml?num=50750\">Elasticsearch<\/a><\/noindex>.<\/p>\n<p>Moloch includes tools for capturing and indexing traffic in standard PCAP format, as well as for quick access to indexed data. For analyzing the accumulated information, a web interface is provided, allowing for navigation, search, and export of samples. It also offers <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/aol\/moloch\/wiki\/API\">API<\/a><\/noindex>, which enables the transfer of data about captured packets in PCAP format and parsed sessions in JSON format to third-party applications. The use of the PCAP format significantly simplifies integration with existing traffic analyzers such as Wireshark.<\/p>\n<p>Moloch consists of three basic components:<\/p>\n<ul>\n<li class=\"l\"> Traffic capture system \u2014 a multithreaded application in C for monitoring traffic, recording dumps in PCAP format to disk, parsing captured packets, and sending metadata about sessions (SPI, Stateful Packet Inspection) and protocols to an Elasticsearch cluster. PCAP files can be stored in encrypted form.\n<li class=\"l\"> Web interface based on the Node.js platform, which runs on each traffic capture server and processes requests related to access to indexed data and transmission of PCAP files through <noindex><a rel=\"nofollow\" href=\"https:\/\/molo.ch\/api\">API<\/a><\/noindex>.\n<li class=\"l\"> Metadata storage based on Elasticsearch.\n<\/ul>\n<p>The web interface offers several viewing modes \u2014 from overall statistics, connection maps, and visual graphs showing changes in network activity to tools for analyzing individual sessions, examining activity by used protocols, and parsing data from PCAP dumps.<\/p>\n<p><center><noindex><a rel=\"nofollow\" href=\"https:\/\/molo.ch\/sessions.png\"><img decoding=\"async\" alt=\"AOL has released the Moloch 2.3 network traffic indexing system\" src=\"\/wp-content\/uploads\/2020\/05\/86274cd49ddc1e853225ec046fff62e2.png\" style=\"display:block;margin: 0 auto;\" \/><\/a><\/noindex><\/center><br \/>\n<center><noindex><a rel=\"nofollow\" href=\"https:\/\/camo.githubusercontent.com\/2bf7015df1a3cbd960df889c1ecdbadf57b5405c\/68747470733a2f2f7261772e6769746875622e636f6d2f77696b692f616f6c2f6d6f6c6f63682f737069766965772e706e67\"><img decoding=\"async\" alt=\"AOL has released the Moloch 2.3 network traffic indexing system\" src=\"\/wp-content\/uploads\/2020\/05\/e6f8f4483549b4501156c80562c4a007.png\" style=\"display:block;margin: 0 auto;\" \/><\/a><\/noindex><\/center><\/p>\n<p><center><noindex><a rel=\"nofollow\" href=\"https:\/\/molo.ch\/spigraph.png\"><img decoding=\"async\" alt=\"AOL has released the Moloch 2.3 network traffic indexing system\" src=\"\/wp-content\/uploads\/2020\/05\/db1e28b5d02583e6d3cbeb9a0925be26.png\" style=\"display:block;margin: 0 auto;\" \/><\/a><\/noindex><\/center><br \/>\n<center><noindex><a rel=\"nofollow\" href=\"https:\/\/molo.ch\/connections.png\"><img decoding=\"async\" alt=\"AOL has released the Moloch 2.3 network traffic indexing system\" src=\"\/wp-content\/uploads\/2020\/05\/e4ab3af3f5f694d9436e8f920fa4cb58.png\" style=\"display:block;margin: 0 auto;\" \/><\/a><\/noindex><\/center><\/p>\n<p>In <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/aol\/moloch\/blob\/master\/CHANGELOG\">new release<\/a><\/noindex>:<\/p>\n<ul>\n<li class=\"l\"> Transitioned to using a type-free format for indexing in Elasticsearch.\n<li class=\"l\"> Examples of traffic capture filters in Lua have been added.\n<li class=\"l\"> Support for the 46-draft version of the QUIC protocol has been implemented.\n<li class=\"l\"> The code for parsing protocols has been revamped, allowing for the creation of parsers for Ethernet and IP layer protocols.\n<li class=\"l\"> New parsers for arp, bgp, igmp, isis, lldp, ospf, and pim protocols have been proposed, as well as parsers for unknown protocols unkEthernet and unkIpProtocol.\n<li class=\"l\"> An option for selectively disabling parsers (disableParsers) has been added.\n<li class=\"l\"> The web interface now includes the ability to display any integer field on graphs, set on the settings page.\n<li class=\"l\"> Graphs and headers can now be pinned and will not shift when scrolling the page.\n<li class=\"l\">  Most navigation panels are hidden or collapsed by default.\n<\/ul>\n<p><noindex><a rel=\"nofollow\" name=\"link\"><\/a><\/noindex><\/p>\n<p>Source: <a \ncontent=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/www.opennet.ru\/opennews\/art.shtml?num=52927\">opennet.ru<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u041a\u043e\u043c\u043f\u0430\u043d\u0438\u044f AOL \u0432\u044b\u043f\u0443\u0441\u0442\u0438\u043b\u0430 \u0440\u0435\u043b\u0438\u0437 \u0441\u0438\u0441\u0442\u0435\u043c\u044b \u0434\u043b\u044f \u0437\u0430\u0445\u0432\u0430\u0442\u0430, \u0445\u0440\u0430\u043d\u0435\u043d\u0438\u044f \u0438 \u0438\u043d\u0434\u0435\u043a\u0441\u0430\u0446\u0438\u0438 \u0441\u0435\u0442\u0435\u0432\u044b\u0445 \u043f\u0430\u043a\u0435\u0442\u043e\u0432 Moloch 2.3, \u043f\u0440\u0435\u0434\u043e\u0441\u0442\u0430\u0432\u043b\u044f\u044e\u0449\u0430\u044f \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u044b \u0434\u043b\u044f \u043d\u0430\u0433\u043b\u044f\u0434\u043d\u043e\u0439 \u043e\u0446\u0435\u043d\u043a\u0438 \u043f\u043e\u0442\u043e\u043a\u043e\u0432 \u0442\u0440\u0430\u0444\u0438\u043a\u0430 \u0438 \u043f\u043e\u0438\u0441\u043a\u0430 \u0441\u0432\u044f\u0437\u0430\u043d\u043d\u043e\u0439 \u0441 \u0441\u0435\u0442\u0435\u0432\u043e\u0439 \u0430\u043a\u0442\u0438\u0432\u043d\u043e\u0441\u0442\u044c\u044e \u0438\u043d\u0444\u043e\u0440\u043c\u0430\u0446\u0438\u0438. \u041a\u043e\u0434 \u043d\u0430\u043f\u0438\u0441\u0430\u043d \u043d\u0430 \u044f\u0437\u044b\u043a\u0435 \u0421\u0438 (\u0438\u043d\u0442\u0435\u0440\u0444\u0435\u0439\u0441 \u043d\u0430 Node.js\/JavaScript) \u0438 \u0440\u0430\u0441\u043f\u0440\u043e\u0441\u0442\u0440\u0430\u043d\u044f\u0435\u0442\u0441\u044f \u043f\u043e\u0434 \u043b\u0438\u0446\u0435\u043d\u0437\u0438\u0435\u0439 Apache 2.0. \u041f\u043e\u0434\u0434\u0435\u0440\u0436\u0438\u0432\u0430\u0435\u0442\u0441\u044f \u0440\u0430\u0431\u043e\u0442\u0430 \u0432 Linux \u0438 FreeBSD. \u0413\u043e\u0442\u043e\u0432\u044b\u0435 \u043f\u0430\u043a\u0435\u0442\u044b \u043f\u043e\u0434\u0433\u043e\u0442\u043e\u0432\u043b\u0435\u043d\u044b \u0434\u043b\u044f \u0440\u0430\u0437\u043d\u044b\u0445 \u0432\u0435\u0440\u0441\u0438\u0439 CentOS [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":81200,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[702],"tags":[],"class_list":["post-81199","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-news"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.2 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u041a\u043e\u043c\u043f\u0430\u043d\u0438\u044f AOL \u0432\u044b\u043f\u0443\u0441\u0442\u0438\u043b\u0430 \u0440\u0435\u043b\u0438\u0437 \u0441\u0438\u0441\u0442\u0435\u043c\u044b \u0434\u043b\u044f \u0437\u0430\u0445\u0432\u0430\u0442\u0430, \u0445\u0440\u0430\u043d\u0435\u043d\u0438\u044f \u0438 \u0438\u043d\u0434\u0435\u043a\u0441\u0430\u0446\u0438\u0438 \u0441\u0435\u0442\u0435\u0432\u044b\u0445 \u043f\u0430\u043a\u0435\u0442\u043e\u0432\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/en\/blog\/news\/aol-opublikoval-sistemu-indeksaczii-setevogo-trafika-moloch-2-3\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.2\" \/>\n\t\t<meta property=\"og:locale\" content=\"en_US\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47AOL \u043e\u043f\u0443\u0431\u043b\u0438\u043a\u043e\u0432\u0430\u043b \u0441\u0438\u0441\u0442\u0435\u043c\u0443 \u0438\u043d\u0434\u0435\u043a\u0441\u0430\u0446\u0438\u0438 \u0441\u0435\u0442\u0435\u0432\u043e\u0433\u043e \u0442\u0440\u0430\u0444\u0438\u043a\u0430 Moloch 2.3 | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u041a\u043e\u043c\u043f\u0430\u043d\u0438\u044f AOL \u0432\u044b\u043f\u0443\u0441\u0442\u0438\u043b\u0430 \u0440\u0435\u043b\u0438\u0437 \u0441\u0438\u0441\u0442\u0435\u043c\u044b \u0434\u043b\u044f \u0437\u0430\u0445\u0432\u0430\u0442\u0430, \u0445\u0440\u0430\u043d\u0435\u043d\u0438\u044f \u0438 \u0438\u043d\u0434\u0435\u043a\u0441\u0430\u0446\u0438\u0438 \u0441\u0435\u0442\u0435\u0432\u044b\u0445 \u043f\u0430\u043a\u0435\u0442\u043e\u0432\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/en\/blog\/news\/aol-opublikoval-sistemu-indeksaczii-setevogo-trafika-moloch-2-3\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-05-11T17:42:01+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-05-11T17:42:01+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47AOL released the network traffic indexing system Moloch 2.3 | ProHoster","description":"AOL has released a system for capturing, storing, and indexing network packets.","canonical_url":"https:\/\/prohoster.info\/en\/blog\/news\/aol-opublikoval-sistemu-indeksaczii-setevogo-trafika-moloch-2-3","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"en_US","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47AOL \u043e\u043f\u0443\u0431\u043b\u0438\u043a\u043e\u0432\u0430\u043b \u0441\u0438\u0441\u0442\u0435\u043c\u0443 \u0438\u043d\u0434\u0435\u043a\u0441\u0430\u0446\u0438\u0438 \u0441\u0435\u0442\u0435\u0432\u043e\u0433\u043e \u0442\u0440\u0430\u0444\u0438\u043a\u0430 Moloch 2.3 | ProHoster","og:description":"\u041a\u043e\u043c\u043f\u0430\u043d\u0438\u044f AOL \u0432\u044b\u043f\u0443\u0441\u0442\u0438\u043b\u0430 \u0440\u0435\u043b\u0438\u0437 \u0441\u0438\u0441\u0442\u0435\u043c\u044b \u0434\u043b\u044f \u0437\u0430\u0445\u0432\u0430\u0442\u0430, \u0445\u0440\u0430\u043d\u0435\u043d\u0438\u044f \u0438 \u0438\u043d\u0434\u0435\u043a\u0441\u0430\u0446\u0438\u0438 \u0441\u0435\u0442\u0435\u0432\u044b\u0445 \u043f\u0430\u043a\u0435\u0442\u043e\u0432","og:url":"https:\/\/prohoster.info\/en\/blog\/news\/aol-opublikoval-sistemu-indeksaczii-setevogo-trafika-moloch-2-3","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-05-11T17:42:01+00:00","article:modified_time":"2020-05-11T17:42:01+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"81199","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 16:03:22","updated":"2022-09-28 02:13:23","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/81199","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/comments?post=81199"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/81199\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media\/81200"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media?parent=81199"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/categories?post=81199"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/tags?post=81199"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}