AOL bipụtara Moloch 2.3 sistemu indexing okporo ụzọ netwọkụ

Ụlọ ọrụ AOL wepụtara ntọhapụ nke usoro maka ịdekọ, ịchekwa na ntinye aha ngwugwu netwọkụ Moloch 2.3, nke na-enye ngwaọrụ maka nlele anya na-enyocha okporo ụzọ okporo ụzọ na ịchọ ozi metụtara ọrụ netwọk. Edere koodu a n'asụsụ C (nhụta na Node.js/JavaScript) na kesara site nyere ikike n'okpuru Apache 2.0. Na-akwado ọrụ na Linux na FreeBSD. Njikere ngwugwu akwadoro maka ụdị dị iche iche nke CentOS na Ubuntu.

Emepụtara oru ngo a na 2012 na ebumnuche nke ịmepụta ngbanwe mepere emepe maka ikpo okwu nhazi ngwugwu netwọk azụmahịa nke nwere ike ịba ụba na mpịakọta okporo ụzọ AOL. Mmejuputa usoro ọhụrụ na AOL mere ka o kwe omume iji nweta njikwa zuru oke na akụrụngwa n'ihi ntinye na sava ya ma belata ọnụ ahịa nke ukwuu - iji Moloch weghara okporo ụzọ kpamkpam na netwọkụ AOL niile na-efu otu ego mgbe ị na-eji ya. ngwọta azụmahịa Na mbụ, a na-eji ya na-ejide okporo ụzọ na naanị otu netwọk. Sistemu ahụ nwere ike ịba ụba iji hazie okporo ụzọ na ọsọ nke iri gigabits kwa nkeji. A na-amachi oke data echekwara naanị site na nha n'usoro diski dị.
Edepụtara metadata oge n'ụyọkọ dabere na injin Elasticsearch.

Moloch na-agụnye ngwaọrụ maka ịdekọ na ịdepụta okporo ụzọ n'ụdị PCAP, yana maka ịnweta data indexed ngwa ngwa. Iji nyochaa ozi chịkọbara, a na-enye ihe ntanetị weebụ nke na-enye gị ohere ịnyagharịa, chọọ na mbupụ samples. Enyerekwa ya API, nke na-enye gị ohere ịnyefe data gbasara ngwugwu ejidere n'ụdị PCAP na oge tụgharịrị n'ụdị JSON gaa na ngwa ndị ọzọ. Ojiji nke usoro PCAP na-eme ka njikọta na ndị nyocha okporo ụzọ dị ugbu a dị ka Wireshark dị mfe.

Moloch nwere ihe atọ bụ isi:

• Usoro njide okporo ụzọ bụ ngwa C nke nwere ọtụtụ eriri maka nyochaa okporo ụzọ, na-ede ihe mkpofu n'ụdị PCAP na diski, nyochaa ngwugwu ejidere na izipu metadata gbasara nnọkọ (SPI, Nyocha ngwugwu Stateful) na protocols na ụyọkọ Elasticsearch. Ọ ga-ekwe omume ịchekwa faịlụ PCAP n'ụdị ezoro ezo.
• Ihe ntanetị weebụ dabere na ikpo okwu Node.js, nke na-agba ọsọ na ihe nkesa na-ejide okporo ụzọ na usoro arịrịọ metụtara ịnweta data indexed na ịnyefe faịlụ PCAP site na. API.
• Nchekwa metadata dabere na Elasticsearch.

Ihe ntanetị weebụ na-enye ọtụtụ ụdị nlele - site na ọnụ ọgụgụ izugbe, maapụ njikọ na eserese ngosi nwere data gbasara mgbanwe na ọrụ netwọk gaa na ngwaọrụ maka ọmụmụ oge nke onye ọ bụla, nyochaa ọrụ n'ọnọdụ nke usoro iwu eji na ịkọwa data sitere na PCAP.

В ọhụrụ ntọhapụ:

• Emeela mgbanwe iji usoro enweghị ụdị maka ntinye aha na Elasticsearch.
• Agbakwunyere ihe atụ nke ihe nzacha njide okporo ụzọ na Lua.
• E mejuputala nkwado maka ụdị 46-draft nke QUIC.
• E degharịala koodu maka ntule usoro, na-eme ka o kwe omume ịde parsers maka ụkpụrụ Ethernet na IP larịị.
• Atụpụtala nzagharị ọhụrụ maka arp, bgp, igmp, isis, lldp, ospf na protocol pim, yana ntụgharị maka usoro unkEthernet amaghi ama na unkIpProtocol.
• Agbakwunyere nhọrọ iji gbanyụọ ntụgharị (disableParsers).
• Agbakwunyela ikike igosipụta mpaghara integer ọ bụla na eserese, nke edobere na ibe ntọala, na interface weebụ.
• Eserese na aha nwere ike gbanwee ugbu a ma ghara ịkwaga mgbe ị na-atụgharị ibe ahụ.
• Ọtụtụ ogwe igodo nsoroụzọ zoro ezo ma ọ bụ daa na ndabara.

isi: opennet.ru