Statistics uye yekutarisa PHP zvinyorwa munguva chaiyo. ClickHouse naGrafana vanouya kuzobatsira Pinba

Muchikamu chino ini ndichakuudza nzira yekushandisa pinba ine clickhouse uye grafana pane pinba_engine uye pinboard.

Pane chirongwa chePHP, pinba ndiyo chete nzira yakavimbika yekunzwisisa zviri kuitika nekuita. Chokwadi, pinba inowanzoitwa chete kana matambudziko atoonekwa uye zvisina kujeka “pekuchera.”

Kazhinji hapana anoziva kuti kangani pasekondi/miniti ichi kana icho chinyorwa chinodanwa uye vanotanga kugadzirisa "nekubata", kutanga kubva kunzvimbo idzo dzinoita senge zvine musoro.

Vamwe vanoongorora matanda enginx, nepo vamwe vachiongorora mibvunzo inononoka yedatabase.

Ehe, pinba haingave yakawandisa, asi pane zvikonzero zvakati nei isiri yega chirongwa inayo.

Uye chikonzero chekutanga ndechekuisa.

Kuti uwedzere kana kushoma kuwana imwe mhando ye "kuneta" kubva pakuitwa kwePinba, zvinonyanya kudiwa kuona metrics kwete ekupedzisira maminetsi, asiwo kwenguva yakareba (kubva pamazuva kusvika kumwedzi).

Kune izvi zvaunoda:

• isa kuwedzera kwe php (uye iwe ungangoda module ye nginx)
• unganidza kuwedzera kwe mysql
• isa pinboard uye gadzirisa cron

Nekuda kwehuwandu hudiki hweruzivo nezvePinba, vanhu vazhinji vane fungidziro yekuti yakangoshanda paPHP5 uye yagara iri chinhu chekare, asi sezvatichaona gare gare, izvi handizvo.

Danho rekutanga ndiro rakapfava, zvese zvaunoda kuita kumhanyisa murairo:

apt install php-pinba

Kuwedzerwa uku kunowanikwa mumatura kusvika uye kusanganisira php 7.3 uye haufanirwe kuunganidza chero chinhu.

Mushure mekuita iyo yekuisa murairo, isu tinobva tagamuchira yekuwedzera yekushanda iyo inounganidza uye inotumira metrics kune yega yega script (inomhanya nguva, ndangariro, nezvimwewo) mufomati. protobuf kuburikidza udp kusvika 127.0.0.1:30002.

Parizvino hapana akambobata kana kugadzirisa mapaketi eUDP aya, asi izvi hazvikanganise kukurumidza kana kugadzikana kwemagwaro ako ePP.

Kusvika nguva pfupi yadarika, iyo chete application yaigona kubata uye kugadzirisa aya eUDP mapaketi aive pinba_engine. Tsanangudzo "nyore uye muchidimbu"kuiswa kunoodza moyo chishuwo chekuzoverenga uye kunyura mazviri zvakare. Iwo makiromita-akareba mazita ekutsamira ane ese mazita emapakeji uye mazita ezvirongwa uye zvinongedzo kumapeji ega ega nekuisa kwawo, uye iwo ane hukama hwavo kune mamwe anotsamira. Hapana ane nguva kana chishuwo chekubata nechinhu ichi.

Kuisa nzira pinba2 haana kunyanya nyore.

Zvichida rimwe zuva zvichave zvichiita kuisa pinba10 nemirairo imwe kana miviri uye haufaniri kuverenga boka rezvinhu kuti unzwisise kuti ungazviita sei, asi ikozvino izvi hazvisizvo.

Kana iwe ukaisa pinba_engine, saka iyi inongova hafu yehondo. Mushure mezvose, pasina pinboard iwe uchafanirwa kuzviganhurira kune data kubva kumaminetsi mashoma apfuura, kana iwe uchafanirwa kuunganidza, kuchengeta, uye kuona iyo data pachako. Zvakanaka kuti pinboard iri nyore kushandisa kuisirwa.

Zvingaite senge, nei kutambura kwakadaro kana ese metrics kubva php atotumirwa kune udp port mune protobuf fomati uye zvese zvaunoda ndezvekunyora application inovabata uye kuiisa mune imwe mhando yekuchengetedza? Sezviri pachena, avo vanogadzira vakauya nepfungwa iyi vakabva vagara pasi kuti vanyore mazano avo, mamwe acho akazoguma paGitHub.

Izvi zvinotevera mhedziso yezvirongwa zvina zvakavhurika sosi inochengetedza metrics mukuchengetedza, kubva iyo data iyi inogona kudzoserwa nyore nekuonekwa, semuenzaniso, kushandisa grafana.

olegfedoseev/pinba-server (Mbudzi 2017)

udp server pakuenda inochengetedza metrics kuOpenTSDB. Zvichida kana iwe uri kutoshandisa OpenTSDB mupurojekiti yako, saka iyi mhinduro ichakukodzera iwe, zvikasadaro ini ndinokurudzira kupfuura nayo.

olegfedoseev/pinba-influxdb (Chikumi 2018)

udp server pakuenda, kubva zvakafanana habrowser, iyo panguva ino inochengeta metrics muInfluxDB. Mapurojekiti mazhinji ari kutoshandisa InfluxDB yekutarisa, saka iyi mhinduro inogona kuvaitira.

Pros:

• InfluxDB Kunoitawo batanidza zviyero zvakagamuchirwa, uye dzima yekutanga mushure menguva yakatarwa.

Cons:

• Iyi mhinduro haichengetedze ruzivo pane zvenguva.
• InfluxDB inochengetedza saiti kero sema tag uye kana uine akawanda akasiyana mapeji kero, izvi zvinotungamira kune kuwedzera kushandiswa random kuwana ndangariro. Kubva pane imwe nguva iye "achatanga kudya ndangariro seanopenga". (tsime)

ClickHouse-Ninja/Proton (Ndira 2019)

udp server pakuenda, iyo inochengetedza metrics muClickHouse. Iyi ndiyo mhinduro yeshamwari yangu. Yakanga iri mushure mekujairana nayo ndipo pandakafunga kuti yaive nguva yekutora Pinbu neClickhouse.

Pros:

• Clickhouse yakanakira mabasa akadaro; inokutendera iwe kudzvanya data zvekuti unogona kuchengeta ese akaomeswa data kunyangwe pasina kuunganidzwa.
• kana zvichidikanwa, unogona kuunganidza nyore metrics inoguma
• yakagadzirira-yakagadzirwa template ye grafana
• inochengetedza ruzivo pane zvinobatika nguva

Cons:

• chirema chinouraya
• Iko hakuna config iyo iwe yaunogona kugadzirisa zita re database uye matafura, kero uye chiteshi che server.
• kana uchichengeta data rakasvibira, tafura yeduramazwi rekubatsira rinoshandiswa kuchengeta peji nemakero edomendi, izvo zvinokanganisa mibvunzo inotevera.
• zvimwe zvinhu zvidiki zvinotevera kubva paminus yekutanga

pinba-server/pinba-server (Kubvumbi 2019)

udp server mu php, iyo inochengetedza metrics muClickHouse. Iyi ndiyo mhinduro yangu, inova mhedzisiro yekuziva pinba, ClickHouse uye protobuf. Ndichiri kuronga boka rese iri, ndakanyora “umbowo hwepfungwa”, izvo, zvisingatarisirwi kwandiri, hazvina kushandisa zviwanikwa zvakakosha (30 MB ye RAM uye isingasviki 1% yeimwe yemasere processor cores), saka ini. akasarudza kuigovera kuvanhu.

Izvo zvakanakira zvakafanana neyakapfuura mhinduro, ini zvakare ndakashandisa akajairika mazita kubva kune yekutanga pinba_engine. Ini ndakawedzerawo config iyo inokutendera kuti utange akati wandei pinbase server zviitiko kamwechete kuitira kuchengetedza metrics mumatafura akasiyana - izvi zvinobatsira kana iwe uchida kuunganidza data kwete kubva php chete, asiwo kubva nginx.
Zvakaipa - "chinouraya chikanganiso" uye izvo zvidiki izvo zvisingaenderane newe pachako, asi mhinduro yangu "yakapusa senge inotsvedza" uye ine chete ingangoita zana mitsara yekodhi, saka chero mugadziri wePHP anogona kushandura zvaasingadi. mumaminetsi mashoma.

Ainoita

UDP port 30002 inoteererwa. Mapaketi ese anouya anotemwa maererano neprotobuf scheme uye akaunganidzwa. Kamwe paminiti, pakiti inoiswa mukati meiyo clickhouse mupinba.requests tafura. (ese ma parameter akagadzirirwa mukati config)

Zvishoma nezve clickhouse

Clickhouse inotsigira akasiyana ekuchengetedza data injini. Iyo inonyanya kushandiswa ndeye MergeTree.

Kana pane imwe nguva iwe ukafunga kuchengetedza data yakaunganidzwa yenguva dzese, uye data raw chete kune yekupedzisira, ipapo iwe unogona kugadzira maonero enyama nemapoka, uye nguva nenguva kuchenesa main pinba.requests tafura, nepo data rese richiramba riri mu maonero enyama. Uyezve, paunenge uchigadzira tafura yepinba.requests, unogona kutsanangura "injini = Null", ipapo data yakasvibiswa haizochengetedzwe ku diski zvachose, uye panguva imwechete icharamba ichiguma mukuona kwenyama uye kuponeswa kwakasanganiswa. . Ini ndinoshandisa chirongwa ichi kune nginx metrics, nekuti panginx ndine 50 times more zvikumbiro pane paphp.

Saka, iwe wabva kure uye ini handidi kukusiya iwe pakati, saka zvinotevera irondedzero yakadzama yekumisikidzwa nekugadzirisa mhinduro yangu uye zvese zvaunoda, pamwe nemakomba akakonzera zvinopfuura imwe ngarava. kuparara. Iyo yese yekuisa maitiro inotsanangurwa kuUbuntu 18.04 LTS uye Centos 7; maitiro anogona kusiyana zvishoma pane kumwe kugovera uye shanduro.

Kuiswa

Ndinoisa mirairo yose inodiwa mukati dockerfile kufambisa kuberekana kwemirairo. Misungo chete ndiyo ichatsanangurwa pasi apa.

php-pinba

Mushure mekuisa, ita shuwa kuti mu /etc/php/7.2/fpm/conf.d/20-pinba.ini faira iwe hauna kutsanangura zvese zvingasarudzwa. Pane kumwe kugoverwa (eg centos) vanogona kutaurwa.

extension=pinba.so
pinba.enabled=1
pinba.server=127.0.0.1:30002

clickhouse

Panguva yekuisa, clickhouse ichakukumbira kuti uise password yemushandisi weiyo default. Nekutadza, mushandisi uyu anowanikwa kubva kune ese IPs, saka kana iwe usina firewall pane yako server, iva nechokwadi chekuisa password yayo. Izvi zvinogona zvakare kuitwa mushure mekuisa mu /etc/clickhouse-server/users.xml faira.

Zvakakoshawo kuziva kuti clickhouse inoshandisa zviteshi zvakawanda, kusanganisira 9000. Ichi chiteshi chinoshandiswawo php-fpm mune mamwe kugovera (semuenzaniso, centos). Kana iwe uchitoshandisa chiteshi ichi, unogona kuchichinjisa kune chimwe mu /etc/clickhouse-server/config.xml faira.

grafana ine clickhouse plugin

Mushure mekuisa Grafana, shandisa login admin uye password admin. Paunopinda kekutanga, Grafana inokukumbira kuti uise password nyowani.

Tevere, enda ku "+" -> pinza menyu uye ratidza nhamba yedashibhodhi yekutumira 10011. Ndakagadzirira uye ndakaisa dashboard iyi kuti usazozviita wega zvakare.

Grafana inotsigira kushanda neclickhouse kuburikidza neyechitatu-bato plugin, asi Grafana haina yambiro kune yechitatu-bato plugins (pave paine tikiti reizvi kwemakore akati wandei).

pinba-server

Kuisa protobuf uye libevent isarudzo, asi inovandudza pinba-server performance. Kana iwe ukaisa pinba-server mune imwe folda kunze kwe / opt, ipapo iwe zvakare uchafanirwa kugadzirisa systemd script file.

pinba module ye nginx

Kuti ugadzire module, iwe unoda masource codes eiyo imwe vhezheni yenginx yakatoiswa pane yako server, pamwe neamwechete sarudzo dzekubatanidza, zvikasadaro kuvaka kuchabudirira, asi kana uchibatanidza module, chikanganiso chinokandwa icho. "Module haienderane nebhinari." Sarudzo dzekubatanidza dzinogona kutariswa uchishandisa iyo nginx -V command

Hupenyu Hacks

Ese masaiti angu anoshanda chete pa https. Iyo schema munda inova isina zvazvinoreva, saka ini ndinoishandisa kuparadzanisa webhu / console.

Mune zvinyorwa zvinowanikwa kubva pawebhu ini ndinoshandisa:

if (ini_get('pinba.enabled')) {
    pinba_schema_set('web');
}

Uye mune console zvinyorwa (semuenzaniso, cron zvinyorwa):

if (ini_get('pinba.enabled')) {
    pinba_schema_set('console');
}

Mudhibhodhi rangu muGrafana pane webhu/console switch yekuona manhamba zvakasiyana.

Iwe unogona zvakare kutumira ma tag ako kuPinba, semuenzaniso:

pinba_tag_set('country', $countryCode);

Ndizvo zvose.

Ndokumbira upindure sarudzo dziri pasi pechinyorwa.

Semazuva ese, ndinokuyambira kuti ini handikurudzire kana kubatsira kuburikidza nemameseji emunhu paHabr uye masocial network.

Gadzira tikiti paGithub.

Ndokumbirawo mutsigire nema likes Shona version chinyorwa ichi pa reddit.

Vashandisi vakanyoresa chete ndivo vanogona kutora chikamu muongororo. Nyorera mu, Munogamuchirwa.

Ndeipi OS yauri kushandisa pane server?

• Ubuntu

• CentOS

• Debian

• Gentoo

• Red Hat

• Fedora

• OpenSUSE

• SIWIZARENDI

• Unix

• Windows

• zvimwe

114 vashandisi vakavhota. 11 vashandisi vakaramba.

Ndeipi vhezheni yephp yauri kushandisa pane server?

• 7.3

• 7.2

• 7.1

• 7.0

• 5

• zvimwe

105 vashandisi vakavhota. 17 vashandisi vakaramba.

Wakamboshandisa pinba here?

• hongu

• kwete, asi ndinoda

• kwete uye handingade

• kwete uye handina kumbonzwa nezvake

100 vashandisi vakavhota. 14 vashandisi vakaramba.

Ndeipi vhezheni yePinba server yaungade kuedza?

• pinba_engine (mysql injini)

• pinba2 (mysql injini)

• pinboard (php + mysql)

• olegfedoseev/pinba-server (enda + OpenTSDB)

• olegfedoseev/pinba-influxdb (enda + influxdb)

• pinba-server/pinba-server (enda + clickhouse)

• pinba-server/pinba-server (php + clickhouse)

• Ndichanyora zvangu ini

• zvimwe

39 vashandisi vakavhota. 47 vashandisi vakaramba.

Source: www.habr.com