ProHoster > Π‘Π»ΠΎΠ³ > Kev tswj hwm > Kev txheeb cais thiab saib xyuas ntawm PHP scripts hauv lub sijhawm. ClickHouse thiab Grafana tuaj pab Pinba

Kev txheeb cais thiab saib xyuas ntawm PHP scripts hauv lub sijhawm. ClickHouse thiab Grafana tuaj pab Pinba

Hauv tsab xov xwm no kuv yuav qhia koj yuav ua li cas siv pinba nrog clickhouse thiab grafana es tsis txhob pinba_engine thiab pinboard.

Ntawm qhov project PHP, pinba yog kab tias tib txoj kev txhim khu kev qha kom nkag siab tias muaj dab tsi tshwm sim nrog kev ua haujlwm. Muaj tseeb, pinba feem ntau yog siv tsuas yog thaum muaj teeb meem twb pom lawm thiab nws tsis meej "qhov twg mus khawb."

Feem ntau tsis muaj leej twg muaj lub tswv yim yuav ua li cas muaj pes tsawg zaus hauv ib ob / feeb qhov no lossis tsab ntawv ntawd raug hu thiab lawv pib ua kom zoo "los ntawm kev kov", pib ntawm cov chaw uas zoo li ntau qhov laj thawj.

Qee qhov txheeb xyuas nginx cav, thaum lwm tus txheeb xyuas cov lus nug qeeb.

Tau kawg, pinba yuav tsis yog superfluous, tab sis muaj ob peb lub laj thawj vim li cas tsis txhua qhov project muaj nws.

Kev txheeb cais thiab saib xyuas ntawm PHP scripts hauv lub sijhawm. ClickHouse thiab Grafana tuaj pab Pinba

Thiab thawj yog vim li cas yog installation.

Yuav kom ntau dua los yog tsawg dua tau txais qee yam "shaus" los ntawm kev siv Pinba, nws yog qhov xav pom kev ntsuas tsis yog rau cov feeb kawg nkaus xwb, tab sis kuj nyob rau lub sijhawm ntev (txij hnub mus rau hli).

Rau qhov koj xav tau:

  • nruab txuas ntxiv rau php (thiab koj yuav xav tau ib qho module rau nginx)
  • compile extension rau mysql
  • nruab pinboard thiab configure cron

Vim muaj cov ntaub ntawv me me ntawm Pinba, ntau tus neeg muaj kev xav tias nws tsuas yog ua haujlwm ntawm PHP5 thiab tau ntev dhau los lawm, tab sis raws li peb yuav pom tom qab, qhov no tsis yog li ntawd.

Thawj kauj ruam yog qhov yooj yim tshaj plaws, txhua yam koj yuav tsum tau ua yog khiav cov lus txib:

apt install php-pinba

Qhov kev txuas ntxiv no muaj nyob rau hauv cov chaw cia khoom mus txog thiab suav nrog php 7.3 thiab koj tsis tas yuav sau dab tsi.

Tom qab ua tiav cov lus txib installation, peb tam sim ntawd tau txais kev txuas ntxiv ua haujlwm uas sau thiab xa cov metrics rau txhua tsab ntawv (lub sijhawm khiav, nco, thiab lwm yam) hauv hom ua protobuf ntawm udp mus rau 127.0.0.1: 30002.

Txog tam sim no tsis muaj leej twg tau ntes lossis ua tiav cov pob ntawv UDP no, tab sis qhov no tsis ua rau muaj kev cuam tshuam rau qhov nrawm lossis ruaj khov ntawm koj cov ntawv PHP.

Txog thaum tsis ntev los no, tib daim ntawv thov uas tuaj yeem ntes thiab ua cov pob ntawv UDP no pinba_engine. Description "yooj yim thiab concise" kev teeb tsa tsis txaus siab rau qhov xav tau nyeem thiab delve rau hauv nws dua. Daim ntawv teev npe ntev-ntev ntawm kev vam khom muaj ob lub npe ntawm cov pob khoom thiab cov npe ntawm cov kev pab cuam thiab txuas rau ib tus nplooj ntawv nrog lawv cov kev teeb tsa, thiab cov uas muaj lawv tus kheej txuas mus rau lwm qhov kev vam khom. Tsis muaj leej twg muaj lub sijhawm los yog lub siab xav daws qhov kev txaj muag no.

Kev Txhim Kho pib 2 tsis tau tshwj xeeb tshaj yog yooj yim dua.

Tej zaum muaj ib hnub nws yuav ua tau rau nruab pinba10 nrog ib lossis ob lo lus txib thiab tsis tas yuav nyeem ib pawg ntawm cov ntaub ntawv kom nkag siab yuav ua li cas, tab sis tam sim no qhov no tsis yog li ntawd.

Yog tias koj nruab pinba_engine, qhov no tsuas yog ib nrab ntawm kev sib ntaus sib tua. Tom qab tag nrho, tsis muaj pinboard koj yuav tsum txwv koj tus kheej rau cov ntaub ntawv los ntawm ob peb feeb dhau los, lossis koj yuav tau sau, khaws cia, thiab pom cov ntaub ntawv koj tus kheej. Nws yog qhov zoo uas pinboard yog qhov yooj yim heev rau siv kev teeb tsa.

Nws yuav zoo li, vim li cas kev txom nyem yog tias tag nrho cov kev ntsuas los ntawm php twb tau xa mus rau qhov chaw nres nkoj udp hauv protobuf hom thiab txhua yam koj xav tau yog sau daim ntawv thov uas yuav ntes lawv thiab muab tso rau hauv qee yam kev cia? Thaj, cov neeg tsim tawm uas tau los nrog lub tswv yim no tam sim ntawd zaum los sau lawv cov tswv yim, qee qhov tau xaus rau GitHub.

Dab tsi hauv qab no yog ib qho kev nthuav dav ntawm plaub qhov chaw qhib qhov project uas txuag kev ntsuas hauv kev khaws cia, los ntawm cov ntaub ntawv no tuaj yeem muab tau yooj yim thiab pom, piv txwv li, siv grafana.

olegfedoseev/pinba-server (Lub Kaum Ib Hlis 2017)

udp server ntawm kev mus uas txuag cov ntsuas rau OpenTSDB. Tej zaum yog tias koj twb tau siv OpenTSDB hauv koj qhov project, ces qhov kev daws teeb meem no yuav haum rau koj, txwv tsis pub kuv pom zoo kom dhau los ntawm.

olegfedoseev/pinba-influxdb (Lub Rau Hli 2018)

udp server ntawm mus, los ntawm tib yam habrowser, uas lub sijhawm no khaws cov ntsuas hauv InfluxDB. Ntau qhov haujlwm twb tau siv InfluxDB rau kev saib xyuas, yog li cov tshuaj no yuav zoo rau lawv.

Tshaj:

  • InfluxDB nws tso cai rau sau cov metrics tau txais, thiab tshem tawm qhov qub tom qab lub sijhawm teev tseg.

Txais:

  • Cov tshuaj no tsis txuag cov ntaub ntawv ntawm timers.
  • InfluxDB yuav khaws cov nplooj ntawv chaw nyob raws li cov cim npe thiab yog tias koj muaj ntau nplooj ntawv tshwj xeeb, qhov no yuav ua rau nce noj random access nco. Los ntawm ib lub sijhawm nws "yuav pib noj lub cim xeeb li vwm". (qhov chaw)

ClickHouse-Ninja/Proton (Lub Ib Hlis 2019)

udp server ntawm kev mus, uas txuag cov ntsuas hauv ClickHouse. Nov yog kuv tus phooj ywg txoj kev daws teeb meem. Nws yog tom qab tau paub nrog nws tias kuv txiav txim siab tias nws yog lub sijhawm los coj Pinbu thiab Clickhouse.

Tshaj:

  • Clickhouse yog qhov zoo tagnrho rau cov haujlwm zoo li no; nws tso cai rau koj los nthuav cov ntaub ntawv ntau heev uas koj tuaj yeem khaws tag nrho cov ntaub ntawv nyoos txawm tias tsis muaj kev sib sau.
  • yog tias xav tau, koj tuaj yeem yooj yim sau cov txiaj ntsig tau
  • npaj-ua template rau grafana
  • txuag cov ntaub ntawv ntawm timers

Txais:

  • tuag lawm
  • tsis muaj config uas koj tuaj yeem teeb tsa lub npe ntawm cov ntaub ntawv thiab cov ntxhuav, qhov chaw nyob thiab chaw nres nkoj ntawm server.
  • Thaum khaws cov ntaub ntawv nyoos, ib lub rooj phau ntawv txhais lus pab tau siv los khaws cov nplooj ntawv thiab chaw nyob, uas cuam tshuam cov lus nug tom ntej
  • lwm yam me me uas ua raws li thawj tus rho tawm

pinba-server/pinba-server (Lub Plaub Hlis 2019)

udp server hauv php, uas txuag kev ntsuas hauv ClickHouse. Nov yog kuv txoj kev daws teeb meem, uas yog qhov tshwm sim ntawm kev paub pinba, ClickHouse thiab protobuf. Thaum kuv tau txheeb xyuas tag nrho cov pawg no, kuv tau sau "cov ntaub ntawv pov thawj ntawm lub tswv yim", uas, poob nthav rau kuv, tsis siv cov peev txheej tseem ceeb (30 MB ntawm RAM thiab tsawg dua 1% ntawm ib qho ntawm yim processors), yog li kuv txiav txim siab los qhia rau pej xeem.

Qhov zoo yog tib yam li cov kev daws teeb meem dhau los, kuv kuj tau siv cov npe ib txwm siv los ntawm tus thawj pinba_engine. Kuv kuj tau ntxiv ib qho kev teeb tsa uas tso cai rau koj los tso ntau qhov pinbase neeg rau zaub mov ib zaug txhawm rau txuag kev ntsuas hauv cov rooj sib txawv - qhov no muaj txiaj ntsig yog tias koj xav khaws cov ntaub ntawv tsis yog los ntawm php, tab sis kuj los ntawm nginx.
Qhov tsis zoo - "qhov tsis txaus ntseeg" thiab cov khoom me me uas yuav tsis haum rau koj tus kheej, tab sis kuv txoj kev daws teeb meem yog "zoo li khau khiab" thiab tsuas yog muaj li ntawm 100 kab code, yog li txhua tus tsim tawm PHP tuaj yeem hloov qhov nws tsis nyiam. hauv ob peb feeb.

Yuav ua li cas nws ua hauj lwm

UDP chaw nres nkoj 30002 tau mloog tag nrho cov pob khoom tuaj yog txiav txim siab raws li cov txheej txheem protobuf thiab sib sau ua ke. Ib zaug ib feeb, ib pob ntawv muab tso rau hauv lub clickhouse rau hauv lub rooj pinba.requests. (tag nrho cov parameter yog configured nyob rau hauv config)

Ib me ntsis txog clickhouse

Clickhouse txhawb nqa cov ntaub ntawv sib txawv. Qhov feem ntau siv ib qho yog MergeTree.

Yog tias qee lub sijhawm koj txiav txim siab khaws cov ntaub ntawv sib sau ua ke rau txhua lub sijhawm, thiab cov ntaub ntawv nyoos tsuas yog rau qhov kawg nkaus xwb, tom qab ntawd koj tuaj yeem tsim cov ntaub ntawv pom nrog kev pab pawg, thiab ua ntu zus ntxuav lub ntsiab pinba.requests lub rooj, thaum tag nrho cov ntaub ntawv yuav nyob twj ywm hauv materialized saib. Ntxiv mus, thaum tsim lub pinba.requests lub rooj, koj tuaj yeem teev "cav = Null", tom qab ntawd cov ntaub ntawv nyoos yuav tsis raug khaws cia rau hauv disk txhua, thiab tib lub sijhawm nws tseem yuav xaus rau hauv cov khoom pom thiab tau txais kev cawmdim aggregated. . Kuv siv cov tswv yim no rau nginx metrics, vim hais tias ntawm nginx kuv muaj 50 zaus thov ntau dua li ntawm php.

Yog li, koj tau los ntev thiab kuv tsis xav tawm ntawm koj ib nrab, yog li hauv qab no yog cov lus piav qhia ntxaws txog kev teeb tsa thiab kev teeb tsa ntawm kuv cov kev daws teeb meem thiab txhua yam koj xav tau, nrog rau cov pitfalls uas tau tshwm sim ntau tshaj ib lub nkoj. mus tsoo. Tag nrho cov txheej txheem kev teeb tsa tau piav qhia rau Ubuntu 18.04 LTS thiab Centos 7; cov txheej txheem yuav txawv me ntsis ntawm lwm qhov kev faib tawm thiab cov qauv.

chaw

Kuv muab tag nrho cov lus txib tsim nyog rau hauv Dockerfile los pab txhawb kev rov tsim dua ntawm cov lus qhia. Tsuas yog cov pitfalls yuav tau piav qhia hauv qab no.

php-pib

Tom qab kev teeb tsa, xyuas kom meej tias hauv /etc/php/7.2/fpm/conf.d/20-pinba.ini cov ntaub ntawv koj tsis tau hais txog tag nrho cov kev xaiv. Ntawm qee qhov kev faib tawm (xws li centos) lawv tuaj yeem hais tawm.

extension=pinba.so
pinba.enabled=1
pinba.server=127.0.0.1:30002

clickhouse

Thaum lub sij hawm installation, clickhouse yuav hais kom koj teem ib lo lus zais rau lub neej ntawd tus neeg siv. Los ntawm lub neej ntawd, tus neeg siv no nkag tau los ntawm txhua tus IPs, yog li yog tias koj tsis muaj firewall ntawm koj lub server, nco ntsoov teem tus password rau nws. Qhov no tuaj yeem ua tiav tom qab kev teeb tsa hauv /etc/clickhouse-server/users.xml cov ntaub ntawv.

Nws tseem tsim nyog sau cia tias clickhouse siv ntau qhov chaw nres nkoj, suav nrog 9000. Qhov chaw nres nkoj no kuj tseem siv rau php-fpm hauv qee qhov kev faib tawm (piv txwv li, centos). Yog tias koj twb siv qhov chaw nres nkoj no lawm, koj tuaj yeem hloov nws mus rau lwm qhov hauv cov ntaub ntawv /etc/clickhouse-server/config.xml.

grafana nrog clickhouse plugin

Tom qab txhim kho Grafana, siv tus ID nkag mus admin thiab password admin. Thaum koj nkag rau hauv thawj zaug, Grafana yuav hais kom koj teeb tus password tshiab.

Tom ntej no, mus rau "+" -> ntshuam ntawv qhia zaub mov thiab qhia tus naj npawb dashboard rau import 10011. Kuv tau npaj thiab xa cov dashboard no kom koj tsis tas yuav ua koj tus kheej dua.

Grafana txhawb kev ua hauj lwm nrog clickhouse los ntawm ib tug thib peb-tog plugin, tab sis Grafana tsis muaj kev ceeb toom rau peb-tog plugins (muaj ib daim pib rau ob peb xyoos).

pinba-server

Txhim kho protobuf thiab libevent yog xaiv tau, tab sis txhim kho pinba-server kev ua tau zoo. Yog tias koj nruab pinba-neeg rau zaub mov hauv daim nplaub tshev uas tsis yog /opt, ces koj kuj yuav tau kho systemd tsab ntawv ntaub ntawv.

pinba module rau nginx

Txhawm rau sau cov qauv, koj xav tau cov lej ntawm tib lub version ntawm nginx uas twb tau teeb tsa rau ntawm koj lub server, nrog rau cov kev xaiv sib sau ua ke, txwv tsis pub tsim yuav ua tiav, tab sis thaum txuas lub module, qhov yuam kev yuav raug muab pov tseg. "lub module tsis yog binary tshaj." Cov kev xaiv muab tso ua ke tuaj yeem pom siv nginx -V hais kom ua

Lub neej hacks

Tag nrho kuv qhov chaw ua haujlwm tsuas yog ntawm https. Daim teb schema dhau los ua qhov tsis muaj qab hau, yog li kuv siv nws los cais lub vev xaib / console.

Hauv scripts uas nkag tau los ntawm lub vev xaib kuv siv:

if (ini_get('pinba.enabled')) {
    pinba_schema_set('web');
}

Thiab hauv console scripts (piv txwv li, cron scripts):

if (ini_get('pinba.enabled')) {
    pinba_schema_set('console');
}

Hauv kuv lub dashboard hauv Grafana muaj lub vev xaib / console hloov rau saib cov txheeb cais cais.

Koj tuaj yeem xa koj cov cim npe rau Pinba, piv txwv li:

pinba_tag_set('country', $countryCode);

Yog tag nrho.

Thov teb cov kev xaiv tsa hauv qab kab lus.

Raws li ib txwm muaj, kuv ceeb toom rau koj tias kuv tsis qhia lossis pab los ntawm tus kheej cov lus ntawm Habr thiab social networks.

Tsim ib daim pib ntawm Github.

Tsis tas li thov txhawb nrog nyiam Lus Askiv version ntawm tsab xov xwm no ntawm reddit.

Tsuas yog cov neeg siv sau npe tuaj yeem koom nrog hauv daim ntawv ntsuam xyuas. Kos npe rau hauvthov.

Koj siv OS dab tsi ntawm server?

  • Ubuntu

  • CentOS

  • Debian

  • Gentoo

  • liab cov kaus mom

  • Fedora

  • OpenSUSE

  • Ntxha

  • Unix

  • lub qhov rais

  • lwm yam

114 cov neeg siv pov npav. 11 cov neeg siv txwv tsis pub siv.

Dab tsi ntawm php koj siv ntawm lub server?

  • 7.3

  • 7.2

  • 7.1

  • 7.0

  • 5

  • lwm yam

105 cov neeg siv pov npav. 17 cov neeg siv txwv tsis pub siv.

Koj puas tau siv pinba?

  • yog

  • tsis yog, tab sis kuv xav tau

  • tsis yog thiab kuv yuav tsis xav

  • tsis tau thiab tsis tau hnov ​​txog nws

100 cov neeg siv pov npav. 14 cov neeg siv txwv tsis pub siv.

Yam twg ntawm Pinba server koj xav sim?

  • pinba_engine (mysql cav)

  • pinba2 (mysql engine)

  • pinboard (php + mysql)

  • olegfedoseev/pinba-server (mus + OpenTSDB)

  • olegfedoseev/pinba-influxdb (go + influxdb)

  • pinba-server/pinba-server (mus + clickhouse)

  • pinba-server/pinba-server (php + clickhouse)

  • Kuv mam sau kuv tus kheej

  • lwm yam

39 cov neeg siv pov npav. 47 cov neeg siv txwv tsis pub siv.

Tau qhov twg los: www.hab.com

Ntxiv ib saib