Kutarisisa kwakajairwa kweiyo sevhisi dhizaini yekutarisisa kutaridzika kwakavakirwa pane neural network

kupinda

Nhai!

Muchikamu chino ini ndichagovera ruzivo rwangu rwekuvaka microservice architecture yepurojekiti inoshandisa neural network.

Ngatitaurei nezvezvinodikanwa zvekuvaka, tarisa kune akasiyana dhizaini, ongorora imwe neimwe yezvikamu zveyakapedzwa zvivakwa, uye zvakare ongorora matekiniki metrics emhinduro.

Enjoy reading!

Mashoko mashoma pamusoro pedambudziko uye mhinduro yaro

Pfungwa huru ndeyekuongorora kukwezva kwemunhu pachiyero chegumi-zvinobva pamufananidzo.

Muchinyorwa chino tichabva pakutsanangura ese ari maviri neural network anoshandiswa uye maitiro ekugadzirisa data uye kudzidziswa. Nekudaro, mune chimwe chezvinyorwa zvinotevera, isu zvirokwazvo tichadzoka pakuongorora pombi yekuongorora padanho rakadzama.

Iye zvino tichapfuura nepaipi yekuongorora padanho repamusoro, uye tichatarisa pakudyidzana kwemicroservices mumamiriro eiyo yakazara purojekiti yekuvaka. 

Paunenge uchishanda pane inokwezva yekuongorora pombi, basa racho rakaderedzwa kuita zvinotevera zvikamu:

1. Kusarudza zviso mumifananidzo
2. Rating yemunhu mumwe nomumwe
3. Rever the result

Yekutanga inogadziriswa nemasimba ekare akadzidziswa MTCNN. Kwechipiri, convolutional neural network yakadzidziswa paPyTorch, ichishandisa ResNet34 - kubva pachiyero "mhando / kumhanya kwekufungidzira paCPU"

Inoshanda dhizaini yepiipi yekuongorora

Ongororo yezvinodikanwa zvekuvaka purojekiti

Mukufamba kwehupenyu ML Matanho epurojekiti ebasa pakuvaka uye otomatiki yekumisikidza modhi kazhinji ari pakati peanonyanya kupedza nguva uye anoshandisa zviwanikwa.

Hupenyu kutenderera kwepurojekiti yeML

Iyi purojekiti haina mutsauko - sarudzo yakaitwa yekuputira pombi yekuongorora kuita sevhisi yepamhepo, yaida kunyudza isu mukuvaka. Zvinotevera zvakakosha zvinodiwa zvakaonekwa:

1. Yakabatana log storage - ese masevhisi anofanirwa kunyora matanda munzvimbo imwechete, anofanirwa kuve nyore kuongorora
2. Kugona kwekuyera kuyera kwesevhisi yekuongorora - seyo ingangoita Bottleneck
3. Huwandu hwakafanana hwezviwanikwa zve processor hunofanirwa kugoverwa kuti uongorore mufananidzo wega wega kuitira kudzivirira kunze kwekugovera nguva yekufungidzirwa.
4. Kurumidza (re) kutumira kweese masevhisi chaiwo uye stack yakazara
5. Iko kugona, kana zvichidikanwa, kushandisa zvinhu zvakajairika mumasevhisi akasiyana

akitekicha

Mushure mekuongorora izvo zvinodikanwa, zvakava pachena kuti iyo microservice architecture inoenderana zvakakwana.

Kuti ubvise kutemwa nemusoro kusingakoshi, iyo Telegraph API yakasarudzwa semberi.

Kutanga, ngatitarisei dhayagiramu yezvimiro zveyakapera dhizaini, tozoenda kune tsananguro yechimwe nechimwe chezvikamu, uye zvakare gadzirisa maitiro ekubudirira kwekugadzirisa mufananidzo.

Chimiro dhiyagiramu yezvakapera zvivakwa

Ngatitaurei zvakadzama nezve chimwe nechimwe chezvikamu zvedhiyagiramu, tichivaratidza Basa Rimwechete mukuita kwekuongorora kwemufananidzo.

Microservice "attrai-telegraph-bot"

Iyi microservice inoputira kudyidzana kwese neTeregiramu API. Pane 2 mamiriro makuru: kushanda nechimiro chetsika uye kushanda nemhedzisiro yepiipi yekuongorora. Ngatitarisei pane ese mascenario mune zvakajairika.

Paunenge uchigamuchira meseji yakasarudzika ine mufananidzo:

1. Kusefa kunoitwa, kunosanganisira cheki dzinotevera:
   • Kuwanikwa kwehukuru hwemufananidzo
   • Nhamba yemifananidzo yevashandisi yatova mumutsara
2. Kana uchipfuura kusefa kwekutanga, chifananidzo chinochengetwa mune docker vhoriyamu
3. Basa rinogadzirwa mumutsara we "to_estimate", unosanganisira, pakati pezvimwe zvinhu, nzira inoenda kumufananidzo uri muvhoriyamu yedu.
4. Kana matanho ari pamusoro akapedzwa zvinobudirira, mushandisi achagamuchira meseji ine nguva yekufungidzira yekugadzira mifananidzo, iyo inoverengerwa zvichienderana nehuwandu hwemabasa ari mumutsara. Kana kukanganisa kukaitika, mushandisi anoziviswa zvakajeka nekutumira meseji ine ruzivo nezve izvo zvingave zvisina kumira zvakanaka.

Zvakare, iyi microservice, semushandi wecelery, inoteerera kune "after_estimate" mutsara, iyo inoitirwa mabasa akapfuura nepaipi yekuongorora.

Kana uchigamuchira basa idzva kubva "after_estimate":

1. Kana mufananidzo ukagadziriswa zvakanaka, tinotumira mhinduro kumushandisi; kana zvisiri, tinozivisa nezve kukanganisa.
2. Kubvisa mufananidzo uri mhedzisiro yepiipi yekuongorora

Evaluation microservice "attrai-estimator"

Iyi microservice mushandi wecelery uye inoputira zvese zvine chekuita nepiipi yekuongorora mufananidzo. Pane imwe chete inoshanda algorithm pano - ngatiiongororei.

Kana uchigamuchira basa idzva kubva ku "to_estimate":

1. Ngatimhanyei mufananidzo kuburikidza nepombi yekuongorora:
   1. Kuisa mufananidzo mundangariro
   2. Isu tinounza mufananidzo kuhukuru hunodiwa
   3. Kutsvaga zviso zvese (MTCNN)
   4. Isu tinoongorora zviso zvese (tinoputira zviso zvinowanikwa munhanho yekupedzisira kuita batch uye inference ResNet34)
   5. Tora mufananidzo wekupedzisira
      1. Ngatidhirowei mabhokisi ekusungira
      2. Kudhirowa zviyero
2. Kudzima mufananidzo wetsika (yepakutanga).
3. Kuchengetedza zvabuda kubva papiipi yekuongorora
4. Isu tinoisa basa mumutsara we "after_estimate", inoteererwa ne "attrai-telegraph-bot" microservice yakurukurwa pamusoro apa.

Greylog (+ mongoDB + Elasticsearch)

Greylog ndiyo mhinduro ye centralized log management. Muchirongwa ichi, yakashandiswa kune chinangwa chayo.

Sarudzo yakawira paari, uye kwete pane yakajairika ELK: stack, nekuda kwekureruka kwekushanda nayo kubva kuPython. Zvese zvaunoda kuti uite kuti utore kuGraylog kuwedzera iyo GELFTCPHndler kubva pasuru greypy kune yakasara yemidzi midzi inobata yepython microservice yedu.

Semunhu aimboshanda chete neiyo ELK stack, ndakave neruzivo rwakakwana ndichishanda naGraylog. Chinhu chega chiri kuodza moyo ndiko kukwirira muKibana maficha pamusoro peGreylog web interface.

TsuroMQ

TsuroMQ ndeye meseji broker yakavakirwa pane AMQP protocol.

Muchirongwa ichi chakashandiswa se iyo yakagadzikana uye yakaedzwa nguva broker yeCelery uye yakashanda mune yakasimba modhi.

Redis

Redis iNoSQL DBMS inoshanda nekiyi-value data zvimiro

Dzimwe nguva pane kudikanwa kwekushandisa zvakajairwa zvinhu zvinoshandisa mamwe data zvimiro mune akasiyana Python microservices.

Semuenzaniso, Redis inochengetedza hashmap yefomu "telegraph_user_id => nhamba yemabasa anoshanda mumutsara," iyo inobvumidza iwe kudzikamisa nhamba yezvikumbiro kubva kune mumwe mushandisi kune imwe kukosha uye, nekudaro, kudzivirira kurwiswa kweDoS.

Ngatigadzirise hurongwa hwekubudirira kwekugadzirisa mufananidzo

1. Mushandisi anotumira mufananidzo kune iyo Telegraph bot
2. "attrai-telegraph-bot" inogamuchira meseji kubva kuTeregiramu API uye inoiburitsa
3. Basa rine mufananidzo rinowedzerwa kune asynchronous queue "to_estimate"
4. Mushandisi anogamuchira meseji ine yakarongwa nguva yekuongorora
5. "attrai-estimator" inotora basa kubva pa "to_estimate" mutsara, inomhanyisa fungidziro nepapombi uye inoburitsa basa mumutsara we "after_estimate"
6. "attrai-telegraph-bot" inoteerera kune "after_estimate" mutsara, inotumira mhedzisiro kumushandisi.

DevOps

Pakupedzisira, mushure mekuongorora zvivakwa, unogona kuenda kune iyo yakaenzana inonakidza chikamu - DevOps

Docker swarm

 

Docker swarm  - a clustering system, kushanda kwayo kunoitwa mukati meDocker Injini uye inowanikwa kunze kwebhokisi.

Kushandisa "swarm", nodes dzose musumbu redu dzinogona kugoverwa mumhando mbiri - mushandi uye maneja. Pamichina yerudzi rwekutanga, mapoka emidziyo (stacks) anoiswa, michina yerudzi rwechipiri inotarisira kuyera, kuyera uye. zvimwe zvinotonhorera. Mamaneja zvakare vashandi nekukasira.

Cluster ine mutungamiri mumwe maneja uye vashandi vatatu

Iyo yakaderera inokwanisika saizi yesumbu ndeye 1 node; muchina mumwechete uchaita panguva imwe chete semutungamiri maneja uye mushandi. Zvichienderana nehukuru hwepurojekiti uye zvishoma zvinodiwa zvekushivirira kukanganisa, zvakasarudzwa kushandisa nzira iyi.

Ndichitarisa mberi, ndichataura kuti kubva pakatanga kubudiswa kwekugadzirwa kwekutanga, kwaiva pakati peJune, hakuna kumbove nezvinetso zvine chokuita nesangano iri remasumbu (asi izvi hazvirevi kuti sangano rakadaro rinogamuchirwa nenzira ipi zvayo mune imwe yepakati-yakakura. mapurojekiti, ayo ari pasi pekutadza kushivirira zvinodiwa).

Docker Stack

Mune swarm mode, iye ane basa rekutumira stacks (seti ye docker services) docker stack

Inotsigira docker-compose configs, ichikubvumidza kuti uwedzere kushandisa deploy paramita.  

Semuenzaniso, tichishandisa aya ma paramita, zviwanikwa zveimwe neimwe yekuongorora microservice zviitiko zvaive zvishoma (isu tinogovera N cores yeN zviitiko, mumicroservice pachayo tinomisa huwandu hwemacores anoshandiswa nePyTorch kune imwe)

attrai_estimator:
  image: 'erqups/attrai_estimator:1.2'
  deploy:
    replicas: 4
    resources:
      limits:
        cpus: '4'
    restart_policy:
      condition: on-failure
      …

Izvo zvakakosha kuti uzive kuti Redis, RabbitMQ uye Graylog masevhisi ane hunyanzvi uye haagone kuyerwa nyore se "attrai-estimator"

Kufanotaura mubvunzo - sei kwete Kubernetes?

Zvinoita sekuti kushandisa Kubernetes mumapurojekiti madiki uye epakati ibasa repamusoro; zvese zvinodikanwa zvinoshanda zvinogona kuwanikwa kubva kuDocker Swarm, iri mushandisi-inoshamwaridzika kune yemudziyo orchestrator uye zvakare ine yakaderera chipingamupinyi chekupinda.

Infrastructure

Zvese izvi zvakaiswa paVDS zvine hunhu hunotevera:

• CPU: 4 core Intel® Xeon® Goridhe 5120 CPU @ 2.20GHz
• RAM: 8 GB
• SSD: 160GB

Mushure mekuyedzwa kwemutoro wenzvimbo, zvaiita sekunge nekuwanda kwevashandisi, muchina uyu waizokwana.

Asi, pakarepo mushure mekutumirwa, ndakaisa chinongedzo kune imwe yeanonyanya kufarirwa mabhodhi emifananidzo muCIS (yup, iyo imwe chete), mushure meizvozvo vanhu vakafarira uye mumaawa mashoma basa rakabudirira kugadzirisa makumi ezviuru zvemifananidzo. Panguva imwecheteyo, panguva dzepamusoro, CPU uye RAM zviwanikwa hazvina kana hafu yakashandiswa.

Mamwe magirafu

Nhamba yevashandisi vakasiyana uye zvikumbiro zvekuongorora kubva pakutumirwa, zvichienderana nezuva

Evaluation pipeline inference yekugovera nguva

zvakawanikwa

Kupfupisa, ndinogona kutaura kuti dhizaini uye nzira yekurongeka kwemidziyo yakazvipembedza zvizere - kunyangwe panguva dzepamusoro pakanga pasina madonhwe kana kudonha munguva yekugadzirisa. 

Ini ndinofunga kuti mapurojekiti madiki uye epakati nepakati anoshandisa chaiyo-nguva inference yeneural network paCPU mukuita kwavo anogona kubudirira kutora maitiro anotsanangurwa muchinyorwa chino.

Ini ndichawedzera kuti pakutanga chinyorwa chaive chakareba, asi kuti ndirege kutumira kuverenga kwenguva refu, ndakafunga kusiya mamwe mapoinzi muchinyorwa chino - tichadzokera kwavari mune ramangwana zvinyorwa.

Unogona kubaya bot paTeregiramu - @AttraiBot, ichashanda kusvika pakupera kwematsutso 2020. Rega ndikuyeuchidze kuti hapana data remushandisi rakachengetwa - kana yepakutanga mifananidzo, kana mhedzisiro yepombi yekuongorora - zvese zvinoputswa mushure mekugadzirisa.

Source: www.habr.com