U haajiridda Nginx una guuritaan Ergeyga Wakiil

Hello, Habr! Waxaan kuu soo jeedinayaa tarjumaada qoraalka: U haajiridda Nginx una guuritaan Ergeyga Wakiil.

Ergaygu waa hawl-qabad heer sare ah oo loo qaybiyey wakiil wakiil (oo ku qoran C++) oo loogu talagalay adeegyada iyo codsiyada shaqsiga ah, sidoo kale waa baska isgaarsiinta iyo "diyaarad xogta caalamiga ah" oo loogu talagalay naqshadaha waaweyn ee adeegga "mesh adeega". Marka la abuurayo, xalalka dhibaatooyinka kacday inta lagu guda jiro horumarinta server-yada sida NGINX, HAProxy, dheellitirka culeyska qalabka iyo dheellitirka culeyska daruuraha ayaa la tixgeliyey. Ergaygu waxa uu la shaqeeyaa codsi kasta oo waxa uu soo koobaa shabakadda si ay u bixiso hawl-qabad guud iyada oo aan loo eegin goobta. Marka dhammaan taraafikada adeegga ee kaabayaasha ay dhex maraan mesh-ka Ergeyga, way fududaanaysaa in lagu sawiro meelaha dhibka leh ee leh indho-indheyn joogto ah, habaynta waxqabadka guud, oo lagu daro waxqabadka asaasiga ah ee meel gaar ah.

Qaababka

  • Nashqada ka baxsan nidaamka: ergeygu waa adeege iskiis u kooban, oo wax qabad sare leh oo qaada qadar yar oo RAM ah. Waxay la shaqaysaa luuqad kasta oo codsi ah ama qaabayn.
  • Taageerada http/2 iyo grpc: ergeygu waxa uu leeyahay heerka koowaad http/2 iyo grpc taageerada xidhiidhada soo galaya iyo kuwa baxaya. Kani waa wakiil hufan oo ka socda http/1.1 ilaa http/2.
  • Dheelitirka Xamuulka Sare: ergeygu waxa uu taageeraa sifooyin dheellitirka culayska sare leh oo ay ku jiraan isku dayo toos ah, silsilad jebin, xadidida heerka caalamiga ah, hadhka codsiga, isu dheellitirnaanta culayska aagga deegaanka, iwm.
  • API Management Configuration: ergeygu waxa uu bixiyaa API adag oo si firfircooni leh loo maareeyo qaabayntaada.
  • U fiirsashada: U fiirsashada qoto dheer ee taraafikada L7, taageerada asalka ah ee raadinta la qaybiyey iyo la socodka mongodb, dynamodb iyo codsiyo kale oo badan.

Tallaabada 1 - Tusaale NGINX Config

Qoraalkani wuxuu isticmaalaa fayl si gaar ah loo farsameeyay nginx.conf, ku salaysan tusaalaha buuxa ee ka NGINX Wiki. Waxaad ka arki kartaa qaabeynta tafatiraha adigoo furaya nginx.conf

nginx isha config

user  www www;
pid /var/run/nginx.pid;
worker_processes  2;

events {
  worker_connections   2000;
}

http {
  gzip on;
  gzip_min_length  1100;
  gzip_buffers     4 8k;
  gzip_types       text/plain;

  log_format main      '$remote_addr - $remote_user [$time_local]  '
    '"$request" $status $bytes_sent '
    '"$http_referer" "$http_user_agent" '
    '"$gzip_ratio"';

  log_format download  '$remote_addr - $remote_user [$time_local]  '
    '"$request" $status $bytes_sent '
    '"$http_referer" "$http_user_agent" '
    '"$http_range" "$sent_http_content_range"';

  upstream targetCluster {
    172.18.0.3:80;
    172.18.0.4:80;
  }

  server {
    listen        8080;
    server_name   one.example.com  www.one.example.com;

    access_log   /var/log/nginx.access_log  main;
    error_log  /var/log/nginx.error_log  info;

    location / {
      proxy_pass         http://targetCluster/;
      proxy_redirect     off;

      proxy_set_header   Host             $host;
      proxy_set_header   X-Real-IP        $remote_addr;
    }
  }
}

Nidaamyada NGINX waxay caadi ahaan leeyihiin saddex walxood oo muhiim ah:

  1. Habaynta server-ka NGINX, qaab-dhismeedka log iyo shaqeynta Gzip. Tan waxaa lagu qeexay si caalami ah dhammaan kiisaska.
  2. Habaynta NGINX si ay u aqbasho codsiyada martigeliyaha hal.tusaale.com dekedda 8080.
  3. Dejinta goobta la beegsanayo, sida loola tacaalo taraafikada qaybaha kala duwan ee URL.

Dhammaan qaabaynta laguma dabaqi doono Ergayga Proxy, uma baahnid inaad dejiso qaar ka mid ah goobaha. Ergaygu wuxuu leeyahay afar nooc oo muhiim ah, kaas oo taageera kaabayaasha asaasiga ah ee ay bixiso NGINX. Xuddunta waa:

  • Dhageystayaal: Waxay go'aamiyaan sida Ergeyga Wakiilku u aqbalo codsiyada soo socda. Ergeyga Proxy wuxuu hadda taageeraa oo keliya dhageystayaasha ku saleysan TCP. Marka xiriir la sameeyo, waxaa loo gudbiyaa shaandhooyin kala duwan si loo habeeyo.
  • Shaandhaynta: Waxay qayb ka yihiin qaab dhismeedka dhuumaha oo farsamayn kara xogta soo socota iyo tan baxaysa. Shaqadan waxaa ku jira filtarrada sida Gzip, kaas oo cadaadiya xogta ka hor inta aan loo dirin macmiilka.
  • Routers: Waxay u gudbiyaan taraafikada meesha loo baahan yahay, oo lagu qeexay koox ahaan.
  • Kooxaha: Waxay qeexayaan barta dhamaadka taraafikada iyo xuduudaha qaabeynta.

Waxaan u isticmaali doonaa afartan qaybood si aan u abuurno qaabeynta wakiilka wakiilka si loo waafajiyo qaabeynta NGINX gaar ah. Hadafka ergaygu waa inuu la shaqeeyo API-yada iyo qaabaynta firfircoon. Xaaladdan oo kale, qaabeynta salka ayaa isticmaali doonta qaab-dhismeed adag oo adag oo ka socda NGINX.

Tallaabada 2 - Habaynta NGINX

Qeybta koowaad nginx.conf qeexayaa qaar ka mid ah NGINX gudaha kuwaas oo u baahan in la habeeyo.

Xiriirinta Shaqaalaha

Qaabeynta hoose ayaa go'aamisa tirada hababka shaqaalaha iyo isku xirka. Tani waxay muujinaysaa sida NGINX ay u cabbiri doonto si loo daboolo baahida.

worker_processes  2;

events {
  worker_connections   2000;
}

Ergeyga Proxy wuxuu u maareeyaa socodka shaqada iyo isku xirka siyaabo kala duwan.

Ergaygu wuxuu u sameeyaa dunta shaqaalaha dunta qalab kasta ee nidaamka. Dufan kasta oo shaqaale ah ayaa fuliya wareeg dhacdo aan xannibin oo mas'uul ka ah

  1. Dhagayste kasta oo dhegaysanaya
  2. Ogolaanshaha isku xidhka cusub
  3. Abuuritaanka shaandhaynta isku xirka
  4. Samee dhammaan hawlgallada I/O inta lagu jiro nolosha xiriirku.

Dhammaan habaynta isku xirka dheeraadka ah waxaa si buuxda loogu maamulaa dunta shaqaalaha, oo ay ku jirto habdhaqan kasta oo gudbin ah.

Dul kasta oo shaqaale ah oo ku jira Ergeyga, waxaa jira barkad isku xidhka. Markaa barkadaha isku xirka HTTP/2 ayaa kaliya dhisa hal xiriir martigeliyaha dibadda ah markiiba, haddii ay jiraan afar xadhig oo shaqaale ah waxaa jiri doona afar xiriiriye HTTP/2 martigeliyaha dibadda ee xaalad deggan. Iyadoo wax walba lagu hayo hal dun oo shaqaale ah, ku dhawaad ​​dhammaan koodka ayaa la qori karaa iyada oo aan la xannibin, sida haddii ay yihiin hal dun. Haddii xadhig shaqaale badan loo qoondeeyo intii loo baahnaa, tani waxay u horseedi kartaa xusuusta luntay, abuurista tiro badan oo xidhiidho shaqo la'aan ah, iyo yaraynta tirada wakhtiyada isku xirka dib loogu celiyo barkadda.

Wixii macluumaad dheeraad ah booqo Ergeyga wakiilka blog.

Habaynta HTTP

Xirmada qaabeynta NGINX ee soo socota ayaa qeexaysa dejinta HTTP sida:

  • Waa maxay noocyada mime ee la taageeray
  • Wakhtiyada kama dambaysta ah
  • Habaynta Gzip

Waxa aad ku habayn kartaa dhinacyadan adiga oo isticmaalaya filtarrada ku jira Ergayga Proxy, kaas oo aan ka hadli doono hadhow.

Talaabada 3 - Habaynta Serverka

Xakamaynta qaabeynta HTTP, qaabeynta NGINX waxay qeexaysaa in lagu dhageysto dekeda 8080 oo looga jawaabo codsiyada soo socda ee domains hal.tusaale.com ΠΈ www.one.example.com.

 server {
    listen        8080;
    server_name   one.example.com  www.one.example.com;

Gudaha Ergeyga, waxaa gacanta ku haya Dhageystayaasha.

Ergeyga dhagaystayaasha

Arrinta ugu muhiimsan ee lagu bilaabayo Ergayga Proxy waa qeexida dhagaystayaashaada. Waxaad u baahan tahay inaad abuurto faylka qaabeynta oo qeexaya sida aad rabto inaad u socodsiiso tusaalaha Ergeyga.

Qaybta hoose waxay abuuri doontaa dhagayste cusub oo waxay ku xidhi doontaa dekedda 8080. Qaabayntu waxay u sheegaysaa Ergeyga Proxy dekedaha ay tahay inay ku xidho codsiyada soo socda.

Ergeyga Proxy wuxuu u isticmaala calaamadda YAML qaabayntiisa. Wixii hordhac ah ee qoraalkan, eeg halkan isku xirka.

Copy to Editorstatic_resources:
  listeners:
  - name: listener_0
    address:
      socket_address: { address: 0.0.0.0, port_value: 8080 }

Looma baahna in la qeexo server_name, mar haddii filtarrada Ergeyga Proxy ay qaban doonaan tan.

Tallaabada 4 - Habaynta Goobta

Marka codsigu yimaado NGINX, goobta xannibaadda ayaa go'aamisa sida loo habeeyo iyo halka loo maro taraafikada. Jajabkan soo socda, dhammaan taraafikada goobta waxaa loo wareejiyaa meel sare (xusuusinta tarjumaha: kor u kaca inta badan waa server-ka codsi) Kooxda bartilmaameedka. Kooxda korka ayaa qeexaysa noodhka ay tahay inay ka baaraandegto codsiga. Tani waxaynu kaga hadli doonaa talaabada xigta.

location / {
    proxy_pass         http://targetCluster/;
    proxy_redirect     off;

    proxy_set_header   Host             $host;
    proxy_set_header   X-Real-IP        $remote_addr;
}

Ergeyga, Filters ayaa tan sameeya.

Shaandhaynta Ergayga

Qaabaynta taagan, filtarrada ayaa go'aamiya sida loo habeeyo codsiyada soo socda. Xaaladdan oo kale waxaan dejinay filtarrada u dhigma server_names tallaabadii hore. Marka ay yimaadaan codsiyada soo socda ee u dhigma xayndaabyada iyo waddooyinka qaarkood, taraafikada waxaa loo rogaa kooxda. Tani waxay u dhigantaa qaabeynta NGINX ee hoose.

Copy to Editor    filter_chains:
    - filters:
      - name: envoy.http_connection_manager
        config:
          codec_type: auto
          stat_prefix: ingress_http
          route_config:
            name: local_route
            virtual_hosts:
            - name: backend
              domains:
                - "one.example.com"
                - "www.one.example.com"
              routes:
              - match:
                  prefix: "/"
                route:
                  cluster: targetCluster
          http_filters:
          - name: envoy.router

magaca ergeyga.http_connection_manager waa shaandheyn ku dhex jirta Ergayga Proxy. filtarrada kale waxaa ka mid ah Redis, Mongo, TCP. Waxaad ka heli kartaa liiska oo dhan at dukumentiyo.

Macluumaad dheeraad ah oo ku saabsan siyaasadaha kale ee dheellitirka culeyska, booqo Dukumentiyada Ergayga.

Talaabada 5 - Proxy and Upstream Configuration

Gudaha NGINX, qaabaynta korka ayaa qeexaysa koox adeegayaal bartilmaameed ah kuwaas oo socodsiin doona taraafikada. Xaaladdan oo kale, laba kooxood ayaa loo qoondeeyay.

  upstream targetCluster {
    172.18.0.3:80;
    172.18.0.4:80;
  }

Ergayga, tan waxaa maamula kooxuhu.

Kooxaha Ergayga

U dhigma korka waxa lagu qeexaa sida kooxuhu. Xaaladdan oo kale, martida u adeegi doonta taraafikada ayaa la aqoonsaday. Qaabka loo galo martida loo yahay, sida wakhtiyada oo dhami, waxaa lagu qeexaa qaabaynta kooxda. Tani waxay u ogolaaneysaa in la xakameeyo granular dheeraad ah oo ku saabsan dhinacyada sida daahitaanka iyo dheelitirka load.

Copy to Editor  clusters:
  - name: targetCluster
    connect_timeout: 0.25s
    type: STRICT_DNS
    dns_lookup_family: V4_ONLY
    lb_policy: ROUND_ROBIN
    hosts: [
      { socket_address: { address: 172.18.0.3, port_value: 80 }},
      { socket_address: { address: 172.18.0.4, port_value: 80 }}
    ]

Marka la isticmaalayo helitaanka adeegga STRICT_DNS Ergaygu waxa uu si joogto ah oo ismaandhaaf ah u xallin doonaa bartilmaameedyada DNS ee la cayimay. Ciwaanka IP kasta oo laga soo celiyo natiijada DNS waxa loo tixgalin doonaa martigeliyaha cad ee kutlada kore. Tani waxay ka dhigan tahay in haddii codsigu soo celiyo laba ciwaan oo IP ah, Ergeygu wuxuu u qaadan doonaa inay jiraan laba martigaliyayaasha kutlada, labaduba waa inay ahaadaan kuwo isku dheelitiran. Haddii martigeliyaha laga saaro natiijada, Ergeygu wuxuu u qaadan doonaa in aanay jirin oo uu taraafikada ka jiidi doono barkadaha isku xidhka ee jira.

Wixii macluumaad dheeraad ah eeg Dukumentiga wakiilka.

Tallaabada 6 - Gelida Gelitaanka iyo Khaladaadka

Qaabeynta kama dambaysta ah waa diiwaangelinta. Halkii aad ku riixi lahayd diiwaannada khaladka saxanka, Ergaygu Proxy wuxuu qaataa hab ku salaysan daruuraha. Dhammaan qoraallada codsiyada waa la soo saaray dhejis ΠΈ stderr.

Marka isticmaalayaashu ay codsadaan, diiwaanka gelitaanka waa ikhtiyaari waana naafo si caadi ah. Si aad awood ugu geliso gelitaanka diiwaanka codsiyada HTTP, karti qaabaynta access_log maamulaha isku xirka HTTP. Waddadu waxay noqon kartaa ama qalab sida dhejis, ama fayl ku jira saxanka, iyadoo ku xiran shuruudahaaga.

Qaabeynta soo socota ayaa u weecin doonta dhammaan diiwaannada gelitaanka dhejis (Xusuusinka tarjumaha - stdout ayaa looga baahan yahay inuu isticmaalo ergeyga gudaha docker. Haddii la isticmaalo docker la'aan, ka dibna ku beddel /dev/stdout dariiqa loo maro faylka log caadiga ah). Nuqul ka koobbi qaybta qaabeynta maamulaha xidhiidhka:

Copy to Clipboardaccess_log:
- name: envoy.file_access_log
  config:
    path: "/dev/stdout"

Natiijadu waa inay u ekaataa sidan:

      - name: envoy.http_connection_manager
        config:
          codec_type: auto
          stat_prefix: ingress_http
          access_log:
          - name: envoy.file_access_log
            config:
              path: "/dev/stdout"
          route_config:

Sida caadiga ah, Ergeygu wuxuu leeyahay xargo qaab ah oo ay ku jiraan faahfaahinta codsiga HTTP:

[%START_TIME%] "%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%" %RESPONSE_CODE% %RESPONSE_FLAGS% %BYTES_RECEIVED% %BYTES_SENT% %DURATION% %RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)% "%REQ(X-FORWARDED-FOR)%" "%REQ(USER-AGENT)%" "%REQ(X-REQUEST-ID)%" "%REQ(:AUTHORITY)%" "%UPSTREAM_HOST%"n

Natiijadii qaabkani waa:

[2018-11-23T04:51:00.281Z] "GET / HTTP/1.1" 200 - 0 58 4 1 "-" "curl/7.47.0" "f21ebd42-6770-4aa5-88d4-e56118165a7d" "one.example.com" "172.18.0.4:80"

Nuxurka wax soo saarka waxaa lagu habeyn karaa iyadoo la dejinayo goobta qaabka. Tusaale ahaan:

access_log:
- name: envoy.file_access_log
  config:
    path: "/dev/stdout"
    format: "[%START_TIME%] "%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%" %RESPONSE_CODE% %RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)% "%REQ(X-REQUEST-ID)%" "%REQ(:AUTHORITY)%" "%UPSTREAM_HOST%"n"

Xariiqda loggu sidoo kale waxaa lagu soo saari karaa qaab JSON iyadoo la dejinayo goobta json_qaab. Tusaale ahaan:

access_log:
- name: envoy.file_access_log
  config:
    path: "/dev/stdout"
    json_format: {"protocol": "%PROTOCOL%", "duration": "%DURATION%", "request_method": "%REQ(:METHOD)%"}

Macluumaad dheeraad ah oo ku saabsan Habka Diiwaangelinta Ergeyga, booqo

https://www.envoyproxy.io/docs/envoy/latest/configuration/access_log#config-access-log-format-dictionaries

Gelintu maaha habka kaliya ee lagu helo aragti ku saabsan la shaqaynta Ergayga Proxy. Waxay leedahay baafinta horumarsan iyo awoodaha cabbir ee lagu dhex dhisay. Waxaad wax dheeraad ah ka heli kartaa dukumentiyada raadinta ama dhex mara Qoraalka raadraaca isdhexgalka.

Tallaabada 7 - Daahfurka

Waxaad hadda ka haajirtay qaabayntaada NGINX una guurtay Wakiil wakiil ah. Talaabada ugu danbeysa waa in la soo saaro tusaale wakiil wakiil ah si loo tijaabiyo.

Orod sidii isticmaale

Xagga sare ee khadka qaabeynta NGINX isticmaale www; waxay qeexaysaa inay u shaqeyso NGINX sidii isticmaale mudnaan hoose leh si loo horumariyo amniga.

Ergayga Proxy wuxuu qaataa hab ku salaysan daruuraha si uu u maareeyo cidda hawsha iska leh. Marka aan ku dhex wadno Ergeyga Proxy weel, waxaan qeexi karnaa isticmaale mudnaan yar leh.

Furitaanka Ergayga Wakiil

Amarka hoose wuxuu ku socodsiin doonaa ergeyga Proxy iyada oo la marinayo weelka Docker-ka ee martida loo yahay. Amarkani wuxuu siinayaa Ergeyga awood u leh inuu dhageysto codsiyada soo socda ee dekedda 80. Si kastaba ha ahaatee, sida lagu qeexay qaabeynta dhegeystaha, Ergey Proxy wuxuu dhageystaa taraafikada soo socota ee dekedda 8080. Tani waxay u oggolaaneysaa geeddi-socodku inuu u shaqeeyo sidii isticmaale mudnaanta hoose leh.

docker run --name proxy1 -p 80:8080 --user 1000:1000 -v /root/envoy.yaml:/etc/envoy/envoy.yaml envoyproxy/envoy

Imtixaanka

Iyadoo wakiilku uu socdo, imminka imminka imminka imminka waa la samayn karaa oo laga baaraandegi karaa imtixaanada. Amarka cURL ee soo socda ayaa soo saara codsi madaxa martigeliyaha ee lagu qeexay qaabeynta wakiilnimada.

curl -H "Host: one.example.com" localhost -i

Codsiga HTTP wuxuu keeni doonaa khalad 503. Tani waa sababta oo ah isku xidhka korku ma shaqaynayaan mana jiraan. Sidaa darteed, Wakiilka Ergaygu ma hayo meel uu u diyaarsan yahay codsiga. Amarka soo socdaa wuxuu bilaabi doonaa taxane adeegyo HTTP ah oo u dhigma qaabeynta lagu qeexay Ergayga.

docker run -d katacoda/docker-http-server; docker run -d katacoda/docker-http-server;

Adeegyada la heli karo, Ergeyga wuxuu si guul leh u matali karaa taraafikada meesha uu u socdo.

curl -H "Host: one.example.com" localhost -i

Waa inaad aragto jawaab muujinaysa weelka Docker ee ka shaqeeyay codsiga. Ergeyga Proxy logs waa inaad sidoo kale ku aragto soo-saarka xargaha gelitaanka.

Madaxda Jawaabta HTTP Dheeraadka ah

Waxaad ku arki doontaa madax HTTP dheeraad ah oo ku jira madax-jawaabeedka codsiga dhabta ah. Madaxa ayaa soo bandhigaya wakhtiga ay martigaliyaha sare ku qaatay socodsiinta codsiga. Lagu sheegay millise seconds. Tani waa faa'iido haddii macmiilku rabo inuu go'aamiyo wakhtiga adeegga marka la barbar dhigo daahitaanka shabakada.

x-envoy-upstream-service-time: 0
server: envoy

Habka ugu dambeeya

static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address: { address: 0.0.0.0, port_value: 8080 }
    filter_chains:
    - filters:
      - name: envoy.http_connection_manager
        config:
          codec_type: auto
          stat_prefix: ingress_http
          route_config:
            name: local_route
            virtual_hosts:
            - name: backend
              domains:
                - "one.example.com"
                - "www.one.example.com"
              routes:
              - match:
                  prefix: "/"
                route:
                  cluster: targetCluster
          http_filters:
          - name: envoy.router
          clusters:
  - name: targetCluster
    connect_timeout: 0.25s
    type: STRICT_DNS
    dns_lookup_family: V4_ONLY
    lb_policy: ROUND_ROBIN
    hosts: [
      { socket_address: { address: 172.18.0.3, port_value: 80 }},
      { socket_address: { address: 172.18.0.4, port_value: 80 }}
    ]

admin:
  access_log_path: /tmp/admin_access.log
  address:
    socket_address: { address: 0.0.0.0, port_value: 9090 }

Macluumaad dheeraad ah oo ka yimid turjumaanka

Tilmaamaha ku rakibida Ergayga Proxy waxaa laga heli karaa mareegaha https://www.getenvoy.io/

Sida caadiga ah, rpm ma laha habayn adeeg oo nidaamsan.

Kudar nidaamka adeegga qaabeynta /etc/systemd/system/envoy.service:

[Unit]
Description=Envoy Proxy
Documentation=https://www.envoyproxy.io/
After=network-online.target
Requires=envoy-auth-server.service
Wants=nginx.service

[Service]
User=root
Restart=on-failure
ExecStart=/usr/bin/envoy --config-path /etc/envoy/config.yaml
[Install]
WantedBy=multi-user.target

Waxaad u baahan tahay inaad abuurto hagaha /etc/envoy/ oo aad geliso config.yaml config.

Waxaa jira wada sheekeysi telegram ah iyadoo la adeegsanayo wakiil wakiil: https://t.me/envoyproxy_ru

Ergeyga Proxy ma taageerayo u adeegida macluumaadka taagan. Sidaa darteed, yaa u codayn kara sifada: https://github.com/envoyproxy/envoy/issues/378

Isticmaalayaasha diiwaangashan oo keliya ayaa ka qaybqaadan kara sahanka. Soo gal, soo dhawoow.

Boostada ma kugu dhiirigelisay inaad rakibto oo tijaabiso wakiilka ergayga?

  • haa

  • no

75 isticmaale ayaa u codeeyay. 18 isticmaale ayaa ka aamusay.

Source: www.habr.com

Add a comment