🥇Banana Pi R64 router — Debian, Wireguard, RKN

Iyo Banana Pi 64 komputa imwe-bhodhi yakafanana neRaspberry Pi, asi ine akati wandei Ethernet ports, izvo zvinoita kuti zvikwanise kuishandura kuita router zvichibva pakugovera-chinangwa Linux.

Ehe, kune Openwrt, asi ine maitiro ayo, GUI yayo neCLI; kune Mikrotik, asi zvakare ine GUI/CLI yayo, uye Wireguard Hazvishande kubva pakutanga... Chaizvoizvo, ndinoda router ine marongero anochinjika, uku ndichiramba ndiri mukati meLinux yakajairika yandinoshanda nayo zuva nezuva.

Muchinyorwa chiri pasi pemazita BPI, R64, single-board, ini ndichareva chinhu chimwe chete - iyo Banana Pi R64 single-bhodhi pachayo.

Kusarudza mufananidzo. Dhaunirodha kuburikidza neEMMC

Hunyanzvi hwekutanga hwaunofanira kuwana paunenge uchishanda nahwo SBC kazhinji, uye neR64 kunyanya, izvi zvinoreva kudzidza kurodha sisitimu yekushandisa mukati mayo uye nekukwanisa kupindirana nayo, nekuti iyo R64 haina chiteshi chekutarisa (HDMI, semuenzaniso). Apo zvinhu zvose zvakadonha - Wifi, Ethernet, Bluetooth, USB, nezvimwewo zvakamira kushanda.Kune UART, kuburikidza nehutachiona hwaunogona kugara uchiona zvakakanganisika, uyewo shandisa mirairo miviri kubva kune console, kana zvichidiwa.

Algorithm yekubatanidza kuR64 kuburikidza ne USB-UART:

tinomhanyira kuchitoro chezvikamu zveredhiyo kune USB-UART tambo (PL2303, Serial-to-USB)
batanidza imwe USB magumo pakombuta, uye imwe, UART, kune iyo R64, ine waya nhatu kubva pane ina, sepamufananidzo pazasi.
mhanya mukombuta console sudo minicom

Mushure meizvi, muzviitiko zvakawanda iyo single-board console ichaonekwa = kubudirira.
Unogona kuona mamwe mashoko pano.

Tevere, nzira iri nyore ndeyekurodha sisitimu yekushandisa kubva kune SD kadhi: dhawunirodha by batanidzo mufananidzo uye uzadze:

unzip -p 2019-08-23-ubuntu-16.04-lite-preview-bpi-r64-sd-emmc.img.zip | pv | sudo dd of=/dev/mmcblk0 bs=10M status=noxfer

Isu tinoisa kadhi muR64 SD slot, batidza, uye tarisa yakabatana console kurodha kutanga uboot, wozojairwa Linux kurodha.

Imwe bhutsu sarudzo iri kushandisa 64Gb kadhi rakatovakwa muR8, inonzi eMMC. Zvinoenderana nemirairo muwiki, tinokopa mufananidzo kune mudziyo
/dev/mmcblk0 kuBPI, reboot, bvisa SD kadhi, vhura BPI zvakare ... uye hazvishande. Nzira yekuenda mberi nekudzoka Boot select usazvinetse.

Icho chokwadi ndechekuti zvirinani zveBPI iwe unofanirwa kuseta yakakosha mureza kuti ukwanise kubhutsura kubva mukati flash drive:

root@bpi-r64:~# ./mmc extcsd read /dev/mmcblk1 | grep 'PARTITION_CONFIG'
Boot configuration bytes [PARTITION_CONFIG: 0x00]
root@bpi-r64:~# ./mmc bootpart enable 1 1 /dev/mmcblk1
root@bpi-r64:~# ./mmc extcsd read /dev/mmcblk1 | grep 'PARTITION_CONFIG'
Boot configuration bytes [PARTITION_CONFIG: 0x48]

Tevere, iwe unofanirwa kunyora preloader mune yakakosha bhutsu partition

root@bpi-r64:~# echo 0 > /sys/block/mmcblk0boot0/force_ro 
root@bpi-r64:~# dd if=preloader_evb7622_64_foremmc.bin of=/dev/mmcblk0boot0

Mugadziri R64 (China) akaisa iyi binary pano. Izvo zvazvinoita hazvizivikanwe (hapana sosi makodhi), asi hazvishande pasina iwo zvakare.

Kazhinji, mushure meizvi, mifananidzo inotanga kurodha kubva kuMMC. Kana iwe uchida kuzvifunga uye kugadzira mifananidzo kubva kutanga, saka kune ese ari maviri kesi (SD/eMMC) unofanirwa kunyora akati wandei mafaera (preloader yeSD kadhi, ATF, u-bhutsu) kuti ungosvika pakurodha kernel. Nyaya iyi ichiripo inovandudza, asi kwatiri chinhu chikuru ndechekuti inoshanda uye yakanaka.

Zvino ini ndinodhawunirodha kuburikidza neEMMC, kutaura chokwadi, ini handirishandise, kadhi reSD rakakwana, asi ndakapedza nguva yakawanda ndichiita kuti rishande, saka ngazvive muchinyorwa.

Kusarudza sisitimu yekushandisa. Armbian

Basa rekutanga rinoshanda nderekuvhura VPN, zviri pachena WireguardNdakabva ndangoona kuti kernel yakanga isina kuunganidzwa zvakanaka, uye pakanga pasina maheaders. Ndakavakazve kernel uye, sezvandaiita ne x86, ndakavaka kernel module ndichishandisa DKMS. Zvisinei, kumhanya kwekuvaka paARM64, kunyangwe kune zvishandiso zvidiki, kwaishamisa zvisingafadzi. Zvadaro imwe kernel module yaidiwa, nezvimwewo. Chaizvoizvo, zvakazoitika kuti chero chinhu chine chekuita nekernel chinonyatsogadzirwa palaptop ye x86 isina kunaka, chobva changokopwa nekunamirwa kuARM64, chotangazve, uye choedzwa.

Chikamu chekushandisa nzvimbo imwe chete. Munyaya yangu, sarudzo Debian, zvese zve arm64 architecture zvatove pamapakeji.debian.org uye hapana chikonzero chekugadzirisazve chero chinhu.

Kuti ndisaburitse rimwe bhasikoro, I ported armbian paBPI R64.
Kana kuti, izvi: chikamu chemushandisi iArmbian, uye kernel inotorwa kubva kune repository va pachena-A. Mufananidzo uchangoburwa unogona kudhaunirodwa pano.

Zvese zviitiko pakuvandudzwa kwesoftware chikamu cheR64 chinoitwa forumKazhinji, mugadziri wacho ari kuedza kuita kuti router yeOpenWrt ifadze, asi nekuda kwebasa raFrank mugadziri wekuGermany, zvese zvinoguma zvasvika padanho repamusoro re Debian-a. Zvinoshamisa kuti Frank anoshanda zvakanyanya muhurukuro dzese dzepaforamu.

Sangano rebasa rekushanda: waya

Takaparadzana, ndinoda kukuudza sei, panguva yekuvandudza / kuyedza, isa SBC (kwete chete BPI) patafura kuitira kuti usamhanye tambo yeEthernet kwairi kubva kuInternet sosi mukati mekamuri yese / hofisi. Ichokwadi ndechokuti, kune rumwe rutivi, unoda kupa chidimbu che hardware neInternet, asi kune rumwe rutivi, zvinhu zvose zviri muchikamu ichocho che hardware zvinogona kuputsa, uye kutanga kweWifi.

Kutanga, ndakasarudza kutenga yakachipa USB-Wifi "muridzo", plug iyo mune chete chiteshi paBPI uye kukanganwa nezve waya. Kuti ndiite izvi, ndakatenga isingadhure TP-LINK TL-WN725N USB 2.0, asi nekukurumidza zvakava pachena kuti yaisazosimuka: kuti muridzo ushande, unoda mutyairi wekernel, uyo, chokwadi, akange asipo. (gare gare ndakaunganidza inodiwa RTL8XXXU mutyairi, asi zvichiri kusashanda). Uye tambo yeEthernet yakashatisa kutaridzika kwekamuri kwechinguva.

Nekuda kweizvozvo, ndakakwanisa kubvisa tambo nerubatsiro rweTenda MW3 (Wifi mesh system): Ndakangoisa cube imwe pasi petafura uye ndikabatanidza iyo BPI kune yekupedzisira LAN port ine mita-yakareba Ethernet tambo. Success.

Wireguard, RKN, Shiri

Chimwe chezvinhu chandinoda kushandisa Banana PI ndechekuva nemahara emahara kunzvimbo dzakavharwa neRKN, kunyanya, kuitira kuti Telegraph uye Slack mafoni ashande. Zvinyorwa paHabré zvakatotaurwa nezvenyaya iyi: nguva, два, vatatu.

Ndakaisa chaizvo mhinduro iyi ndichishandisa Ansible: ссылка.

Zvinofungidzirwa kuti VPS iri kushanda pasi pe Ubuntu Kubvumbi 18.04. Ndakaedza mashandiro emakambani maviri anopa mawebhusaiti ekugamuchira mawebhusaiti muEurope: Amazon neDigital Ocean.

Saka, isu takaisa iri pamusoro Armbian paR64, inowanikwa kuburikidza ne ssh pasi pezita hm-bananapi-1 uye ine internet access. Isu tinogara tichitumira Ansible, otomatiki zvinyorwa uye totangisa iyo yekumisikidza pachayo paR64:

# зависимости для Debian-based дистрибутивов
$ sudo apt install --no-install-recommends python3-pip python3-setuptools python3-wheel git
$ which pip3
/usr/bin/pip3

# ansible с pybook, скриптование на Python
$ pip3 install https://github.com/muravjov/ansible/archive/ansible-2.10.0.dev0-pybook2019.tar.gz

$ export PATH=~/.local/bin:$PATH
$ which ansible-playbook
/home/sa/.local/bin/ansible-playbook

$ git clone https://github.com/muravjov/ansible-bpi-r64.git
$ cd ansible-bpi-r64

$ git submodule update --init

# убеждаемся в доступности hm-bananapi-1
$ ssh hm-bananapi-1 which python3
/usr/bin/python3

# собственно установка
$ ansible-playbook ./router.py -l hm-bananapi-1

Tevere, iwe unofanirwa kuendesa VPN yedu kuVPS nenzira imwecheteyo:

ansible-playbook ./router.py -l current-vpn

Pano iyo nharo inogara iripo-vpn, uye iyo chaiyo VPS zita rinogadziriswa mune shanduko (munyaya iyi iri paris-vpn-aws-t2-micro-1):

$ grep current_vpn group_vars/all 
current_vpn: paris-vpn-aws-t2-micro-1
#current_vpn: frankfurt-vpn-d0-starter-1

Ehe, usati waita zvese izvi unofanirwa kugadzira zvakavanzika (kunyanya makiyi) Wireguard) kuenda kufolda ./secrets, dhairekitori rinofanira kutaridzika saka.

Ansible otomatiki muPython

Unogona kuona kuti pachinzvimbo chekuve muYAML fomati, iyo Ansible mirairo yakavharirwa muPython zvinyorwa. Kuenzanisa, maitiro ekugonesa daemon yeshiri nenzira yakajairika:

- name: start bird
  systemd:
    name: bird
    state: started
    enabled: yes

uye maitiro ekuita zvakafanana kuburikidza nePython:

with mapping:
    append("name", "start bird")
    with mapping("systemd"):
        append("name",  "bird")
        append("state", "started")
        append("enabled", "yes")

Kunyora Mirairo Inogoneka muPython inokutendera kuti ushandisezve kodhi, uye kazhinji inovhura zvese zvingave zvemutauro-chinangwa mutauro. Semuenzaniso, kuisa shiri paR64 uye VPS:

install_bird("router/bird.conf.j2")
install_bird("vpn/bird.conf.j2")

ona kodhi yebasa install_shiri().

Chimiro ichi chakadanwa pybook itwa pano. Iko hakuna zvinyorwa papybook parizvino, asi ini ndichagadzirisa nyaya iyi gare gare.

Anofungei nechokumusoro kwerwizi pachiitiko ichi.

Monitoring. Prometheus

Yakazara: telegraph inoshanda, linkedin uye pornhub zvakare, kazhinji ruzivo rwemushandisi rwakanaka. Asi zvese zvinogona kutyora, kusanganisira Chinese Hardware.

Kugadziriswa kwekernel kunogonawo kunakidza: semuenzaniso, ndaida kugadzirisa kernel 5.4 => 5.6, zvakanaka, ipapo Wireguard Kunze kwebhokisi, hapana chikonzero chekusunga... Pasina nguva ndapedza: Ndakatamisa mapetch kubva pa5.4 kuenda pa5.6, kernel yakatanga, tunnel kuenda kuVPS pings, asi Bird haana kukwanisa kubatana ne "BGP Error"... "Nekutya, ndakadzokera kumashure" (c) kuenda ku5.4; kutamira ku5.6 kwaiswa muTODO.

Saka, pamusoro pekuisa router neVPS, ndakawedzera kutarisa (pa x86) Ubuntu 18.04), iyo yakaiswa pane imwe host yakasiyana ine zvikamu zvinotevera:

prometheus, alertmanager, blackbox_exporter - zvese zviri mudocker
Yambiro inotumirwa kuchiteshi cheteregiramu uchishandisa iyo metalmatze/alertmanager-bot bot - zvakare muDocker.
tor yebhoti, kuitira kuti bhoti igone kunyevera mamiriro kana paine Internet, asi teregiramu haisati yashanda, uye iyo bot pachayo haigone kubatana.
kushandiswa alerts: NodeVPNMatambudziko (hapana ping kuVPS), ShiriVPNMatambudziko (hapana Shiri musangano), AntifilterDownloadTroubles (kukanganisa kurodha yakavharirwa IP kero), SiteTroubles (yakashata-fated telegraph haisipo)
system yambiro, semuenzaniso, HostGrowingDiskReadLatency (yakachipa SD kadhi inova isingaverengeki)

Monitoring installation muenzaniso:

ansible-playbook ./monitoring.py -l monitoring-preprod

Auto Discovery yePrometheus inogadziriswa mu /etc/prometheus/auto_http folda, muenzaniso wekuwedzera muenzi kutarisisa (vatenzi havatariswe nekusarudzika):

bash << 'EOF'
HOSTNAME=hm-bananapi-1
IP_ADDRESS=`ssh -G $HOSTNAME | awk '/^hostname / { print $2 }'`

ssh monitoring-preprod sudo sponge /etc/prometheus/auto_http/$HOSTNAME.json << EOF2
[
  {
    "targets": ["$IP_ADDRESS:9100"],
    "labels": {
      "env": "prod",
      "hostname": "$HOSTNAME"
    }
  }
]
EOF2
EOF

TODO: 2 vanopa, 2 BPI, anycast failover

Mukuwedzera kune zvose, ndakaronga kubatanidza kune vanopa vaviri kuitira kuti Indaneti irambe ichishanda, kunyange kana mumwe mupi ane matambudziko netiweki, kana kuti vakakanganwa kubhadhara Indaneti, nezvimwewo, nezvimwe zvinhu zvevanhu.

Iyo yakanyanya kukwirisa mushandisi ruzivo pane iyo yakawanda-wan inotsanangurwa pano yeMwan3 system pasi peOpenwrt. Iyi mhinduro ine hupfumi hwekuita, asi kumisikidza uye kuishandisa mune zvakajairika kune akawanda-wan zvinonetsa. Muenzaniso mumwe chete: kana iwe ukauya kune dzimwe nzvimbo kubva kune mbiri IP kero kamwechete, vangave vasingazvifarire, vanorega kushanda => "Internet haisi kushanda."

Tichifunga nezve chiitiko ichi, ndakafunga kuti multihoming haisati iri yekutanga, chete failerover. Kunyangwe, zvinoita sekunge mushanduro dzazvino dzeLinux zvese zvinofanirwa kushanda nemurairo mumwechete senge:

ip route add default 
    nexthop via 192.168.1.1 weight 10 
    nexthop via 192.168.2.1 weight 5

Saka, kuitira kuti tidzivise imwe pfungwa yekukundikana, tinotora 2 BPIs, kubatanidza imwe neimwe kune mumwe mupi, kubatanidza kune mumwe kune mumwe uye kuita kuwirirana kune imwe neimwe nzira ine simba kuburikidza neshiri / OSPF.

Tevere, isu tinoshambadzira yakafanana IP kero pane imwe neimwe kana sevhisi iripo (Internet, DNS). Ndiko kuti, isu hatisi kuzoisa nzira yekusarudzika isu pachedu, asi kuburikidza neshiri. Ndakatarisa mhinduro pano .

Kuita uku hakusati kwaitwa, iyo insidious coronavirus yakatamba hunyengeri pano (kwete zvese zvakasvika kubva kuAliexpress; chimwe chitoro chepamhepo, Layta, akavimbisa kuendesa muvhiki, asi kupfuura mwedzi yapfuura; wechipiri mupi anga asina nguva. kuwedzera tambo isati yavharirwa, yakangokwanisa kuwana gomba rekubooresa kumadziro kwetambo).

Mahodha sei R64

Bhodhi pachayo iri muchitoro chepamutemo SinoVoip.
Zviri nani zvakare kuodha nekukurumidza:

hutano + zivisa EU kana US plug standard
kupisa kunyura: radiators / mafeni; nekuti zvese zviri zviviri CPU uye switch chip zviri kupisa
antenna ye wifi, somuenzaniso

Iko kune nuance - mutengo wekutakura wave wakakwira zvisina kufanira muchitoro chepamutemo kwenguva yakati. Maneja Judy Huang akandisimbisa kuti pakanga pasina kukanganisa, uye unogona kusarudza ePacket yemadhora mashanu, asi ndakaona kuti kuRussia kune chete EMS ye> $5. Hazvinakidze, asi kwete kutsoropodza. Uyezve, kana ukasarudza chero imwe nyika yekuendesa (ndakapfuura nemakondinendi ese), kuendesa kunodhura ~ $ 33. Russophobes?.. Asi ndakazoona kuti kuFrance mutengo wekutakura uriwo ~ 5 $, uye ndakadzikama.

Nekuda kweizvozvo, Judy akazvipira kuisa odha, asi kwete kubhadhara (hint: isa zvishoma pane kadhi kuitira kuti kubhadhara otomatiki kusapfuura); munyorere uye achadzikisa mutengo wekutumira kune zvakajairika. Success.

Issues

Hazvisi zvese zviri kushanda zvakakwana parizvino.

Kubudirira

Ansible=Mirairo yePython inoitwa zvishoma nezvishoma, kunyangwe isina basa, kwemasekonzi makumi maviri-makumi matatu; kurongeka kwehukuru hwakareba kupfuura pane x20 laptop. Uyezve, pakutanga vanourayiwa nekukurumidza, ~ 30 masekonzi, vobva vanonoka zvakanyanya. Izvi zvinogona kunge zviri nekuda kweCPU inodziya kumusoro (throttling). Iyo Go kodhi zvakare inotora nguva yakareba kushanda:

# запрос метрик для прометея из node_exporter на Go
$ time curl -s http://172.30.1.1:9100/metrics > /dev/null

real    0m6,118s
user    0m0,005s
sys     0m0,009s

# однако температура 51 градус, не так и много
sa@bananapir64:~$ cat /sys/devices/virtual/thermal/thermal_zone0/temp
51700

Wifi

Wifi inoshanda, asi paArmbian inomira mushure mezuva rimwechete, inonyora kuti:

sa@bananapir64:~$ dmesg | grep -E 'mt7622_wmac.*timeout'
[470303.802539] mt7622_wmac 18000000.wmac: Message 38 (seq 3) timeout
[470314.042508] mt7622_wmac 18000000.wmac: Message 50 (seq 4) timeout
...

Kutangazve chete kunobatsira. Tinofanira kuenderera mberi gadzirisa.