Sawubona wonke umntu!
Umsebenzi umi ngolu hlobo lulandelayo - kukho ukuhamba okuboniswe kumfanekiso ongentla, ekufuneka ukhutshelwe ngaphandle kwiiseva ze-N nge
Indawo ye-NiFi kwiSiza (i-S2S) yindlela ekhuselekileyo, enokwenziwa ngokwezifiso kakhulu yokudlulisa idatha phakathi kweemeko ze-NiFi. Jonga ukuba isebenza njani i-S2S
Xa kuziwa ekudluliselweni kwedatha usebenzisa i-S2S, umzekelo omnye ubizwa ngokuba ngumxhasi, owesibini ngumncedisi. Umxhasi uthumela idatha, umncedisi uyayifumana. Iindlela ezimbini zokuseta ukudluliselwa kwedatha phakathi kwazo:
- Dudula. Idatha ithunyelwa kumzekelo womxhasi kusetyenziswa iQela leNkqubo eLawulwa kude (RPG). Kumzekelo weseva, idatha ifunyenwe kusetyenziswa i-Input Port
- Tsala. Umncedisi ufumana idata usebenzisa i RPG, umxhasi uthumela usebenzisa Imveliso yezibuko.
Ukuhamba kokuqengqeleka kugcinwa kwiRejistri yeApache.
I-Apache NiFi Registry yi-subproject ye-Apache NiFi ebonelela ngokugcinwa kokuhamba kunye nesixhobo soguqulelo. Uhlobo lwe-GIT. Ulwazi malunga nokufaka, ukuqwalasela kunye nokusebenza kunye nerejista inokufumaneka kwi
Ekuqaleni, xa i-N iyinani elincinci, ukuhamba kuhanjiswe kwaye kuhlaziywe ngesandla ngexesha elifanelekileyo.
Kodwa njengoko i-N ikhula, kukho iingxaki ezininzi:
- kuthatha ixesha elininzi ukuhlaziya ukuhamba. Kufuneka uye kuzo zonke iiseva
- kukho iimpazamo zokuhlaziya itemplates. Apha bahlaziya, kodwa apha balibale
- impazamo yomntu xa kusenziwa inani elikhulu lemisebenzi efanayo
Konke oku kusizisa kwinto yokuba kuyimfuneko ukwenza inkqubo ngokuzenzekelayo. Ndizame ezi ndlela zilandelayo ukusombulula le ngxaki:
- Sebenzisa iMiNiFi endaweni yeNiFi
- NiFi CLI
- NiPyAPI
Ukusebenzisa iMiNiFi
Enye i-subproject, i-MiNiFi C2 Server, iya kunceda ukuxazulula le ngxaki. Le mveliso ijoliswe ukuba ibe yindawo ephambili kwi-architecture yokuthunyelwa. Uyiqwalasela njani imeko-bume-ichazwe kwi
Inketho echazwe kwinqaku elingentla iyasebenza kwaye akunzima ukuyiphumeza, kodwa akufanele silibale oku kulandelayo:
- minifi ayinazo zonke iiprosesa ezivela kwi-nifi
- Iinguqulelo ze-CPU kwi-Minifi lag ngasemva kweenguqulelo ze-CPU kwi-NiFi.
Ngexesha lokubhala, inguqulo yakamuva ye-NiFi yi-1.9.2. Inguqulo yeprosesa yenguqulo yakutshanje ye-MiNiFi yi-1.7.0. Iiprosesa zinokongezwa kwi-MiNiFi, kodwa ngenxa yokungafani kwenguqulelo phakathi kwe-NiFi kunye ne-MiNiFi processors, oku kungasebenzi.
NiFi CLI
Ukugweba
Qhuba usetyenziso
./bin/cli.sh
_ ___ _
Apache (_) .' ..](_) ,
_ .--. __ _| |_ __ )
[ `.-. | [ |'-| |-'[ | /
| | | | | | | | | | ' '
[___||__][___][___] [___]', ,'
`'
CLI v1.9.2
Type 'help' to see a list of available commands, use tab to auto-complete.
Ukuze silayishe ukuhamba okuyimfuneko ukusuka kwirejista, kufuneka sazi izichazi zebhaskithi (isazisi sebhakethi) kunye nokuhamba ngokwazo (isazisi sokuhamba). Le datha inokufunyanwa mhlawumbi nge-cli okanye kwi-interface yewebhu ye-NiFi yobhaliso. Ujongano lwewebhu lujongeka ngolu hlobo:
Ukusebenzisa i-CLI, wenza oku:
#> registry list-buckets -u http://nifi-registry:18080
# Name Id Description
- -------------- ------------------------------------ -----------
1 test_bucket 709d387a-9ce9-4535-8546-3621efe38e96 (empty)
#> registry list-flows -b 709d387a-9ce9-4535-8546-3621efe38e96 -u http://nifi-registry:18080
# Name Id Description
- ------------ ------------------------------------ -----------
1 test_flow d27af00a-5b47-4910-89cd-9c664cd91e85
Qhuba iqela lenkqubo yokuthatha ngaphandle kubhaliso:
#> nifi pg-import -b 709d387a-9ce9-4535-8546-3621efe38e96 -f d27af00a-5b47-4910-89cd-9c664cd91e85 -fv 1 -u http://nifi:8080
7f522a13-016e-1000-e504-d5b15587f2f3
Inqaku elibalulekileyo kukuba nawuphi na umzekelo we-nifi unokuchazwa njengomamkeli apho siqengqela khona iqela lenkqubo.
Iqela lenkqubo elongeziweyo kunye nabaqhubekisi abamisiweyo, kufuneka baqalwe
#> nifi pg-start -pgid 7f522a13-016e-1000-e504-d5b15587f2f3 -u http://nifi:8080
Kulungile, iiprosesa ziqalile. Nangona kunjalo, ngokwemiqathango yengxaki, sifuna iimeko ze-NiFi ukuthumela idatha kwezinye iimeko. Makhe sicinge ukuba indlela yokuTyhala yakhethwa ukudlulisela idatha kumncedisi. Ukuze ulungelelanise ukuhanjiswa kwedatha, kuyimfuneko ukwenza ukuhanjiswa kwedatha (Vumela ukuhanjiswa) kwiQela leNkqubo yeRemote eyongeziweyo (RPG), esele ibandakanyiwe ekuhambeni kwethu.
Kumaxwebhu kwi-CLI nakweminye imithombo, andizange ndifumane indlela yokwenza ukudluliselwa kwedatha. Ukuba uyayazi indlela yokwenza oku, nceda ubhale kwizimvo.
Kuba sine-bash kwaye sikulungele ukuya esiphelweni, siya kufumana indlela yokuphuma! Ungasebenzisa i-NiFi API ukusombulula le ngxaki. Masisebenzise le ndlela ilandelayo, sithatha i-ID kwimizekelo engentla (kwimeko yethu yi-7f522a13-016e-1000-e504-d5b15587f2f3). Inkcazo yeeNdlela ze-NiFi API
Emzimbeni, kufuneka udlule i-JSON, yale fomu ilandelayo:
{
"revision": {
"clientId": "value",
"version": 0,
"lastModifier": "value"
},
"state": "value",
"disconnectedNodeAcknowledged": true
}
Iiparamitha ekufuneka zizaliswe ukuze "usebenze":
Lumente β imo yogqithiso lwedatha. Ifumaneka TRANSMITTING ukwenza ugqithiso lwedatha, STOPED ukukhubaza
Inguqulelo - inguqulo yeprosesa
uguqulelo lungagqibekanga ukuya ku-0 xa yenziwe, kodwa ezi parameters zinokufunyanwa kusetyenziswa indlela
Kubathandi bezikripthi ze-bash, le ndlela inokubonakala ifanelekile, kodwa inzima kum - izikripthi ze-bash azikho intando yam. Indlela elandelayo inomdla ngakumbi kwaye ilungele ngakumbi kwimbono yam.
NiPyAPI
I-NiPyAPI yilayibrari yePython yokunxibelelana neemeko zeNiFi.
Isikripthi sethu sokukhupha uqwalaselo yinkqubo yePython. Masiqhubele phambili kwikhowudi.
Cwangcisa uqwalaselo lomsebenzi oqhubekayo. Siya kufuna ezi parameters zilandelayo:
nipyapi.config.nifi_config.host = 'http://nifi:8080/nifi-api' #ΠΏΡΡΡ Π΄ΠΎ nifi-api ΠΈΠ½ΡΡΠ°Π½ΡΠ°, Π½Π° ΠΊΠΎΡΠΎΡΠΎΠΌ ΡΠ°Π·Π²ΠΎΡΠ°ΡΠΈΠ²Π°Π΅ΠΌ process group
nipyapi.config.registry_config.host = 'http://nifi-registry:18080/nifi-registry-api' #ΠΏΡΡΡ Π΄ΠΎ nifi-registry-api registry
nipyapi.config.registry_name = 'MyBeutifulRegistry' #Π½Π°Π·Π²Π°Π½ΠΈΠ΅ registry, ΠΊΠ°ΠΊ Π±ΡΠ΄Π΅Ρ Π½Π°Π·ΡΠ²Π°ΡΡΡΡ Π² ΠΈΠ½ΡΡΠ°Π½ΡΠ΅ nifi
nipyapi.config.bucket_name = 'BucketName' #Π½Π°Π·Π²Π°Π½ΠΈΠ΅ bucket, ΠΈΠ· ΠΊΠΎΡΠΎΡΠΎΠ³ΠΎ ΠΏΠΎΠ΄ΡΡΠ³ΠΈΠ²Π°Π΅ΠΌ flow
nipyapi.config.flow_name = 'FlowName' #Π½Π°Π·Π²Π°Π½ΠΈΠ΅ flow, ΠΊΠΎΡΠΎΡΠΎΠ΅ ΠΏΠΎΠ΄ΡΡΠ³ΠΈΠ²Π°Π΅ΠΌ
Ngapha koko ndiza kufaka amagama eendlela zeli thala leencwadi, ezichaziweyo
Sidibanisa irejistri kumzekelo we-nifi usebenzisa
nipyapi.versioning.create_registry_client
Kweli nyathelo, unokongeza kwakhona itshekhi ukuba ubhaliso sele longezwe kumzekelo, kuba oku ungasebenzisa indlela.
nipyapi.versioning.list_registry_clients
Sifumana ibhakethi ukukhangela ngakumbi ukuhamba kwebhasikithi
nipyapi.versioning.get_registry_bucket
Ngokwebhakethi efunyenweyo, sijonge ukuhamba
nipyapi.versioning.get_flow_in_bucket
Okulandelayo, kubalulekile ukuqonda ukuba eli qela lenkqubo sele longeziwe. Iqela lenkqubo libekwe yi-coordinates kwaye imeko inokuvela xa isibini sibekwe ngaphezulu kwenye. Ndijongile, inokuba π Ukufumana lonke iqela lenkqubo eyongeziweyo, sebenzisa indlela
nipyapi.canvas.list_all_process_groups
kwaye ke sinokukhangela, umzekelo, ngegama.
Andiyi kuchaza inkqubo yokuhlaziywa kwethemplate, ndiya kuthetha kuphela ukuba ngaba iiprosesa zongezwa kwinguqu entsha yethemplate, ngoko akukho ngxaki kunye nobukho bemiyalezo kwimigca. Kodwa ukuba iiprosesa zisusiwe, ke iingxaki zinokuvela (i-nifi ayivumeli ukususwa kweprosesa ukuba umgca womyalezo uqokelele phambi kwayo). Ukuba unomdla kwindlela endiyicombulula ngayo le ngxaki - bhala kum, nceda, siza kuxubusha le ngongoma. Abafowunelwa ekupheleni kwenqaku. Masiqhubele phambili kwinyathelo lokongeza iqela lenkqubo.
Xa kulungiswa iskripthi, ndifumene into yokuba inguqulelo yamva nje yokuhamba ayisoloko inyuswa, ke ndincoma ukuba uqale uyicacise le nguqulelo:
nipyapi.versioning.get_latest_flow_ver
Sebenzisa iqela lenkqubo:
nipyapi.versioning.deploy_flow_version
Siqala iiprosesa:
nipyapi.canvas.schedule_process_group
Kwibhloko malunga ne-CLI, kubhaliwe ukuba ukuhanjiswa kwedatha akwenziwanga ngokuzenzekelayo kwiqela lenkqubo ekude? Xa ndiphumeza iskripthi, nam ndidibene nale ngxaki. Ngelo xesha, andizange ndikwazi ukuqalisa ukudluliselwa kwedatha usebenzisa i-API kwaye ndagqiba ekubeni ndibhalele kumthuthukisi welayibrari ye-NiPyAPI kwaye ndicele iingcebiso / uncedo. Umphuhlisi wandiphendula, saxoxa ngengxaki kwaye wabhala ukuba ufuna ixesha "lokukhangela into". Kwaye ngoku, kwiintsuku ezimbalwa kamva, i-imeyile ifika apho kubhalwe umsebenzi wePython osombulula ingxaki yam yokuqalisa !!! Ngelo xesha, inguqulo ye-NiPyAPI yayingu-0.13.3 kwaye, ngokuqinisekileyo, kwakungekho nto yolo hlobo kuyo. Kodwa kwinguqulo 0.14.0, ekhutshwe kutshanje, lo msebenzi sele ufakiwe kwithala leencwadi. Hlangana
nipyapi.canvas.set_remote_process_group_transmission
Ngoko, ngoncedo lwelayibrari ye-NiPyAPI, sadibanisa irejistri, saqengqeleka ukuhamba, saze saqalisa iiprosesa kunye nokudluliselwa kwedatha. Emva koko ungadibanisa ikhowudi, udibanise zonke iintlobo zeetshekhi, ukugawulwa kwemithi, kwaye yiloo nto. Kodwa elo libali elahluke ngokupheleleyo.
Kwiinketho ezizenzekelayo endiziqwalaseleyo, ezi zamva zabonakala kum zisebenza kakhulu. Okokuqala, oku kuseyikhowudi yepython, apho unokubethelela ikhowudi yenkqubo yokuncedisa kwaye ujabulele zonke izibonelelo zolwimi lweprogram. Okwesibini, iprojekthi ye-NiPyAPI iphuhlisa ngokusebenzayo kwaye kwimeko yeengxaki ungabhalela umphuhlisi. Okwesithathu, i-NiPyAPI isesisixhobo esiguquguqukayo ngakumbi sokusebenzisana ne-NiFi ekusombululeni iingxaki ezinzima. Umzekelo, ekumiseleni ukuba ngaba imigca yomyalezo ayinanto ngoku ekuqukuqeleni kunye nokuba kunokwenzeka ukuhlaziya inkqubo yeqela.
Kuko konke. Ndachaza iindlela ze-3 zokuzenzekelayo ukuhanjiswa kokuhamba kwi-NiFi, imigibe enokuthi umphuhlisi ahlangabezane nayo kwaye unikeze ikhowudi yokusebenza yokuhanjiswa ngokuzenzekelayo. Ukuba unomdla kwesi sihloko njengoko ndinjalo -
umthombo: www.habr.com