O fa'ata'ita'iga fa'apitoa e fa'atupuina e fa'atuputeleina ai le fa'asilisili o le tau o punaoa o lo'o fa'aaogaina aua e fa'aaogaina na'o le taimi e mana'omia ai. E tele filifiliga ile fa'atinoina o lenei mea ae le o le fa'atupuina o fa'aputuga o ao fa'aopoopo e pei o talosaga a tagata faigaluega. Ma o le asō o le a ou le talanoa e uiga i le FaaS, ae e uiga i webhooks. O le a ou faʻaalia se faʻataʻitaʻiga faʻataʻitaʻiga o le taulimaina o mea tutupu e faʻaaoga ai mea e teu ai mea i luga ole laiga.
O nai upu e uiga i le teuina o mea faitino ma webhooks. O le teuina o mea e mafai ai ona e teuina soʻo se faʻamatalaga i le ao i foliga o mea faitino, e mafai ona maua e ala ile S3 poʻo se isi API (faʻalagolago ile faʻatinoga) e ala ile HTTP/HTTPS. Webhooks e masani lava HTTP callbacks. E masani ona fa'aosoina e se mea na tupu, e pei o le fa'ailoga o lo'o tuleia i se fale teu oloa po'o se fa'amatalaga o lo'o fa'apipi'i i luga ole blog. A tupu se mea na tupu, e auina atu e le nofoaga autu se talosaga HTTP i le URL ua faʻamaonia mo le webhook. O le iʻuga, e mafai ona e faia mea i luga o le tasi nofoaga e faʻaosofia ai gaioiga i luga o le isi (tasi e faaaogaina). I le tulaga o le nofoaga autu o se mea e teu ai mea, o mea e tutupu e fai ma suiga i mea o loʻo i totonu.
Faʻataʻitaʻiga o mataupu faigofie pe a mafai ona faʻaogaina ia masini:
Fausiaina o kopi o mea uma i totonu o se isi ao teu. E tatau ona faia kopi i luga ole laiga pe a faaopoopo faila pe suia.
Fausia otometi o se faasologa o ata o faila kalafi, faʻaopoopo faʻailoga vai i ata, ma isi suiga ata.
Fa'asilasilaga e uiga i le taunu'u mai o pepa fou (mo se fa'ata'ita'iga, e tu'uina atu e le au'aunaga fa'atausitusi tu'ufa'atasi lipoti i le ao, ma le mata'ituina o tupe e maua ai fa'amatalaga e uiga i lipoti fou, siaki ma su'esu'eina).
O mataupu e sili atu ona lavelave e aofia ai, mo se faʻataʻitaʻiga, faʻatupuina o se talosaga i Kubernetes, lea e fatuina ai se pusa faʻatasi ai ma atigipusa manaʻomia, pasi atu i ai tapulaʻa galuega, ma a maeʻa le faʻagasologa e malepe le koneteina.
Mo se faʻataʻitaʻiga, o le a matou faia se fesuiaiga o le galuega 1, pe a faʻafetaui suiga i le Mail.ru Cloud Solutions (MCS) pakete teuina mea faitino i le teuina o mea AWS e faʻaaoga ai webhooks. I se tulaga faʻapipiʻiina moni, e tatau ona tuʻuina atu galuega asynchronous e ala i le resitalaina o webhooks i se laina, ae mo le galuega aʻoaʻoga o le a matou faia le faʻatinoga e aunoa ma lenei.
Polokalame galuega
O loʻo faʻamatalaina auiliili le faʻasalalauga faʻasalalauga i Taiala ile S3 webhooks ile MCS. O le polokalame galuega e aofia ai elemene nei:
Auaunaga fa'asalalau, lea o loʻo i luga o le itu S3 teuina ma faʻasalalau talosaga HTTP pe a faʻaosoina le webnhook.
Webhook e mauaina le server, lea e faʻalogo i talosaga mai le auaunaga faʻasalalau HTTP ma faʻatino gaioiga talafeagai. E mafai ona tusia le 'auʻaunaga i soʻo se gagana; i la matou faʻataʻitaʻiga, matou te tusia le server i Go.
O se vaega faʻapitoa o le faʻatinoina o webhooks i le S3 API o le resitalaina o le upega tafaʻilagi e maua ai le server i luga o le auaunaga faʻasalalau. Aemaise lava, e tatau i le webhook receiving server ona faʻamaonia le saofaga i feʻau mai le auaunaga faʻasalalau (i isi faʻatinoga webhook, faʻamaoniga o le saofaga e masani lava e le manaʻomia).
E tusa ai, e tatau ona lagolagoina e le 'upega tafaʻilagi le mauaina o galuega autu e lua:
tali i le talosaga a le auaunaga lomitusi e faʻamaonia le resitalaina,
fa'agasolo mea tutupu.
Fa'apipi'i se 'upega tafa'ilagi e maua ai le server
Ina ia faʻatautaia le webhook receiving server, e te manaʻomia se Linux server. I lenei tusiga, e fai ma faʻataʻitaʻiga, matou te faʻaogaina se faʻataʻitaʻiga faʻapitoa matou te faʻapipiʻiina i luga o le MCS.
Se'i o tatou fa'apipi'i le polokalama talafeagai ma fa'alauiloa le 'au'aunaga e maua ai le webhook.
ubuntu@ubuntu-basic-1-2-10gb:~$ sudo apt-get install git
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
bc dns-root-data dnsmasq-base ebtables landscape-common liblxc-common
liblxc1 libuv1 lxcfs lxd lxd-client python3-attr python3-automat
python3-click python3-constantly python3-hyperlink
python3-incremental python3-pam python3-pyasn1-modules
python3-service-identity python3-twisted python3-twisted-bin
python3-zope.interface uidmap xdelta3
Use 'sudo apt autoremove' to remove them.
Suggested packages:
git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui
gitk gitweb git-cvs git-mediawiki git-svn
The following NEW packages will be installed:
git
0 upgraded, 1 newly installed, 0 to remove and 46 not upgraded.
Need to get 3915 kB of archives.
After this operation, 32.3 MB of additional disk space will be used.
Get:1 http://MS1.clouds.archive.ubuntu.com/ubuntu bionic-updates/main
amd64 git amd64 1:2.17.1-1ubuntu0.7 [3915 kB]
Fetched 3915 kB in 1s (5639 kB/s)
Selecting previously unselected package git.
(Reading database ... 53932 files and directories currently installed.)
Preparing to unpack .../git_1%3a2.17.1-1ubuntu0.7_amd64.deb ...
Unpacking git (1:2.17.1-1ubuntu0.7) ...
Setting up git (1:2.17.1-1ubuntu0.7) ...
Fa'apipi'i le faila i le webhook e mauaina le server:
ubuntu@ubuntu-basic-1-2-10gb:~$ cd s3-webhook/
ubuntu@ubuntu-basic-1-2-10gb:~/s3-webhook$ sudo ./s3-webhook -port 80
Fa'asoa ile auaunaga lomitusi
E mafai ona e resitalaina lau 'upega tafaʻilagi receiving server e ala i le API poʻo le upega tafaʻilagi. Mo le faigofie, matou te lesitala e ala i le upega tafaʻilagi:
Alu i le pakete lea o le a matou faʻapipiʻi ai webhooks ma kiliki i luga o le mea e faʻaoga ai:
Alu i le Webhooks tab ma kiliki Add:
Faatumu fanua:
ID - le igoa o le webhook.
Mea na tutupu - o mea tutupu e fa'asalalau. Ua matou setiina le faʻasalalauina o mea uma e tutupu pe a galulue ma faila (faʻaopoopo ma tape).
URL — webhook mauaina tuatusi server.
Filter prefix/suffix o se faamama lea e mafai ai ona e fa'atupuina webhooks mo na'o mea faitino e fetaui lelei o latou igoa i tulafono patino. Mo se faʻataʻitaʻiga, ina ia mafai e le webhook ona faʻaoso naʻo faila e iai le faʻaopoopoga .png, i Filifili suffix e tatau ona e tusia le "png".
I le taimi nei, na'o ports 80 ma 443 o lo'o lagolagoina mo le mauaina o le webhook receiving server.
Tatou kiliki Faaopopo le matau ma o le a tatou vaʻai i mea nei:
Ua faaopoopo le matau.
O lo'o fa'aalia i totonu o ana ogalaau e le 'upega tafa'ilagi le fa'agasologa o le resitalaina o matau:
Ping() - o se auala e tali mai ile URL/ping, ole fa'atinoga sili ona faigofie ole su'esu'ega ola.
Webhook() - auala autu, URL/pu'u webhook:
faʻamaonia le resitalaina i luga o le auaunaga faʻasalalau (alu i le SubscriptionConfirmation function),
fa'agasolo mai upega tafa'ilagi (Gorecords function).
Galuega HmacSha256 ma HmacSha256hex o faʻatinoga o le HMAC-SHA256 ma le HMAC-SHA256 faʻailoga algorithms faʻatasi ai ma gaioiga o se manoa o numera hexadecimal mo le fuafuaina o le saini.
autu o le galuega autu, fa'agasolo laina laina fa'atonu ma resitalaina tagata e fa'afoeina URL.
Fa'atonu laina laina e talia e le 'au'aunaga:
-taulaga o le taulaga lea o le a faʻalogo ai le 'auʻaunaga.
-tuatusi - tuatusi IP o le a faʻalogo i ai le 'auʻaunaga.
-script o se polokalame i fafo e valaʻau mo matau taʻitasi e sau.
Sei o tatou tilotilo totoa i nisi o galuega tauave:
//Webhook
func Webhook(w http.ResponseWriter, req *http.Request) {
// Read body
body, err := ioutil.ReadAll(req.Body)
defer req.Body.Close()
if err != nil {
http.Error(w, err.Error(), 500)
return
}
// log request
log.Printf("[%s] incoming HTTP request from %sn", req.Method, req.RemoteAddr)
// check if we got subscription confirmation request
if strings.Contains(string(body),
""Type":"SubscriptionConfirmation"") {
SubscriptionConfirmation(w, req, body)
} else {
GotRecords(w, req, body)
}
}
O lenei galuega e iloa ai pe o se talosaga e faʻamaonia le resitalaina poʻo se webhook ua taunuu. E pei ona mulimuli mai mai fa'amaumauga, pe a faʻamaonia le resitalaina, o le fausaga Json o loʻo i lalo e maua i le talosaga a le Post:
POST http://test.com HTTP/1.1
x-amz-sns-messages-type: SubscriptionConfirmation
content-type: application/json
{
"Timestamp":"2019-12-26T19:29:12+03:00",
"Type":"SubscriptionConfirmation",
"Message":"You have chosen to subscribe to the topic $topic. To confirm the subscription you need to response with calculated signature",
"TopicArn":"mcs2883541269|bucketA|s3:ObjectCreated:Put",
"SignatureVersion":1,
"Token":«RPE5UuG94rGgBH6kHXN9FUPugFxj1hs2aUQc99btJp3E49tA»
}
E tusa ai, faʻalagolago i le talosaga, e tatau ona e malamalama i le faʻaogaina o faʻamaumauga. Sa ou filifilia le tusi e fai ma faailoga "Type":"SubscriptionConfirmation", talu ai o loʻo i ai i le talosaga faʻamaonia o le saofaga ma e le o iai i le webhook. Faʻavae i luga o le iai / leai o lenei faʻamatalaga i le POST talosaga, o le faʻatinoina o le polokalame e alu i le galuega SubscriptionConfirmation, pe i totonu o se galuega GotRecords.
O le a matou le iloiloina auiliili le galuega o le SubscriptionConfirmation; o loʻo faʻatinoina e tusa ai ma mataupu faavae o loʻo tuʻuina atu i totonu fa'amaumauga. E mafai ona e va'ai i le source code mo lenei galuega ile poloketi git faleoloa.
O le galuega a le GotRecords e faʻasalalau le talosaga o loʻo oʻo mai ma mo mea faʻamaumauga taʻitasi e taʻua se faʻamatalaga fafo (o lona igoa na pasia i le -script parameter) faʻatasi ai ma tapulaʻa:
igoa pakete
mea ki
gaioiga:
kopi - pe afai i le uluai talosaga EventName = ObjectCreated | PutObject | PutObjectCopy
tape - pe afai i le uluai talosaga EventName = ObjectRemoved | DeleteObject
O lea la, afai e sau se matau ma se talosaga Post, e pei ona faamatalaina maualuga, ma le parakalafa -script=script.sh ona taʻua lea o le tusitusiga e pei ona taua i lalo:
script.sh bucketA some-file-to-bucket copy
E tatau ona malamalama o lenei webhook receiving server e le o se fofo atoatoa o le gaosiga, ae o se faʻataʻitaʻiga faigofie o se faʻatinoga talafeagai.
Faataitaiga o galuega
Sei o tatou faamaopoopo faila mai le pakete autu i le MCS i le pakete faaleoleo i le AWS. O le pakete autu e taʻua o myfiles-ash, o le faʻamaumauga e taʻua myfiles-backup (faʻapipiʻi pakete i le AWS e sili atu i le lautele o lenei tusiga). E tusa ai, pe a tuʻuina se faila i totonu o le pakete autu, e tatau ona faʻaalia lana kopi i le faʻamaumauga, ma pe a tape ese mai le autu, e tatau ona tape i le faʻamaumauga.
O le a matou galulue faʻatasi ma pakete e faʻaaoga ai le awscli utility, lea e fetaui ma MCS cloud storage ma AWS cloud storage.
ubuntu@ubuntu-basic-1-2-10gb:~$ sudo apt-get install awscli
Reading package lists... Done
Building dependency tree
Reading state information... Done
After this operation, 34.4 MB of additional disk space will be used.
Unpacking awscli (1.14.44-1ubuntu1) ...
Setting up awscli (1.14.44-1ubuntu1) ...
Sei o tatou fetuutuunai le avanoa i le S3 MCS API:
ubuntu@ubuntu-basic-1-2-10gb:~$ aws configure --profile mcs
AWS Access Key ID [None]: hdywEPtuuJTExxxxxxxxxxxxxx
AWS Secret Access Key [None]: hDz3SgxKwXoxxxxxxxxxxxxxxxxxx
Default region name [None]:
Default output format [None]:
Sei o tatou fetuutuunai le avanoa i le AWS S3 API:
ubuntu@ubuntu-basic-1-2-10gb:~$ aws configure --profile aws
AWS Access Key ID [None]: AKIAJXXXXXXXXXXXX
AWS Secret Access Key [None]: dfuerphOLQwu0CreP5Z8l5fuXXXXXXXXXXXXXXXX
Default region name [None]:
Default output format [None]:
Sei o tatou siaki avanoa:
I le AWS:
ubuntu@ubuntu-basic-1-2-10gb:~$ aws s3 ls --profile aws
2020-07-06 08:44:11 myfiles-backup
Mo MCS, pe a faʻatinoina le poloaiga e te manaʻomia e faʻaopoopo -endpoint-url:
Sei o tatou vaai pe faapefea ona aoga. E ala i MCS upega tafa'ilagi fa'aopoopo le faila test.txt i le pakete myfiles-ash. O fa'amaumauga o fa'amafanafanaga o lo'o fa'aalia ai na faia se talosaga i le 'au'aunaga webhook:
2020/07/06 09:43:08 [POST] incoming HTTP request from
95.163.216.92:56612
download: s3://myfiles-ash/test.txt to ../../../tmp/myfiles-ash/test.txt
upload: ../../../tmp/myfiles-ash/test.txt to
s3://myfiles-backup/test.txt
Sei o tatou siaki mea o loʻo i totonu o le pakete myfiles-backup i le AWS:
I le taimi nei, e ala i le upega tafaʻilagi, matou te tapeina le faila mai le pakete myfiles-ash.
Fa'amaumauga o le server:
2020/07/06 09:44:46 [POST] incoming HTTP request from
95.163.216.92:58224
delete: s3://myfiles-backup/test.txt
pakete i totonu:
ubuntu@ubuntu-basic-1-2-10gb:~/s3-webhook$ aws s3 --profile aws ls
myfiles-backup
ubuntu@ubuntu-basic-1-2-10gb:~$
Ua tape le faila, ua foia le faafitauli.
Fa'ai'uga ma Fai
O tulafono uma o loʻo faʻaaogaina i lenei tusiga o le i totonu o laʻu fale teu oloa. E iai foʻi faʻataʻitaʻiga o tusitusiga ma faʻataʻitaʻiga o le faitauina o saini mo le resitalaina o webhooks.
O lenei tulafono e leai se mea e sili atu nai lo se faʻataʻitaʻiga o le auala e mafai ai ona e faʻaogaina S3 webhooks i au gaioiga. E pei ona ou fai atu i le amataga, afai e te fuafua e faʻaaoga sea server i le gaosiga, e tatau ona e toe tusi le server mo galuega asynchronous: resitala webhooks sau i totonu o se laina (RabbitMQ poʻo NATS), ma mai iina faʻasalalau ma faʻagasolo. ma talosaga a tagata faigaluega. A leai, pe a o'o mai webhooks tele, e te ono feagai ma le le lava o punaoa a le server e fa'auma ai galuega. O le i ai o laina e mafai ai e oe ona tufatufa atu le 'auʻaunaga ma tagata faigaluega, faʻapea foʻi ma le foia o faʻafitauli i le toe faia o galuega i le tulaga o le toilalo. E fautuaina foi le suia o le logging i se mea e sili atu ona auiliili ma sili atu ona faʻamaonia.