ProHoster > Pūnaewele > Nā Administration > ʻO kahi laʻana o kahi noi i alakaʻi ʻia i ka hanana e pili ana i nā webhooks i ka mālama ʻana i nā mea S3 Mail.ru Cloud Solutions

ʻO kahi laʻana o kahi noi i alakaʻi ʻia i ka hanana e pili ana i nā webhooks i ka mālama ʻana i nā mea S3 Mail.ru Cloud Solutions

ʻO kahi laʻana o kahi noi i alakaʻi ʻia i ka hanana e pili ana i nā webhooks i ka mālama ʻana i nā mea S3 Mail.ru Cloud Solutions
ʻO ka mīkini kope ʻo Rube Goldberg

Hoʻonui ka hoʻolālā hanana hanana i ka maikaʻi o ke kumukūʻai o nā kumuwaiwai i hoʻohana ʻia no ka mea hoʻohana wale ʻia i ka manawa e pono ai. Nui nā koho e pili ana i ka hoʻokō ʻana i kēia a ʻaʻole e hana i nā hui kapua hou e like me nā noi limahana. A i kēia lā ʻaʻole wau e kamaʻilio e pili ana i ka FaaS, akā e pili ana i nā webhooks. E hōʻike wau i kahi laʻana aʻoaʻo o ka lawelawe ʻana i nā hanana me ka hoʻohana ʻana i nā webhooks mālama mea.

He mau huaʻōlelo e pili ana i ka mālama ʻana i nā mea a me ka hoʻopaʻa pūnaewele. Hiki iā ʻoe ke mālama i nā ʻikepili i ke ao ma ke ʻano o nā mea, hiki ke loaʻa ma o S3 a i ʻole kekahi API (ma muli o ka hoʻokō) ma HTTP/HTTPS. ʻO nā Webhooks ka maʻamau HTTP callbacks. Hoʻomaka pinepine ʻia lākou e kahi hanana, e like me ke kaomi ʻana i ke code i kahi waihona a i ʻole kahi ʻōlelo i kau ʻia ma kahi blog. Ke loaʻa kahi hanana, hoʻouna ka pūnaewele kumu i kahi noi HTTP i ka URL i kuhikuhi ʻia no ka webhook. ʻO ka hopena, hiki iā ʻoe ke hana i nā hanana ma kahi pūnaewele e hoʻoulu i nā hana ma kekahi (wiki). Inā he waihona mea ka waihona kumu, hana nā hanana ma ke ʻano he hoʻololi i kāna ʻike.

ʻO nā hiʻohiʻona o nā hihia maʻalahi ke hoʻohana ʻia ia ʻano automation:

  1. Ke hana nei i kope o nā mea a pau i loko o kahi waihona kapuaʻi ʻē aʻe. Pono e hana ʻia nā kope ma ka lele ke hoʻohui ʻia a hoʻololi ʻia paha nā faila.
  2. Hoʻokumu ʻakomi i nā ʻano liʻiliʻi o nā faila kiʻi, hoʻohui i nā kaha wai i nā kiʻi, a me nā hoʻololi kiʻi ʻē aʻe.
  3. Hoʻolaha e pili ana i ka hōʻea ʻana mai o nā palapala hou (no ka laʻana, hoʻoili ʻia kahi lawelawe helu helu i hāʻawi ʻia i nā hōʻike i ke ao, a loaʻa ka nānā kālā i nā leka e pili ana i nā hōʻike hou, nānā a nānā ʻana iā lākou).
  4. ʻO nā hihia paʻakikī liʻiliʻi e pili ana, no ka laʻana, ka hoʻopuka ʻana i kahi noi iā Kubernetes, ka mea i hana i kahi pod me nā ipu e pono ai, hāʻawi i nā ʻāpana hana iā ia, a ma hope o ka hana ʻana e hāʻule ka ipu.

E like me ka laʻana, e hana mākou i kahi ʻokoʻa o ka hana 1, i ka wā e hoʻololi ʻia ai ka pahu mālama mea Mail.ru Cloud Solutions (MCS) i loko o ka waihona mea AWS me ka hoʻohana ʻana i nā webhooks. Ma kahi hihia i hoʻouka ʻia, pono e hāʻawi ʻia ka hana asynchronous ma ke kau inoa ʻana i nā webhooks i kahi pila, akā no ka hana aʻo e hana mākou i ka hoʻokō me ka ʻole o kēia.

Papa hana o ka hana

Ua wehewehe kikoʻī ʻia ka protocol interaction ma Alakaʻi i nā S3 webhooks ma MCS. Aia ka papahana hana i kēia mau mea:

  • Oihana hoolaha, aia ma ka ʻaoʻao mālama S3 a hoʻopuka i nā noi HTTP ke hoʻomaka ka webnhook.
  • Ke kikowaena loaʻa pūnaewele, e hoʻolohe ana i nā noi mai ka lawelawe hoʻopuka HTTP a hana i nā hana kūpono. Hiki ke kākau ʻia ke kikowaena ma kekahi ʻōlelo; ma kā mākou hiʻohiʻona, e kākau mākou i ke kikowaena ma Go.

ʻO kahi hiʻohiʻona kūikawā o ka hoʻokō ʻana i nā webhooks ma ka S3 API ʻo ia ke kākau inoa ʻana o ka server loaʻa webhook ma ka lawelawe hoʻolaha. ʻO ka mea nui, pono e hoʻopaʻa inoa ka mea hoʻohana pūnaewele hoʻolaha i ka hoʻopaʻa inoa ʻana i nā memo mai ka lawelawe hoʻopuka (ma nā hoʻokō webhook ʻē aʻe, ʻaʻole koi ʻia ka hōʻoia o ke kau inoa).

No laila, pono e kākoʻo ke kikowaena loaʻa webhook i ʻelua hana nui:

  • e pane i ka noi a ka lawelawe hoʻopuka e hōʻoia i ke kākau inoa ʻana,
  • kaʻina hana e hiki mai ana.

Ke kau ʻana i kahi kikowaena loaʻa pūnaewele

No ka holo ʻana i ke kikowaena loaʻa webhook, pono ʻoe i kahi kikowaena Linux. Ma kēia ʻatikala, ma ke ʻano he laʻana, hoʻohana mākou i kahi hiʻohiʻona virtual a mākou e kau ai ma MCS.

E hoʻokomo i ka lako polokalamu pono a hoʻomaka i ke kikowaena loaʻa webhook.

ubuntu@ubuntu-basic-1-2-10gb:~$ sudo apt-get install git
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
  bc dns-root-data dnsmasq-base ebtables landscape-common liblxc-common 
liblxc1 libuv1 lxcfs lxd lxd-client python3-attr python3-automat 
python3-click python3-constantly python3-hyperlink
  python3-incremental python3-pam python3-pyasn1-modules 
python3-service-identity python3-twisted python3-twisted-bin 
python3-zope.interface uidmap xdelta3
Use 'sudo apt autoremove' to remove them.
Suggested packages:
  git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui 
gitk gitweb git-cvs git-mediawiki git-svn
The following NEW packages will be installed:
  git
0 upgraded, 1 newly installed, 0 to remove and 46 not upgraded.
Need to get 3915 kB of archives.
After this operation, 32.3 MB of additional disk space will be used.
Get:1 http://MS1.clouds.archive.ubuntu.com/ubuntu bionic-updates/main 
amd64 git amd64 1:2.17.1-1ubuntu0.7 [3915 kB]
Fetched 3915 kB in 1s (5639 kB/s)
Selecting previously unselected package git.
(Reading database ... 53932 files and directories currently installed.)
Preparing to unpack .../git_1%3a2.17.1-1ubuntu0.7_amd64.deb ...
Unpacking git (1:2.17.1-1ubuntu0.7) ...
Setting up git (1:2.17.1-1ubuntu0.7) ...

Hoʻopili i ka waihona me ke kikowaena loaʻa webhook:

ubuntu@ubuntu-basic-1-2-10gb:~$ git clone
https://github.com/RomanenkoDenys/s3-webhook.git
Cloning into 's3-webhook'...
remote: Enumerating objects: 48, done.
remote: Counting objects: 100% (48/48), done.
remote: Compressing objects: 100% (27/27), done.
remote: Total 114 (delta 20), reused 45 (delta 18), pack-reused 66
Receiving objects: 100% (114/114), 23.77 MiB | 20.25 MiB/s, done.
Resolving deltas: 100% (49/49), done.

E hoʻomaka kākou i ke kikowaena:

ubuntu@ubuntu-basic-1-2-10gb:~$ cd s3-webhook/
ubuntu@ubuntu-basic-1-2-10gb:~/s3-webhook$ sudo ./s3-webhook -port 80

E kau inoa i ka lawelawe hoʻolaha

Hiki iā ʻoe ke hoʻopaʻa inoa i kāu kikowaena loaʻa webhook ma o ka API a i ʻole ke kikowaena pūnaewele. No ka maʻalahi, e kākau inoa mākou ma o ka pūnaewele pūnaewele:

  1. E hele kāua i ka ʻāpana bakeke i loko o ka lumi hoʻomalu.
  2. E hele i ka bākeke a mākou e hoʻonohonoho ai i nā webhooks a kaomi i ka mea hana:

ʻO kahi laʻana o kahi noi i alakaʻi ʻia i ka hanana e pili ana i nā webhooks i ka mālama ʻana i nā mea S3 Mail.ru Cloud Solutions

E hele i ka ʻaoʻao Webhooks a kaomi iā Add:

ʻO kahi laʻana o kahi noi i alakaʻi ʻia i ka hanana e pili ana i nā webhooks i ka mālama ʻana i nā mea S3 Mail.ru Cloud Solutions
E hoʻopiha i nā kahua:

ʻO kahi laʻana o kahi noi i alakaʻi ʻia i ka hanana e pili ana i nā webhooks i ka mālama ʻana i nā mea S3 Mail.ru Cloud Solutions

ID - ka inoa o ka webhook.

Ka hanana - nā hanana e hoʻouna. Ua hoʻonohonoho mākou i ka hoʻouna ʻana i nā hanana āpau i ka wā e hana ai me nā faila (hoʻohui a holoi ʻia).

HKH - loaʻa ka helu kikowaena pūnaewele hoʻopaʻa inoa.

ʻO ka kānana prefix/suffix kahi kānana e hiki ai iā ʻoe ke hana i nā webhooks wale nō no nā mea nona nā inoa i kūlike i kekahi mau lula. No ka laʻana, i mea e hoʻomaka ai ka webhook i nā faila wale nō me ka hoʻonui .png, ma kānana suffix pono ʻoe e kākau i ka "png".

I kēia manawa, kākoʻo wale ʻia nā awa 80 a me 443 no ke komo ʻana i ke kikowaena loaʻa webhook.

E kaomi kāua Hoʻohui i ka makau a e ike kakou i keia.

ʻO kahi laʻana o kahi noi i alakaʻi ʻia i ka hanana e pili ana i nā webhooks i ka mālama ʻana i nā mea S3 Mail.ru Cloud Solutions
Hoʻohui ʻia ʻo Hook.

Hōʻike ke kikowaena loaʻa pūnaewele i loko o kāna mau lāʻau i ka holomua o ke kaʻina hoʻopaʻa inoa hook:

ubuntu@ubuntu-basic-1-2-10gb:~/s3-webhook$ sudo ./s3-webhook -port 80
2020/06/15 12:01:14 [POST] incoming HTTP request from 
95.163.216.92:42530
2020/06/15 12:01:14 Got timestamp: 2020-06-15T15:01:13+03:00 TopicArn: 
mcs5259999770|myfiles-ash|s3:ObjectCreated:*,s3:ObjectRemoved:* Token: 
E2itMqAMUVVZc51pUhFWSp13DoxezvRxkUh5P7LEuk1dEe9y URL: 
http://89.208.199.220/webhook
2020/06/15 12:01:14 Generate responce signature: 
3754ce36636f80dfd606c5254d64ecb2fd8d555c27962b70b4f759f32c76b66d

Ua pau ke kakau inoa. Ma ka ʻāpana aʻe, e nānā pono mākou i ka algorithm o ka hana ʻana o ka server loaʻa webhook.

ʻO ka wehewehe ʻana i ke kikowaena loaʻa pūnaewele

I kā mākou laʻana, ua kākau ʻia ke kikowaena ma Go. E nānā kākou i nā kumu kumu o kāna hana.

package main

// Generate hmac_sha256_hex
func HmacSha256hex(message string, secret string) string {
}

// Generate hmac_sha256
func HmacSha256(message string, secret string) string {
}

// Send subscription confirmation
func SubscriptionConfirmation(w http.ResponseWriter, req *http.Request, body []byte) {
}

// Send subscription confirmation
func GotRecords(w http.ResponseWriter, req *http.Request, body []byte) {
}

// Liveness probe
func Ping(w http.ResponseWriter, req *http.Request) {
    // log request
    log.Printf("[%s] incoming HTTP Ping request from %sn", req.Method, req.RemoteAddr)
    fmt.Fprintf(w, "Pongn")
}

//Webhook
func Webhook(w http.ResponseWriter, req *http.Request) {
}

func main() {

    // get command line args
    bindPort := flag.Int("port", 80, "number between 1-65535")
    bindAddr := flag.String("address", "", "ip address in dot format")
    flag.StringVar(&actionScript, "script", "", "external script to execute")
    flag.Parse()

    http.HandleFunc("/ping", Ping)
    http.HandleFunc("/webhook", Webhook)

log.Fatal(http.ListenAndServe(*bindAddr+":"+strconv.Itoa(*bindPort), nil))
}

E noʻonoʻo i nā hana nui:

  • Ping () - he ala e pane ai ma o URL / ping, ka hoʻokō maʻalahi o kahi ʻimi ola.
  • Webhook() - ala nui, URL/mea hoʻohana pūnaewele:
    • hōʻoia i ka hoʻopaʻa inoa ma ka lawelawe hoʻopuka (e hele i ka hana SubscriptionConfirmation),
    • kaʻina hana i nā webhooks (hana Gorecords).
  • ʻO nā hana HmacSha256 a me HmacSha256hex nā hoʻokō o ka HMAC-SHA256 a me HMAC-SHA256 algorithms encryption me ka hoʻopuka ma ke ʻano he kaula o nā helu hexadecimal no ka helu ʻana i ka pūlima.
  • ʻO ka mea nui ka hana nui, kaʻina hana i nā palena laina kauoha a hoʻopaʻa inoa i nā mea lawelawe URL.

ʻAe ʻia nā ʻāpana laina kauoha e ke kikowaena:

  • -port ke awa kahi e hoʻolohe ai ke kikowaena.
  • -address - IP address e hoʻolohe ai ke kikowaena.
  • -script he polokalamu waho ia i kapa ʻia no kēlā me kēia makau e komo mai.

E nānā pono kākou i kekahi o nā hana:

//Webhook
func Webhook(w http.ResponseWriter, req *http.Request) {

    // Read body
    body, err := ioutil.ReadAll(req.Body)
    defer req.Body.Close()
    if err != nil {
        http.Error(w, err.Error(), 500)
        return
    }

    // log request
    log.Printf("[%s] incoming HTTP request from %sn", req.Method, req.RemoteAddr)
    // check if we got subscription confirmation request
    if strings.Contains(string(body), 
""Type":"SubscriptionConfirmation"") {
        SubscriptionConfirmation(w, req, body)
    } else {
        GotRecords(w, req, body)
    }

}

Hoʻoholo kēia hana inā ua hiki mai kahi noi e hōʻoia i ke kākau inoa a i ʻole kahi webhook. Penei mai palapala, inā hoʻopaʻa inoa ʻia, loaʻa kēia ʻano Json ma ka noi Post:

POST http://test.com HTTP/1.1
x-amz-sns-messages-type: SubscriptionConfirmation
content-type: application/json

{
    "Timestamp":"2019-12-26T19:29:12+03:00",
    "Type":"SubscriptionConfirmation",
    "Message":"You have chosen to subscribe to the topic $topic. To confirm the subscription you need to response with calculated signature",
    "TopicArn":"mcs2883541269|bucketA|s3:ObjectCreated:Put",
    "SignatureVersion":1,
    "Token":«RPE5UuG94rGgBH6kHXN9FUPugFxj1hs2aUQc99btJp3E49tA»
}

Pono e pane ʻia kēia nīnau:

content-type: application/json

{"signature":«ea3fce4bb15c6de4fec365d36bcebbc34ccddf54616d5ca12e1972f82b6d37af»}

Kahi i helu ʻia ai ka pūlima penei:

signature = hmac_sha256(url, hmac_sha256(TopicArn, 
hmac_sha256(Timestamp, Token)))

Inā hiki mai kahi webhook, ua like ke ʻano o ka noi Post:

POST <url> HTTP/1.1
x-amz-sns-messages-type: SubscriptionConfirmation

{ "Records":
    [
        {
            "s3": {
                "object": {
                    "eTag":"aed563ecafb4bcc5654c597a421547b2",
                    "sequencer":1577453615,
                    "key":"some-file-to-bucket",
                    "size":100
                },
            "configurationId":"1",
            "bucket": {
                "name": "bucketA",
                "ownerIdentity": {
                    "principalId":"mcs2883541269"}
                },
                "s3SchemaVersion":"1.0"
            },
            "eventVersion":"1.0",
            "requestParameters":{
                "sourceIPAddress":"185.6.245.156"
            },
            "userIdentity": {
                "principalId":"2407013e-cbc1-415f-9102-16fb9bd6946b"
            },
            "eventName":"s3:ObjectCreated:Put",
            "awsRegion":"ru-msk",
            "eventSource":"aws:s3",
            "responseElements": {
                "x-amz-request-id":"VGJR5rtJ"
            }
        }
    ]
}

No laila, ma muli o ka noi, pono ʻoe e hoʻomaopopo i ka hana ʻana i ka ʻikepili. Ua koho au i ke komo ʻana ma ke ʻano he hōʻailona "Type":"SubscriptionConfirmation", no ka mea aia i loko o ka noi hōʻoia inoa a ʻaʻole i loaʻa i ka webhook. Ma muli o ka loaʻa ʻana o kēia komo ʻana i ka noi POST, hele hou ka hoʻokō ʻana o ka papahana i ka hana. SubscriptionConfirmation, a i ʻole i loko o ka hana GotRecords.

ʻAʻole mākou e noʻonoʻo i ka hana SubscriptionConfirmation i nā kikoʻī; ua hoʻokō ʻia e like me nā loina i hoʻonohonoho ʻia palapala. Hiki iā ʻoe ke nānā i ke kumu kumu no kēia hana ma papahana git waihona.

Hoʻopili ka hana GotRecords i kahi noi komo a no kēlā me kēia mea Record e kāhea i kahi palapala waho (nona ka inoa i hāʻawi ʻia ma ka -script parameter) me nā ʻāpana:

  • inoa bakeke
  • mea kī
  • hana:
    • kope - inā ma ka noi kumu EventName = ObjectCreated | PutObject | PutObjectCopy
    • holoi - inā ma ka noi kumu EventName = ObjectRemoved | Holoi mea

No laila, inā hiki mai kahi makau me kahi noi Post, e like me ka mea i wehewehe ʻia kiekie, a me ka palena -script=script.sh a laila e kapa ʻia ka palapala penei:

script.sh  bucketA some-file-to-bucket copy

Pono e hoʻomaopopo ʻia ʻaʻole kēia kikowaena loaʻa webhook he hopena hana piha, akā he laʻana maʻalahi o kahi hoʻokō hiki.

Laʻana o ka hana

E hoʻonohonoho like i nā faila mai ka bākeke nui ma MCS a i ka bakeke hoʻihoʻi ma AWS. Ua kapa ʻia ka pahu nui ʻo myfiles-ash, ʻo ka mea hoʻihoʻi i kapa ʻia ʻo myfiles-backup (ʻo ka hoʻonohonoho ʻana o ka bakeke ma AWS ma waho o ke kiko o kēia ʻatikala). No laila, ke waiho ʻia kahi faila i loko o ka bākeke nui, pono e ʻike ʻia kāna kope i ka waihona kope, a i ka wā e holoi ʻia ai mai ka mea nui, pono e holoi ʻia ma ka waihona.

E hana mākou me nā bākeke me ka hoʻohana ʻana i ka pono awscli, i kūpono me ka MCS cloud storage a me AWS cloud storage.

ubuntu@ubuntu-basic-1-2-10gb:~$ sudo apt-get install awscli
Reading package lists... Done
Building dependency tree
Reading state information... Done
After this operation, 34.4 MB of additional disk space will be used.
Unpacking awscli (1.14.44-1ubuntu1) ...
Setting up awscli (1.14.44-1ubuntu1) ...

E hoʻonohonoho i ke komo ʻana i ka S3 MCS API:

ubuntu@ubuntu-basic-1-2-10gb:~$ aws configure --profile mcs
AWS Access Key ID [None]: hdywEPtuuJTExxxxxxxxxxxxxx
AWS Secret Access Key [None]: hDz3SgxKwXoxxxxxxxxxxxxxxxxxx
Default region name [None]:
Default output format [None]:

E hoʻonohonoho i ke komo ʻana i ka AWS S3 API:

ubuntu@ubuntu-basic-1-2-10gb:~$ aws configure --profile aws
AWS Access Key ID [None]: AKIAJXXXXXXXXXXXX
AWS Secret Access Key [None]: dfuerphOLQwu0CreP5Z8l5fuXXXXXXXXXXXXXXXX
Default region name [None]:
Default output format [None]:

E nānā kākou i nā mea komo:

I AWS:

ubuntu@ubuntu-basic-1-2-10gb:~$ aws s3 ls --profile aws
2020-07-06 08:44:11 myfiles-backup

No MCS, i ka holo ʻana i ke kauoha pono ʻoe e hoʻohui i —endpoint-url:

ubuntu@ubuntu-basic-1-2-10gb:~$ aws s3 ls --profile mcs --endpoint-url 
https://hb.bizmrg.com
2020-02-04 06:38:05 databasebackups-0cdaaa6402d4424e9676c75a720afa85
2020-05-27 10:08:33 myfiles-ash

Loaʻa.

I kēia manawa, e kākau kāua i palapala no ka hoʻoponopono ʻana i ka makau e komo mai ana, e kapa ʻia ʻo s3_backup_mcs_aws.sh

#!/bin/bash
# Require aws cli
# if file added — copy it to backup bucket
# if file removed — remove it from backup bucket
# Variables
ENDPOINT_MCS="https://hb.bizmrg.com"
AWSCLI_MCS=`which aws`" --endpoint-url ${ENDPOINT_MCS} --profile mcs s3"
AWSCLI_AWS=`which aws`" --profile aws s3"
BACKUP_BUCKET="myfiles-backup"

SOURCE_BUCKET=""
SOURCE_FILE=""
ACTION=""

SOURCE="s3://${SOURCE_BUCKET}/${SOURCE_FILE}"
TARGET="s3://${BACKUP_BUCKET}/${SOURCE_FILE}"
TEMP="/tmp/${SOURCE_BUCKET}/${SOURCE_FILE}"

case ${ACTION} in
    "copy")
    ${AWSCLI_MCS} cp "${SOURCE}" "${TEMP}"
    ${AWSCLI_AWS} cp "${TEMP}" "${TARGET}"
    rm ${TEMP}
    ;;

    "delete")
    ${AWSCLI_AWS} rm ${TARGET}
    ;;

    *)
    echo "Usage: 
#!/bin/bash
# Require aws cli
# if file added — copy it to backup bucket
# if file removed — remove it from backup bucket
# Variables
ENDPOINT_MCS="https://hb.bizmrg.com"
AWSCLI_MCS=`which aws`" --endpoint-url ${ENDPOINT_MCS} --profile mcs s3"
AWSCLI_AWS=`which aws`" --profile aws s3"
BACKUP_BUCKET="myfiles-backup"
SOURCE_BUCKET="${1}"
SOURCE_FILE="${2}"
ACTION="${3}"
SOURCE="s3://${SOURCE_BUCKET}/${SOURCE_FILE}"
TARGET="s3://${BACKUP_BUCKET}/${SOURCE_FILE}"
TEMP="/tmp/${SOURCE_BUCKET}/${SOURCE_FILE}"
case ${ACTION} in
"copy")
${AWSCLI_MCS} cp "${SOURCE}" "${TEMP}"
${AWSCLI_AWS} cp "${TEMP}" "${TARGET}"
rm ${TEMP}
;;
"delete")
${AWSCLI_AWS} rm ${TARGET}
;;
*)
echo "Usage: ${0} sourcebucket sourcefile copy/delete"
exit 1
;;
esac
 sourcebucket sourcefile copy/delete"
    exit 1
    ;;
esac

E hoʻomaka kākou i ke kikowaena:

ubuntu@ubuntu-basic-1-2-10gb:~/s3-webhook$ sudo ./s3-webhook -port 80 -
script scripts/s3_backup_mcs_aws.sh

E ʻike kākou pehea e hana ai. Ma o MCS punaewele e hoʻohui i ka faila test.txt i ka bakeke myfiles-ash. Hōʻike nā moʻolelo console ua noi ʻia i ke kikowaena webhook:

2020/07/06 09:43:08 [POST] incoming HTTP request from 
95.163.216.92:56612
download: s3://myfiles-ash/test.txt to ../../../tmp/myfiles-ash/test.txt
upload: ../../../tmp/myfiles-ash/test.txt to 
s3://myfiles-backup/test.txt

E nānā kākou i nā mea o ka waihona myfiles-backup ma AWS:

ubuntu@ubuntu-basic-1-2-10gb:~/s3-webhook$ aws s3 --profile aws ls 
myfiles-backup
2020-07-06 09:43:10       1104 test.txt

I kēia manawa, ma o ka pūnaewele pūnaewele, e holoi mākou i ka faila mai ka myfiles-ash bucket.

Nā moʻolelo kikowaena:

2020/07/06 09:44:46 [POST] incoming HTTP request from 
95.163.216.92:58224
delete: s3://myfiles-backup/test.txt

Pākeke maʻiʻo:

ubuntu@ubuntu-basic-1-2-10gb:~/s3-webhook$ aws s3 --profile aws ls 
myfiles-backup
ubuntu@ubuntu-basic-1-2-10gb:~$

Holoi ʻia ka faila, hoʻopau ʻia ka pilikia.

Ka hopena a me ka hana

ʻO nā code a pau i hoʻohana ʻia ma kēia ʻatikala i loko o koʻu waihona. Aia kekahi mau laʻana o nā palapala a me nā laʻana o ka helu ʻana i nā pūlima no ka hoʻopaʻa inoa ʻana i nā webhooks.

ʻAʻole kēia code ma mua o kahi hiʻohiʻona pehea e hiki ai iā ʻoe ke hoʻohana i nā webhooks S3 i kāu mau hana. E like me kaʻu i ʻōlelo ai i ka hoʻomaka, inā ʻoe e hoʻolālā e hoʻohana i ia kikowaena i ka hana ʻana, pono ʻoe e kākau hou i ka server no ka hana asynchronous: e hoʻopaʻa inoa i nā webhooks e komo mai ana i kahi queue (RabbitMQ a i ʻole NATS), a mai laila e hoʻokaʻawale iā lākou a hana iā lākou. me nā noi hana. A i ʻole, i ka hiki ʻana mai o nā webhooks nui, hiki iā ʻoe ke hālāwai me ka nele o nā kumuwaiwai kikowaena e hoʻopau i nā hana. ʻO ka hele ʻana o nā queues e ʻae iā ʻoe e puʻunaue i ke kikowaena a me nā limahana, a me ka hoʻoponopono ʻana i nā pilikia me ka hana hou ʻana i nā hana i ka wā o nā hemahema. He mea maikaʻi hoʻi e hoʻololi i ka logging i kahi kikoʻī a ʻoi aku ka maʻamau.

Pomaikai!

Heluhelu hou aku ma ke kumuhana:

Source: www.habr.com

Pākuʻi i ka manaʻo hoʻopuka