Apeere ti ohun elo-iṣẹlẹ ti o da lori awọn kio wẹẹbu ni ibi ipamọ ohun S3 Mail.ru Awọn solusan awọsanma

Rube Goldberg kofi ẹrọ

Iṣẹlẹ-ìṣó faaji mu ki awọn iye owo ṣiṣe ti awọn oro ti a lo nitori won ti wa ni lilo nikan ni akoko nigba ti won nilo. Awọn aṣayan pupọ lo wa lori bii o ṣe le ṣe eyi ati pe ko ṣẹda awọn ohun elo awọsanma ni afikun bi awọn ohun elo oṣiṣẹ. Ati loni Emi kii yoo sọrọ nipa FaaS, ṣugbọn nipa webhooks. Emi yoo ṣe afihan apẹẹrẹ ikẹkọ ti mimu awọn iṣẹlẹ mu nipa lilo awọn oju opo wẹẹbu ibi ipamọ ohun.

Awọn ọrọ diẹ nipa ibi ipamọ ohun ati awọn kio wẹẹbu. Ibi ipamọ ohun n gba ọ laaye lati tọju eyikeyi data sinu awọsanma ni irisi awọn nkan, wiwọle nipasẹ S3 tabi API miiran (da lori imuse) nipasẹ HTTP/HTTPS. Awọn kio wẹẹbu jẹ awọn atunyin HTTP aṣa ni gbogbogbo. Wọn jẹ okunfa nigbagbogbo nipasẹ iṣẹlẹ kan, gẹgẹbi koodu titari si ibi ipamọ tabi asọye ti a firanṣẹ lori bulọọgi kan. Nigbati iṣẹlẹ kan ba waye, aaye ipilẹṣẹ nfi ibeere HTTP ranṣẹ si URL ti a pato fun kio wẹẹbu naa. Bi abajade, o le ṣe awọn iṣẹlẹ lori aaye kan nfa awọn iṣe lori omiiran (wiki). Ninu ọran nibiti aaye orisun jẹ ibi ipamọ ohun kan, awọn iṣẹlẹ ṣiṣẹ bi awọn ayipada si awọn akoonu rẹ.

Awọn apẹẹrẹ ti awọn ọran ti o rọrun nigbati iru adaṣe le ṣee lo:

1. Ṣiṣẹda awọn ẹda ti gbogbo awọn nkan ni ibi ipamọ awọsanma miiran. Awọn adakọ gbọdọ wa ni ṣiṣẹda lori fo nigbakugba ti awọn faili ti wa ni afikun tabi yipada.
2. Ṣiṣẹda adaṣe ti lẹsẹsẹ awọn eekanna atanpako ti awọn faili ayaworan, fifi awọn ami omi kun awọn fọto, ati awọn iyipada aworan miiran.
3. Ifitonileti nipa dide ti awọn iwe aṣẹ tuntun (fun apẹẹrẹ, iṣẹ ṣiṣe iṣiro pinpin pinpin awọn ijabọ si awọsanma, ati ibojuwo owo n gba awọn iwifunni nipa awọn ijabọ tuntun, ṣayẹwo ati itupalẹ wọn).
4. Awọn ọran ti o ni idiju diẹ diẹ pẹlu, fun apẹẹrẹ, ti ipilẹṣẹ ibeere kan si Kubernetes, eyiti o ṣẹda podu kan pẹlu awọn apoti to wulo, gbe awọn aye-ṣiṣe iṣẹ-ṣiṣe lọ si rẹ, ati lẹhin sisẹ naa ṣubu eiyan naa.

Gẹgẹbi apẹẹrẹ, a yoo ṣe iyatọ ti iṣẹ-ṣiṣe 1, nigbati awọn iyipada ninu apo ipamọ ohun elo Mail.ru Cloud Solutions (MCS) ti wa ni mimuuṣiṣẹpọ ni ibi ipamọ ohun elo AWS nipa lilo awọn oju-iwe ayelujara. Ninu ọran ti kojọpọ gidi, iṣẹ asynchronous yẹ ki o pese nipasẹ fiforukọṣilẹ awọn oju opo wẹẹbu ni isinyi, ṣugbọn fun iṣẹ ikẹkọ a yoo ṣe imuse laisi eyi.

Eto iṣẹ

Ilana ibaraenisepo jẹ apejuwe ni awọn alaye ni Itọsọna si S3 webhooks lori MCS. Ilana iṣẹ ni awọn eroja wọnyi:

• Iṣẹ atẹjade, eyiti o wa ni ẹgbẹ ibi ipamọ S3 ati titẹjade awọn ibeere HTTP nigbati webnhook ti nfa.
• Webhook gbigba olupin, eyiti o tẹtisi awọn ibeere lati iṣẹ titẹjade HTTP ati ṣiṣe awọn iṣe ti o yẹ. Olupin naa le kọ ni eyikeyi ede; ninu apẹẹrẹ wa, a yoo kọ olupin ni Go.

Ẹya pataki ti imuse ti webhooks ni S3 API ni iforukọsilẹ ti olupin gbigba wẹẹbu lori iṣẹ titẹjade. Ni pataki, olupin gbigba wẹẹbu gbọdọ jẹrisi ṣiṣe-alabapin si awọn ifiranṣẹ lati iṣẹ titẹjade (ninu awọn imuṣẹ webhook miiran, ijẹrisi ṣiṣe alabapin ko nilo nigbagbogbo).

Nitorinaa, olupin gbigba wẹẹbu gbọdọ ṣe atilẹyin awọn iṣẹ akọkọ meji:

• dahun si ibeere iṣẹ titẹjade lati jẹrisi iforukọsilẹ,
• ilana ti nwọle iṣẹlẹ.

Fifi sori ẹrọ olupin gbigba wẹẹbu kan

Lati ṣiṣẹ olupin gbigba wẹẹbu, o nilo olupin Linux kan. Ninu nkan yii, gẹgẹbi apẹẹrẹ, a lo apẹẹrẹ foju kan ti a gbe sori MCS.

Jẹ ki a fi sọfitiwia pataki sori ẹrọ ki o ṣe ifilọlẹ olupin gbigba webhook.

ubuntu@ubuntu-basic-1-2-10gb:~$ sudo apt-get install git
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
  bc dns-root-data dnsmasq-base ebtables landscape-common liblxc-common 
liblxc1 libuv1 lxcfs lxd lxd-client python3-attr python3-automat 
python3-click python3-constantly python3-hyperlink
  python3-incremental python3-pam python3-pyasn1-modules 
python3-service-identity python3-twisted python3-twisted-bin 
python3-zope.interface uidmap xdelta3
Use 'sudo apt autoremove' to remove them.
Suggested packages:
  git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui 
gitk gitweb git-cvs git-mediawiki git-svn
The following NEW packages will be installed:
  git
0 upgraded, 1 newly installed, 0 to remove and 46 not upgraded.
Need to get 3915 kB of archives.
After this operation, 32.3 MB of additional disk space will be used.
Get:1 http://MS1.clouds.archive.ubuntu.com/ubuntu bionic-updates/main 
amd64 git amd64 1:2.17.1-1ubuntu0.7 [3915 kB]
Fetched 3915 kB in 1s (5639 kB/s)
Selecting previously unselected package git.
(Reading database ... 53932 files and directories currently installed.)
Preparing to unpack .../git_1%3a2.17.1-1ubuntu0.7_amd64.deb ...
Unpacking git (1:2.17.1-1ubuntu0.7) ...
Setting up git (1:2.17.1-1ubuntu0.7) ...

Di folda naa pẹlu olupin gbigba wẹẹbu:

ubuntu@ubuntu-basic-1-2-10gb:~$ git clone
https://github.com/RomanenkoDenys/s3-webhook.git
Cloning into 's3-webhook'...
remote: Enumerating objects: 48, done.
remote: Counting objects: 100% (48/48), done.
remote: Compressing objects: 100% (27/27), done.
remote: Total 114 (delta 20), reused 45 (delta 18), pack-reused 66
Receiving objects: 100% (114/114), 23.77 MiB | 20.25 MiB/s, done.
Resolving deltas: 100% (49/49), done.

Jẹ ki a bẹrẹ olupin naa:

ubuntu@ubuntu-basic-1-2-10gb:~$ cd s3-webhook/
ubuntu@ubuntu-basic-1-2-10gb:~/s3-webhook$ sudo ./s3-webhook -port 80

Ṣiṣe alabapin si iṣẹ titẹjade

O le forukọsilẹ olupin gbigba wẹẹbu rẹ nipasẹ API tabi wiwo wẹẹbu. Fun irọrun, a yoo forukọsilẹ nipasẹ wiwo wẹẹbu:

1. Jẹ ki a lọ si apakan awọn buckets ninu yara iṣakoso.
2. Lọ si garawa fun eyiti a yoo tunto webhooks ki o tẹ lori jia:

Lọ si Webhooks taabu ki o tẹ Fikun-un:

Fọwọsi awọn aaye:

ID - orukọ ti webhook.

Iṣẹlẹ - eyi ti awọn iṣẹlẹ lati atagba. A ti ṣeto awọn gbigbe ti gbogbo awọn iṣẹlẹ ti o waye nigba ṣiṣẹ pẹlu awọn faili (fikun ati pipaarẹ).

URL — webhook gbigba adirẹsi olupin.

Àlẹmọ ìpele / suffix jẹ àlẹmọ ti o fun ọ laaye lati ṣe ina awọn kio wẹẹbu nikan fun awọn nkan ti orukọ wọn baamu awọn ofin kan. Fun apẹẹrẹ, ni ibere fun webhook lati ma nfa awọn faili nikan pẹlu itẹsiwaju .png, ni Àlẹmọ suffix o nilo lati kọ "png".

Lọwọlọwọ, awọn ebute oko oju omi 80 ati 443 nikan ni atilẹyin fun iraye si olupin gbigba wẹẹbu.

Jẹ ki a tẹ Fi ìkọ kun ati pe a yoo rii nkan wọnyi:

Kio kun.

Olupin gbigba webhook fihan ninu awọn akọọlẹ rẹ ilọsiwaju ti ilana iforukọsilẹ kio:

ubuntu@ubuntu-basic-1-2-10gb:~/s3-webhook$ sudo ./s3-webhook -port 80
2020/06/15 12:01:14 [POST] incoming HTTP request from 
95.163.216.92:42530
2020/06/15 12:01:14 Got timestamp: 2020-06-15T15:01:13+03:00 TopicArn: 
mcs5259999770|myfiles-ash|s3:ObjectCreated:*,s3:ObjectRemoved:* Token: 
E2itMqAMUVVZc51pUhFWSp13DoxezvRxkUh5P7LEuk1dEe9y URL: 
http://89.208.199.220/webhook
2020/06/15 12:01:14 Generate responce signature: 
3754ce36636f80dfd606c5254d64ecb2fd8d555c27962b70b4f759f32c76b66d

Iforukọsilẹ ti pari. Ni apakan ti o tẹle, a yoo wo isunmọ si algorithm ti iṣiṣẹ ti olupin gbigba wẹẹbu.

Apejuwe ti webhook gbigba olupin

Ninu apẹẹrẹ wa, olupin ti kọ ni Go. Jẹ ki a wo awọn ilana ipilẹ ti iṣẹ rẹ.

package main

// Generate hmac_sha256_hex
func HmacSha256hex(message string, secret string) string {
}

// Generate hmac_sha256
func HmacSha256(message string, secret string) string {
}

// Send subscription confirmation
func SubscriptionConfirmation(w http.ResponseWriter, req *http.Request, body []byte) {
}

// Send subscription confirmation
func GotRecords(w http.ResponseWriter, req *http.Request, body []byte) {
}

// Liveness probe
func Ping(w http.ResponseWriter, req *http.Request) {
    // log request
    log.Printf("[%s] incoming HTTP Ping request from %sn", req.Method, req.RemoteAddr)
    fmt.Fprintf(w, "Pongn")
}

//Webhook
func Webhook(w http.ResponseWriter, req *http.Request) {
}

func main() {

    // get command line args
    bindPort := flag.Int("port", 80, "number between 1-65535")
    bindAddr := flag.String("address", "", "ip address in dot format")
    flag.StringVar(&actionScript, "script", "", "external script to execute")
    flag.Parse()

    http.HandleFunc("/ping", Ping)
    http.HandleFunc("/webhook", Webhook)

log.Fatal(http.ListenAndServe(*bindAddr+":"+strconv.Itoa(*bindPort), nil))
}

Jẹ ki a wo awọn iṣẹ akọkọ:

• Ping () - ipa ọna ti o dahun nipasẹ URL/ping, imuse ti o rọrun julọ ti iwadii igbesi aye.
• Webhook() - ipa ọna akọkọ, URL/olutọju wẹẹbu:
  • jẹrisi iforukọsilẹ lori iṣẹ titẹjade (lọ si iṣẹ Ijẹrisi Alabapin),
  • lakọkọ ti nwọle webhooks (Gorecords iṣẹ).
• Awọn iṣẹ HmacSha256 ati HmacSha256hex jẹ awọn imuse ti HMAC-SHA256 ati HMAC-SHA256 algorithms fifi ẹnọ kọ nkan pẹlu iṣelọpọ bi okun ti awọn nọmba hexadecimal fun ṣiṣe iṣiro ibuwọlu naa.
• akọkọ jẹ iṣẹ akọkọ, awọn ilana laini laini aṣẹ ati awọn oluṣakoso URL forukọsilẹ.

Awọn paramita laini aṣẹ ti olupin gba:

• -ibudo ni ibudo lori eyi ti awọn olupin yoo gbọ.
• -adirẹsi - adiresi IP ti olupin yoo gbọ.
• -akosile jẹ ẹya ita eto ti o ti wa ni a npe ni fun kọọkan ti nwọle kio.

Jẹ ki a ṣe akiyesi diẹ ninu awọn iṣẹ ṣiṣe:

//Webhook
func Webhook(w http.ResponseWriter, req *http.Request) {

    // Read body
    body, err := ioutil.ReadAll(req.Body)
    defer req.Body.Close()
    if err != nil {
        http.Error(w, err.Error(), 500)
        return
    }

    // log request
    log.Printf("[%s] incoming HTTP request from %sn", req.Method, req.RemoteAddr)
    // check if we got subscription confirmation request
    if strings.Contains(string(body), 
""Type":"SubscriptionConfirmation"") {
        SubscriptionConfirmation(w, req, body)
    } else {
        GotRecords(w, req, body)
    }

}

Iṣẹ yii pinnu boya ibeere lati jẹrisi iforukọsilẹ tabi kio wẹẹbu kan ti de. Bi atẹle lati iwe, ti iforukọsilẹ ba jẹ idaniloju, ilana Json atẹle ni a gba ninu ibeere Ifiweranṣẹ naa:

POST http://test.com HTTP/1.1
x-amz-sns-messages-type: SubscriptionConfirmation
content-type: application/json

{
    "Timestamp":"2019-12-26T19:29:12+03:00",
    "Type":"SubscriptionConfirmation",
    "Message":"You have chosen to subscribe to the topic $topic. To confirm the subscription you need to response with calculated signature",
    "TopicArn":"mcs2883541269|bucketA|s3:ObjectCreated:Put",
    "SignatureVersion":1,
    "Token":«RPE5UuG94rGgBH6kHXN9FUPugFxj1hs2aUQc99btJp3E49tA»
}

Ibeere yii nilo lati dahun:

content-type: application/json

{"signature":«ea3fce4bb15c6de4fec365d36bcebbc34ccddf54616d5ca12e1972f82b6d37af»}

Nibo ti ibuwọlu ti ṣe iṣiro bi:

signature = hmac_sha256(url, hmac_sha256(TopicArn, 
hmac_sha256(Timestamp, Token)))

Ti webhook kan ba de, eto ti ibeere Ifiweranṣẹ dabi eyi:

POST <url> HTTP/1.1
x-amz-sns-messages-type: SubscriptionConfirmation

{ "Records":
    [
        {
            "s3": {
                "object": {
                    "eTag":"aed563ecafb4bcc5654c597a421547b2",
                    "sequencer":1577453615,
                    "key":"some-file-to-bucket",
                    "size":100
                },
            "configurationId":"1",
            "bucket": {
                "name": "bucketA",
                "ownerIdentity": {
                    "principalId":"mcs2883541269"}
                },
                "s3SchemaVersion":"1.0"
            },
            "eventVersion":"1.0",
            "requestParameters":{
                "sourceIPAddress":"185.6.245.156"
            },
            "userIdentity": {
                "principalId":"2407013e-cbc1-415f-9102-16fb9bd6946b"
            },
            "eventName":"s3:ObjectCreated:Put",
            "awsRegion":"ru-msk",
            "eventSource":"aws:s3",
            "responseElements": {
                "x-amz-request-id":"VGJR5rtJ"
            }
        }
    ]
}

Nitorinaa, da lori ibeere naa, o nilo lati loye bi o ṣe le ṣe ilana data naa. Mo yan titẹ sii bi itọkasi "Type":"SubscriptionConfirmation", niwọn bi o ti wa ninu ibeere ijẹrisi ṣiṣe alabapin ati pe ko si ninu kio wẹẹbu naa. Da lori wiwa / isansa ti titẹsi yii ninu ibeere POST, ipaniyan siwaju sii ti eto naa lọ boya si iṣẹ naa SubscriptionConfirmation, tabi sinu iṣẹ naa GotRecords.

A kii yoo gbero iṣẹ ṣiṣe Ṣiṣe alabapin ni awọn alaye; o ti ṣe imuse ni ibamu si awọn ipilẹ ti a ṣeto sinu iwe. O le wo koodu orisun fun iṣẹ yii ni ise agbese git ibi ipamọ.

Iṣẹ GotRecords ṣe alaye ibeere ti nwọle ati fun ohun Igbasilẹ kọọkan n pe iwe afọwọkọ ita (ẹniti orukọ rẹ ti kọja ni paramita -script) pẹlu awọn paramita:

• garawa orukọ
• bọtini ohun
• igbese:
  • daakọ - ti o ba wa ninu atilẹba ìbéèrè EventName = Nkan ti Ṣẹda | PutObject | PutObjectCopy
  • paarẹ - ti o ba wa ninu atilẹba ibeere EventName = Nkan Yiyọ | Nkan Parẹ

Nitorinaa, ti kio kan ba de pẹlu ibeere Ifiweranṣẹ, bi a ti ṣalaye ti o ga, ati paramita -script=script.sh lẹhinna ao pe akosile naa bi atẹle:

script.sh  bucketA some-file-to-bucket copy

O yẹ ki o loye pe olupin gbigba wẹẹbu yii kii ṣe ojutu iṣelọpọ pipe, ṣugbọn apẹẹrẹ irọrun ti imuse ti o ṣeeṣe.

Apẹẹrẹ ti iṣẹ

Jẹ ki a muuṣiṣẹpọ awọn faili lati inu garawa akọkọ ni MCS si garawa afẹyinti ni AWS. Garawa akọkọ ni a pe ni myfiles-ash, afẹyinti ni a pe ni myfiles-afẹyinti (iṣeto garawa ni AWS ti kọja ipari ti nkan yii). Gẹgẹ bẹ, nigba ti a ba gbe faili kan sinu garawa akọkọ, ẹda rẹ yẹ ki o han ninu apo afẹyinti, ati nigbati o ba ti paarẹ lati akọkọ, o yẹ ki o paarẹ ni afẹyinti.

A yoo ṣiṣẹ pẹlu awọn buckets nipa lilo ohun elo awscli, eyiti o ni ibamu pẹlu mejeeji MCS ipamọ awọsanma ati ibi ipamọ awọsanma AWS.

ubuntu@ubuntu-basic-1-2-10gb:~$ sudo apt-get install awscli
Reading package lists... Done
Building dependency tree
Reading state information... Done
After this operation, 34.4 MB of additional disk space will be used.
Unpacking awscli (1.14.44-1ubuntu1) ...
Setting up awscli (1.14.44-1ubuntu1) ...

Jẹ ki a tunto iraye si S3 MCS API:

ubuntu@ubuntu-basic-1-2-10gb:~$ aws configure --profile mcs
AWS Access Key ID [None]: hdywEPtuuJTExxxxxxxxxxxxxx
AWS Secret Access Key [None]: hDz3SgxKwXoxxxxxxxxxxxxxxxxxx
Default region name [None]:
Default output format [None]:

Jẹ ki a tunto iraye si AWS S3 API:

ubuntu@ubuntu-basic-1-2-10gb:~$ aws configure --profile aws
AWS Access Key ID [None]: AKIAJXXXXXXXXXXXX
AWS Secret Access Key [None]: dfuerphOLQwu0CreP5Z8l5fuXXXXXXXXXXXXXXXX
Default region name [None]:
Default output format [None]:

Jẹ ki a ṣayẹwo awọn wiwọle:

Si AWS:

ubuntu@ubuntu-basic-1-2-10gb:~$ aws s3 ls --profile aws
2020-07-06 08:44:11 myfiles-backup

Fun MCS, nigbati o ba nṣiṣẹ aṣẹ o nilo lati ṣafikun — endpoint-url:

ubuntu@ubuntu-basic-1-2-10gb:~$ aws s3 ls --profile mcs --endpoint-url 
https://hb.bizmrg.com
2020-02-04 06:38:05 databasebackups-0cdaaa6402d4424e9676c75a720afa85
2020-05-27 10:08:33 myfiles-ash

Wọle si.

Bayi jẹ ki a kọ iwe afọwọkọ kan fun sisẹ kio ti nwọle, jẹ ki a pe ni s3_backup_mcs_aws.sh

#!/bin/bash
# Require aws cli
# if file added — copy it to backup bucket
# if file removed — remove it from backup bucket
# Variables
ENDPOINT_MCS="https://hb.bizmrg.com"
AWSCLI_MCS=`which aws`" --endpoint-url ${ENDPOINT_MCS} --profile mcs s3"
AWSCLI_AWS=`which aws`" --profile aws s3"
BACKUP_BUCKET="myfiles-backup"

SOURCE_BUCKET=""
SOURCE_FILE=""
ACTION=""

SOURCE="s3://${SOURCE_BUCKET}/${SOURCE_FILE}"
TARGET="s3://${BACKUP_BUCKET}/${SOURCE_FILE}"
TEMP="/tmp/${SOURCE_BUCKET}/${SOURCE_FILE}"

case ${ACTION} in
    "copy")
    ${AWSCLI_MCS} cp "${SOURCE}" "${TEMP}"
    ${AWSCLI_AWS} cp "${TEMP}" "${TARGET}"
    rm ${TEMP}
    ;;

    "delete")
    ${AWSCLI_AWS} rm ${TARGET}
    ;;

    *)
    echo "Usage: 
#!/bin/bash
# Require aws cli
# if file added — copy it to backup bucket
# if file removed — remove it from backup bucket
# Variables
ENDPOINT_MCS="https://hb.bizmrg.com"
AWSCLI_MCS=`which aws`" --endpoint-url ${ENDPOINT_MCS} --profile mcs s3"
AWSCLI_AWS=`which aws`" --profile aws s3"
BACKUP_BUCKET="myfiles-backup"
SOURCE_BUCKET="${1}"
SOURCE_FILE="${2}"
ACTION="${3}"
SOURCE="s3://${SOURCE_BUCKET}/${SOURCE_FILE}"
TARGET="s3://${BACKUP_BUCKET}/${SOURCE_FILE}"
TEMP="/tmp/${SOURCE_BUCKET}/${SOURCE_FILE}"
case ${ACTION} in
"copy")
${AWSCLI_MCS} cp "${SOURCE}" "${TEMP}"
${AWSCLI_AWS} cp "${TEMP}" "${TARGET}"
rm ${TEMP}
;;
"delete")
${AWSCLI_AWS} rm ${TARGET}
;;
*)
echo "Usage: ${0} sourcebucket sourcefile copy/delete"
exit 1
;;
esac
 sourcebucket sourcefile copy/delete"
    exit 1
    ;;
esac

Jẹ ki a bẹrẹ olupin naa:

ubuntu@ubuntu-basic-1-2-10gb:~/s3-webhook$ sudo ./s3-webhook -port 80 -
script scripts/s3_backup_mcs_aws.sh

Jẹ ká wo bi o ti ṣiṣẹ. Nipasẹ MCS ayelujara ni wiwo fi test.txt faili si myfiles-ash garawa. Awọn akọọlẹ console fihan pe a ṣe ibeere kan si olupin webhook:

2020/07/06 09:43:08 [POST] incoming HTTP request from 
95.163.216.92:56612
download: s3://myfiles-ash/test.txt to ../../../tmp/myfiles-ash/test.txt
upload: ../../../tmp/myfiles-ash/test.txt to 
s3://myfiles-backup/test.txt

Jẹ ki a ṣayẹwo awọn akoonu ti apo-afẹyinti myfiles ni AWS:

ubuntu@ubuntu-basic-1-2-10gb:~/s3-webhook$ aws s3 --profile aws ls 
myfiles-backup
2020-07-06 09:43:10       1104 test.txt

Bayi, nipasẹ wiwo wẹẹbu, a yoo paarẹ faili naa lati inu garawa myfiles-ash.

Awọn akọọlẹ olupin:

2020/07/06 09:44:46 [POST] incoming HTTP request from 
95.163.216.92:58224
delete: s3://myfiles-backup/test.txt

Awọn akoonu inu garawa:

ubuntu@ubuntu-basic-1-2-10gb:~/s3-webhook$ aws s3 --profile aws ls 
myfiles-backup
ubuntu@ubuntu-basic-1-2-10gb:~$

Faili naa ti paarẹ, iṣoro naa ti yanju.

Ipari ati ToDo

Gbogbo koodu ti a lo ninu nkan yii jẹ ninu mi ibi ipamọ. Awọn apẹẹrẹ awọn iwe afọwọkọ tun wa ati awọn apẹẹrẹ ti kika awọn ibuwọlu fun iforukọsilẹ awọn kio wẹẹbu.

Koodu yii kii ṣe diẹ sii ju apẹẹrẹ ti bii o ṣe le lo awọn iwo wẹẹbu S3 ninu awọn iṣẹ rẹ. Gẹgẹbi Mo ti sọ ni ibẹrẹ, ti o ba gbero lati lo iru olupin bẹ ni iṣelọpọ, o nilo lati tun atunkọ olupin naa fun iṣẹ asynchronous: forukọsilẹ awọn oju opo wẹẹbu ti nwọle ni isinyi (RabbitMQ tabi NATS), ati lati ibẹ sọ wọn ki o ṣe ilana wọn. pẹlu Osise ohun elo. Bibẹẹkọ, nigbati awọn kio wẹẹbu ba de lọpọlọpọ, o le ba pade aini awọn orisun olupin lati pari awọn iṣẹ ṣiṣe. Iwaju awọn ila n gba ọ laaye lati kaakiri olupin ati awọn oṣiṣẹ, bi daradara bi yanju awọn iṣoro pẹlu awọn iṣẹ ṣiṣe atunwi ni ọran ti awọn ikuna. O tun ni imọran lati yi gedu pada si alaye diẹ sii ati idiwon diẹ sii.

Ti o dara orire!

Kika diẹ sii lori koko:

• Nṣiṣẹ pẹlu Mail.ru Awọn solusan awọsanma S3 ipamọ ohun elo bi eto faili kan.
• Bawo ni ipamọ ohun ti a lo lati lu idije naa.

orisun: www.habr.com