Kukhazikitsa ma docker pull and docker push commands popanda kasitomala wa docker pogwiritsa ntchito zopempha za HTTP

Tidali ndi matumba awiri a udzu, 2 mapiritsi a mescaline unix chilengedwe, malo osungiramo docker komanso ntchito yokhazikitsa malamulo a docker pull and docker push popanda kasitomala wa docker.

Kukhazikitsa ma docker pull and docker push commands popanda kasitomala wa docker pogwiritsa ntchito zopempha za HTTP

UPD:
Funso: Kodi zonsezi ndi za chiyani?
Yankho: Kuyesa kwa katundu (OSATI kugwiritsa ntchito bash, zolembazo zimaperekedwa pazolinga zamaphunziro). Zinasankhidwa kuti asagwiritse ntchito kasitomala wa docker kuti achepetse zigawo zowonjezera (m'malire oyenera) ndipo, motero, kutsanzira katundu wapamwamba. Zotsatira zake, kuchedwa konse kwa kasitomala wa Docker kudachotsedwa. Tinalandira katundu waukhondo mwachindunji pa mankhwala.
Nkhaniyi idagwiritsa ntchito zida za GNU.

Choyamba, tiyeni tione zimene malamulowa amachita.

Ndiye kodi docker pull imagwiritsidwa ntchito chiyani? Malinga ndi zolemba:

"Kokani chithunzi kapena chosungira kuchokera ku registry".

Kumeneko timapezanso ulalo wa kumvetsetsa zithunzi, zotengera, ndi madalaivala osungira.

Kukhazikitsa ma docker pull and docker push commands popanda kasitomala wa docker pogwiritsa ntchito zopempha za HTTP

Kuchokera apa titha kumvetsetsa kuti chithunzi cha docker ndi gulu la zigawo zina zomwe zili ndi chidziwitso chokhudza kusintha kwaposachedwa pachithunzichi, zomwe mwachiwonekere ndizomwe timafunikira. Kenako timayang'ana registry API.

Akuti:

"Chithunzi" ndi chophatikizira cha mawonekedwe a JSON ndi mafayilo osanjikiza paokha. Njira yokoka > chithunzi imayang'ana pochotsa zigawo ziwirizi."

Chifukwa chake gawo loyamba molingana ndi zolembazo ndi "Kukoka Chiwonetsero Chachithunzi".

Inde, sitidzawombera, koma timafunikira deta kuchokera pamenepo. Chotsatira ndi chitsanzo chopempha: GET /v2/{name}/manifests/{reference}

"Dzina ndi zolozerazo zimazindikiritsa chithunzicho ndipo zikufunika. Bukuli likhoza kukhala ndi tag kapena digest."

Malo athu a docker atumizidwa kwanuko, tiyeni tiyese kuchita zomwe tapempha:

curl -s -X GET "http://localhost:8081/link/to/docker/registry/v2/centos-11-10/manifests/1.1.1" -H "header_if_needed"

Kukhazikitsa ma docker pull and docker push commands popanda kasitomala wa docker pogwiritsa ntchito zopempha za HTTP

Poyankha, timalandira json komwe tili ndi chidwi ndi njira zamoyo, kapena ma hashi awo. Titawalandira, titha kudutsa iliyonse ndikuchita zotsatirazi: "GET /v2/{name}/blobs/{digest}"

"Kufikira pagawo kudzadziwika ndi dzina la malo osungiramo zinthu koma amadziwika mwapadera mu registry ndi digest."

Digest munkhaniyi ndi hashi yomwe tidalandira.

Kuyesera

curl -s -X GET "http://localhost:8081/link/to/docker/registry/v2/centos-11-10/blobs/sha256:f972d139738dfcd1519fd2461815651336ee25a8b54c358834c50af094bb262f" -H "header_if_needed" --output firstLayer

Kukhazikitsa ma docker pull and docker push commands popanda kasitomala wa docker pogwiritsa ntchito zopempha za HTTP

Tiyeni tiwone mtundu wa fayilo yomwe tidalandira pomaliza ngati njira yoyamba yopulumukira.

file firstLayer

Kukhazikitsa ma docker pull and docker push commands popanda kasitomala wa docker pogwiritsa ntchito zopempha za HTTP

izo. njanji ndi malo osungiramo phula, kuwamasula m'njira yoyenera tipeza zomwe zili pachithunzichi.

Tiyeni tilembe kalembedwe kakang'ono ka bash kuti zonsezi zitha kukhala zokha

#!/bin/bash -eu

downloadDir=$1
# url as http://localhost:8081/link/to/docker/registry
url=$2
imageName=$3
tag=$4

# array of layers
layers=($(curl -s -X GET "$url/v2/$imageName/manifests/$tag" | grep -oP '(?<=blobSum" : ").+(?=")'))

# download each layer from array
for layer in "${layers[@]}"; do
    echo "Downloading ${layer}"
    curl -v -X GET "$url/v2/$imageName/blobs/$layer" --output "$downloadDir/$layer.tar"
done

# find all layers, untar them and remove source .tar files
cd "$downloadDir" && find . -name "sha256:*" -exec tar xvf {} ;
rm sha256:*.tar
exit 0

Tsopano titha kuyendetsa ndi magawo omwe tikufuna ndikupeza zomwe zili pachithunzi chofunikira

./script.sh dirName β€œhttp://localhost:8081/link/to/docker/registry” myAwesomeImage 1.0

Gawo 2 - kukankha docker

Izi zidzakhala zovuta kwambiri.

Tiyeni tiyambenso zolemba. Chifukwa chake tiyenera kutsitsa mtsogoleri aliyense, sonkhanitsani chiwonetsero chofananira ndikutsitsanso. Zikuwoneka zosavuta.

Pambuyo pophunzira zolembedwa, titha kugawa njira yotsitsa m'njira zingapo:

  • Kuyambitsa ndondomeko - "POST /v2/{repoName}/blobs/uploads/"
  • Kukweza njira yopulumutsira moyo (tidzagwiritsa ntchito kukweza kwa monolithic, mwachitsanzo, timatumiza njira iliyonse yopulumutsira moyo) - "PUT /v2/{repoName}/blobs/uploads/{uuid}?digest={digest}
    Utali Wazinthu: {size of layer}
    Mtundu-Zamkati: kugwiritsa ntchito/octet-stream
    Layer Binary Data".
  • Kutsegula chiwonetsero - "PUT /v2/{repoName}/manifests/{reference}".

Koma zolembazo zimaphonya sitepe imodzi, popanda zomwe sizingagwire ntchito. Pakutsitsa kwa monolithic, komanso pang'ono (chunked), musanakweze njanji, muyenera kuchita pempho la PATCH:

"PATCH /v2/{repoName}/blobs/uploads/{uuid}
Utali Wazinthu: {kukula kwa chunk}
Mtundu-Zamkati: kugwiritsa ntchito/octet-stream
{Layer Chunk Binary Data}".

Apo ayi, simudzatha kupitirira mfundo yoyamba, chifukwa ... M'malo mwa nambala yoyembekezeka yoyankha 202, mudzalandira 4xx.

Tsopano algorithm ikuwoneka ngati:

  • Kuyambitsa
  • Patch njanji
  • Kutsegula panja
  • Kutsegula chiwonetsero
    Mfundo 2 ndi 3, motsatana, zidzabwerezedwa kangapo momwe mizere iyenera kuyikidwa.

Choyamba, timafunikira chithunzi chilichonse. Ndigwiritsa ntchito archlinux:posachedwa

docker pull archlinux

Kukhazikitsa ma docker pull and docker push commands popanda kasitomala wa docker pogwiritsa ntchito zopempha za HTTP

Tsopano tiyeni tisunge kwanuko kuti tiwunikenso

docker save c24fe13d37b9 -o savedArch

Kukhazikitsa ma docker pull and docker push commands popanda kasitomala wa docker pogwiritsa ntchito zopempha za HTTP

Tsegulani zosungidwa zomwe zatuluka mu chikwatu chomwe chilipo

tar xvf savedArch

Kukhazikitsa ma docker pull and docker push commands popanda kasitomala wa docker pogwiritsa ntchito zopempha za HTTP

Monga mukuwonera, mzere uliwonse wamoyo uli mufoda yosiyana. Tsopano tiyeni tione kamangidwe ka chiwonetsero chomwe tidalandira

cat manifest.json | json_pp

Kukhazikitsa ma docker pull and docker push commands popanda kasitomala wa docker pogwiritsa ntchito zopempha za HTTP

Osati kwenikweni. Tiyeni tiwone zomwe zimafunikira kuti zitsegulidwe, malinga ndi zolemba.

Kukhazikitsa ma docker pull and docker push commands popanda kasitomala wa docker pogwiritsa ntchito zopempha za HTTP

Mwachiwonekere, manifesto yomwe ilipo sagwirizana ndi ife, kotero tidzipanga tokha ndi blackjack ndi courtesans, lifelines and configs.

Nthawi zonse tidzakhala ndi fayilo imodzi yosinthira ndi mizere yamoyo. Scheme version 2 (panopa panthawi yolemba), mediaType idzasiyidwa yosasinthika:

echo β€˜{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
   "config": {
      "mediaType": "application/vnd.docker.container.image.v1+json",
      "size": config_size,
      "digest": "config_hash"
   },
   "layers": [
      ’ > manifest.json

Mukapanga chiwonetsero choyambira, muyenera kudzaza ndi data yoyenera. Kuti tichite izi, timagwiritsa ntchito json template ya chinthu cha njanji:

{
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": ${layersSizes[$i]},
         "digest": "sha256:${layersNames[$i]}"
      },

Tiziwonjezera ku chiwonetsero cha njanji iliyonse.

Kenako, tifunika kudziwa kukula kwa fayilo ya config ndikusintha ma stubs mu chiwonetserocho ndi deta yeniyeni

sed -i "s/config_size/$configSize/g; s/config_hash/$configName/g" $manifestFile

Tsopano mutha kuyambitsa kutsitsa ndikudzisungira nokha uuid, yomwe iyenera kutsagana ndi zopempha zonse zotsatila.

Script yathunthu ikuwoneka motere:

#!/bin/bash -eux

imageDir=$1
# url as http://localhost:8081/link/to/docker/registry
url=$2
repoName=$3
tag=$4
manifestFile=$(readlink -f ${imageDir}/manifestCopy)
configFile=$(readlink -f $(find $imageDir -name "*.json" ! -name "manifest.json"))

# calc layers sha 256 sum, rename them accordingly, and add info about each to manifest file
function prepareLayersForUpload() {
  info_file=$imageDir/info
  # lets calculate layers sha256 and use it as layers names further
  layersNames=($(find $imageDir -name "layer.tar" -exec shasum -a 256 {} ; | cut -d" " -f1))

  # rename layers according to shasums. !!!Set required amount of fields for cut command!!!
  # this part definitely can be done easier but i didn't found another way, sry
  find $imageDir -name "layer.tar" -exec bash -c 'mv {} "$(echo {} | cut -d"/" -f1,2)/$(shasum -a 256 {} | cut -d" " -f1)"' ;

  layersSizes=($(find $imageDir -name "*.tar" -exec ls -l {} ; | awk '{print $5}'))

  for i in "${!layersNames[@]}"; do
    echo "{
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": ${layersSizes[$i]},
         "digest": "sha256:${layersNames[$i]}"
      }," >> $manifestFile
  done
  # remove last ','
  truncate -s-2 $manifestFile
  # add closing brakets to keep json consistent
  printf "nt]n}" >> $manifestFile
}

# calc config sha 256 sum and add info about it to manifest
function setConfigProps() {
  configSize=$(ls -l $configFile | awk '{print $5}')
  configName=$(basename $configFile | cut -d"." -f1)

  sed -i "s/config_size/$configSize/g; s/config_hash/$configName/g" $manifestFile
}

#prepare manifest file
prepareLayersForUpload
setConfigProps
cat $manifestFile

# initiate upload and get uuid
uuid=$(curl -s -X POST -I "$url/v2/$repoName/blobs/uploads/" | grep -oP "(?<=Docker-Upload-Uuid: ).+")

# patch layers
# in data-binary we're getting absolute path to layer file
for l in "${!layersNames[@]}"; do
  pathToLayer=$(find $imageDir -name ${layersNames[$l]} -exec readlink -f {} ;)
    curl -v -X PATCH "$url/v2/$repoName/blobs/uploads/$uuid" 
  -H "Content-Length: ${layersSizes[$i]}" 
  -H "Content-Type: application/octet-stream" 
  --data-binary "@$pathToLayer"

# put layer
  curl -v -X PUT "$url/v2/$repoName/blobs/uploads/$uuid?digest=sha256:${layersNames[$i]}" 
  -H 'Content-Type: application/octet-stream' 
  -H "Content-Length: ${layersSizes[$i]}" 
  --data-binary "@$pathToLayer"
done

# patch and put config after all layers
curl -v -X PATCH "$url/v2/$repoName/blobs/uploads/$uuid" 
  -H "Content-Length: $configSize" 
  -H "Content-Type: application/octet-stream" 
  --data-binary "@$configFile"

  curl -v -X PUT "$url/v2/$repoName/blobs/uploads/$uuid?digest=sha256:$configName" 
  -H 'Content-Type: application/octet-stream' 
  -H "Content-Length: $configSize" 
  --data-binary "@$configFile"

# put manifest
curl -v -X PUT "$url/v2/$repoName/manifests/$tag" 
  -H 'Content-Type: application/vnd.docker.distribution.manifest.v2+json' 
  --data-binary "@$manifestFile"

exit 0

titha kugwiritsa ntchito script yopangidwa kale:

./uploadImage.sh "~/path/to/saved/image" "http://localhost:8081/link/to/docker/registry" myRepoName 1.0

UPD:
Nanga tinapeza zotani?
Choyamba, deta yeniyeni yowunikira, popeza mayesero amayendetsedwa mu blazemeter ndipo deta pa zopempha za makasitomala a docker sizophunzitsa kwambiri, mosiyana ndi zopempha zenizeni za HTTP.

Kachiwiri, kusinthaku kunatilola kuti tiwonjezere kuchuluka kwa ogwiritsa ntchito pa docker pafupifupi 150% ndikupeza nthawi yoyankhira 20-25% mwachangu. Pakutsitsa kwa docker, takwanitsa kuonjezera chiwerengero cha ogwiritsa ntchito ndi 500%, pomwe nthawi yoyankha yatsika ndi pafupifupi 60%.

Zikomo chifukwa tcheru chanu.

Source: www.habr.com

Kuwonjezera ndemanga