Ua aʻo au i kēia mau haʻawina 6 o ka hana ʻana me ka cloudformation no ke koena o koʻu ola.

Ua hoʻomaka wau e hana me ʻōpua 4 makahiki aku nei. Mai ia manawa ua uhai au i nā ʻōnaehana he nui, ʻo nā mea i hana mua ʻia. Akā, i kēlā me kēia manawa aʻu e hana hewa ai, aʻo wau i kahi mea hou. Ma o kēia ʻike, e kaʻana aku au i kekahi o nā haʻawina koʻikoʻi aʻu i aʻo ai.

Haʻawina 1: E hoʻāʻo i nā loli ma mua o ka hoʻohana ʻana iā lākou

Ua aʻo wau i kēia haʻawina ma hope koke o koʻu hoʻomaka ʻana e hana me ʻōpua. ʻAʻole wau e hoʻomanaʻo i ka mea aʻu i haki ai i kēlā manawa, akā hoʻomanaʻo maoli wau ua hoʻohana wau i ke kauoha hōʻano hou aws cloudformation. Hoʻopuka wale kēia kauoha i ka template me ka ʻole o ka hōʻoia ʻana i nā loli e kau ʻia. Manaʻo wau ʻaʻole pono kahi wehewehe no ke kumu e hoʻāʻo ai ʻoe i nā hoʻololi āpau ma mua o ka kau ʻana iā lākou.

Ma hope o kēia hāʻule, ua hoʻololi koke wau ka pipeline pili, e hoʻololi ana i ke kauoha hou me ke kauoha hana-hoʻololi-hoʻonoho

# OPERATION is either "UPDATE" or "CREATE"
changeset_id=$(aws cloudformation create-change-set 
    --change-set-name "$CHANGE_SET_NAME" 
    --stack-name "$STACK_NAME" 
    --template-body "$TPL_PATH" 
    --change-set-type "$OPERATION" 
    --parameters "$PARAMETERS" 
    --output text 
    --query Id)

aws cloudformation wait 
    change-set-create-complete --change-set-name "$changeset_id"

Ke hoʻokumu ʻia kahi hoʻololi, ʻaʻohe hopena i ka waihona i loaʻa. ʻAʻole like me ke kauoha hou, ʻaʻole hoʻomaka ka hoʻololi ʻana i ka hoʻonohonoho maoli. Akā, hana ia i kahi papa inoa o nā loli i hiki iā ʻoe ke loiloi ma mua o ka hoʻolaha ʻana. Hiki iā ʻoe ke nānā i nā hoʻololi i ka aws console interface. Akā inā makemake ʻoe e hoʻokaʻawale i nā mea āpau āu e hiki ai, a laila e nānā iā lākou i ka CLI:

# this command is presented only for demonstrational purposes.
# the real command should take pagination into account
aws cloudformation describe-change-set 
    --change-set-name "$changeset_id" 
    --query 'Changes[*].ResourceChange.{Action:Action,Resource:ResourceType,ResourceId:LogicalResourceId,ReplacementNeeded:Replacement}' 
    --output table

Pono kēia kauoha e hoʻopuka i nā huahana e like me kēia:

--------------------------------------------------------------------
|                         DescribeChangeSet                        |
+---------+--------------------+----------------------+------------+
| Action  | ReplacementNeeded  |      Resource        | ResourceId |
+---------+--------------------+----------------------+------------+
|  Modify | True               |  AWS::ECS::Cluster   |  MyCluster |
|  Replace| True               |  AWS::RDS::DBInstance|  MyDB      |
|  Add    | None               |  AWS::SNS::Topic     |  MyTopic   |
+---------+--------------------+----------------------+------------+

E nānā pono i nā hoʻololi i kahi o Action e puku i, e kāpae i aiʻole ma hea Pono ka hoʻololi - ʻoiaʻiʻo. ʻO kēia nā hoʻololi weliweli loa a alakaʻi pinepine i ka nalowale o ka ʻike.

Ke nānā ʻia nā hoʻololi, hiki ke kau ʻia

aws cloudformation execute-change-set --change-set-name "$changeset_id"

operation_lowercase=$(echo "$OPERATION" | tr '[:upper:]' '[:lower:]')
aws cloudformation wait "stack-${operation_lowercase}-complete" 
    --stack-name "$STACK_NAME"

Haʻawina 2: E hoʻohana i ke kulekele hoʻopaʻa no ka pale ʻana i nā kumuwaiwai kūlana mai ka hoʻololi ʻana a i ʻole ka wehe ʻana

I kekahi manawa ʻaʻole lawa ka nānā wale ʻana i nā loli. He kanaka mākou a pau a hana hewa mākou. Ma hope koke iho o ko mākou hoʻomaka ʻana i ka hoʻohana ʻana i nā hoʻololi, ua hana koʻu hoa hui me ka ʻike ʻole i kahi hoʻolālā i hopena i kahi hōʻano hou. ʻAʻohe mea ʻino i hana ʻia no ka mea he wahi hoʻāʻo.

ʻOiai ua hōʻike kā mākou mau palapala i kahi papa inoa o nā hoʻololi a noi ʻia no ka hōʻoia ʻana, ua hoʻokuʻu ʻia ka hoʻololi Hoʻololi no ka nui o ka papa inoa o nā hoʻololi ʻaʻole i kūpono i ka pale. A no ka mea he mea maʻamau kēia i ka hoʻāʻo ʻana, ʻaʻole nui ka nānā ʻana i nā loli.

Aia nā kumuwaiwai āu e makemake ʻole e pani a wehe paha. He mau lawelawe piha keia, e like me ka RDS database instance a i ole elasticsearch cluster, etc. ʻO ka mea pōmaikaʻi, loaʻa i ka cloudformation kahi ala i kūkulu ʻia e hana ai i kēia. Kapa ʻia kēia i ka stack policy, a hiki iā ʻoe ke heluhelu hou aʻe e pili ana iā ia ma palapala:

STACK_NAME=$1
RESOURCE_ID=$2

POLICY_JSON=$(cat <<EOF
{
    "Statement" : [{
        "Effect" : "Deny",
        "Action" : [
            "Update:Replace",
            "Update:Delete"
        ],
        "Principal": "*",
        "Resource" : "LogicalResourceId/$RESOURCE_ID"
    }]
}
EOF
)

aws cloudformation set-stack-policy --stack-name "$STACK_NAME" 
    --stack-policy-body "$POLICY_JSON"

Haʻawina 3: E hoʻohana i UsePreviousValue i ka wā e hoʻohou ana i kahi ahu me nā ʻāpana huna

Ke hana ʻoe i kahi hui RDS mysql, koi ʻo AWS iā ʻoe e hāʻawi i kahi MasterUsername a me MasterUserPassword. No ka mea ʻoi aku ka maikaʻi o ka mālama ʻole ʻana i nā mea huna i ka code kumu a makemake wau e hoʻokaʻawale i nā mea āpau, ua hoʻokō wau i kahi "mekanika akamai" kahi ma mua o ka hoʻoili ʻia ʻana e loaʻa ai nā hōʻoia mai s3, a inā ʻaʻole ʻike ʻia nā hōʻoia, hana ʻia nā hōʻoia hou a mālama ʻia ma s3.

A laila e hāʻawi ʻia kēia mau hōʻoia e like me nā ʻāpana i ke kauoha cloudformation create-change-set. ʻOiai e hoʻāʻo ana me ka palapala, ua nalowale ka pilina me s3, a ua mālama ʻia kaʻu "mekanika akamai" i hōʻailona e hana i nā hōʻoia hou.

Inā hoʻomaka wau e hoʻohana i kēia palapala i ka hana ʻana a ua hoʻi hou ka pilikia pili, e hōʻano hou i ka waihona me nā hōʻoia hou. I kēia hihia, ʻaʻohe mea ʻino e hiki mai. Eia naʻe, haʻalele wau i kēia ala a hoʻomaka e hoʻohana i kahi ʻē aʻe, e hāʻawi ana i nā hōʻoia i hoʻokahi wale nō - i ka wā e hana ai i ka waihona. A ma hope aku, i ka wā e pono ai ka hoʻopaʻa ʻana i ka hoʻonui ʻana, ma kahi o ka wehewehe ʻana i ka waiwai huna o ka parameter, e hoʻohana wale wau UsePreviousValue=ʻoiaʻiʻo:

aws cloudformation create-change-set 
    --change-set-name "$CHANGE_SET_NAME" 
    --stack-name "$STACK_NAME" 
    --template-body "$TPL_PATH" 
    --change-set-type "UPDATE" 
    --parameters "ParameterKey=MasterUserPassword,UsePreviousValue=true"

Haʻawina 4: E hoʻohana i ka hoʻonohonoho rollback

Ua hoʻohana kekahi hui aʻu i hana ai i ka hana ʻōpua, kaheaia hoʻonohonoho hoʻihoʻi. ʻAʻole au i ʻike ma mua a ua ʻike koke wau e ʻoi aku ka maʻalili o ka lawe ʻana i kaʻu mau puʻupuʻu. I kēia manawa ke hoʻohana nei au i kēlā me kēia manawa aʻu e kau ai i kaʻu code i lambda a i ʻole ECS me ka hoʻohana ʻana i ka cloudformation.

Pehea e hana ai: kuhikuhi ʻoe CloudWatch alarm ma ka palena --rollback-hoʻonohonohoke hana ʻoe i kahi hoʻololi. Ma hope aku, ke hoʻokō ʻoe i kahi hoʻololi, nānā ʻo aws i ka ʻōhua no hoʻokahi minuke. Hoʻihoʻi ia i ka hoʻolālā inā hoʻololi ke ala i ka ALARM i kēia manawa.

Aia ma lalo kahi hiʻohiʻona o kahi ʻāpana template ʻōpuakahi a'u e hana ai ʻōpua ʻōpua, ka mea e hahai ana i ka metric mea hoʻohana kapua e like me ka helu o nā hewa i loko o nā moʻolelo ao (hoʻokumu ʻia ka metric ma o MetricFilter):

Resources:
  # this metric tracks number of errors in the cloudwatch logs. In this
  # particular case it's assumed logs are in json format and the error logs are
  # identified by level "error". See FilterPattern
  ErrorMetricFilter:
    Type: AWS::Logs::MetricFilter
    Properties:
      LogGroupName: !Ref LogGroup
      FilterPattern: !Sub '{$.level = "error"}'
      MetricTransformations:
      - MetricNamespace: !Sub "${AWS::StackName}-log-errors"
        MetricName: Errors
        MetricValue: 1
        DefaultValue: 0

  ErrorAlarm:
    Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmName: !Sub "${AWS::StackName}-errors"
      Namespace: !Sub "${AWS::StackName}-log-errors"
      MetricName: Errors
      Statistic: Maximum
      ComparisonOperator: GreaterThanThreshold
      Period: 1 # 1 minute
      EvaluationPeriods: 1
      Threshold: 0
      TreatMissingData: notBreaching
      ActionsEnabled: yes

Ano alarm hiki ke hoʻohana e like me huli hou hoʻomaka i ka wā e hoʻokō ai i ka pahu mea hana:

ALARM_ARN=$1

ROLLBACK_TRIGGER=$(cat <<EOF
{
  "RollbackTriggers": [
    {
      "Arn": "$ALARM_ARN",
      "Type": "AWS::CloudWatch::Alarm"
    }
  ],
  "MonitoringTimeInMinutes": 1
}
EOF
)

aws cloudformation create-change-set 
    --change-set-name "$CHANGE_SET_NAME" 
    --stack-name "$STACK_NAME" 
    --template-body "$TPL_PATH" 
    --change-set-type "UPDATE" 
    --rollback-configuration "$ROLLBACK_TRIGGER"

Haʻawina 5: E hōʻoia ʻoe e kau i ka mana hou loa o ka template

He mea maʻalahi ke kau ʻana i kahi mana liʻiliʻi ma mua o ka mea hou loa o ka template cloudformation, akā ʻo ka hana ʻana pēlā e hōʻeha nui ai. Ua loaʻa kēia iā mākou i hoʻokahi manawa: ʻaʻole i koi ka mea hoʻomohala i nā hoʻololi hou loa mai Git a me ka ʻike ʻole i kau ʻia i kahi mana mua o ka waihona. Ua hopena kēia i ka manawa haʻahaʻa no ka noi i hoʻohana i kēia waihona.

ʻO kahi mea maʻalahi e like me ka hoʻohui ʻana i kahi kikoʻī e ʻike ai inā paʻa ka lālā ma mua o ka hoʻokō ʻana iā ia e maikaʻi (manaʻo ʻia ʻo git kāu mea hana mana mana):

git fetch
HEADHASH=$(git rev-parse HEAD)
UPSTREAMHASH=$(git rev-parse master@{upstream})

if [[ "$HEADHASH" != "$UPSTREAMHASH" ]] ; then
   echo "Branch is not up to date with origin. Aborting"
   exit 1
fi

Haʻawina 6: Mai hana hou i ka huila

Me he mea lā e kau ʻia ana me ʻōpua - he maʻalahi. Pono ʻoe i kahi pūʻulu o nā palapala bash e hoʻokō nei i nā kauoha aws cli.

4 mau makahiki i hala aku nei ua hoʻomaka wau me nā palapala maʻalahi i kapa ʻia ʻo aws cloudformation create-stack command. ʻAʻole maʻalahi ka palapala. ʻO kēlā me kēia haʻawina i aʻo ʻia i ʻoi aku ka paʻakikī o ka palapala. ʻAʻole paʻakikī wale nō, akā piha pū kekahi i nā pōpoki.

Ke hana nei au ma kahi keʻena IT liʻiliʻi. Ua hōʻike ʻia ka ʻike e loaʻa i kēlā me kēia hui kona ala ponoʻī e hoʻokau i nā pūʻulu cloudformation. A ʻino kēlā. ʻOi aku ka maikaʻi inā e hana like nā mea a pau. ʻO ka mea pōmaikaʻi, he nui nā mea hana e kōkua iā ʻoe e hoʻonohonoho a hoʻonohonoho i nā waihona cloudformation.

E kōkua kēia mau haʻawina iā ʻoe e pale i nā hewa.

Source: www.habr.com