ืืื ืืืืจ!
ืฆื ืืืจ ืืื ืคืืืขื ืืืง ืขืจืคึผืืืื ื? ืืื ืืึธื ืืึธืก ืืื, ืึธืืขืจ ืืขืฉืึทืก ืืื-ืืคืืขืืื ืืขืจืืงืืื ืืื ืืืื ืืขืคืืื ืืื ืืืืข ืืื ืึทื ืึทืืืืืื ื ืืึทืื ืืืืฃ ืืืคื ืืืงืืฅ ืคืื ืืืื ืืึทืืืืกื ืืืื - Aviasales.
ืืืึทื ื ืืืจ ืืืขืื ืึทื ืึทืืืื ืื ืึทืจืืขื ืคืื Amazon Kinesis, ืืืืขื ืึท ืกืืจืืืื ื ืกืืกืืขื ืืื ืคืึทืงืืืฉ-ืฆืืื ืึทื ืึทืืืืืงืก, ืื ืกืืึทืืืจื ืื Amazon DynamoDB NoSQL ืืึทืืึทืืืืก ืืื ืื ืืืืคึผื ืืึทืื ืกืืึธืจืืืืฉ ืืื ืฉืืขืื SMS ื ืึธืืืึทืคืึทืงืืืฉืึทื ื ืคึฟืึทืจ ืืฉืืงืึทืืืข ืืืงืืฅ.
ืึทืืข ืื ืคืจืืื ืืขื ืขื ืืื ืืขืจ ืื ืฉื ืืึทืื! ืืื!
ืืงืืื
ืคึฟืึทืจ ืื ืืืึทืฉืคึผืื, ืืืจ ืืึทืจืคึฟื ืฆืืืจืื ืฆื . ืึทืงืกืขืก ืฆื ืขืก ืืื ืฆืืืขืฉืืขืื ืคืจืื ืืื ืึธื ืจืืกืืจืืงืฉืึทื ื; ืืืจ ื ืึธืจ ืืึทืจืคึฟื ืฆื ืคืึทืจืฉืจืืึทืื ืืื ืื "ืืขืืืขืืึธืคึผืขืจืก" ืึธืคึผืืืืืื ื ืฆื ืืึทืงืืืขื ืืืื ืึทืคึผื ืกืืืขื ืฆื ืึทืงืกืขืก ืื ืืึทืื.
ืืขืจ ืืืืคึผื ืฆืื ืคืื ืืขื ืึทืจืืืงื ืืื ืฆื ืืขืื ืึท ืืขื ืขืจืึทื ืคืืจืฉืืื ื ืคืื ืื ื ืืฆื ืคืื ืืื ืคึฟืึธืจืืึทืฆืืข ืกืืจืืืื ื ืืื AWS; ืืืจ ื ืขืืขื ืืื ืืฉืืื ืึทื ืื ืืึทืื ืืืืืขืงืขืจื ืืืจื ืื ืึทืคึผื ืืขืืืืื ื ืืขื ืขื ื ืืฉื ืฉืืจืขื ื ืืขืจืืืึทื ืืืงื ืืื ืืขื ืขื ืืจืึทื ืกืืืืืขื ืคึฟืื ืื ืงืึทืฉ. ืืขืืจืื ืืขื ืืืืืจื ืืืืฃ ืึธื ืคึฟืจืขืื ืืืจื ื ืืฆืขืจืก ืคืื ืื Aviasales.ru ืืื Jetradar.com ืืืืืืขื ืคึฟืึทืจ ืืขืฆืืข 48 ืฉืขื.
Kinesis-ืึทืืขื ื, ืืื ืกืืึทืืืจื ืืืืฃ ืื ืคึผืจืึทืืืกืื ื ืืึทืฉืื, ืืืงืืืขื ืืืจื ืื API ืืืขื ืืืืืึธืืึทืืืฉ ืคึผืึทืจืก ืืื ืืืขืจืฉืืงื ืืึทืื ืฆื ืื ืืขืืขืื ืืืึทื ืืืจื Kinesis Data Analytics. ืื ืจืื ืืืขืจืกืืข ืคืื โโืืขื ืืืึทื ืืืขื ืืืื ืืขืฉืจืืื ืืืืื ืฆื ืื ืงืจืึธื. ืื ืจืื ืืึทืื ืกืืึธืจืืืืฉ ืืืคึผืืืื ืืื DynamoDB ืืืขื ืืึธืื ืืืคึผืขืจ ืืืืขื ืึทื ืึทืืืกืืก ืืืจื BI ืืืฉืืจืื, ืึทืืึท ืืื AWS ืงืืืืง ืกืืื.
ืืืจ ืืืขืื ืืึทืืจืึทืืื ืฆืืืื ืึธืคึผืฆืืขืก ืคึฟืึทืจ ืืืคึผืืืืื ื ืื ืืื ืฆืข ืื ืคืจืึทืกืืจืึทืงืืฉืขืจ:
- ืืึทื ืืึทื - ืืืจื AWS ืืึทื ืึทืืขืืขื ื ืงืึทื ืกืึธืื;
- ืื ืคืจืึทืกืืจืึทืงืืฉืขืจ ืคึฟืื ืืขืจืจืึทืคืึธืจื ืงืึธื ืืื ืคึฟืึทืจ ืคืืื ืึธืืึทืืืืืขืจื;
ืึทืจืงืึทืืขืงืืฉืขืจ ืคืื ืื ืืขืืืขืืึธืคึผืขื ืกืืกืืขื
ืืื ืืฆื ืงืึทืืคึผืึธืื ืึทื ืฅ:
- - ืื ืืึทืื ืืืืืขืงืขืจื ืืืจื ืืขื ืึทืคึผื ืืืขื ืืืื ืืขืืืืื ื ืคึฟืึทืจ ืึทืืข ืกืึทืืกืึทืงืืืึทื ื ืึทืจืืขื;
- - ืึท ืจืขืืืืขืจ ืืืืจืืืึทื ืืึทืฉืื ืืื ืื ืืืึธืืงื ืืืืฃ ืืืึธืก ืื ืึทืจืืึทื ืฉืจืืึทื ืืึทืื ืืืึทื ืืืขื ืืืื ืืืฉืขื ืขืจืืืืึทื:
- ืืื ืึท Java ืึทืคึผืืึทืงืืืฉืึทื ืืื ืกืืึทืืืจื ืืึธืืงืึทืื ืืืืฃ ืื ืืึทืฉืื ืืืึธืก ืืื ืึทื ืืจืื ื ืืืขื ืฆื ืืึทืืืขื ืืื ืฉืืงื ืืึทืื ืฆื Kinesis (Kinesis Data Streams ืึธืืขืจ Kinesis Firehose). ืืขืจ ืึทืืขื ื ืงืขืกืืืืขืจ ืืึธื ืืืึธืจืก ืึท ืกืืื ืคืื ืืขืงืขืก ืืื ืื ืกืคึผืขืกืืคืืขื ืืืืจืขืงืืขืจืื ืืื ืกืขื ืื ื ืืึท ืืึทืื ืฆื ืงืื ืขืกืืก;
- - ื ืคึผืืืืึธื ืฉืจืืคื ืืืึธืก ืืืื ืจืืงืืืขืก ืฆื ืื ืึทืคึผื ืืื ืืืืื ืื ืขื ืืคืขืจ ืืื ืึท ืืขืงืข ืืืึธืก ืืื ืืึธื ืืืึธืจืขื ืืืจื ืื ืงืื ืขืกืืก ืึทืืขื ื;
- - ืคืึทืงืืืฉ-ืฆืืื ืืึทืื ืกืืจืืืื ื ืืื ืกื ืืื ืืจืืื ืกืงืืืืื ื ืงืืืคึผืึทืืืืึทืืื;
- ืืื ืึท ืกืขืจืืืขืจืืขืกืก ืืื ืกื ืืืึธืก ืกืืืคึผืืึทืคืืื ืื ืึทื ืึทืืืกืืก ืคืื ืกืืจืืืื ื ืืึทืื ืืื ืคืึทืงืืืฉ ืฆืืื. Amazon Kinesis Data Analytics ืงืึทื ืคืืืืขืจ ืึทืคึผืืึทืงืืืฉืึทื ืจืขืกืืจืกื ืืื ืืืืืึธืืึทืืืฉ ืืืึธื ืฆื ืฉืขืคึผื ืงืืื ืืึทื ื ืคืื ืื ืงืึทืืื ื ืืึทืื;
- - ืึท ืืื ืกื ืืืึธืก ืึทืืึทืื ืืืจ ืฆื ืืืืคื ืงืึธื ืึธื ืืึทืงืื ื ืึธืืขืจ ืืึทืฉืืขืืืงื ืกืขืจืืืขืจืก. ืึทืืข ืงืึทืืคึผืืืืื ื ืืึทืื ืืื ืืืืืึธืืึทืืืฉ ืกืงืืืื ืคึฟืึทืจ ืืขืืขืจ ืจืืคื;
- - ื ืืึทืืึทืืืืก ืคืื ืฉืืืกื-ืืืขืจื ืคึผืขืจื ืืื ืืึธืงืืืขื ืื ืืืึธืก ืืื ืืืืืึทื ืกื ืคืื ืืืืื ืืงืขืจ ืืื 10 ืืืืืกืขืงืึทื ืื ืืืขื ืคืืืกื ืืืง ืืื ืงืืื ืืืึธื. ืืืขื ืืืจ ื ืืฆื DynamoDB, ืืืจ ืืึธื ื ืื ืืึทืจืคึฟื ืฆื ืฆืืฉืืขืื, ืืึทืืข ืึธืืขืจ ืคืืจื ืงืืื ืกืขืจืืืขืจืก. DynamoDB ืกืงืึทืืขืก ืืืืืึธืืึทืืืฉ ืืืฉื ืฆื ืกืืจืืืขืจื ืื ืกืืืข ืคืื โโืคืึทืจืึทื ืขื ืจืขืกืืจืกื ืืื ืืึทืืื ืืืื ืคืึธืจืฉืืขืืื ื. ืงืืื ืกืืกืืขื ืึทืืืื ืืกืืจืึทืฆืืข ืืื ืคืืจืืื ืื;
- - ืึท ืืึธืจ ืืขืจืืื ืืื ืกื ืคึฟืึทืจ ืฉืืงื ืึทืจืืืงืืขื ืืื ืื ืึทืจืืืกืืขืืขืจ-ืึทืืึธื ืขื ื (ืคึผืื / ืกืื) ืืึธืืขื, ืืื ืืืึธืก ืืืจ ืงืขื ืขื ืืืึธืืืจื ืืืงืจืึธ ืืึทืืื ืื ืืก, ืคืื ืื ืืขืจืืขืืืืื ืกืืกืืขืืขื ืืื ืกืขืจืืืขืจืืขืกืก ืึทืคึผืืึทืงืืืฉืึทื ื. SNS ืงืขื ืขื ืืืขืจื ืืขื ืืฆื ืฆื ืฉืืงื ืืื ืคึฟืึธืจืืึทืฆืืข ืฆื ืกืืฃ ื ืืฆืขืจืก ืืืจื ืจืืจืขืืืืืง ืฉืืืคึผื ื ืึธืืืึทืคืึทืงืืืฉืึทื ื, SMS ืึทืจืืืงืืขื ืืื ืืืืืื.
ืขืจืฉื ืืจืืื ืื ื
ืฆื ืขืืืึทืืืื ืื ืืึทืื ืืืืคื, ืืื ืืึทืฉืืึธืกื ืฆื ื ืืฆื ืื ืขืจืืืื ืืืืขื ืืื ืคึฟืึธืจืืึทืฆืืข ืืืืืขืงืขืจื ืืืจื ืื Aviasales API. ืืื ืืึทื ืฅ ืึท ืืจืืื ืจืฉืืื ืคืื ืคืึทืจืฉืืืขื ืข ืืขืืืึธืืก, ืืึธืื ืืื ืื ื ืขืืขื ืืืื ืขืจ ืคืื ืืื - "ืืืืืขืฉืืขื ืคึผืจืืืก ืงืึทืืขื ืืึทืจ", ืืืึธืก ืงืขืจื ืคึผืจืืืกืึทื ืคึฟืึทืจ ืืขืืขืจ ืืึธื ืคืื ืื ืืืืฉ, ืืจืืคึผื ืืืจื ืื ื ืืืขืจ ืคืื ืืจืึทื ืกืคืขืจืก. ืืืื ืืืจ ืืึธื ื ืื ืกืคึผืขืฆืืคืืฆืืจื ืืขื ืืืื ืืืืฉ ืืื ืื ืืขืื, ืืื ืคึฟืึธืจืืึทืฆืืข ืืืขื ืืืื ืืืืืขืงืขืจื ืคึฟืึทืจ ืื ืืืืฉ ื ืึธื ืืขื ืงืจืึทื ื.
ืึทืืื, ืืึธืื ืืื ืื ืคืึทืจืฉืจืืึทืื ืืื ืืึทืงืืืขื ืืื ืืืขืจ ืกืืืขื.
ื ืืืืฉืคึผืื ืืงืฉื ืืื ืืื ืื:
http://api.travelpayouts.com/v2/prices/month-matrix?currency=rub&origin=LED&destination=HKT&show_to_affiliates=true&token=TOKEN_APIืืขืจ ืืืืื ืืืคึฟื ืคืื ืจืืกืืืืื ื ืืึทืื ืคืื ืื ืึทืคึผื ืืืจื ืกืคึผืขืฆืืคืืฆืืจื ืึท ืกืืืขื ืืื ืื ืืขืื ืืืขื ืึทืจืืขืื, ืึธืืขืจ ืืื ืืขืกืขืจ ืฆื ืคืึธืจื ืื ืึทืงืกืขืก ืกืืืขื ืืืจื ืื ืืขืืขืจ, ืึทืืื ืืืจ ืืืขืื ื ืืฆื ืืขื ืืืคึฟื ืืื ืื api_caller.py ืฉืจืืคื.
ืืืกืืขืจ ืขื ืืคืขืจ:
{{
"success":true,
"data":[{
"show_to_affiliates":true,
"trip_class":0,
"origin":"LED",
"destination":"HKT",
"depart_date":"2015-10-01",
"return_date":"",
"number_of_changes":1,
"value":29127,
"found_at":"2015-09-24T00:06:12+04:00",
"distance":8015,
"actual":true
}]
}
ืื ืืืืฉืคึผืื ืึทืคึผื ืขื ืืคืขืจ ืืืืื ืืืืืื ืึท ืืืืขื ืคืื ืคืขืืขืจืืืจื ืฆื ืคืืง... ืืื, ืืืึธืก ืึท ืืืื...
ืืื ื ืืื ืืื ืคึฟืื ืงืึทืืึทื, ืืื ืคืืงืขื ืืื ืืืฆื "ืืืืื ืึท ืืืื", ืืึธืืืจ ืืืื ืืืงืืฅ ืคืื ืกื ืคืขืืขืจืืืจื ืฆื ืงืึทืืึทื.
ืขืก ืึทืกืืื ืึทื ืืืจ ืฉืืื ืืึธืื ืึท AWS ืืฉืืื. ืืื ืืืึธืื ืืื ืฆื ืืืืื ืืคืืขืจืงืืึทืืงืืื ืฆื ืืขื ืคืึทืงื ืึทื ืงืื ืขืกืืก ืืื ืฉืืงื ื ืึธืืืึทืคืึทืงืืืฉืึทื ื ืืืจื SMS ืืขื ืขื ื ืืฉื ืึทืจืืึทื ืืขืจืขืื ื ืืื ืื ืืขืจืืขื . ืืืขืจ ืืคืืื ืืจืึธืฅ ืืขื, ืืื ืึท ืคึผืึธืจ ืคืื ืืึธืืืึทืจืก ืืื ืืื ืขื, ืขืก ืืื ืืึทื ืฅ ืืขืืืขื ืฆื ืืืืขื ืื ืคืืจืืขืืืืื ืกืืกืืขื ืืื ืฉืคึผืืื ืืื ืืื. ืืื, ืคืื ืงืืจืก, ืืึธื ื ืื ืคืึทืจืืขืกื ืฆื ืืืกืืขืงื ืึทืืข ืจืขืกืืจืกื ื ืึธื ืืื ืืขื ืขื ื ืื ืืขืจ ืืืจืฃ.
ืฆืื ืืืืง, DynamoDb ืืื lambda ืคืึทื ืืงืฉืึทื ื ืืืขื ืืืื ืคืจืื ืคึฟืึทืจ ืืื ืื ืืืื ืืืจ ืืจืขืคื ืืื ืืืขืจ ืืืืืขืฉืืขื ืคืจืื ืืืืึทืฅ. ืืืฉื, ืคึฟืึทืจ DynamoDB: 25 ืืืืืืืื ืกืืึธืจืืืืฉ, 25 WCU / RCU ืืื 100 ืืืืืึธื ืคึฟืจืืื. ืืื ืึท ืืืืืึธื ืืึทืืืืึท ืคืื ืงืฆืืข ืงืึทืืืก ืคึผืขืจ ืืืืฉ.
ืืึทื ืืึทื ืกืืกืืขื ืืืคึผืืืืืึทื ื
ืืึทืฉืืขืืืงื Kinesis Data Streams
ืืึธืืืจ ืืืื ืฆื ืื Kinesis Data Streams ืืื ืกื ืืื ืฉืึทืคึฟื ืฆืืืื ื ืืึทืข ืกืืจืืื, ืืืื ืฉืึทืจื ืคึฟืึทืจ ืืขืืขืจ.
ืืืึธืก ืืื ืึท ืฉืึทืจืื?
ื ืฉืึทืจื ืืื ืื ืืงืขืจืืืง ืืึทืื ืึทืจืืืขืจืคืืจื ืึทืคึผืึทืจืึทื ืคืื ืึทื ืึทืืึทืืึธื ืงืื ืขืกืืก ืืืึทื. ืืืื ืึธืคึผืฉื ืื ืืื ืึทืจืืึทื ืฉืจืืึทื ืืึทืื ืึทืจืืืขืจืคืืจื ืืื ืึท ืืืืงืืึทื ืคืื 1 MB / s ืืื ืจืขืืืืืึทื ืืึทืื ืึทืจืืืขืจืคืืจื ืืื ืึท ืืืืงืืึทื ืคืื 2 MB / s. ืืืื ืึธืคึผืฉื ืื ืฉืืืฆื ืึทืจืืืฃ ืฆื 1000 PUT ืืืื ืกื ืคึผืขืจ ืกืขืงืื ืืข. ืืืขื ืงืจืืืืืื ื ืึท ืืึทืื ืืืึทื, ืืืจ ืืึทืจืคึฟื ืฆื ืกืคึผืขืฆืืคืืฆืืจื ืื ืคืืจืืื ืื ื ืืืขืจ ืคืื ืกืขืืืึทื ืฅ. ืคึฟืึทืจ ืืืึทืฉืคึผืื, ืืืจ ืงืขื ืขื ืืึทืื ืึท ืืึทืื ืืืึทื ืืื ืฆืืืื ืกืขืืืึทื ืฅ. ืืขืจ ืืึทืื ืืืึทื ืืืขื ืฆืืฉืืขืื ืึทืจืืึทื ืฉืจืืึทื ืืึทืื ืึทืจืืืขืจืคืืจื ืืื 2 MB / s ืืื ืจืขืืืืืึทื ืืึทืื ืึทืจืืืขืจืคืืจื ืืื 4 MB / s, ืืืึธืก ืฉืืืฆื ืึทืจืืืฃ ืฆื 2000 PUT ืจืขืงืึธืจืืก ืคึผืขืจ ืกืขืงืื ืืข.
ืื ืืขืจ ืฉืึทืจืื ืืื ืืืื ืืืึทื, ืื ืืจืขืกืขืจ ืื ืืจืืคึผืื. ืืื ืคึผืจืื ืฆืืคึผ, ืืึธืก ืืื ืืื ืคืืึธืื ืืขื ืขื ืกืงืืืื - ืืืจื ืึทืืื ื ืฉืึทืจืื. ืืืขืจ ืื ืืขืจ ืฉืึทืจืื ืืืจ ืืึธืื, ืื ืืขืืขืจ ืื ืคึผืจืืึทื. ืืขืืขืจ ืฉืขืจื ืงืึธืก 1,5 ืกืขื ืก ืคึผืขืจ ืฉืขื ืืื ื ืึธื 1.4 ืกืขื ืก ืคึฟืึทืจ ืืขืืขืจ ืืืืืึธื PUT ืคึผืืืืึธืื ืื ืืฅ.
ืืึธืืืจ ืืึทืื ืึท ื ืืึทืข ืืืึทื ืืื ืืขื ื ืึธืืขื ืขืจืืืื_ืืืงืืฅ, 1 ืฉืืืืจืฆืข ืืืขื ืืืื ืืขื ืื ืคืืจ ืืื:
ืืขืฆื ืืืืืจ ืืืื ื ืื ื ืืฉืืื ืืืื ื ืืืขื ืกืคึผืขืฆืืขืืข_ืกืืจืื:
ืคึผืจืึธืืืฆืืจืขืจ ืกืขืืึทืคึผ
ืฆื ืึทื ืึทืืืื ืึท ืึทืจืืขื, ืขืก ืืื ืืขื ืื ืฆื ื ืืฆื ืึท ืจืขืืืืขืจ EC2 ืืืึทืฉืคึผืื ืืื ืึท ืืึทืื ืคึผืจืึธืืืฆืืจืขืจ. ืขืก ืืึทืจืฃ ื ืื ืืืื ืึท ืฉืืึทืจืง, ืืืึทืขืจ ืืืืจืืืึทื ืืึทืฉืื; ืึท ืึธืจื t2.micro ืืืขื ืืืื ืืื.
ืืืืืืืง ืืึธื: ืคึฟืึทืจ ืืืึทืฉืคึผืื, ืืืจ ืืึธื ื ืืฆื ืืืื - Amazon Linux AMI 2018.03.0, ืขืก ืืื ืืืืื ืืงืขืจืข ืกืขืืืื ืืก ืคึฟืึทืจ ืืขืฉืืืื ื ืงืึทืืขืจ ืื ืงืื ืขืกืืก ืึทืืขื ื.
ืืืื ืฆื ืื EC2 ืกืขืจืืืืก, ืฉืึทืคึฟื ืึท ื ืืึทืข ืืืืจืืืึทื ืืึทืฉืื, ืกืขืืขืงืืืจื ืืขื ืืขืืืืื ืึทืื ืืื ืืืคึผ t2.micro, ืืืึธืก ืืื ืึทืจืืึทื ืืขืจืขืื ื ืืื ืื Free Tier:
ืึผืื ืื ื ืื-ืืขืฉืึทืคึฟืืข ืืืืจืืืขื ืืึทืฉืื ืืึธื ืงืขื ืขื ืื ืืขืจืึทืงื ืืื ืื ืงืื ืขืกืืก ืืื ืกื, ืขืก ืืืื ืืืื ืืขืืขืื ืจืขืื ืฆื ืืึธื ืืึธืก. ืืขืจ ืืขืกืืขืจ ืืืขื ืฆื ืืึธื ืืึธืก ืืื ืฆื ืืึทืฉืืืืขื ืึทื IAM ืจืึธืืข. ืืขืจืืืขืจ, ืืืืฃ ืื ืฉืจืื 3: ืงืึทื ืคืืืืขืจ ืื ืกืืึทื ืกืข ืืขืืึทืืืก ืคืึทืจืฉืืขืื, ืืืจ ืืึธื ืืืืกืงืืืึทืื ืฉืึทืคึฟื ืึท ื ืืึทืข IAM ืจืึธืืข:
ืฉืึทืคึฟื ืึทื IAM ืจืึธืืข ืคึฟืึทืจ EC2
ืืื ืื ืคึฟืขื ืฆืืขืจ ืืืึธืก ืึธืคึผืขื ืก, ืกืขืืขืงืืืจื ืึทื ืืืจ ืฉืึทืคึฟื ืึท ื ืืึทืข ืจืึธืืข ืคึฟืึทืจ EC2 ืืื ืืืื ืฆื ืื ืคึผืขืจืืืฉืึทื ื ืึธืคึผืืืืืื ื:
ื ืืฆื ืื ืืจืืื ืื ื ืืืืฉืคึผืื, ืืืจ ืืึธื ื ืื ืืึธืื ืฆื ืืืื ืืื ืึทืืข ืื ืื ืืจืึทืงืึทืกืื ืคืื ืืจืึทื ืืึทืืขืจ ืงืึทื ืคืืืืขืจืืืฉืึทื ืคืื ืืืื ืจืขืื, ืึทืืื ืืืจ ืืืขืื ืืืืกืงืืืึทืื ืื ืคึผืึทืืึทืกืื ืคืึทืจ-ืงืึทื ืคืืืืขืจื ืืืจื ืึทืืึทืืึธื: AmazonKinesisFullAccess ืืื CloudWatchFullAccess.
ืืื ืืื ืื ืืขืื ืึท ืืึทืืืึทืืืง ื ืึธืืขื ืคึฟืึทืจ ืืขื ืจืึธืืข, ืืืฉื: EC2-KinesisStreams-FullAccess. ืืขืจ ืจืขืืืืืึทื ืืึธื ืืืื ืื ืืขืืืข ืืื ืืขืืืืื ืืื ืื ืืืื ืืื ืื:
ื ืึธื ืงืจืืืืืื ื ืืขื ื ืืึทืข ืจืึธืืข, ืืึธื ื ืื ืคืึทืจืืขืกื ืฆื ืฆืืืฉืขืคึผืขื ืขืก ืฆื ืื ืืืฉืืคื ืืืืจืืืึทื ืืึทืฉืื ืืืึทืฉืคึผืื:
ืืืจ ืืึธื ื ืื ืืืืฉื ืขืคึผืขืก ืึทื ืืขืจืฉ ืืืืฃ ืืขื ืขืงืจืึทื ืืื ืืึทื ืืืืฃ ืฆื ืื ืืืืึทืืขืจ ืคึฟืขื ืฆืืขืจ.
ืื ืฉืืืขืจ ืคืึธืจ ืกืขืืืื ืืก ืงืขื ืขื ืืืื ืืื ืงืก ืืื ืคืขืืืงืืึทื, ืืื ืืขืืื ื ืืื ืื ืืึทืืก (ืืึธืืฉ ืขืก ืืื ืืื ืคืืจ ืฆื ื ืืฆื ืืึทืืก, ืืคึผืืืช ืืขืื ืืขื ืืืึทืฉืคึผืื ืึท ื ืึธืืขื ืืื ืึธื ืืืืึทืื ืื ืกืืืืืืข).
ืืืฆื ืืืจ ืืขื ืขื ืืืืฃ ืื ืฉืจืื 6: ืงืึทื ืคืืืืขืจ ืืืืขืจืืืื ืืจืืคืข ืงืืืืื, ืืื ืืืจ ืืึทืจืคึฟื ืฆื ืฉืึทืคึฟื ืึท ื ืืึทืข ืึธืืขืจ ืกืคึผืขืฆืืคืืฆืืจื ืืืื ืืืืืกืืื ื ืืืืขืจืืืื ืืจืืคึผืข, ืืืึธืก ืึทืืึทืื ืืืจ ืฆื ืคืึทืจืืื ืื ืืืจื ssh (ืคึผืึธืจื 22) ืฆื ืืขื ืืืึทืฉืคึผืื. ืกืขืืขืงืืืจื ืืงืืจ -> ืืืึทื IP ืืึธืจื ืืื ืืืจ ืงืขื ืขื ืงืึทืืขืจ ืืขื ืืืึทืฉืคึผืื.
ืืื ืืึทืื ืืื ืขืก ืกืืืืืฉืื ืฆื ืคืืืกื ืืืง ืกืืึทืืืก, ืืืจ ืงืขื ืขื ืคึผืจืืืืจื ืฆื ืคืึทืจืืื ืื ืฆื ืขืก ืืืจื ssh.
ืฆื ืงืขื ืขื ืึทืจืืขืื ืืื Kinesis Agent, ื ืึธื ืืฆืืื ืงืึทื ืขืงืืื ื ืฆื ืื ืืึทืฉืื, ืืืจ ืืืื ืึทืจืืึทื ืื ืคืืืืขื ืืข ืงืึทืืึทื ืื ืืื ืื ืืืึธืงืืึทื:
sudo yum -y update
sudo yum install -y python36 python36-pip
sudo /usr/bin/pip-3.6 install --upgrade pip
sudo yum install -y aws-kinesis-agent
ืืึธืืืจ ืฉืึทืคึฟื ืึท ืืขืงืข ืฆื ืจืึทืืขืืืขื ืึทืคึผื ืจืขืกืคึผืึธื ืกืขืก:
sudo mkdir /var/log/airline_ticketsืืืืืขืจ ืืืจ ืึธื ืืืืื ืืขื ืึทืืขื ื, ืืืจ ืืึทืจืคึฟื ืฆื ืงืึทื ืคืืืืขืจ ืื ืงืึทื ืคืืืืขืจืืืฉืึทื:
sudo vim /etc/aws-kinesis/agent.jsonืืขืจ ืืื ืืึทืื ืคืื ืื agent.json ืืขืงืข ืืึธื ืงืืงื ืืื ืืึธืก:
{
"cloudwatch.emitMetrics": true,
"kinesis.endpoint": "",
"firehose.endpoint": "",
"flows": [
{
"filePattern": "/var/log/airline_tickets/*log",
"kinesisStream": "airline_tickets",
"partitionKeyOption": "RANDOM",
"dataProcessingOptions": [
{
"optionName": "CSVTOJSON",
"customFieldNames": ["cost","trip_class","show_to_affiliates",
"return_date","origin","number_of_changes","gate","found_at",
"duration","distance","destination","depart_date","actual","record_id"]
}
]
}
]
}
ืืื ืืืจ ืงืขื ื ืืขื ืคึฟืื ืื ืงืึทื ืคืืืืขืจืืืฉืึทื ืืขืงืข, ืืขืจ ืึทืืขื ื ืืืขื ืืึธื ืืืึธืจ ืืขืงืขืก ืืื ืื .ืืึธื ืืขืฉืคึผืจืืื ืืื ืื /var/log/airline_tickets/ ืืืขืืืืืึทืืขืจ, ืคึผืึทืจืก ืืื ืืื ืึทืจืืืขืจืคืืจื ืืื ืฆื ืื ืขืจืืืื_ืืืงืงืขืฅ ืืืึทื.
ืืืจ ืจืืกืืึทืจื ืื ืืื ืกื ืืื ืืึทืื ืืืืขืจ ืึทื ืขืก ืืื ืึทืจืืืฃ ืืื ืคืืืกื ืืืง:
sudo service aws-kinesis-agent restartืืืฆื ืืึธืื ืืื ืื ืืจืืคืงืืคืืข ืื Python ืฉืจืืคื ืืืึธืก ืืืขื ืืขืื ืืึทืื ืคึฟืื ืื API:
REPO_PATH=https://raw.githubusercontent.com/igorgorbenko/aviasales_kinesis/master/producer
wget $REPO_PATH/api_caller.py -P /home/ec2-user/
wget $REPO_PATH/requirements.txt -P /home/ec2-user/
sudo chmod a+x /home/ec2-user/api_caller.py
sudo /usr/local/bin/pip3 install -r /home/ec2-user/requirements.txt
ืื api_caller.py ืฉืจืืคื ืจืืงืืืขืก ืืึทืื ืคืื Aviasales ืืื ืกืึทืืืขืก ืื ืืืงืืืขื ืขื ืืคืขืจ ืืื ืื ืืืขืืืืืึทืืขืจ ืืืึธืก ืื ืงืื ืขืกืืก ืึทืืขื ื ืกืงืึทื ื. ืื ืืืคึผืืึทืืขื ืืืืฉืึทื ืคืื ืืขื ืฉืจืืคื ืืื ืืึทื ืฅ ื ืึธืจืืึทื, ืขืก ืืื ืึท ืืืงืงืขืฅืึทืคึผื ืงืืึทืก, ืขืก ืึทืืึทืื ืืืจ ืฆื ืืืกืื ืืงืจืึทื ืึทืกืื ืฆืืขื ืื ืึทืคึผื. ืืืจ ืคืึธืจื ืึท ืืขืืขืจ ืืื ืึท ืกืืืขื ืืื ืืขืื ืคึผืึทืจืึทืืขืืขืจืก ืฆื ืืขื ืงืืึทืก:
class TicketsApi:
"""Api caller class."""
def __init__(self, headers):
"""Init method."""
self.base_url = BASE_URL
self.headers = headers
async def get_data(self, data):
"""Get the data from API query."""
response_json = {}
async with ClientSession(headers=self.headers) as session:
try:
response = await session.get(self.base_url, data=data)
response.raise_for_status()
LOGGER.info('Response status %s: %s',
self.base_url, response.status)
response_json = await response.json()
except HTTPError as http_err:
LOGGER.error('Oops! HTTP error occurred: %s', str(http_err))
except Exception as err:
LOGGER.error('Oops! An error ocurred: %s', str(err))
return response_json
def prepare_request(api_token):
"""Return the headers and query fot the API request."""
headers = {'X-Access-Token': api_token,
'Accept-Encoding': 'gzip'}
data = FormData()
data.add_field('currency', CURRENCY)
data.add_field('origin', ORIGIN)
data.add_field('destination', DESTINATION)
data.add_field('show_to_affiliates', SHOW_TO_AFFILIATES)
data.add_field('trip_duration', TRIP_DURATION)
return headers, data
async def main():
"""Get run the code."""
if len(sys.argv) != 2:
print('Usage: api_caller.py <your_api_token>')
sys.exit(1)
return
api_token = sys.argv[1]
headers, data = prepare_request(api_token)
api = TicketsApi(headers)
response = await api.get_data(data)
if response.get('success', None):
LOGGER.info('API has returned %s items', len(response['data']))
try:
count_rows = log_maker(response)
LOGGER.info('%s rows have been saved into %s',
count_rows,
TARGET_FILE)
except Exception as e:
LOGGER.error('Oops! Request result was not saved to file. %s',
str(e))
else:
LOGGER.error('Oops! API request was unsuccessful %s!', response)
ืฆื ืคึผืจืืืืจื ืื ืจืืืืืง ืกืขืืืื ืืก ืืื ืคืึทื ืืงืฉืึทื ืึทืืืื ืคืื ืืขื ืึทืืขื ื, ืืึธืืืจ ืคึผืจืืืืจื ืื api_caller.py ืฉืจืืคื:
sudo ./api_caller.py TOKEN
ืืื ืืืจ ืงืืงื ืืื ืืขืจ ืจืขืืืืืึทื ืคืื ืื ืึทืจืืขื ืืื ืื ืึทืืขื ื ืืึธืืก ืืื ืืืืฃ ืื ืืึธื ืืืึธืจืื ื ืงืืืืื ืืื ืื Airline_tickets ืืึทืื ืืืึทื:
tail -f /var/log/aws-kinesis-agent/aws-kinesis-agent.log
ืืื ืืืจ ืงืขื ืขื ืืขื, ืึทืืฅ ืึทืจืืขื ืืื ืื ืงืื ืขืกืืก ืึทืืขื ื ืกืขื ืื ืืึทืื ืฆื ืื ืืืึทื ืืฆืืื. ืืืฆื ืืึธืื ืก ืงืึทื ืคืืืืขืจ ืงืึทื ืกืืืขืจ.
ืืึทืฉืืขืืืงื Kinesis Data Analytics
ืืึธืืืจ ืืืื ืฆื ืื ืืืืคื ืงืึธืืคึผืึธื ืขื ื ืคืื ืื ืืื ืฆืข ืกืืกืืขื - ืฉืึทืคึฟื ืึท ื ืืึทืข ืึทืคึผืืึทืงืืืฉืึทื ืืื Kinesis Data Analytics ืืืื ื ืึธืืขื kinesis_analytics_airlines_app:
Kinesis Data Analytics ืึทืืึทืื ืืืจ ืฆื ืืืจืืคืืจื ืคืึทืงืืืฉ-ืฆืืื ืืึทืื ืึทื ืึทืืืืืงืก ืคึฟืื Kinesis ืกืืจืขืึทืืก ื ืืฆื ืื SQL ืฉืคึผืจืึทื. ืขืก ืืื ืึท ืืึธืจ ืึทืืืึธืกืงืึทืืื ื ืืื ืกื (ื ืื ืขื ืืขื ืงืื ืขืกืืก ืกืืจืขืึทืืก) ืึทื:
- ืึทืืึทืื ืืืจ ืฆื ืฉืึทืคึฟื ื ืืึทืข ืกืืจืืื (ืึธืืืคึผืื ืกืืจืื) ืืืืืจื ืืืืฃ ืจืืงืืืขืก ืฆื ืืงืืจ ืืึทืื;
- ืืื ืึท ืืืึทื ืืื ืขืจืจืึธืจืก ืึทื ืคืืจืืขืงืืืขื ืืฉืขืช ืึทืคึผืืึทืงืืืฉืึทื ื ืืขื ืขื ืคืืืกื ืืืง (ืืขืืช ืกืืจืื);
- ืงืขื ืขื ืืืืืึธืืึทืืืฉ ืืึทืฉืืืืขื ืื ืึทืจืืึทื ืฉืจืืึทื ืืึทืื ืกืืขืืข (ืขืก ืงืขื ืขื ืืืื ืืึทื ืืืึทืื ืจืืืืคืืื ื ืืืื ื ืืืืืง).
ืืึธืก ืืื ื ืืฉื ืึท ืืืืืง ืืื ืกื - 0.11 ืืกื ืคึผืขืจ ืฉืขื ืคืื ืึทืจืืขื, ืึทืืื ืืืจ ืืึธื ื ืืฆื ืขืก ืงืขืจืคืึทืื ืืื ืืืกืืขืงื ืขืก ืืืขื ืืืจ ืืขื ื ืคืึทืจืืืง.
ืืึธืืืจ ืคืึทืจืืื ืื ืื ืึทืคึผืืึทืงืืืฉืึทื ืฆื ืื ืืึทืื ืืงืืจ:
ืกืขืืขืงืืืจื ืืขื ืืืึทื ืืื ืืืึธืก ืืืจ ืืืขืื ืคืึทืจืืื ืื (airline_tickets):
ืืขืจื ืึธื, ืืืจ ืืึทืจืคึฟื ืฆื ืฆืืืฉืขืคึผืขื ืึท ื ืืึทืข IAM ืจืึธืืข ืึทืืื ืึทื ืื ืึทืคึผืืึทืงืืืฉืึทื ืงืขื ืขื ืืืืขื ืขื ืคึฟืื ืืขื ืืืึทื ืืื ืฉืจืืึทืื ืฆื ืืขื ืืืึทื. ืฆื ืืึธื ืืึธืก, ืขืก ืืื ืืขื ืื ื ืืฉื ืฆื ืืืืฉื ืขืคึผืขืก ืืื ืื ืึทืงืกืขืก ืคึผืขืจืืืฉืึทื ื ืืืึธืง:
ืืืฆื ืืึธืืืจ ืืขืื ืืคืืขืงืื ื ืคืื ืื ืืึทืื ืกืืฉืขืืึท ืืื ืืขื ืืืึทื; ืฆื ืืึธื ืืึธืก, ืืจืืงื ืืืืฃ ืื "ืึทื ืืืขืงื ืกืืฉืขืืึท" ืงื ืขืคึผื. ืืื ืึท ืจืขืืืืืึทื, ืื IAM ืจืึธืืข ืืืขื ืืืื ืืขืจืืืึทื ืืืงื (ืึท ื ืืึทืข ืืืขื ืืืื ืืืฉืืคื) ืืื ืกืืฉืขืืึท ืืืืขืงืฉืึทื ืืืขื ืืืื ืืึธื ืืฉื ืคึฟืื ืื ืืึทืื ืืืึธืก ืืขื ืขื ืฉืืื ืื ืืขืงืืืขื ืืื ืื ืืืึทื:
ืืืฆื ืืืจ ืืึทืจืคึฟื ืฆื ืืืื ืฆื ืื SQL ืจืขืืึทืงืืึธืจ. ืืืขื ืืืจ ืืจืืงื ืืืืฃ ืืขื ืงื ืขืคึผื, ืึท ืคึฟืขื ืฆืืขืจ ืืืขื ืืืื ืึทืกืงืื ื ืืืจ ืฆื ืงืึทืืขืจ ืื ืึทืคึผืืึทืงืืืฉืึทื - ืกืขืืขืงืืืจื ืืืึธืก ืืืจ ืืืืื ืฆื ืงืึทืืขืจ:
ืึทืจืืึทื ืืืืื ืื ืคืืืืขื ืืข ืคึผืฉืื ืึธื ืคึฟืจืขื ืืื ืื ืกืงื ืจืขืืึทืงืืึธืจ ืคึฟืขื ืฆืืขืจ ืืื ืืื ืืื ืืื ืจืึทื ืกืงื:
CREATE OR REPLACE STREAM "DESTINATION_SQL_STREAM" ("cost" DOUBLE, "gate" VARCHAR(16));
CREATE OR REPLACE PUMP "STREAM_PUMP" AS INSERT INTO "DESTINATION_SQL_STREAM"
SELECT STREAM "cost", "gate"
FROM "SOURCE_SQL_STREAM_001"
WHERE "cost" < 5000
and "gate" = 'Aeroflot';
ืืื ืจืืืืืฉืึทื ืึทื ืืึทืืึทืืืืกืื, ืืืจ ืึทืจืืขื ืืื ืืืฉื ื ืืฆื INSERT ืกืืืืืืึทื ืฅ ืฆื ืืืืื ืจืขืงืึธืจืืก ืืื ืึท ืืืืกืงืืืึทืื ืืืกืืึธืืื ื ืฆื ืึธื ืคึฟืจืขื ืืึทืื. ืืื Amazon Kinesis Data Analytics, ืืืจ ืึทืจืืขื ืืื ืกืืจืืื (ืกืืจืืื) ืืื ืคึผืึทืืคึผืก (ืคึผืึทืืคึผืก) - ืงืขืกืืืืขืจืืืง ืึทืจืืึทื ืืืืื ืจืืงืืืขืก ืืืึธืก ืึทืจืืึทื ืืืืื ืืึทืื ืคืื ืืืื ืืืึทื ืืื ืึท ืึทืคึผืืึทืงืืืฉืึทื ืืื ืื ืื ืืขืจ ืืืึทื.
ืื SQL ืึธื ืคึฟืจืขื ืืขืจืืื ืื ืืืืื ืึธื ืคึฟืจืขืื ืคึฟืึทืจ ืึทืขืจืึธืคืืึธื ืืืงืืฅ ืฆื ืึท ืคึผืจืืึทื ืืื ืืขืจ ืคืื ืฃ ืืืืื ื ืจืืื. ืึทืืข ืจืขืงืึธืจืืก ืืืึธืก ืืจืขืคื ืื ืืืืื ืืื ืืขื ืืืขื ืืืื ืืขืฉืืขืื ืืื ืื DESTINATION_SQL_STREAM ืืืึทื.
ืืื ืื ืืขืกืืื ืืืฉืึทื ืืืึธืง, ืืืืกืงืืืึทืื ืื ืกืคึผืขืฆืืขืืข_ืกืืจืื ืืืึทื, ืืื ืืื ืื ืืื-ืึทืคึผืืึทืงืืืฉืึทื ืืืึทื ื ืึธืืขื DESTINATION_SQL_STREAM ืคืึทื-ืึทืจืึธืคึผ ืจืฉืืื:
ืืขืจ ืจืขืืืืืึทื ืคืื ืึทืืข ืืึทื ืืคึผืืึทืืืืฉืึทื ื ืืึธื ืืืื ืขืคึผืขืก ืขื ืืขื ืฆื ืื ืืืื ืืื ืื:
ืฉืึทืคึฟื ืืื ืึทืืึธื ืืจื ืฆื ืึท SNS ืืขืืข
ืืืื ืฆื ืื ืกืืืคึผืืข ื ืึธืืืคืืงืึทืืืึธื ืกืขืจืืืืก ืืื ืฉืึทืคึฟื ืึท ื ืืึทืข ืืขืืข ืืื ืื ื ืึธืืขื ืึทืืจืืื ืขืก:
ืึทืืึธื ืืจื ืฆื ืืขื ืืขืืข ืืื ืึธื ืืืืึทืื ืื ืจืืจืขืืืืืง ืืขืืขืคืึธื ื ืืืขืจ ืฆื ืืืึธืก SMS ื ืึธืืืึทืคืึทืงืืืฉืึทื ื ืืืขื ืืืื ืืขืฉืืงื:
ืฉืึทืคึฟื ืึท ืืืฉ ืืื DynamoDB
ืฆื ืงืจืึธื ืื ืจืื ืืึทืื ืคืื ืืืืขืจ ืขืจืืืื_ืืืงืงืขืฅ ืืืึทื, ืืึธืื ืืื ืื ืฉืึทืคึฟื ืึท ืืืฉ ืืื DynamoDB ืืื ืื ืืขืืืข ื ืึธืืขื. ืืืจ ืืืขืื ื ืืฆื ืจืขืงืึธืจื_ืื ืืื ืื ืขืจืฉืืืง ืฉืืืกื:
ืงืจืืืืืื ื ืึท ืืึทืืืึท ืคึฟืื ืงืฆืืข ืงืึทืืขืงืืขืจ
ืืึธืืืจ ืืึทืื ืึท ืืึทืืืืึท ืคึฟืื ืงืฆืืข ืืขืจืืคึฟื ืงืึทืืขืงืืขืจ, ืืืขืืขื ืก ืึทืจืืขื ืืืขื ืืืื ืฆื ืืึทืงืืืขื ืื ืึทืืจืืื ืข_ืืืงืึทืฅ ืืืึทื ืืื, ืืืื ื ืืึท ืจืขืงืึธืจืืก ืืขื ืขื ืืขืคึฟืื ืขื ืืึธืจื, ืึทืจืืึทื ืืืืื ืื ืจืขืงืึธืจืืก ืืื ืื DynamoDB ืืืฉ. ืืึธื, ืืื ืึทืืืฉืึทื ืฆื ืื ืคืขืืืงืืึทื ืจืขืื, ืืขื ืืึทืืืืึท ืืืื ืืึธืื ืืืืขื ืขื ืึทืงืกืขืก ืฆื ืื ืงืื ืขืกืืก ืืึทืื ืืืึทื ืืื ืฉืจืืึทืื ืึทืงืกืขืก ืฆื DynamoDB.
ืฉืึทืคึฟื ืึทื IAM ืจืึธืืข ืคึฟืึทืจ ืื ืงืึทืืขืงืืขืจ ืืึทืืืืึท ืคืื ืงืฆืืข
ืขืจืฉืืขืจ, ืืึธืื ืืื ืื ืืึทืื ืึท ื ืืึทืข IAM ืจืึธืืข ืคึฟืึทืจ ืื ืืึทืืืืึท ืืขืืืืกื Lambda-TicketsProcessingRole:
ืคึฟืึทืจ ืื ืคึผืจืึธืืข ืืืืฉืคึผืื, ืื ืคืึทืจ-ืงืึทื ืคืืืืขืจื AmazonKinesisReadOnlyAccess ืืื AmazonDynamoDBFullAccess ืคึผืึทืืึทืกืื ืืขื ืขื ืืึทื ืฅ ืคึผืึทืกืืง, ืืื ืืขืืืืื ืืื ืื ืืืื ืืื ืื:
ืื ืืึทืืืืึท ืืึธื ืืืื ืืึธื ืืฉื ืืืจื ืึท ืฆืื ืื ืคึฟืื Kinesis ืืืขื ื ืืึทืข ืืืื ืกื ืึทืจืืึทื ืื ืขืจืืืื_ืกืืจืื, ืึทืืื ืืืจ ืืึทืจืคึฟื ืฆื ืืืืื ืึท ื ืืึทืข ืฆืื ืื:
ืึทืืข ืืืึธืก ืืืืืื ืืื ืฆื ืึทืจืืึทื ืืืืื ืื ืงืึธื ืืื ืจืึทืืขืืืขื ืื ืืึทืืืืึท.
"""Parsing the stream and inserting into the DynamoDB table."""
import base64
import json
import boto3
from decimal import Decimal
DYNAMO_DB = boto3.resource('dynamodb')
TABLE_NAME = 'airline_tickets'
class TicketsParser:
"""Parsing info from the Stream."""
def __init__(self, table_name, records):
"""Init method."""
self.table = DYNAMO_DB.Table(table_name)
self.json_data = TicketsParser.get_json_data(records)
@staticmethod
def get_json_data(records):
"""Return deserialized data from the stream."""
decoded_record_data = ([base64.b64decode(record['kinesis']['data'])
for record in records])
json_data = ([json.loads(decoded_record)
for decoded_record in decoded_record_data])
return json_data
@staticmethod
def get_item_from_json(json_item):
"""Pre-process the json data."""
new_item = {
'record_id': json_item.get('record_id'),
'cost': Decimal(json_item.get('cost')),
'trip_class': json_item.get('trip_class'),
'show_to_affiliates': json_item.get('show_to_affiliates'),
'origin': json_item.get('origin'),
'number_of_changes': int(json_item.get('number_of_changes')),
'gate': json_item.get('gate'),
'found_at': json_item.get('found_at'),
'duration': int(json_item.get('duration')),
'distance': int(json_item.get('distance')),
'destination': json_item.get('destination'),
'depart_date': json_item.get('depart_date'),
'actual': json_item.get('actual')
}
return new_item
def run(self):
"""Batch insert into the table."""
with self.table.batch_writer() as batch_writer:
for item in self.json_data:
dynamodb_item = TicketsParser.get_item_from_json(item)
batch_writer.put_item(dynamodb_item)
print('Has been added ', len(self.json_data), 'items')
def lambda_handler(event, context):
"""Parse the stream and insert into the DynamoDB table."""
print('Got event:', event)
parser = TicketsParser(TABLE_NAME, event['Records'])
parser.run()
ืงืจืืืืืื ื ืึท ื ืึธืืืืคืืขืจ ืคืื ืืึทืืืึท ืคืื ืงืฆืืข
ืื ืจืืข ืืึทืืืืึท ืคึฟืื ืงืฆืืข, ืืืึธืก ืืืขื ืืึธื ืืืึธืจ ืื ืจืืข ืืืึทื (ืกืคึผืขืฆืืขื_ืกืืจืื) ืืื ืฉืืงื ืึท ืึธื ืืึธื ืฆื SNS, ืืื ืืืฉืืคื ืืืืฃ ืึท ืขื ืืขื ืืืขื. ืืขืจืืืขืจ, ืื ืืึทืืืืึท ืืืื ืืึธืื ืฆืืืจืื ืฆื ืืืืขื ืขื ืคึฟืื ืงืื ืขืกืืก ืืื ืฉืืงื ืึทืจืืืงืืขื ืฆื ืึท ืืขืืขืื SNS ืืขืืข, ืืืึธืก ืืืขื ืืืื ืืขืฉืืงื ืืืจื ืื SNS ืืื ืกื ืฆื ืึทืืข ืืืื ืขื ืื ืคืื ืืขื ืืขืืข (ืขืืึทืื, SMS, ืืื"ื ื).
ืฉืึทืคึฟื ืึทื IAM ืจืึธืืข
ืขืจืฉืืขืจ, ืืืจ ืืึทืื ืื IAM ืจืึธืืข Lambda-KinesisAlarm ืคึฟืึทืจ ืืขื ืืึทืืืืึท, ืืื ืืึทื ืืึทืฉืืืืขื ืืขื ืจืึธืืข ืฆื ืื ืึทืืึทืจื_ื ืึธืืืคืืขืจ ืืึทืืืืึท ืืืฉืืคื:
ืื ืืึทืืืืึท ืืึธื ืึทืจืืขืื ืืืืฃ ืึท ืฆืื ืื ืคึฟืึทืจ ื ืืึทืข ืจืขืงืึธืจืืก ืฆื ืึทืจืืึทื ืื ืกืคึผืขืฆืืขืืข_ืกืืจืื, ืึทืืื ืืืจ ืืึทืจืคึฟื ืฆื ืงืึทื ืคืืืืขืจ ืื ืฆืื ืื ืืื ืื ืืขืืืข ืืืขื ืืื ืืืจ ืืึธืื ืืขืืื ืคึฟืึทืจ ืื ืงืึทืืขืงืืขืจ ืืึทืืืืึท.
ืฆื ืืึทืื ืขืก ืืจืื ืืขืจ ืฆื ืงืึทื ืคืืืืขืจ ืื ืืึทืืืืึท, ืืึธืื ืืื ืื ืืึทืงืขื ืขื ืึท ื ืืึทืข ืื ืืืืืจืึทื ืืขื ืึทื ืืืึทืืขืืืืืง - TOPIC_ARN, ืืื ืืืจ ืฉืืขืื ืื ANR (Amazon Recourse Names) ืคืื ืื ืึทืืจืืื ืขืก ืืขืืข:
ืืื ืึทืจืืึทื ืืืืื ืื ืืึทืืืืึท ืงืึธื, ืขืก ืืื ื ืืฉื ืงืึธืืคึผืืืฆืืจื:
import boto3
import base64
import os
SNS_CLIENT = boto3.client('sns')
TOPIC_ARN = os.environ['TOPIC_ARN']
def lambda_handler(event, context):
try:
SNS_CLIENT.publish(TopicArn=TOPIC_ARN,
Message='Hi! I have found an interesting stuff!',
Subject='Airline tickets alarm')
print('Alarm message has been successfully delivered')
except Exception as err:
print('Delivery failure', str(err))
ืขืก ืืืื ื ืึทื ืืึธืก ืืื ืืื ืื ืืึทื ืืึทื ืกืืกืืขื ืงืึทื ืคืืืืขืจืืืฉืึทื ืืื ืืขืขื ืืืงื. ืึทืืข ืืืึธืก ืืืืืื ืืื ืฆื ืคึผืจืืืืจื ืืื ืืึทืื ืืืืขืจ ืึทื ืืืจ ืืึธืื ืงืึทื ืคืืืืขืจื ืึทืืฅ ืจืืืืืง.
ืฆืขืืืืงืืขื ืคึฟืื Terraform ืงืึธื
ื ืืืืืง ืฆืืืจืืืืื ื
ืืื ืึท ืืืืขืจ ืืึทืงืืืขื ืึธืคึฟื ืืงืืจ ืืขืฆืืึทื ืคึฟืึทืจ ืืืคึผืืืืื ื ืื ืคืจืึทืกืืจืึทืงืืฉืขืจ ืคึฟืื ืงืึธื. ืขืก ืืื ืืืื ืืืืืขื ืข ืกืื ืืึทืงืก ืืืึธืก ืืื ืืจืื ื ืฆื ืืขืจื ืขื ืืื ืืื ืคืืืข ืืืืฉืคืืื ืคืื ืืื ืืื ืืืึธืก ืฆื ืฆืขืืืืงืืขื. ืืขืจ Atom ืจืขืืึทืงืืึธืจ ืึธืืขืจ ืืืืกืืึทื ืกืืืืืึธ ืงืึธืื ืืื ืคืืืข ืืึทื ืืืง ืคึผืืืืื ืก ืืืึธืก ืืึทืื ืืจืืขืื ืืื Terraform ืืจืื ืืขืจ.
ืืืจ ืงืขื ืขื ืืจืืคืงืืคืืข ืื ืคืึทืจืฉืคึผืจืืืืื ื . ื ืืืืืืื ืึทื ืึทืืืกืืก ืคืื ืึทืืข Terraform ืงืืืคึผืึทืืืืึทืืื ืืื ืืืืึทืืขืจ ืคืื ืืขื ืคืึทืจื ืขื ืคืื ืืขื ืึทืจืืืงื, ืึทืืื ืืืจ ืืืขืื ืืึทืืจืขื ืขืฆื ืืื ืฆื ืื ืืืืคึผื ืคืื ืงืื.
ืืื ืฆื ืึธื ืืืืื
ืื ืคืื ืงืึธื ืคืื ืื ืคึผืจืืืขืงื ืืื . ืืืจ ืงืืึธืื ืื ืจืืคึผืึทืืึทืืึธืจื ืฆื ืืื. ืืืืืขืจ ืืืจ ืึธื ืืืืื, ืืืจ ืืึทืจืคึฟื ืฆื ืืึทืื ืืืืขืจ ืึทื AWS CLI ืืื ืืื ืกืืึทืืืจื ืืื ืงืึทื ืคืืืืขืจื ืืืืึทื ... ืืขืจืจืึทืคืึธืจื ืืืขื ืงืืงื ืคึฟืึทืจ ืงืจืึทืืขื ืืฉืึทืื ืืื ืื ~/.aws/credentials ืืขืงืข.
ื ืืืืข ืคืืจ ืืื ืฆื ืืืืคื ืื ืคึผืืึทื ืืึทืคึฟืขื ืืืืืขืจ ืืืคึผืืืืื ื ืื ืืื ืฆืข ืื ืคืจืึทืกืืจืึทืงืืฉืขืจ ืฆื ืืขื ืืืึธืก Terraform ืืื ืืืฆื ืงืจืืืืืื ื ืคึฟืึทืจ ืืื ืื ืืื ืื ืืืึธืืงื:
terraform.exe planืืืจ ืืืขื ืืืื ืคึผืจืึทืืคึผืืื ืฆื ืึทืจืืึทื ืึท ืืขืืขืคืึธื ื ืืืขืจ ืฆื ืฉืืงื ื ืึธืืืึทืคืึทืงืืืฉืึทื ื. ืขืก ืืื ื ืื ื ืืืืืง ืฆื ืึทืจืืึทื ืขืก ืืื ืืขื ืืื ืข.
ื ืึธื ืึทื ืึทืืืืื ืื ืึธืคึผืขืจืึทืฆืืข ืคึผืืึทื ืคืื ืื ืคึผืจืึธืืจืึทื, ืืืจ ืงืขื ืขื ืึธื ืืืืื ืงืจืืืืืื ื ืจืขืกืืจืกื:
terraform.exe applyื ืึธื ืฉืืงื ืืขื ืืึทืคึฟืขื, ืืืจ ืืืขื ืืืืืขืจ ืืืื ืืขืืขืื ืฆื ืึทืจืืึทื ืึท ืืขืืขืคืึธื ื ืืืขืจ; ืจืขืื "ืืึธ" ืืืขื ืึท ืงืฉืื ืืืขืื ืึทืงืืฉืึทืืืึทืื ืืืจืืคืืจื ืื ืึทืงืฉืึทื ื ืืื ืืขืืืืื. ืืึธืก ืืืขื ืืึธืื ืืืจ ืฆื ืฉืืขืื ืื ืืื ืฆืข ืื ืคืจืึทืกืืจืึทืงืืฉืขืจ, ืืืจืืคืืจื ืึทืืข ืื ื ืืืืืง ืงืึทื ืคืืืืขืจืืืฉืึทื ืคืื EC2, ืฆืขืืืืงืืขื ืืึทืืืืึท ืคืึทื ืืงืฉืึทื ื, ืขืืง.
ื ืึธื ืึทืืข ืจืขืกืืจืกื ืืขื ืขื ืืฆืืื ืืืฉืืคื ืืืจื ืื Terraform ืงืึธื, ืืืจ ืืึทืจืคึฟื ืฆื ืืืื ืืื ืื ืืขืืึทืืืก ืคืื ืื ืงืื ืขืกืืก ืึทื ืึทืืืืืงืก ืึทืคึผืืึทืงืืืฉืึทื (ืฆืื ืืึทืืืืขืจื, ืืื ืงืขื ื ืืฉื ืืขืคึฟืื ืขื ืืื ืฆื ืืึธื ืืึธืก ืืืืื ืคึฟืื ืื ืงืึธื).
ืงืึทืืขืจ ืื ืึทืคึผืืึทืงืืืฉืึทื:
ื ืึธื ืืขื, ืืืจ ืืืื ืืคืืจืืฉ ืฉืืขืื ืื ืืื-ืึทืคึผืืึทืงืืืฉืึทื ืืืึทื ื ืึธืืขื ืืืจื ืกืึทืืขืงืืื ื ืคืื ืื ืคืึทื-ืึทืจืึธืคึผ ืจืฉืืื:
ืืืฆื ืึทืืฅ ืืื ืืจืืื ืฆื ืืืื.
ืคึผืจืืืืจื ืื ืึทืคึผืืึทืงืืืฉืึทื
ืจืขืืึทืจืืืขืกืก ืคืื ืืื ืืืจ ืืืคึผืืืื ืื ืกืืกืืขื, ืืึทื ืืืึทืื ืึธืืขืจ ืืืจื Terraform ืงืึธื, ืขืก ืืืขื ืึทืจืืขืื ืื ืืขืืืข.
ืืืจ ืงืืึธืฅ ืืื ืืืจื SSH ืฆื ืื EC2 ืืืืจืืืึทื ืืึทืฉืื ืืื ืงืื ืขืกืืก ืึทืืขื ื ืืื ืืื ืกืืึทืืืจื ืืื ืืืืคื ืื api_caller.py ืฉืจืืคื
sudo ./api_caller.py TOKENืื ืืืจ ืืึธืื ืฆื ืืึธื ืืื ืฆื ืืืึทืจืื ืคึฟืึทืจ ืึท SMS ืฆื ืืืื ื ืืืขืจ:
SMS - ืืขืจ ืึธื ืืึธื ืงืืื ืืืืฃ ืื ืืขืืขืคืึธื ืืื ืึผืืขื 1 ืืื ืื:
ืขืก ืืืืืื ืฆื ืืขื ืฆื ืื ืจืขืงืึธืจืืก ืืขื ืขื ืืขืจืืืขืืืขื ืืื ืื DynamoDB ืืึทืืึทืืืืก ืคึฟืึทืจ ืกืึทืืกืึทืงืืืึทื ื, ืืขืจ ืืืืืืื ืึทื ืึทืืืกืืก. ืื Airline_tickets ืืืฉ ืึผืืื ืืขืขืจืขื ืื ืคืืืืขื ืืข ืืึทืื:
ืกืึธืฃ
ืืื ืื ืืืืฃ ืคืื ืื ืึทืจืืขื, ืึทื ืึธื ืืืื ืืึทืื ืคึผืจืึทืกืขืกืื ื ืกืืกืืขื ืืื ืืขืืืื ืืืืืจื ืืืืฃ Amazon Kinesis. ืึธืคึผืฆืืขืก ืคึฟืึทืจ ื ืืฆื ืื Kinesis ืึทืืขื ื ืืื ืงืึทื ืืืฉืึทื ืืงืฉืึทื ืืื Kinesis Data Streams ืืื ืคืึทืงืืืฉ-ืฆืืื ืึทื ืึทืืืืืงืก Kinesis Analytics ื ืืฆื SQL ืงืึทืืึทื ืื, ืืื ืืขืืื ื ืืื ืื ืื ืืขืจืึทืงืฉืึทื ืคืื Amazon Kinesis ืืื ืื ืืขืจืข AWS ืืึทืืื ืื ืืก.
ืืืจ ืืึธืื ืืืคึผืืืื ืื ืืืืื ืกืืกืืขื ืืื ืฆืืืื ืืืขืื: ืึท ืืึทื ืฅ ืืึทื ื ืืึทื ืืึทื ืืืื ืขืจ ืืื ืึท ืฉื ืขื ืืืื ืขืจ ืคึฟืื ืื Terraform ืงืึธื.
ืึทืืข ืคึผืจืืืขืงื ืืงืืจ ืงืึธื ืืื ืื ืืืฆื , ืืื ืคึฟืึธืจืฉืืึธืื ืืืจ ืืืงืขื ืขื ืืื ืืื ืืื.
ืืื ืืื ืฆืืคืจืืื ืฆื ืืืกืงืืืืจื ืืขื ืึทืจืืืงื, ืืื ืงืืง ืคืึธืจืืืก ืฆื ืืืื ืืึทืืขืจืงืื ืืขื. ืืื ืืึธืคึฟื ืคึฟืึทืจ ืงืึทื ืกืืจืึทืงืืืื ืงืจืืืืง.
ืืื ืืืื ืืฉื ืืืจ ืืฆืืื!
ืืงืืจ: www.habr.com