ื“ืขืจืคืึทืจื•ื ื’ ื™ื ืกื˜ืึธืœื™ื ื’ Apache Airflow ืื•ื™ืฃ Windows 10

Preamble: ืœื•ื™ื˜ ื“ืขื ื’ื•ืจืœ ืคื•ืŸ ื’ื•ืจืœ, ืคึฟื•ืŸ ื“ืขืจ ื•ื•ืขืœื˜ ืคื•ืŸ ืึทืงืึทื“ืขืžื™ืง ื•ื•ื™ืกื ืฉืึทืคึฟื˜ (ืžืขื“ื™ืฆื™ืŸ) ืื™ืš ื’ืขืคึฟื•ื ืขืŸ ื–ื™ืš ืื™ืŸ ื“ืขืจ ื•ื•ืขืœื˜ ืคื•ืŸ ืื™ื ืคึฟืึธืจืžืึทืฆื™ืข ื˜ืขื›ื ืึธืœืึธื’ื™ืข, ื•ื•ื• ืื™ืš ื”ืึธื‘ืŸ ืฆื• ื ื•ืฆืŸ ืžื™ื™ืŸ ื•ื•ื™ืกืŸ ืคื•ืŸ ื“ื™ ืžืขื˜ืึทื“ืึทืœืึทื“ื–ืฉื™ ืคื•ืŸ ืงืึทื ืกื˜ืจืึทืงื˜ื™ื ื’ ืึทืŸ ืขืงืกืคึผืขืจื™ืžืขื ื˜ ืื•ืŸ ืกื˜ืจืึทื˜ืขื’ื™ืขืก ืคึฟืึทืจ ืึทื ืึทืœื™ื™ื–ื™ื ื’ ื™ืงืกืคึผืขืจืžืขื ืึทืœ ื“ืึทื˜ืŸ, ืึธื‘ืขืจ, ืฆื•ืœื™ื™ื’ืŸ ืึท ื˜ืขื›ื ืึธืœืึธื’ื™ืข ืึธื ืœื™ื™ื’ืŸ ื•ื•ืึธืก ืื™ื– ื ื™ื™ึท ืคึฟืึทืจ ืžื™ืจ. ืื™ืŸ ื“ืขื ืคึผืจืึธืฆืขืก ืคื•ืŸ ืžืึทืกื˜ืขืจื™ื ื’ ื“ื™ ื˜ืขืงื ืึทืœืึทื“ื–ืฉื™ื–, ืื™ืš ื˜ืจืขืคืŸ ืึท ื ื•ืžืขืจ ืคื•ืŸ ืฉื•ื•ืขืจื™ืงื™ื™ื˜ืŸ, ื•ื•ืึธืก, ื’ืœื™ืง, ื”ืึธื‘ืŸ ืฉื•ื™ืŸ ื‘ืึทืงื•ืžืขืŸ. ื˜ืึธืžืขืจ ื“ืขื ืคึผืึธืกื˜ืŸ ื•ื•ืขื˜ ื–ื™ื™ืŸ ื ื•ืฆื™ืง ืคึฟืึทืจ ื™ืขื ืข ื•ื•ืึธืก ื–ืขื ืขืŸ ืื•ื™ืš ืคึผื•ื ืงื˜ ืกื˜ืึทืจื˜ื™ื ื’ ืฆื• ืึทืจื‘ืขื˜ืŸ ืžื™ื˜ ืึทืคึผืึทื˜ืฉื™ ืคึผืจืึทื“ื–ืฉืขืงืก.

ืึทื–ื•ื™, ืฆื• ื“ื™ ืคื•ื ื˜. ื™ื ืกืคึผื™ืจืขื“ ืืจื˜ื™ืงืœืขืŸ ื™ื•ืจื™ ืขืžืขืœื™ืึทื ืึธื•ื• ื•ื•ืขื’ืŸ ื“ื™ ืงื™ื™ืคึผืึทื‘ื™ืœืึทื˜ื™ื– ืคื•ืŸ ืึทืคึผืึทื˜ืฉื™ ืึทื™ืจืคืœืึธื•ื• ืื™ืŸ ื“ื™ ืคืขืœื“ ืคื•ืŸ ืึธื˜ืึทืžื™ื™ืฉืึทืŸ ืคื•ืŸ ืึทื ืึทืœื™ื˜ื™ืงืึทืœ ืคึผืจืึธื•ืกื™ื“ื–ืฉืขืจื–, ืื™ืš ื’ืขื•ื•ืืœื˜ ืฆื• ืึธื ื”ื™ื™ื‘ืŸ ื ื™ืฆืŸ ื“ื™ ืคืืจื’ืขืœื™ื™ื’ื˜ ื’ืึทื ื’ ืคื•ืŸ ืœื™ื™ื‘ืจืขืจื™ื– ืื™ืŸ ืžื™ื™ืŸ ืึทืจื‘ืขื˜. ื™ืขื ืข ื•ื•ืึธืก ื–ืขื ืขืŸ ื ืึธืš ื ื™ืฉื˜ ื‘ืึทืงืึทื ื˜ ืžื™ื˜ Apache Airflow ืงืขืŸ ื–ื™ื™ืŸ ืื™ื ื˜ืขืจืขืกื™ืจื˜ ืื™ืŸ ืึท ืงื•ืจืฅ ืื™ื‘ืขืจื‘ืœื™ืง ืึทืจื˜ื™ืงืœ ืื•ื™ืฃ ื“ืขืจ ื•ื•ืขื‘ื–ื™ื™ื˜ืœ ืคื•ืŸ ื“ืขืจ ื ืึทืฉืึทื ืึทืœ ื‘ื™ื‘ืœื™ืึธื˜ืขืง. ื .ืข. ื‘ืึทื•ืžืึทืŸ.

ื–ื™ื ื˜ ื“ื™ ื’ืขื•ื•ื™ื™ื ื˜ืœืขืš ืื™ื ืกื˜ืจื•ืงืฆื™ืขืก ืคึฟืึทืจ ืคืœื™ืกื ื“ื™ืง ืึทื™ืจืคืœืึธื•ื• ื˜ืึธืŸ ื ื™ื˜ ืึทืคึผืœื™ื™ื– ืื™ืŸ ืึท Windows ืกื•ื•ื™ื•ื•ืข, ื ื•ืฆืŸ ื“ืขื ืฆื• ืกืึธืœื•ื•ืข ื“ืขื ืคึผืจืึธื‘ืœืขื ื“ืึธืงืขืจ ืื™ืŸ ืžื™ื™ืŸ ืคืึทืœ ืขืก ื•ื•ืึธืœื˜ ื–ื™ื™ืŸ ื™ื‘ืขืจื™ืง, ืื™ืš ืกื˜ืึทืจื˜ืขื“ ืงื•ืงืŸ ืคึฟืึทืจ ืื ื“ืขืจืข ืกืึทืœื•ืฉืึทื ื–. ืฆื•ืž ื’ืœื™ืง ืคึฟืึทืจ ืžื™ืจ, ืื™ืš ืื™ื– ื ื™ืฉื˜ ื“ืขืจ ืขืจืฉื˜ืขืจ ืื•ื™ืฃ ื“ืขื ื•ื•ืขื’, ืึทื–ื•ื™ ืื™ืš ื’ืขืจืื˜ืŸ ืฆื• ื’ืขืคึฟื™ื ืขืŸ ืึท ื•ื•ื•ื ื“ืขืจืœืขืš ื•ื•ื™ื“ืขื ืื™ื ืกื˜ืจื•ืงืฆื™ืขืก ื•ื•ื™ ืฆื• ื™ื ืกื˜ืึทืœื™ืจืŸ ืึทืคึผืึทื˜ืฉื™ ืึทื™ืจืคืœืึธื•ื• ืื•ื™ืฃ Windows 10 ืึธืŸ ื ื™ืฆืŸ ื“ืึธืงืงืขืจ. ืึธื‘ืขืจ, ื•ื•ื™ ืึธืคื˜ ื›ืึทืคึผืึทื ื–, ื•ื•ืขืŸ ืื™ืจ ื ืึธื›ืคืึธืœื’ืŸ ื“ื™ ืจืขืงืึทืžืขื ื“ื™ื“ ืกื˜ืขืคึผืก, ืขืก ื–ืขื ืขืŸ ืฉื•ื•ืขืจื™ืงื™ื™ื˜ืŸ ืื•ืŸ, ืื™ืš ื’ืœื•ื™ื‘ืŸ, ื ื™ืฉื˜ ื‘ืœื•ื™ื– ืคึฟืึทืจ ืžื™ืจ. ื“ืขืจื™ื‘ืขืจ, ืื™ืš ื•ื•ืึธืœื˜ ื•ื•ื™ ืฆื• ืจืขื“ืŸ ื•ื•ืขื’ืŸ ืžื™ื™ืŸ ื“ืขืจืคืึทืจื•ื ื’ ืคื•ืŸ ื™ื ืกื˜ืึธืœื™ื ื’ Apache Airflow, ืืคึฟืฉืจ ืขืก ื•ื•ืขื˜ ืจืึทื˜ืขื•ื•ืขืŸ ืขืžืขืฆืขืจ ืึท ื‘ื™ืกืœ ืฆื™ื™ื˜.

ืœืึธืžื™ืจ ื’ื™ื™ืŸ ื“ื•ืจืš ื“ื™ ืกื˜ืขืคึผืก ืคื•ืŸ ื“ื™ ื™ื ืกื˜ืจืึทืงืฉืึทื ื– (ืกืคึผื•ื™ืœืขืจ - ืึทืœืฅ ืื™ื– ื’ืขื•ื•ืขืŸ ื’ื•ื˜ ืื•ื™ืฃ ื“ื™ 5 ืฉืจื™ื˜):

1. ื™ื ืกื˜ืึธืœื™ื ื’ ื“ื™ ื•ื•ื™ื ื“ืึธื•ื– ืกื•ื‘ืกื™ืกื˜ืขื ืคึฟืึทืจ ืœื™ื ื•ืงืก ืคึฟืึทืจ ืกืึทื‘ืกืึทืงื•ื•ืึทื ื˜ ื™ื ืกื˜ืึทืœื™ืจื•ื ื’ ืคื•ืŸ ืœื™ื ื•ืงืก ื“ื™ืกื˜ืจื™ื‘ื™ื•ืฉืึทื ื–

ื“ืึธืก ืื™ื– ื“ืขืจ ืงืœืขื ืกื˜ืขืจ ืคื•ืŸ ื“ื™ ืคืจืื‘ืœืขืžืขืŸ, ื•ื•ื™ ื–ื™ื™ ื–ืึธื’ืŸ:

ืงืึธื ื˜ืจืึธืœ ืคึผืึทื ืขืœ โ†’ ืžื’ื™ืœื” โ†’ ืžื’ื™ืœื” ืื•ืŸ ืคึฟืขื™ึดืงื™ื™ื˜ืŸ โ†’ ืงืขืจ Windows ืคึฟืขื™ึดืงื™ื™ื˜ืŸ ืื•ื™ืฃ ืึธื“ืขืจ ืึทื•ื•ืขืง โ†’ Windows ืกื•ื‘ืกื™ืกื˜ืขื ืคึฟืึทืจ ืœื™ื ื•ืงืก

2. ื™ื ืกื˜ืึทืœื™ืจืŸ ื“ื™ ืœื™ื ื•ืงืก ืคืึทืจืฉืคึผืจื™ื™ื˜ื•ื ื’ ืคื•ืŸ ื“ื™ื™ืŸ ื‘ืจื™ืจื”

ืื™ืš ื’ืขื•ื•ื™ื™ื ื˜ ื“ื™ ืึทืคึผืœืึทืงื™ื™ืฉืึทืŸ ื•ื‘ื•ื ื˜ื•.

3. ื™ื ืกื˜ืึทืœื™ืจื•ื ื’ ืื•ืŸ ื“ืขืจื”ื™ื™ึทื ื˜ื™ืงืŸ ืคึผื™ืคึผ

sudo apt-get install software-properties-common
sudo apt-add-repository universe
sudo apt-get update
sudo apt-get install python-pip

4. ื™ื ืกื˜ืึธืœื™ื ื’ ืึทืคึผืึทื˜ืฉื™ ืึทื™ืจืคืœืึธื•ื•

export SLUGIFY_USES_TEXT_UNIDECODE=yes
pip install apache-airflow

5. ื“ืึทื˜ืึทื‘ืึทืกืข ื™ื ื™ื˜ื™ืึทืœื™ื–ืึทื˜ื™ืึธืŸ

ืื•ืŸ ื“ืึธ ื”ืึธื‘ืŸ ื–ื™ืš ืึธื ื’ืขื”ื•ื™ื‘ืŸ ืžืฒึทื ืข ืงืœื™ื™ื ืข ืฉื•ื•ืขืจื™ืงื™ื™ื˜ืŸ. ื“ื™ ื™ื ืกื˜ืจืึทืงืฉืึทื ื– ื“ืึทืจืคืŸ ืื™ืจ ืึทืจื™ื™ึทืŸ ื“ื™ ื‘ืึทืคึฟืขืœ airflow initdb ืื•ืŸ ืžืึทืš ืื•ื™ืฃ ืฆื• ื“ืขืจ ื•ื•ื™ื™ึทื˜ืขืจ ืฉืจื™ื˜. ืึธื‘ืขืจ, ืื™ืš ืฉื˜ืขื ื“ื™ืง ื‘ืึทืงื•ืžืขืŸ ืึทืŸ ืขื ื˜ืคืขืจ airflow: command not found. ืขืก ืื™ื– ืœืึทื“ื–ืฉื™ืงืึทืœ ืฆื• ื™ื‘ืขืจื ืขืžืขืŸ ืึทื– ืฉื•ื•ืขืจื™ืงื™ื™ื˜ืŸ ื–ืขื ืขืŸ ืื•ื™ืคื’ืขืฉื˜ืื ืขืŸ ื‘ืขืฉืึทืก ื“ื™ ื™ื™ึทื ืžืึธื ื˜ื™ืจื•ื ื’ ืคื•ืŸ Apache Airflow ืื•ืŸ ื“ื™ ื ื™ื™ื˜ื™ืง ื˜ืขืงืขืก ืคืฉื•ื˜ ื–ืขื ืขืŸ ื ื™ืฉื˜ ื‘ื ื™ืžืฆื. ื ืึธืš ืžืึทื›ืŸ ื–ื™ื›ืขืจ ืึทื– ืึทืœืฅ ืื™ื– ื•ื•ื• ืขืก ื–ืึธืœ ื–ื™ื™ืŸ, ืื™ืš ื‘ืึทืฉืœืึธืกืŸ ืฆื• ืคึผืจื•ื‘ื™ืจืŸ ืฆื• ืกืคึผืขืฆื™ืคื™ืฆื™ืจืŸ ื“ื™ ืคื•ืœ ื“ืจืš ืฆื• ื“ื™ ืึทื™ืจืคืœืึธื•ื• ื˜ืขืงืข (ืขืก ื–ืึธืœ ืงื•ืงืŸ ื•ื•ื™ ื“ืึธืก: ะŸะพะปะฝั‹ะน/ะฟัƒั‚ัŒ/ะดะพ/ั„ะฐะนะปะฐ/airflow initdb). ืื‘ืขืจ ื“ืขืจ ื ืก ืื™ื– ื ื™ืฉื˜ ื’ืขืฉืขืŸ ืื•ืŸ ื“ืขืจ ืขื ื˜ืคืขืจ ืื™ื– ื’ืขื•ื•ืขืŸ ื“ื™ ื–ืขืœื‘ืข airflow: command not found. ืื™ืš ื’ืขืคืจื•ืื•ื•ื˜ ื ื™ืฆืŸ ืึท ืงืึธืจืขื•ื• ื“ืจืš ืฆื• ื“ืขืจ ื˜ืขืงืข (./.local/bin/airflow initdb), ื•ื•ืึธืก ื’ืขืคึฟื™ืจื˜ ืฆื• ืึท ื ื™ื™ึทืข ื˜ืขื•ืช ModuleNotFoundError: No module named json'ื•ื•ืึธืก ืงืขื ืขืŸ ื–ื™ื™ืŸ ื‘ืึทืงื•ืžืขืŸ ื“ื•ืจืš ืึทืคึผื“ื™ื™ื˜ื™ื ื’ ื“ื™ ื‘ื™ื‘ืœื™ืึธื˜ืขืง ื’ืขืฆื™ื™ึทื’ (ืื™ืŸ ืžื™ื™ืŸ ืคืึทืœ ืึทืจื•ื™ืฃ ืฆื• ื•ื•ืขืจืกื™ืข 0.15.4):

pip install werkzeug==0.15.4

ืื™ืจ ืงืขื ื˜ ืœื™ื™ืขื ืขืŸ ืžืขืจ ื•ื•ืขื’ืŸ werkzeug ื“ืึธ.

ื ืึธืš ื“ืขื ืคึผืฉื•ื˜ ืžืึทื ื™ืคึผื™ืึทืœื™ื™ืฉืึทืŸ ื“ื™ ื‘ืึทืคึฟืขืœ ./.local/bin/airflow initdb ืื™ื– ื’ืขื•ื•ืขืŸ ื’ืขืขื ื“ื™ืงื˜ ื”ืฆืœื—ื”.

6. ืœืึธื ื˜ืฉื™ื ื’ ื“ื™ ืึทื™ืจืคืœืึธื•ื• ืกืขืจื•ื•ืขืจ

ื“ืึธืก ืื™ื– ื ื™ืฉื˜ ื“ืขืจ ืกื•ืฃ ืคื•ืŸ ื“ื™ ืฉื•ื•ืขืจื™ืงื™ื™ื˜ืŸ ืžื™ื˜ ืึทืงืกืขืก ื“ื™ ืœื•ืคื˜ ืœื•ื™ืคืŸ. ืœื•ื™ืคืŸ ืึท ื‘ืึทืคึฟืขืœ ./.local/bin/airflow webserver -p 8080 ืจื™ื–ืึทืœื˜ื™ื“ ืื™ืŸ ืึท ื˜ืขื•ืช No such file or directory. ืžื™ืกื˜ืึธืžืข, ืึท ื™ืงืกืคึผื™ืจื™ืึทื ืกื˜ Ubuntu ื‘ืึทื ื™ืฆืขืจ ื•ื•ืึธืœื˜ ืžื™ื“ ืคึผืจื•ื‘ื™ืจืŸ ืฆื• ื‘ืึทืงื•ืžืขืŸ ืึทื–ืึท ืฉื•ื•ืขืจื™ืงื™ื™ื˜ืŸ ืžื™ื˜ ืึทืงืกืขืก ื“ื™ ื˜ืขืงืข ื“ื•ืจืš ื ื™ืฆืŸ ื“ื™ ื‘ืึทืคึฟืขืœ. export PATH=$PATH:~/.local/bin/ (ื“"ื”, ืึทื“ื™ื ื’ /.local/bin/ ืฆื• ื“ื™ ื™ื’ื–ื™ืกื˜ื™ื ื’ PATH ืขืงืกืขืงื•ื˜ืึทื‘ืœืข ื–ื•ื›ืŸ ื“ืจืš), ืึธื‘ืขืจ ื“ืขื ืคึผืึธืกื˜ืŸ ืื™ื– ื‘ื“ืขื” ืคึฟืึทืจ ื™ืขื ืข ื•ื•ืึธืก ื‘ืคึฟืจื˜ ืึทืจื‘ืขื˜ ืžื™ื˜ Windows ืื•ืŸ ืงืขืŸ ื ื™ืฉื˜ ื˜ืจืึทื›ื˜ืŸ ืึทื– ื“ื™ ืœื™ื™ื–ื•ื ื’ ืื™ื– ืงืœืึธืจ ื•ื•ื™ ื“ืขืจ ื˜ืึธื’.

ื ืึธืš ื“ื™ ืžืึทื ื™ืคึผื™ืึทืœื™ื™ืฉืึทืŸ ื“ื™ืกืงืจื™ื™ื‘ื“ ืื•ื™ื‘ืŸ, ื“ื™ ื‘ืึทืคึฟืขืœ ./.local/bin/airflow webserver -p 8080 ืื™ื– ื”ืฆืœื—ื” ื’ืขืขื ื“ื™ืงื˜.

7. URL: ืœืึธืงืึทืœื”ืึธืกื˜: 8080/

ืื•ื™ื‘ ืึทืœืฅ ืื™ื– ื’ืขื•ื•ืขืŸ ื’ื•ื˜ ืื™ืŸ ื“ื™ ืคืจื™ืขืจื“ื™ืงืข ืกื˜ืึทื’ืขืก, ืื™ืจ ื–ืขื ื˜ ื’ืจื™ื™ื˜ ืฆื• ืงืึทื ื’ืงืขืจ ืึทื ืึทืœื™ืกื™ืก ืคึผื™ืงืก.

ืื™ืš ื”ืึธืคึฟืŸ ืึทื– ื“ื™ ื“ืขืจืคืึทืจื•ื ื’ ื“ื™ืกืงืจื™ื™ื‘ื“ ืื•ื™ื‘ืŸ ืื™ืŸ ื™ื ืกื˜ืึธืœื™ื ื’ Apache Airflow ืื•ื™ืฃ Windows 10 ื•ื•ืขื˜ ื–ื™ื™ืŸ ื ื•ืฆื™ืง ืคึฟืึทืจ ืึธื ื”ื™ื™ื‘ืขืจ ื™ื•ื–ืขืจื– ืื•ืŸ ื•ื•ืขื˜ ืคืึทืจื’ื™ื›ืขืจืŸ ื–ื™ื™ืขืจ ืคึผืึธื–ื™ืฆื™ืข ืื™ืŸ ื“ื™ ืึทืœื•ื•ืขืœื˜ ืคื•ืŸ ืžืึธื“ืขืจืŸ ืึทื ืึทืœื™ื˜ื™ืงืก ืžื›ืฉื™ืจื™ื.

ื•ื•ื™ื™ึทื˜ืขืจ ืžืึธืœ ืื™ืš ื•ื•ืึธืœื˜ ื•ื•ื™ ืฆื• ืคืึธืจื–ืขืฆืŸ ื“ื™ ื˜ืขืžืข ืื•ืŸ ืจืขื“ืŸ ื•ื•ืขื’ืŸ ื“ื™ ื“ืขืจืคืึทืจื•ื ื’ ืคื•ืŸ ื ื™ืฆืŸ Apache Airflow ืื™ืŸ ื“ื™ ืคืขืœื“ ืคื•ืŸ ืึทื ืึทืœื™ื™ื–ื™ื ื’ ื‘ืึทื ื™ืฆืขืจ ื ืึทื˜ื•ืจ ืคื•ืŸ ืจื™ืจืขื•ื•ื“ื™ืง ืึทืคึผืœืึทืงื™ื™ืฉืึทื ื–.

ืžืงื•ืจ: www.habr.com

ืœื™ื™ื’ืŸ ืึท ื‘ืึทืžืขืจืงื•ื ื’