PgGraph ืื™ื– ืึท ื ื•ืฆืŸ ืคึฟืึทืจ ืึทืจืงื™ื™ื•ื•ื™ื ื’ ืื•ืŸ ื’ืขืคึฟื™ื ืขืŸ ื˜ื™ืฉ ื“ื™ืคึผืขื ื“ืึทื ืกื™ื– ืื™ืŸ PostgreSQL

PgGraph ืื™ื– ืึท ื ื•ืฆืŸ ืคึฟืึทืจ ืึทืจืงื™ื™ื•ื•ื™ื ื’ ืื•ืŸ ื’ืขืคึฟื™ื ืขืŸ ื˜ื™ืฉ ื“ื™ืคึผืขื ื“ืึทื ืกื™ื– ืื™ืŸ PostgreSQL
ื”ื™ื™ึทื ื˜ ืื™ืš ื•ื•ื™ืœืŸ ืฆื• ืคืึธืจืฉื˜ืขืœืŸ Habr ืœื™ื™ืขื ืขืจ ืžื™ื˜ ืึท ื ื•ืฆืŸ ื’ืขืฉืจื™ื‘ืŸ ืื™ืŸ ืคึผื™ื˜ื”ืึธืŸ ืคึฟืึทืจ ืืจื‘ืขื˜ืŸ ืžื™ื˜ ื˜ื™ืฉ ื“ื™ืคึผืขื ื“ืึทื ืกื™ื– ืื™ืŸ ื“ื™ PostgreSQL DBMS.

ื“ื™ API ืคื•ืŸ ื“ื™ ื ื•ืฆืŸ ืื™ื– ืคึผืฉื•ื˜ ืื•ืŸ ื‘ืืฉื˜ื™ื™ื˜ ืคื•ืŸ ื“ืจื™ื™ ืžืขื˜ื”ืึธื“ืก:

  • archive_table - ืจืขืงื•ืจืกื™ื•ื•ืข ืึทืจื˜ืฉื™ื•ื•ื™ื ื’ / ื“ื™ืœื™ื˜ื™ื ื’ ืจืึธื•ื– ืžื™ื˜ ืกืคึผืขืกื™ืคื™ืขื“ ืขืจืฉื˜ื™ืง ืงื™ื–
  • ื‘ืึทืงื•ืžืขืŸ_ื˜ืึทื‘ืœืข_ืจืขืคืขืจืขื ืกืขืก - ื–ื•ื›ืŸ ืคึฟืึทืจ ื“ื™ืคึผืขื ื“ืึทื ืกื™ื– ืคึฟืึทืจ ืึท ื˜ื™ืฉ (ื•ื•ืขื˜ ื•ื•ื™ื™ึทื–ืŸ ื˜ื™ืฉืŸ ืจืขืคืขืจืขื ืกื˜ ื“ื•ืจืš ื“ื™ ืกืคึผืขืกื™ืคื™ืขื“ ืื™ื™ื ืขืจ ืื•ืŸ ื“ื™ ืจืขืคืขืจืขื ืกื™ื ื’ ืขืก)
  • ื‘ืึทืงื•ืžืขืŸ_ืจืึธื•ื•ืก_ืจืขืคืขืจืขื ืกืขืก - ื–ื•ื›ืŸ ืคึฟืึทืจ ืจืึธื•ื– ืื™ืŸ ืื ื“ืขืจืข ื˜ื™ืฉืŸ ื•ื•ืึธืก ื“ืขืจืžืึธื ืขืŸ ื“ื™ ืจืึธื•ื– ืื™ืŸ ื“ื™ ื’ืขื‘ืขื˜ืŸ ื˜ื™ืฉ

ืคึผืจืขื”ื™ืกื˜ืึธืจื™

ืžื™ื™ึทืŸ ื ืึธืžืขืŸ ืื™ื– ืึธืœืขื’ ื‘ืึธืจืกืึธื•ื•, ืื™ืš ื‘ื™ืŸ ืึท ื“ืขื•ื•ืขืœืึธืคึผืขืจ ืื™ืŸ ื“ื™ CRM ืžืึทื ืฉืึทืคึฟื˜ ืคึฟืึทืจ ื”ื™ืคึผืึธื˜ืขืง ืœืขื ื“ื™ื ื’ ืžืึทื ืึทื“ื–ืฉืขืจื– ืื™ืŸ ื“ืึธืžืงืœื™ืง.

ื“ื™ ื”ื•ื™ืคึผื˜ ื“ืึทื˜ืึทื‘ื™ื™ืก ืคื•ืŸ ืื•ื ื“ื–ืขืจ CRM ืกื™ืกื˜ืขื ืื™ื– ืื™ื™ื ืขืจ ืคื•ืŸ ื“ื™ ื’ืจืขืกื˜ืข ืื™ืŸ ื˜ืขืจืžื™ื ืขืŸ ืคื•ืŸ ื‘ืึทื ื“ ืื™ืŸ ื“ื™ ืคื™ืจืžืข. ืขืก ืื™ื– ืื•ื™ืš ืื™ื™ื ืขืจ ืคื•ืŸ ื“ื™ ืึธื•ืœื“ืึทืกื˜: ืขืก ืื™ื– ืืจื•ื™ืก ื‘ื™ื™ ื“ื™ ืงืึทื˜ืขืจ ืคื•ืŸ ื“ื™ ืคึผืจื•ื™ืขืงื˜, ื•ื•ืขืŸ ื“ื™ ื‘ื™ื™ืžืขืจ ื–ืขื ืขืŸ ื’ืขื•ื•ืขืŸ ื’ืจื•ื™ืก, Domklik ืื™ื– ื’ืขื•ื•ืขืŸ ืึท ืกื˜ืึทืจื˜ืึทืคึผ, ืื•ืŸ ืึทื ืฉื˜ืึธื˜ ืคื•ืŸ ืึท ืžื™ืงืจืึธ ืกืขืจื•ื•ื™ืก ืื•ื™ืฃ ืึท ืžืึธื“ืขืจืŸ ืคึผื™ื˜ื”ืึธืŸ ื™ื™ืกื™ื ื’ืงืจืึทื ืึทืก ืคืจื™ื™ืžื•ื•ืขืจืง, ืขืก ืื™ื– ื’ืขื•ื•ืขืŸ ืึท ืจื™ื–ื™ืง ืžืึทื ืึทืœื™ื˜ ืื™ืŸ PHP.

ื“ื™ ื™ื‘ืขืจื’ืึทื ื’ ืคื•ืŸ PHP ืฆื• Python ืื™ื– ื’ืขื•ื•ืขืŸ ื–ื™ื™ืขืจ ืœืึทื ื’ ืื•ืŸ ืคืืจืœืื ื’ื˜ ืกื™ื™ืžืึทืœื˜ื™ื™ื ื™ืึทืก ืฉื˜ื™ืฆืŸ ืคื•ืŸ ื‘ื™ื™ื“ืข ืกื™ืกื˜ืขืžืขืŸ, ื•ื•ืึธืก ืึทืคืขืงื˜ืึทื“ ื“ื™ ืคึผืœืึทืŸ ืคื•ืŸ ื“ื™ ื“ืึทื˜ืึทื‘ื™ื™ืก.

ื•ื•ื™ ืึท ืจืขื–ื•ืœื˜ืึทื˜, ืžื™ืจ ื”ืึธื‘ืŸ ืึท ื“ืึทื˜ืึทื‘ื™ื™ืก ืžื™ื˜ ืึท ื’ืจื•ื™ืก ื ื•ืžืขืจ ืคื•ืŸ ื”ืขื›ืกื˜ ืงืึธื ื ืขืงื˜ืขื“ ืื•ืŸ ืจื™ื–ื™ืง ื˜ื™ืฉืŸ ืžื™ื˜ ืึท ื‘ื™ื ื˜ืœ ืคื•ืŸ ื™ื ื“ืขืงืกื™ื– ืคึฟืึทืจ ืคืึทืจืฉื™ื“ืขื ืข ื˜ื™ื™ืคึผืก ืคื•ืŸ ืงื•ื•ื™ืจื™ื–. ืึทืœืข ื“ืขื ืึทืคืขืงืฅ ื“ื™ ืคืึธืจืฉื˜ืขืœื•ื ื’ ืคื•ืŸ ื“ื™ ื“ืึทื˜ืึทื‘ื™ื™ืก ื ืขื’ืึทื˜ื™ื•ื•ืœื™: ืจืขื›ื˜ ืฆื• ื’ืจื•ื™ืก ื˜ื™ืฉืŸ ืื•ืŸ ืึท ื‘ื™ื ื˜ืœ ืคื•ืŸ ื‘ืึทืฆื™ื•ื ื’ืขืŸ ืฆื•ื•ื™ืฉืŸ ื–ื™ื™, ื“ื™ ืงืึทืžืคึผืœืขืงืกื™ื˜ื™ ืคื•ืŸ ืงื•ื•ื™ืจื™ื– ืื™ื– ืงืขืกื™ื™ื“ืขืจ ื™ื ืงืจื™ืกื™ื ื’, ื•ื•ืึธืก ืื™ื– ืกืคึผืขืฆื™ืขืœ ืงืจื™ื˜ื™ืฉ ืคึฟืึทืจ ื“ื™ ืžืขืจืกื˜ ืœืึธื•ื“ื™ื“ ื˜ื™ืฉืŸ.

ืฆื• ืจืขื“ื•ืฆื™ืจืŸ ื“ื™ ืžืึทืกืข ืื•ื™ืฃ ื“ื™ ื“ืึทื˜ืึทื‘ื™ื™ืก, ืžื™ืจ ื‘ืึทืฉืœืึธืกืŸ ืฆื• ืฉืจื™ื™ึทื‘ืŸ ืึท ืฉืจื™ืคื˜ ื•ื•ืึธืก ื•ื•ืึธืœื˜ ืึทืจื™ื‘ืขืจืคื™ืจืŸ ืึทืœื˜ ืจืขืงืึธืจื“ืก ืคื•ืŸ ื“ื™ ืžืขืจืกื˜ ื•ื•ืึทืœื•ืžืึทื ืึทืก ืื•ืŸ ืœืึธื•ื“ื™ื“ ื˜ื™ืฉืŸ ืฆื• ืึทืจื˜ืฉื™ื•ื•ืขื“ (ืœืžืฉืœ, ืคึฟื•ืŸ task ะฒ task_archive).

ื“ืขื ืึทืจื‘ืขื˜ ืื™ื– ืงืึธืžืคึผืœื™ืฆื™ืจื˜ ื“ื•ืจืš ื“ื™ ื’ืจื•ื™ืก ื ื•ืžืขืจ ืคื•ืŸ ื‘ืึทืฆื™ื•ื ื’ืขืŸ ืฆื•ื•ื™ืฉืŸ ื˜ื™ืฉืŸ: ืคืฉื•ื˜ ืžืึทืš ืจืึธื•ื– ืคื•ืŸ task ะฒ task_archive ืื™ื– ื ื™ืฉื˜ ื’ืขื ื•ื’, ืื™ื™ื“ืขืจ ืื™ืจ ื“ืึทืจืคึฟืŸ ืฆื• ื˜ืึธืŸ ื“ื™ ื–ืขืœื‘ืข ืจืขืงื•ืจืกื™ื•ื•ืœื™ ืžื™ื˜ ืึทืœืข ื“ื™ ืจืขืคืขืจืขื ืกื™ื ื’ task ื˜ื™ืฉืŸ.

ืื™ืš ื•ื•ืขืœ ื“ืขืžืึธื ืกื˜ืจื™ืจืŸ ืžื™ื˜ ืึท ื‘ื™ื™ืฉืคึผื™ืœ ื“ืขืžืึธ ื“ืึทื˜ืึทื‘ื™ื™ืก ืคื•ืŸ ื“ื™ ืคึผืœืึทืฅ postgrespro.ru:

PgGraph ืื™ื– ืึท ื ื•ืฆืŸ ืคึฟืึทืจ ืึทืจืงื™ื™ื•ื•ื™ื ื’ ืื•ืŸ ื’ืขืคึฟื™ื ืขืŸ ื˜ื™ืฉ ื“ื™ืคึผืขื ื“ืึทื ืกื™ื– ืื™ืŸ PostgreSQL
ื–ืืœ ืก ื–ืึธื’ืŸ ืžื™ืจ ื“ืึทืจืคึฟืŸ ืฆื• ื•ื™ืกืžืขืงืŸ ืจืขืงืึธืจื“ืก ืคื•ืŸ ืึท ื˜ื™ืฉ Flights. Postgres ื•ื•ืขื˜ ื ื™ืฉื˜ ืœืึธื–ืŸ ืื•ื ื“ื– ืฆื• ื˜ืึธืŸ ื“ืึธืก ืคึผื•ื ืงื˜ ืึทื–ื•ื™: ืžื™ืจ ืขืจืฉื˜ืขืจ ื“ืึทืจืคึฟืŸ ืฆื• ื•ื™ืกืžืขืงืŸ ืจืขืงืึธืจื“ืก ืคื•ืŸ ืึทืœืข ืจืขืคืขืจืขื ืกื™ื ื’ ื˜ื™ืฉืŸ, ืื•ืŸ ืึทื–ื•ื™ ืื•ื™ืฃ ืจืขืงื•ืจืกื™ื•ื•ืขืœื™ ืึทืจืึธืคึผ ืฆื• ื˜ื™ืฉืŸ ื•ื•ืึธืก ื–ืขื ืขืŸ ื ื™ืฉื˜ ืจืขืคืขืจืขื ืกืขื“ ื“ื•ืจืš ื•ื•ืขืจ ืขืก ื™ื–.

ืื™ืŸ ืื•ื ื“ื–ืขืจ ื‘ื™ื™ึทืฉืคึผื™ืœ ื‘ื™ื™ึท Flights ืจืขืคืขืจืก Ticket_flightsืื•ืŸ ืื•ื™ืฃ ืื™ืจ - Boarding_passes.

ื“ืขืจื™ื‘ืขืจ, ืื™ืจ ื“ืึทืจืคึฟืŸ ืฆื• ื•ื™ืกืžืขืงืŸ ืขืก ืื™ืŸ ื“ืขื ืกื“ืจ:

  1. ืžื™ืจ ื‘ืึทืงื•ืžืขืŸ ื“ื™ ืขืจืฉื˜ื™ืง ืฉืœื™ืกืœืขืŸ (PK) ื•ื•ืึทืœื•ืขืก ืคื•ืŸ ืจืึธื•ื– ืื™ืŸ Ticket_flights, ื•ื•ืึธืก ืึธืคึผืฉื™ืงืŸ ืฆื• ื“ื™ ืจืึธื•ื– ืฆื• ื•ื•ืขืจืŸ ืื•ื™ืกื’ืขืžืขืงื˜ ืื™ืŸ Flights.
  2. ืžื™ืจ ื‘ืึทืงื•ืžืขืŸ ืคึผืง ืจืึธื•ื– Boarding_passes, ื•ื•ืึธืก ืึธืคึผืฉื™ืงืŸ ืฆื• Ticket_flights.
  3. ืžื™ืจ ื•ื™ืกืžืขืงืŸ ืจืึธื•ื– ื“ื•ืจืš ืคึผืง ืคึฟื•ืŸ ืฉืจื™ื˜ 2 ืื™ืŸ ื“ื™ ื˜ื™ืฉ Boarding_passes.
  4. ื•ื™ืกืžืขืงืŸ ืฉื•ืจื•ืช ื“ื•ืจืš ืคึผืง ืคึฟื•ืŸ ืฉืจื™ื˜ 1 ืื™ืŸ Ticket_flights.
  5. ืจื™ืžื•ื•ื•ื™ื ื’ ืฉื•ืจื•ืช ืคื•ืŸ Flights.

ื“ืขืจ ืจืขื–ื•ืœื˜ืึทื˜ ืื™ื– ื’ืขื•ื•ืขืŸ ืึท ื ื•ืฆืŸ ื’ืขืจื•ืคืŸ PgGraph, ื•ื•ืึธืก ืžื™ืจ ื‘ืึทืฉืœืึธืกืŸ ืฆื• ืžืึทื›ืŸ ืึธืคึฟืŸ ืžืงื•ืจ.

ื•ื•ื™ ืฆื• ื ื•ืฆืŸ

ื“ื™ ื ื•ืฆืŸ ืฉื˜ื™ืฆื˜ ืฆื•ื•ื™ื™ ืžืึธื“ืขืก ืคื•ืŸ ื ื•ืฆืŸ:

  • ืจื•ืคืŸ ืคื•ืŸ ื“ื™ ื‘ืึทืคึฟืขืœืŸ ืฉื•ืจื” (pggraph โ€ฆ).
  • ื‘ืึทื ื™ืฅ ืื™ืŸ ืคึผื™ื˜ื”ืึธืŸ ืงืึธื“ (ืงืœืึทืก PgGraphApi).

ื™ื ืกื˜ืึทืœื™ืจื•ื ื’ ืื•ืŸ ืงืึทื ืคื™ื’ื™ืขืจื™ื™ืฉืึทืŸ

ืขืจืฉื˜ืขืจ ืื™ืจ ื“ืึทืจืคึฟืŸ ืฆื• ื™ื ืกื˜ืึทืœื™ืจืŸ ื“ื™ ื ื•ืฆืŸ ืคื•ืŸ ื“ื™ Pypi ืจื™ืคึผืึทื–ืึทื˜ืึธืจื™:

pip3 install pggraph

ื“ืขืจื ืึธืš ืฉืึทืคึฟืŸ ืึท config.ini ื˜ืขืงืข ืื•ื™ืฃ ื“ื™ ื”ื™ื’ืข ืžืึทืฉื™ืŸ ืžื™ื˜ ื“ื™ ืงืึทื ืคื™ื’ื™ืขืจื™ื™ืฉืึทืŸ ืคื•ืŸ ื“ื™ ื“ืึทื˜ืึทื‘ื™ื™ืก ืื•ืŸ ื“ื™ ืึทืจื˜ืฉื™ื•ื•ื™ื ื’ ืฉืจื™ืคื˜:

[db]
host = localhost
port = 5432
user = postgres
password = postgres
dbname = postgres
schema = public ; ะะตะพะฑัะทะฐั‚ะตะปัŒะฝั‹ะน ะฟะฐั€ะฐะผะตั‚ั€, ัƒะบะฐะทะฐะฝะพ ะทะฝะฐั‡ะตะฝะธะต ะฟะพ ัƒะผะพะปั‡ะฐะฝะธัŽ

[archive]  ; ะ”ะฐะฝะฝั‹ะน ั€ะฐะทะดะตะป ะทะฐะฟะพะปะฝัั‚ัŒ ะฝะตะพะฑัะทะฐั‚ะตะปัŒะฝะพ, ะฝะธะถะต ัƒะบะฐะทะฐะฝั‹ ะทะฝะฐั‡ะตะฝะธั ะฟะพ ัƒะผะพะปั‡ะฐะฝะธัŽ
is_debug = false
chunk_size = 1000
max_depth = 20
to_archive = true
archive_suffix = 'archive'

ืœื•ื™ืคืŸ ืคึฟื•ืŸ ืงืึทื ืกืึธื•ืœ

ืคึผืึทืจืึทืžืขื˜ืขืจืก

$ pggraph -h
usage: pggraph action [-h] --table TABLE [--ids IDS] [--config_path CONFIG_PATH]
positional arguments:
  action        required action: archive_table, get_table_references, get_rows_references

optional arguments:
  -h, --help                    show this help message and exit
  --table TABLE                 table name
  --ids IDS                     primary key ids, separated by comma, e.g. 1,2,3
  --config_path CONFIG_PATH     path to config.ini
  --log_path LOG_PATH           path to log dir
  --log_level LOG_LEVEL         log level (debug, info, error)

ืคืื–ื™ืฆื™ืื ืืœืข ื˜ืขื ื•ืช:

  • action - ืคืืจืœืื ื’ื˜ ืงืึทืžืฃ: archive_table, get_table_references ืึธื“ืขืจ get_rows_references.

ืึทืจื’ื•ืžืขื ื˜ืŸ:

  • --config_path - ื“ืจืš ืฆื• ื“ื™ ืงืึธื ืคื™ื’ ื˜ืขืงืข;
  • --table - ืึท ื˜ื™ืฉ ืžื™ื˜ ื•ื•ืึธืก ืื™ืจ ื“ืึทืจืคึฟืŸ ืฆื• ื“ื•ืจื›ืคื™ืจืŸ ืึท ืงืึทืžืฃ;
  • --ids - ืจืฉื™ืžื” ืคื•ืŸ ID ืืคื’ืขืฉื™ื™ื“ื˜ ื“ื•ืจืš ืงืึธืžืข, ืœืžืฉืœ, 1,2,3 (ืึทืคึผืฉืึทื ืึทืœ ืคึผืึทืจืึทืžืขื˜ืขืจ);
  • --log_path - ื“ืจืš ืฆื• ื“ืขืจ ื˜ืขืงืข ืคึฟืึทืจ ืœืึธื’ืก (ืึทืคึผืฉืึทื ืึทืœ ืคึผืึทืจืึทืžืขื˜ืขืจ, ื“ื•ืจืš ืคืขืœื™ืงื™ื™ึทื˜ - ื”ื™ื™ื ื˜ืขืงืข);
  • --log_level - ืœืึธื’ื™ื ื’ ืžื“ืจื’ื” (ืึทืคึผืฉืึทื ืึทืœ ืคึผืึทืจืึทืžืขื˜ืขืจ, ืคืขืœื™ืงื™ื™ึทื˜ ืื™ื– INFO).

ื‘ืึทืคึฟืขืœ ื‘ื™ื™ืฉืคื™ืœืŸ

ืึทืจื˜ืฉื™ื•ื•ื™ื ื’ ืึท ื˜ื™ืฉ

ื“ื™ ื”ื•ื™ืคึผื˜ ืคึฟื•ื ืงืฆื™ืข ืคื•ืŸ โ€‹โ€‹ื“ื™ ื ื•ืฆืŸ ืื™ื– ื“ืึทื˜ืŸ ืึทืจื˜ืฉื™ื•ื•ื™ื ื’, ื“.ื”. ื˜ืจืึทื ืกืคืขืจื™ื ื’ ืจืึธื•ื– ืคื•ืŸ ื“ื™ ื”ื•ื™ืคึผื˜ ื˜ื™ืฉ ืฆื• ื“ื™ ืึทืจืงื™ื™ื•ื• ื˜ื™ืฉ (ืœืžืฉืœ, ืคึฟื•ืŸ ื“ื™ ื˜ื™ืฉ ืกืคืจื™ื ะฒ books_archive).

ื“ื™ืœื™ืฉืึทืŸ ืึธืŸ ืึทืจื˜ืฉื™ื•ื•ื™ื ื’ ืื™ื– ืื•ื™ืš ื’ืขืฉื˜ื™ืฆื˜: ืคึฟืึทืจ ื“ืขื ืื™ืจ ื“ืึทืจืคึฟืŸ ืฆื• ืฉื˜ืขืœืŸ ื“ืขื ืคึผืึทืจืึทืžืขื˜ืขืจ ืื™ืŸ config.ini to_archive = ืคืึทืœืฉ).

ืคืืจืœืื ื’ื˜ ืคึผืึทืจืึทืžืขื˜ืขืจืก - config_path, ื˜ื™ืฉ ืื•ืŸ ื™ื“ืก.

ื ืึธืš ืงืึทื˜ืขืจ, ืจืขืงืึธืจื“ืก ื•ื•ืขื˜ ื–ื™ื™ืŸ ืจืขืงื•ืจืกื™ื•ื•ืขืœื™ ืื•ื™ืกื’ืขืžืขืงื˜ ids ืื™ืŸ ื“ื™ ื˜ื™ืฉ table ืื•ืŸ ืื™ืŸ ืึทืœืข ื˜ื™ืฉืŸ ื•ื•ืึธืก ืจื™ืคืขืจื“ ืฆื• ืื™ื.

$ pggraph archive_table --config_path config.hw.local.ini --table flights --ids 1,2,3
2020-06-20 19:27:44 INFO: flights - START
2020-06-20 19:27:44 INFO: flights - start archive_recursive 3 rows (depth=0)
2020-06-20 19:27:44 INFO:       START ARCHIVE REFERRING TABLES
2020-06-20 19:27:44 INFO:       ticket_flights - start archive_recursive 3 rows (depth=1)
2020-06-20 19:27:44 INFO:               START ARCHIVE REFERRING TABLES
2020-06-20 19:27:44 INFO:               boarding_passes - start archive_recursive 3 rows (depth=2)
2020-06-20 19:27:44 INFO:                       START ARCHIVE REFERRING TABLES
2020-06-20 19:27:44 INFO:                       END ARCHIVE REFERRING TABLES
2020-06-20 19:27:44 INFO:               boarding_passes - archive_by_ids 3 rows by ticket_no, flight_id
2020-06-20 19:27:44 INFO:               boarding_passes - start archive_recursive 3 rows (depth=2)
2020-06-20 19:27:44 INFO:                       START ARCHIVE REFERRING TABLES
2020-06-20 19:27:44 INFO:                       END ARCHIVE REFERRING TABLES
2020-06-20 19:27:44 INFO:               boarding_passes - archive_by_ids 3 rows by ticket_no, flight_id
2020-06-20 19:27:44 INFO:               boarding_passes - start archive_recursive 3 rows (depth=2)
2020-06-20 19:27:44 INFO:                       START ARCHIVE REFERRING TABLES
2020-06-20 19:27:44 INFO:                       END ARCHIVE REFERRING TABLES
2020-06-20 19:27:44 INFO:               boarding_passes - archive_by_ids 3 rows by ticket_no, flight_id
2020-06-20 19:27:44 INFO:               boarding_passes - start archive_recursive 3 rows (depth=2)
2020-06-20 19:27:44 INFO:                       START ARCHIVE REFERRING TABLES
2020-06-20 19:27:44 INFO:                       END ARCHIVE REFERRING TABLES
2020-06-20 19:27:44 INFO:               boarding_passes - archive_by_ids 3 rows by ticket_no, flight_id
2020-06-20 19:27:44 INFO:               END ARCHIVE REFERRING TABLES
2020-06-20 19:27:44 INFO:       ticket_flights - archive_by_ids 3 rows by ticket_no, flight_id
2020-06-20 19:27:44 INFO:       END ARCHIVE REFERRING TABLES
2020-06-20 19:27:44 INFO: flights - archive_by_ids 3 rows by id
2020-06-20 19:27:44 INFO: flights - END

ื’ืขืคึฟื™ื ืขืŸ ื“ื™ืคึผืขื ื“ืึทื ืกื™ื– ืคึฟืึทืจ ืึท ืกืคึผืขืกืึทืคื™ื™ื“ ื˜ื™ืฉ

ืคึฟื•ื ืงืฆื™ืข ืฆื• ื’ืขืคึฟื™ื ืขืŸ ื“ื™ืคึผืขื ื“ืึทื ืกื™ื– ืคื•ืŸ ืึท ืกืคึผืขืกื™ืคื™ืขื“ ื˜ื™ืฉ table. ืคืืจืœืื ื’ื˜ ืคึผืึทืจืึทืžืขื˜ืขืจืก - config_path ะธ table.

ื ืึธืš ืงืึทื˜ืขืจ, ืึท ื•ื•ืขืจื˜ืขืจื‘ื•ืš ื•ื•ืขื˜ ื–ื™ื™ืŸ ื’ืขื•ื•ื™ื–ืŸ ืื•ื™ืฃ ื“ืขื ืขืงืจืึทืŸ, ื•ื•ื•:

  • in_refs - ืึท ื•ื•ืขืจื˜ืขืจื‘ื•ืš ืคื•ืŸ ื˜ื™ืฉืŸ ืจืขืคืขืจืขื ืกื™ื ื’ ืึท ื’ืขื’ืขื‘ืŸ ืื™ื™ื ืขืจ, ื•ื•ื• ื“ืขืจ ืฉืœื™ืกืœ ืื™ื– ื“ื™ ื ืึธืžืขืŸ ืคื•ืŸ ื“ื™ ื˜ื™ืฉ, ื“ื™ ื•ื•ืขืจื˜ ืื™ื– ืึท ืจืฉื™ืžื” ืคื•ืŸ ืคืจืขืžื“ ืฉืœื™ืกืœ ืึทื‘ื“ื–ืฉืขืงืฅ (pk_main - ืขืจืฉื˜ื™ืง ืฉืœื™ืกืœ ืื™ืŸ ื“ื™ ื”ื•ื™ืคึผื˜ ื˜ื™ืฉ, pk_ref - ืขืจืฉื˜ื™ืง ืฉืœื™ืกืœ ืื™ืŸ ื“ื™ ืจืขืคืขืจืขื ืกื™ื ื’ ื˜ื™ืฉ, fk_ref - ื“ืขืจ ื ืึธืžืขืŸ ืคื•ืŸ ื“ืขืจ ื–ื™ื™ึทืœ ื•ื•ืึธืก ืื™ื– ื“ืขืจ ืคืจืขืžื“ ืฉืœื™ืกืœ ืฆื• ื“ื™ ืžืงื•ืจ ื˜ื™ืฉ);
  • out_refs โ€” ื ื•ื•ืขืจื˜ืขืจื‘ื•ืš ืคื•ืŸ ื˜ื™ืฉืŸ ืื•ื™ืฃ ื“ืขื ืจืขืคืขืจื™ืจื˜ ืžืขืŸ.

$ pggraph get_table_references --config_path config.hw.local.ini --table flights
{'in_refs': {'ticket_flights': [ForeignKey(pk_main='flight_id', pk_ref='ticket_no, flight_id', fk_ref='flight_id')]},
 'out_refs': {'aircrafts': [ForeignKey(pk_main='aircraft_code', pk_ref='flight_id', fk_ref='aircraft_code')],
              'airports': [ForeignKey(pk_main='airport_code', pk_ref='flight_id', fk_ref='arrival_airport'),
                           ForeignKey(pk_main='airport_code', pk_ref='flight_id', fk_ref='departure_airport')]}}

ื’ืขืคึฟื™ื ืขืŸ ืจืขืคืขืจืขื ืฆืŸ ืฆื• ืกื˜ืจื™ื ื’ืก ืžื™ื˜ ื“ื™ ืกืคึผืขืกืึทืคื™ื™ื“ ืขืจืฉื˜ื™ืง ืฉืœื™ืกืœ

ืคื•ื ืงืฆื™ืข ืฆื• ื–ื•ื›ืŸ ืคึฟืึทืจ ืจืึธื•ื– ืื™ืŸ ืื ื“ืขืจืข ื˜ื™ืฉืŸ ื•ื•ืึธืก ืึธืคึผืฉื™ืงืŸ ืฆื• ืจืึธื•ื– ื“ื•ืจืš ืคืจืขืžื“ ืฉืœื™ืกืœ ids ื˜ื™ืฉืŸ table. ืคืืจืœืื ื’ื˜ ืคึผืึทืจืึทืžืขื˜ืขืจืก - config_path, table ะธ ids.

ื ืึธืš ืงืึทื˜ืขืจ, ืึท ื•ื•ืขืจื˜ืขืจื‘ื•ืš ืžื™ื˜ ื“ื™ ืคืืœื’ืขื ื“ืข ืกื˜ืจื•ืงื˜ื•ืจ ื•ื•ืขื˜ ื–ื™ื™ืŸ ื’ืขื•ื•ื™ื–ืŸ ืื•ื™ืฃ ื“ืขื ืขืงืจืึทืŸ:

{
	pk_id_1: {
		reffering_table_name_1: {
			foreign_key_1: [
				{row_pk_1: value, row_pk_2: value},
				...
			], 
			...
		},
		...
	},
	pk_id_2: {...},
	...
}

ื‘ื™ื™ึทืฉืคึผื™ืœ ืจื•ืคืŸ:

$ pggraph get_rows_references --config_path config.hw.local.ini --table flights --ids 1,2,3
{1: {'ticket_flights': {'flight_id': [{'flight_id': 1,
                                       'ticket_no': '0005432816945'},
                                      {'flight_id': 1,
                                       'ticket_no': '0005432816941'}]}},
 2: {'ticket_flights': {'flight_id': [{'flight_id': 2,
                                       'ticket_no': '0005433101832'},
                                      {'flight_id': 2,
                                       'ticket_no': '0005433101864'},
                                      {'flight_id': 2,
                                       'ticket_no': '0005432919715'}]}},
 3: {'ticket_flights': {'flight_id': [{'flight_id': 3,
                                       'ticket_no': '0005432817560'},
                                      {'flight_id': 3,
                                       'ticket_no': '0005432817568'},
                                      {'flight_id': 3,
                                       'ticket_no': '0005432817559'}]}}}

ื ื•ืฆืŸ ืื™ืŸ ืงืึธื“

ืื™ืŸ ืึทื“ื™ืฉืึทืŸ ืฆื• ืœื•ื™ืคืŸ ืขืก ืื™ืŸ ื“ื™ ืงืึทื ืกืึธื•ืœ, ื“ื™ ื‘ื™ื‘ืœื™ืึธื˜ืขืง ืงืขื ืขืŸ ื–ื™ื™ืŸ ื’ืขื•ื•ื™ื™ื ื˜ ืื™ืŸ ืคึผื™ื˜ื”ืึธืŸ ืงืึธื“. ื‘ื™ื™ืฉืคื™ืœืŸ ืคื•ืŸ ืงืึทืœืœืก ืื™ืŸ ื“ื™ iPython ื™ื ื˜ืขืจืึทืงื˜ื™ื•ื• ืกื•ื•ื™ื•ื•ืข ื–ืขื ืขืŸ ื’ืขื•ื•ื™ื–ืŸ ืื•ื ื˜ืŸ.

ืึทืจื˜ืฉื™ื•ื•ื™ื ื’ ืึท ื˜ื™ืฉ

>>> from pg_graph.main import setup_logging
>>> setup_logging(log_level='DEBUG')
>>> from pg_graph.api import PgGraphApi
>>> api = PgGraphApi('config.hw.local.ini')
>>> api.archive_table('flights', [4,5])
2020-06-20 23:12:08 INFO: flights - START
2020-06-20 23:12:08 INFO: flights - start archive_recursive 2 rows (depth=0)
2020-06-20 23:12:08 INFO: 	START ARCHIVE REFERRING TABLES
2020-06-20 23:12:08 DEBUG: 	ticket_flights - ForeignKey(pk_main='flight_id', pk_ref='flight_id, ticket_no', fk_ref='flight_id')
2020-06-20 23:12:08 DEBUG: 	SQL('SELECT flight_id, ticket_no FROM bookings.ticket_flights WHERE (flight_id) IN (%s, %s)')
2020-06-20 23:12:08 INFO: 	ticket_flights - start archive_recursive 30 rows (depth=1)
2020-06-20 23:12:08 INFO: 		START ARCHIVE REFERRING TABLES
2020-06-20 23:12:08 DEBUG: 		boarding_passes - ForeignKey(pk_main='flight_id, ticket_no', pk_ref='flight_id, ticket_no', fk_ref='flight_id, ticket_no')
2020-06-20 23:12:08 INFO: 		boarding_passes - archive_by_fk 30 rows by ForeignKey(pk_main='flight_id, ticket_no', pk_ref='flight_id, ticket_no', fk_ref='flight_id, ticket_no')
2020-06-20 23:12:08 DEBUG: 		SQL('CREATE TABLE IF NOT EXISTS bookings.boarding_passes_archive (LIKE bookings.boarding_passes)')
2020-06-20 23:12:08 DEBUG: 		DELETE FROM boarding_passes by FK flight_id, ticket_no - 30 rows
2020-06-20 23:12:08 INFO: 		END ARCHIVE REFERRING TABLES
2020-06-20 23:12:08 INFO: 	ticket_flights - archive_by_ids 30 rows by flight_id, ticket_no
2020-06-20 23:12:08 DEBUG: 	SQL('CREATE TABLE IF NOT EXISTS bookings.ticket_flights_archive (LIKE bookings.ticket_flights)')
2020-06-20 23:12:08 DEBUG: 	DELETE FROM ticket_flights by flight_id, ticket_no - 30 rows
2020-06-20 23:12:08 DEBUG: 	INSERT INTO ticket_flights_archive - 30 rows
2020-06-20 23:12:08 INFO: 	ticket_flights - start archive_recursive 30 rows (depth=1)
2020-06-20 23:12:08 INFO: 		START ARCHIVE REFERRING TABLES
2020-06-20 23:12:08 DEBUG: 		boarding_passes - ForeignKey(pk_main='flight_id, ticket_no', pk_ref='flight_id, ticket_no', fk_ref='flight_id, ticket_no')
2020-06-20 23:12:08 INFO: 		boarding_passes - archive_by_fk 30 rows by ForeignKey(pk_main='flight_id, ticket_no', pk_ref='flight_id, ticket_no', fk_ref='flight_id, ticket_no')
2020-06-20 23:12:08 DEBUG: 		SQL('CREATE TABLE IF NOT EXISTS bookings.boarding_passes_archive (LIKE bookings.boarding_passes)')
2020-06-20 23:12:08 DEBUG: 		DELETE FROM boarding_passes by FK flight_id, ticket_no - 30 rows
2020-06-20 23:12:08 INFO: 		END ARCHIVE REFERRING TABLES
2020-06-20 23:12:08 INFO: 	ticket_flights - archive_by_ids 30 rows by flight_id, ticket_no
2020-06-20 23:12:08 DEBUG: 	SQL('CREATE TABLE IF NOT EXISTS bookings.ticket_flights_archive (LIKE bookings.ticket_flights)')
2020-06-20 23:12:08 DEBUG: 	DELETE FROM ticket_flights by flight_id, ticket_no - 30 rows
2020-06-20 23:12:08 DEBUG: 	INSERT INTO ticket_flights_archive - 30 rows
2020-06-20 23:12:08 INFO: 	ticket_flights - start archive_recursive 30 rows (depth=1)
2020-06-20 23:12:08 INFO: 		START ARCHIVE REFERRING TABLES
2020-06-20 23:12:08 DEBUG: 		boarding_passes - ForeignKey(pk_main='flight_id, ticket_no', pk_ref='flight_id, ticket_no', fk_ref='flight_id, ticket_no')
2020-06-20 23:12:08 INFO: 		boarding_passes - archive_by_fk 30 rows by ForeignKey(pk_main='flight_id, ticket_no', pk_ref='flight_id, ticket_no', fk_ref='flight_id, ticket_no')
2020-06-20 23:12:08 DEBUG: 		SQL('CREATE TABLE IF NOT EXISTS bookings.boarding_passes_archive (LIKE bookings.boarding_passes)')
2020-06-20 23:12:08 DEBUG: 		DELETE FROM boarding_passes by FK flight_id, ticket_no - 30 rows
2020-06-20 23:12:08 INFO: 		END ARCHIVE REFERRING TABLES
2020-06-20 23:12:08 INFO: 	ticket_flights - archive_by_ids 30 rows by flight_id, ticket_no
2020-06-20 23:12:08 DEBUG: 	SQL('CREATE TABLE IF NOT EXISTS bookings.ticket_flights_archive (LIKE bookings.ticket_flights)')
2020-06-20 23:12:08 DEBUG: 	DELETE FROM ticket_flights by flight_id, ticket_no - 30 rows
2020-06-20 23:12:08 DEBUG: 	INSERT INTO ticket_flights_archive - 30 rows
2020-06-20 23:12:08 INFO: 	ticket_flights - start archive_recursive 3 rows (depth=1)
2020-06-20 23:12:08 INFO: 		START ARCHIVE REFERRING TABLES
2020-06-20 23:12:08 DEBUG: 		boarding_passes - ForeignKey(pk_main='flight_id, ticket_no', pk_ref='flight_id, ticket_no', fk_ref='flight_id, ticket_no')
2020-06-20 23:12:08 INFO: 		boarding_passes - archive_by_fk 3 rows by ForeignKey(pk_main='flight_id, ticket_no', pk_ref='flight_id, ticket_no', fk_ref='flight_id, ticket_no')
2020-06-20 23:12:08 DEBUG: 		SQL('CREATE TABLE IF NOT EXISTS bookings.boarding_passes_archive (LIKE bookings.boarding_passes)')
2020-06-20 23:12:08 DEBUG: 		DELETE FROM boarding_passes by FK flight_id, ticket_no - 3 rows
2020-06-20 23:12:08 INFO: 		END ARCHIVE REFERRING TABLES
2020-06-20 23:12:08 INFO: 	ticket_flights - archive_by_ids 3 rows by flight_id, ticket_no
2020-06-20 23:12:08 DEBUG: 	SQL('CREATE TABLE IF NOT EXISTS bookings.ticket_flights_archive (LIKE bookings.ticket_flights)')
2020-06-20 23:12:08 DEBUG: 	DELETE FROM ticket_flights by flight_id, ticket_no - 3 rows
2020-06-20 23:12:08 DEBUG: 	INSERT INTO ticket_flights_archive - 3 rows
2020-06-20 23:12:08 INFO: 	END ARCHIVE REFERRING TABLES
2020-06-20 23:12:08 INFO: flights - archive_by_ids 2 rows by flight_id
2020-06-20 23:12:09 DEBUG: SQL('CREATE TABLE IF NOT EXISTS bookings.flights_archive (LIKE bookings.flights)')
2020-06-20 23:12:09 DEBUG: DELETE FROM flights by flight_id - 2 rows
2020-06-20 23:12:09 DEBUG: INSERT INTO flights_archive - 2 rows
2020-06-20 23:12:09 INFO: flights - END

ื’ืขืคึฟื™ื ืขืŸ ื“ื™ืคึผืขื ื“ืึทื ืกื™ื– ืคึฟืึทืจ ืึท ืกืคึผืขืกืึทืคื™ื™ื“ ื˜ื™ืฉ

>>> from pg_graph.api import PgGraphApi
>>> from pprint import pprint
>>> api = PgGraphApi('config.hw.local.ini')
>>> res = api.get_table_references('flights')
>>> pprint(res)
{'in_refs': {'ticket_flights': [ForeignKey(pk_main='flight_id', pk_ref='flight_id, ticket_no', fk_ref='flight_id')]},
 'out_refs': {'aircrafts': [ForeignKey(pk_main='aircraft_code', pk_ref='flight_id', fk_ref='aircraft_code')],
              'airports': [ForeignKey(pk_main='airport_code', pk_ref='flight_id', fk_ref='arrival_airport'),
                           ForeignKey(pk_main='airport_code', pk_ref='flight_id', fk_ref='departure_airport')]}}

ื’ืขืคึฟื™ื ืขืŸ ืจืขืคืขืจืขื ืฆืŸ ืฆื• ืกื˜ืจื™ื ื’ืก ืžื™ื˜ ื“ื™ ืกืคึผืขืกืึทืคื™ื™ื“ ืขืจืฉื˜ื™ืง ืฉืœื™ืกืœ

>>> from pg_graph.api import PgGraphApi
>>> from pprint import pprint
>>> api = PgGraphApi('config.hw.local.ini')
>>> rows = api.get_rows_references('flights', [1,2,3])
>>> pprint(rows)
{1: {'ticket_flights': {'flight_id': [{'flight_id': 1,
                                       'ticket_no': '0005432816945'},
                                      {'flight_id': 1,
                                       'ticket_no': '0005432816941'}]}},
 2: {'ticket_flights': {'flight_id': [{'flight_id': 2,
                                       'ticket_no': '0005433101832'},
                                      {'flight_id': 2,
                                       'ticket_no': '0005433101864'},
                                      {'flight_id': 2,
                                       'ticket_no': '0005432919715'}]}},
 3: {'ticket_flights': {'flight_id': [{'flight_id': 3,
                                       'ticket_no': '0005432817560'},
                                      {'flight_id': 3,
                                       'ticket_no': '0005432817568'},
                                      {'flight_id': 3,
                                       'ticket_no': '0005432817559'}]}}}

ื“ืขืจ ื‘ื™ื‘ืœื™ืึธื˜ืขืง ืžืงื•ืจ ืงืึธื“ ืื™ื– ื‘ื ื™ืžืฆื ืื™ืŸ ื’ื™ื˜ื”ื•ื‘ ืื•ื ื˜ืขืจ MIT ื“ืขืจืœื•ื™ื‘ืขื ื™ืฉ, ื•ื•ื™ ื’ืขื–ื•ื ื˜ ื•ื•ื™ ืื™ืŸ ื“ื™ ืจื™ืคึผืึทื–ืึทื˜ืึธืจื™ ืคึผืคึผื™.

ืื™ืš ื•ื•ืขืœ ื–ื™ื™ืŸ ืฆื•ืคืจื™ื“ืŸ ืžื™ื˜ ื‘ืึทืžืขืจืงื•ื ื’ืขืŸ, ืงืึทืžื™ืฅ ืื•ืŸ ืคึฟื™ืจืœื™ื™ื’ืŸ.

ืื™ืš ื•ื•ืขืœ ืคึผืจื•ื‘ื™ืจืŸ ืฆื• ืขื ื˜ืคึฟืขืจืŸ ืคึฟืจืื’ืŸ ืฆื• ื“ื™ ื‘ืขืกื˜ืขืจ ืคื•ืŸ ืžื™ื™ืŸ ืคื™ื™ื™ืงื™ื™ื˜ ื“ืึธ ืื•ืŸ ืื™ืŸ ื“ื™ ืจื™ืคึผืึทื–ืึทื˜ืึธืจื™.

ืžืงื•ืจ: www.habr.com

ืœื™ื™ื’ืŸ ืึท ื‘ืึทืžืขืจืงื•ื ื’