Kuv tuaj ua neej zoo li cas?
Tsis ntev tas los no kuv yuav tsum tau ua hauj lwm nyob rau hauv lub backend ntawm ib tug heev loaded project, nyob rau hauv uas nws yog tsim nyog los npaj cov kev ua tsis tu ncua ntawm ib tug loj tus naj npawb ntawm cov hauj lwm tom qab nrog complex xam thiab thov rau peb-tog cov kev pab cuam. Qhov project yog asynchronous thiab ua ntej kuv tuaj, nws muaj ib tug yooj yim mechanism rau cron-launching hauj lwm: lub voj tshawb xyuas lub sij hawm tam sim no thiab launching pab pawg neeg ntawm coroutines los ntawm kev sib sau ua ke - qhov no mus kom ze tau txais mus txog rau thaum muaj kaum ob thiab pua pua ntawm xws li coroutines. , txawm li cas los xij, thaum lawv tus lej ntau tshaj li ob txhiab, kuv yuav tsum xav txog kev teeb tsa ib txwm ua haujlwm nrog tus neeg ua haujlwm, ntau tus neeg ua haujlwm, thiab lwm yam.
Ua ntej kuv txiav txim siab sim tawm Celery, uas kuv tau siv ua ntej. Vim lub asynchronous xwm ntawm qhov project, kuv dhia rau hauv cov lus nug thiab pom raws li zoo li , tsim los ntawm tus sau tsab xov xwm.
Kuv yuav hais qhov no, qhov project yog qhov nthuav heev thiab ua haujlwm zoo heev hauv lwm cov ntawv thov ntawm peb pab neeg, thiab tus sau nws tus kheej hais tias nws muaj peev xwm yob nws tawm mus rau hauv kev tsim khoom los ntawm kev siv lub pas dej asynchronous. Tab sis, hmoov tsis, nws yeej tsis haum kuv, raws li nws tau tawm nrog pab pawg pib ua haujlwm (saib. ). Thaum lub sijhawm sau ntawv twb kaw lawm, txawm li cas los xij, kev ua haujlwm tau mus rau ib hlis. Txawm li cas los xij, hmoov zoo rau tus sau thiab txhua qhov zoo tshaj plaws, txij li muaj twb tau ua haujlwm ntawm lub lib ... feem ntau, cov ntsiab lus yog nyob rau hauv kuv thiab lub cuab yeej tig tawm los ua damp rau kuv. Tsis tas li ntawd, qee qhov haujlwm muaj 2-3 http thov rau cov kev pabcuam sib txawv, yog li txawm tias thaum ua haujlwm kom zoo, peb tsim 4 txhiab TCP kev sib txuas, kwv yees li txhua 2 teev - tsis zoo heev ... Kuv xav tsim kev sib tham rau ib hom. ua haujlwm thaum pib ua haujlwm. Ib me ntsis ntxiv txog cov neeg thov ntau ntawm aiohttp .
Hauv qhov no, kuv pib saib lwm txoj thiab pom nws! Cov neeg tsim ntawm celery, tshwj xeeb, raws li kuv nkag siab nws , tau tsim , Ameslikas rau qhov project . Faust yog kev tshoov siab los ntawm Kafka Streams thiab ua haujlwm nrog Kafka ua tus broker, rocksdb kuj tseem siv los khaws cov txiaj ntsig los ntawm kev ua haujlwm ntawm cov neeg ua haujlwm, thiab qhov tseem ceeb tshaj plaws yog lub tsev qiv ntawv yog asynchronous.
Tsis tas li, koj tuaj yeem saib celery thiab faust los ntawm cov creators ntawm lub tom kawg: lawv qhov sib txawv, qhov sib txawv ntawm cov neeg ua haujlwm, kev ua haujlwm ntawm theem pib. Txhua yam yog yooj yim heev, txawm li cas los xij, qhov zoo nkauj hauv faust nyiam mloog - ntaus cov ntaub ntawv rau kev xa mus rau lub ntsiab lus.
Peb ua li cas?
Yog li, hauv cov lus luv luv, kuv yuav qhia koj yuav ua li cas sau cov ntaub ntawv los ntawm cov haujlwm tom qab siv Faust. Lub hauv paus rau peb qhov project piv txwv yuav yog, raws li lub npe qhia, . Kuv yuav ua qauv qhia yuav ua li cas sau cov neeg sawv cev (lub dab dej, cov ntsiab lus, kev faib), yuav ua li cas ua haujlwm (cron) kev ua haujlwm, qhov yooj yim tshaj plaws faust cli commands (ib wrapper dhau nias), pawg yooj yim, thiab thaum kawg peb yuav xa ib datadog ( ua hauj lwm tawm ntawm lub thawv) thiab sim ua ib yam dab tsi kom pom. Txhawm rau khaws cov ntaub ntawv khaws tseg peb yuav siv mongodb thiab lub cev muaj zog rau kev sib txuas.
PS Kev txiav txim siab los ntawm kev ntseeg siab uas cov ntsiab lus hais txog kev soj ntsuam tau sau, Kuv xav tias tus nyeem ntawv kawg ntawm tsab xov xwm kawg tseem yuav zoo li no:
Qhov xav tau ntawm qhov project
Vim qhov kuv tau cog lus tseg lawm, cia peb ua ib daim ntawv teev npe me me ntawm qhov kev pabcuam yuav tsum ua tau:
- Upload kev ruaj ntseg thiab cov ntsiab lus ntawm lawv (nrog rau cov txiaj ntsig thiab kev poob, daim ntawv tshuav nyiaj, cov nyiaj ntsuab ntws - rau xyoo tas los) - tsis tu ncua
- Upload cov ntaub ntawv keeb kwm (rau txhua xyoo kev ua lag luam, nrhiav qhov tseem ceeb ntawm tus nqi kaw ntawm kev lag luam) - tsis tu ncua
- Upload tshiab trading cov ntaub ntawv - tsis tu ncua
- Upload ib daim ntawv teev cov ntsuas ntsuas rau txhua qhov kev ruaj ntseg - tsis tu ncua
Raws li xav tau, peb xaiv lub npe rau qhov project los ntawm kos: Horton
Peb tab tom npaj cov infrastructure
Lub npe yeej muaj zog, txawm li cas los xij, txhua yam koj yuav tsum tau ua yog sau ib qho me me rau docker-compose nrog kafka (thiab zookeeper - hauv ib lub thawv), kafdrop (yog tias peb xav saib cov lus hauv cov ncauj lus), mongodb. Peb tau [docker-compose.yml]() ntawm daim ntawv hauv qab no:
version: '3'
services:
db:
container_name: horton-mongodb-local
image: mongo:4.2-bionic
command: mongod --port 20017
restart: always
ports:
- 20017:20017
environment:
- MONGO_INITDB_DATABASE=horton
- MONGO_INITDB_ROOT_USERNAME=admin
- MONGO_INITDB_ROOT_PASSWORD=admin_password
kafka-service:
container_name: horton-kafka-local
image: obsidiandynamics/kafka
restart: always
ports:
- "2181:2181"
- "9092:9092"
environment:
KAFKA_LISTENERS: "INTERNAL://:29092,EXTERNAL://:9092"
KAFKA_ADVERTISED_LISTENERS: "INTERNAL://kafka-service:29092,EXTERNAL://localhost:9092"
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: "INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT"
KAFKA_INTER_BROKER_LISTENER_NAME: "INTERNAL"
KAFKA_ZOOKEEPER_SESSION_TIMEOUT: "6000"
KAFKA_RESTART_ATTEMPTS: "10"
KAFKA_RESTART_DELAY: "5"
ZOOKEEPER_AUTOPURGE_PURGE_INTERVAL: "0"
kafdrop:
container_name: horton-kafdrop-local
image: 'obsidiandynamics/kafdrop:latest'
restart: always
ports:
- '9000:9000'
environment:
KAFKA_BROKERCONNECT: kafka-service:29092
depends_on:
- kafka-serviceTsis muaj dab tsi nyuab ntawm no txhua. Ob tus neeg mloog tau tshaj tawm rau kafka: ib qho (sab hauv) rau kev siv hauv lub network sib xyaw, thiab qhov thib ob (sab nraud) rau kev thov los ntawm sab nraud, yog li lawv tau xa mus rau sab nraud. 2181 — zookeeper port. Tus so, kuv xav tias, yog qhov tseeb.
Npaj lub cev pob txha ntawm qhov project
Nyob rau hauv lub hauv paus version, tus qauv ntawm peb tes num yuav tsum zoo li no:
horton
├── docker-compose.yml
└── horton
├── agents.py *
├── alphavantage.py *
├── app.py *
├── config.py
├── database
│ ├── connect.py
│ ├── cruds
│ │ ├── base.py
│ │ ├── __init__.py
│ │ └── security.py *
│ └── __init__.py
├── __init__.py
├── records.py *
└── tasks.py **Txhua yam kuv sau tseg Peb tseem tsis tau kov nws, peb tsuas yog tsim cov ntaub ntawv khoob.**
Peb tsim ib qho qauv. Tam sim no cia peb ntxiv qhov tsim nyog dependencies, sau cov config thiab txuas rau mongodb. Kuv yuav tsis muab tag nrho cov ntawv nyeem ntawm cov ntaub ntawv nyob rau hauv tsab xov xwm, thiaj li tsis mus ncua nws, tab sis kuv yuav muab txuas mus rau lub tsim nyog versions.
Cia peb pib nrog dependencies thiab meta txog qhov project -
Tom ntej no, peb pib txhim kho kev vam khom thiab tsim ib lub virtualenv (los yog koj tuaj yeem tsim lub venv nplaub tshev koj tus kheej thiab qhib ib puag ncig):
pip3 install poetry (если ещё не установлено)
poetry installTam sim no cia peb tsim - Daim ntawv pov thawj thiab qhov twg los khob. Koj tuaj yeem tso cov ntaub ntawv tam sim ntawd rau alphavantage muaj. Zoo, cia peb txav mus rau - rho tawm cov ntaub ntawv rau daim ntawv thov los ntawm peb config. Yog, Kuv lees txim, Kuv siv kuv lib - .
Thaum txuas rau Mongo, txhua yam yooj yim heev. tshaj tawm mus txuas thiab rau cruds, kom nws yooj yim dua los ua cov lus nug ntawm kev sau.
Dab tsi yuav tshwm sim tom ntej?
Cov kab lus tsis ntev dhau los, txij li ntawm no kuv tsuas yog hais txog kev txhawb nqa thiab kev npaj, yog li tsis txhob cem kuv - Kuv cog lus tias ntu tom ntej yuav muaj kev nqis tes ua thiab cov duab.
Yog li ntawd, nyob rau hauv no tsab xov xwm peb yuav tham txog:
- Cia peb sau tus neeg siv khoom me me rau alphavantage ntawm aiohttp nrog kev thov rau qhov kawg peb xav tau.
- Cia peb tsim ib tus neeg sawv cev uas yuav sau cov ntaub ntawv ntawm kev nyab xeeb thiab cov nqi keeb kwm rau lawv.
Tau qhov twg los: www.hab.com