Lus Qhia Tshab
Kuv lub vev xaib, uas kuv ua haujlwm ua haujlwm, yog tsim los tuav cov nplooj ntawv nthuav dav hauv tsev thiab qhov chaw ntawm tus kheej. Cov ncauj lus no tau pib txaus siab rau kuv thaum pib ntawm kuv txoj kev mus ncig ua haujlwm; lub sijhawm ntawd kuv txaus siab los ntawm kev nrhiav cov kws tshaj lij uas sau txog lawv tus kheej, lawv txoj haujlwm nyiam thiab cov haujlwm. Tus cwj pwm ntawm kev tshawb pom lawv rau kuv tus kheej tseem nyob rau niaj hnub no: ntawm yuav luag txhua qhov chaw lag luam thiab tsis yog qhov chaw lag luam, kuv txuas ntxiv mus saib hauv qab hauv qab hauv kev tshawb nrhiav cov ntawv txuas mus rau cov kws sau ntawv.
Kev siv lub tswv yim
Thawj version tsuas yog nplooj ntawv html ntawm kuv tus kheej lub vev xaib, qhov twg kuv muab cov ntawv txuas nrog kos npe rau hauv daim ntawv teev npe. Tau ntaus 20 nplooj ntawv dhau ib lub sijhawm, kuv pib xav tias qhov no tsis zoo heev thiab txiav txim siab los sim ua kom cov txheej txheem automate. Ntawm stackoverflow, kuv pom tias muaj coob tus neeg qhia qhov chaw hauv lawv cov profiles, yog li kuv tau sau ib qho parser hauv php, uas yooj yim mus dhau cov profiles, pib nrog thawj (qhov chaw nyob ntawm SO rau hnub no yog qhov zoo li no: "/users/1" ), rho tawm cov ntawv txuas los ntawm qhov xav tau thiab ntxiv rau hauv SQLite.
Qhov no tuaj yeem hu ua qhov thib ob version: kev sau ntawm kaum tawm txhiab tus URLs hauv lub rooj SQLite, uas hloov cov npe zoo li qub hauv html. Kuv tau tshawb nrhiav yooj yim ntawm daim ntawv teev npe no. Vim muaj tsuas yog URLs, ces qhov kev tshawb fawb tsuas yog raws li lawv.
Nyob rau theem no kuv tau tso tseg qhov project thiab rov qab mus rau nws tom qab lub sijhawm ntev. Nyob rau theem no, kuv qhov kev paub dhau los ua haujlwm tau ntau dua peb xyoos thiab kuv xav tias kuv tuaj yeem ua qee yam hnyav dua. Tsis tas li ntawd, muaj lub siab xav ua kom paub cov thev naus laus zis tshiab.
Niaj hnub nimno version
xa mus rau Docker, cov ntaub ntawv tau pauv mus rau mongoDb, thiab tsis ntev los no, radish tau ntxiv, uas thaum xub thawj tsuas yog rau caching. Ib qho ntawm PHP microframeworks yog siv los ua lub hauv paus.
teeb meem
Cov chaw tshiab tau ntxiv los ntawm console hais kom ua synchronously ua cov hauv qab no:
- Downloads cov ntsiab lus los ntawm URL
- Teem ib tus chij qhia seb HTTPS puas muaj
- Khaws qhov tseem ceeb ntawm lub vev xaib
- Lub hauv paus HTML thiab headers tau txais kev cawmdim nyob rau hauv "indexing" keeb kwm
- Parses cov ntsiab lus, rho tawm lub npe thiab piav qhia
- Txuag cov ntaub ntawv mus rau ib qho kev sau cais
Qhov no yog txaus los cia li khaws cov chaw thiab muab tso rau hauv ib daim ntawv teev npe:
Tab sis lub tswv yim ntawm automatically indexing, categorizing thiab ranking txhua yam, ua kom txhua yam mus txog hnub, tsis haum zoo rau hauv no paradigm. Txawm tias tsuas yog ntxiv ib txoj hauv kev los ntxiv cov nplooj ntawv uas yuav tsum tau ua kom duplication thiab thaiv kom tsis txhob muaj peev xwm DDoS.
Feem ntau, tau kawg, txhua yam tuaj yeem ua tiav synchronously, thiab hauv lub vev xaib txoj hauv kev koj tuaj yeem txuag tau qhov URL kom cov monstrous daemon ua txhua yam haujlwm rau URLs los ntawm cov npe. Tab sis tseem, txawm tias ntawm no lo lus "queue" qhia nws tus kheej. Thiab yog hais tias ib tug queue yog siv, ces tag nrho cov hauj lwm yuav muab faib thiab ua yam tsawg kawg nkaus asynchronously.
kev txiav txim siab
Siv cov kab ke thiab ua ib qho kev tshwm sim-tsav system los ua txhua yam haujlwm. Thiab kuv tau xav sim Redis Stream ntev.
Siv Redis kwj hauv PHP
Vim Txij li thaum kuv lub moj khaum tsis yog ib qho ntawm peb lub loj loj Symfony, Laravel, Yii, kuv xav nrhiav ib lub tsev qiv ntawv ywj pheej. Tab sis, raws li nws tau muab tawm (ntawm kev ntsuam xyuas thawj zaug), nws tsis tuaj yeem nrhiav cov tsev qiv ntawv loj ntawm tus kheej. Txhua yam cuam tshuam nrog cov kab ke yog ib qhov project los ntawm 3 commits tsib xyoos dhau los, los yog khi rau lub moj khaum.
Kuv tau hnov ββββntau txog Symfony ua tus xa khoom ntawm tus kheej cov khoom siv tau zoo, thiab kuv twb tau siv qee yam ntawm lawv. Thiab tseem qee yam ntawm Laravel tuaj yeem siv tau, piv txwv li lawv ORM, tsis muaj lub hauv paus ntawm nws tus kheej.
symfony/messenger
Thawj tus neeg sib tw tam sim ntawd zoo li zoo tagnrho thiab tsis muaj qhov tsis ntseeg kuv tau nruab nws. Tab sis nws tau dhau los ua nyuaj rau google piv txwv ntawm kev siv sab nraum Symfony. Yuav ua li cas sib sau ua ke lub tsheb npav kom dhau cov lus los ntawm pawg ntawm cov chav kawm nrog thoob ntiaj teb, cov npe tsis muaj qab hau, thiab txawm nyob hauv Redis?
Cov ntaub ntawv ntawm lub vev xaib raug cai tau nthuav dav heev, tab sis qhov pib tsuas yog piav qhia rau Symfony siv lawv nyiam YML thiab lwm txoj hauv kev ua khawv koob rau cov tsis yog symphonist. Kuv tsis muaj kev txaus siab rau cov txheej txheem installation nws tus kheej, tshwj xeeb tshaj yog thaum lub caij so Xyoo Tshiab. Tab sis kuv yuav tsum tau ua qhov no rau lub sijhawm ntev uas tsis tau xav txog.
Sim los txiav txim seb yuav ua li cas thiaj li tsim tau ib qho system siv Symfony cov peev txheej kuj tsis yog txoj haujlwm tseem ceeb tshaj plaws rau lub sijhawm kaw:
Tom qab delving rau hauv tag nrho cov no thiab sim ua ib yam dab tsi nrog kuv txhais tes, kuv tuaj mus rau lub xaus hais tias kuv tau ua ib co hom crutches thiab txiav txim siab mus sim lwm yam.
illuminated/queue
Nws tau muab tawm tias lub tsev qiv ntawv no tau raug khi nruj rau Laravel infrastructure thiab ib pawg ntawm lwm qhov kev vam khom, yog li kuv tsis siv sijhawm ntau rau nws: Kuv tau teeb tsa nws, saib nws, pom qhov kev vam khom thiab tshem tawm nws.
yiisoft/yii2-queue
Zoo, ntawm no nws tau tam sim ntawd los ntawm lub npe, dua, kev sib txuas nruj rau Yii2. Kuv yuav tsum siv lub tsev qiv ntawv no thiab tsis yog qhov phem, tab sis kuv tsis xav txog qhov tseeb tias nws nyob ntawm Yii2 kiag li.
Tus so
Txhua yam uas kuv pom ntawm GitHub yog qhov tsis muaj kev ntseeg siab, dhau los thiab tso tseg cov haujlwm yam tsis muaj hnub qub, rab rawg thiab ntau qhov kev cog lus.
Rov qab mus rau symfony/messenger, cov ntsiab lus qhia
Kuv yuav tsum xav txog lub tsev qiv ntawv no thiab, tom qab siv qee lub sijhawm, kuv muaj peev xwm ua tau. Nws muab tawm hais tias txhua yam yog heev concise thiab yooj yim. Kom instantiate lub tsheb npav, kuv ua ib lub Hoobkas me me, vim ... Kuv yuav tsum muaj ob peb lub log tsheb thiab nrog ntau tus tuav.
Tsuas yog ob peb kauj ruam:
- Peb tsim cov lus handlers uas yuav tsum tau yooj yim hu tau
- Peb qhwv lawv hauv HandlerDescriptor (chav kawm los ntawm lub tsev qiv ntawv)
- Peb qhwv cov "Descriptors" hauv qhov piv txwv HandlersLocator
- Ntxiv HandlersLocator rau MessageBus piv txwv
- Peb dhau ib txheej ntawm 'SenderInterface' rau SenderLocator, hauv kuv qhov xwm txheej ntawm 'RedisTransport' cov chav kawm, uas tau teeb tsa hauv txoj hauv kev pom tseeb
- Ntxiv SenderLocator rau MessageBus piv txwv
MessageBus muaj ib txoj kev "-> xa tawm ()" uas saib cov neeg ua haujlwm tsim nyog hauv HandlersLocator thiab xa cov lus rau lawv, siv qhov sib thooj 'SenderInterface' xa ntawm lub npav (Redis kwj).
Nyob rau hauv lub thawv configuration (hauv qhov no php-di), tag nrho cov pob no tuaj yeem teeb tsa zoo li no:
CONTAINER_REDIS_TRANSPORT_SECRET => function (ContainerInterface $c) {
return new RedisTransport(
$c->get(CONTAINER_REDIS_STREAM_CONNECTION_SECRET),
$c->get(CONTAINER_SERIALIZER))
;
},
CONTAINER_REDIS_TRANSPORT_LOG => function (ContainerInterface $c) {
return new RedisTransport(
$c->get(CONTAINER_REDIS_STREAM_CONNECTION_LOG),
$c->get(CONTAINER_SERIALIZER))
;
},
CONTAINER_REDIS_STREAM_RECEIVER_SECRET => function (ContainerInterface $c) {
return new RedisReceiver(
$c->get(CONTAINER_REDIS_STREAM_CONNECTION_SECRET),
$c->get(CONTAINER_SERIALIZER)
);
},
CONTAINER_REDIS_STREAM_RECEIVER_LOG => function (ContainerInterface $c) {
return new RedisReceiver(
$c->get(CONTAINER_REDIS_STREAM_CONNECTION_LOG),
$c->get(CONTAINER_SERIALIZER)
);
},
CONTAINER_REDIS_STREAM_BUS => function (ContainerInterface $c) {
$sendersLocator = new SendersLocator([
AppMessagesSecretJsonMessages::class => [CONTAINER_REDIS_TRANSPORT_SECRET],
AppMessagesDaemonLogMessage::class => [CONTAINER_REDIS_TRANSPORT_LOG],
], $c);
$middleware[] = new SendMessageMiddleware($sendersLocator);
return new MessageBus($middleware);
},
CONTAINER_REDIS_STREAM_CONNECTION_SECRET => function (ContainerInterface $c) {
$host = 'bu-02-redis';
$port = 6379;
$dsn = "redis://$host:$port";
$options = [
'stream' => 'secret',
'group' => 'default',
'consumer' => 'default',
];
return Connection::fromDsn($dsn, $options);
},
CONTAINER_REDIS_STREAM_CONNECTION_LOG => function (ContainerInterface $c) {
$host = 'bu-02-redis';
$port = 6379;
$dsn = "redis://$host:$port";
$options = [
'stream' => 'log',
'group' => 'default',
'consumer' => 'default',
];
return Connection::fromDsn($dsn, $options);
},
Ntawm no koj tuaj yeem pom tau tias hauv SenderLocator peb tau muab "kev thauj mus los" sib txawv rau ob qho lus sib txawv, txhua tus muaj nws tus kheej txuas rau cov kwj deg.
Kuv tau ua ib txoj haujlwm demo cais qhia txog daim ntawv thov ntawm peb daemons sib txuas lus nrog ib leeg siv lub npav hauv qab no: .
Tab sis kuv yuav qhia koj li cas tus neeg siv khoom tuaj yeem tsim kho:
use AppMessagesDaemonLogMessage;
use SymfonyComponentMessengerHandlerHandlerDescriptor;
use SymfonyComponentMessengerHandlerHandlersLocator;
use SymfonyComponentMessengerMessageBus;
use SymfonyComponentMessengerMiddlewareHandleMessageMiddleware;
use SymfonyComponentMessengerMiddlewareSendMessageMiddleware;
use SymfonyComponentMessengerTransportSenderSendersLocator;
require_once __DIR__ . '/../vendor/autoload.php';
/** @var PsrContainerContainerInterface $container */
$container = require_once('config/container.php');
$handlers = [
DaemonLogMessage::class => [
new HandlerDescriptor(
function (DaemonLogMessage $m) {
error_log('DaemonLogHandler: message handled: / ' . $m->getMessage());
},
['from_transport' => CONTAINER_REDIS_TRANSPORT_LOG]
)
],
];
$middleware = [];
$middleware[] = new HandleMessageMiddleware(new HandlersLocator($handlers));
$sendersLocator = new SendersLocator(['*' => [CONTAINER_REDIS_TRANSPORT_LOG]], $container);
$middleware[] = new SendMessageMiddleware($sendersLocator);
$bus = new MessageBus($middleware);
$receivers = [
CONTAINER_REDIS_TRANSPORT_LOG => $container->get(CONTAINER_REDIS_STREAM_RECEIVER_LOG),
];
$w = new SymfonyComponentMessengerWorker($receivers, $bus, $container->get(CONTAINER_EVENT_DISPATCHER));
$w->run();
Kev siv cov txheej txheem no hauv ib daim ntawv thov
Tom qab siv lub tsheb npav hauv kuv qhov backend, kuv tau cais cov theem ntawm tus kheej los ntawm cov lus txib synchronous qub thiab tsim cov neeg ua haujlwm sib cais, txhua tus ua lawv tus kheej.
Lub raj xa dej ntxiv rau qhov chaw tshiab rau hauv cov ntaub ntawv zoo li no:
Thiab tam sim ntawd tom qab ntawd, nws tau yooj yim dua rau kuv ntxiv kev ua haujlwm tshiab, piv txwv li, rho tawm thiab txheeb xyuas Rss. Vim Cov txheej txheem no tseem xav tau cov ntsiab lus tseem ceeb, tom qab ntawd RSS txuas tus neeg tuav haujlwm, zoo li WebsiteIndexHistoryPersistor, sau npe rau "Cov ntsiab lus / HtmlContent" cov lus, ua tiav nws thiab hla cov lus xav tau raws nws cov raj xa dej ntxiv.
Thaum kawg, peb tau xaus nrog ntau lub daemons, txhua tus tswj kev sib txuas tsuas yog rau cov peev txheej tsim nyog. Piv txwv li dab neeg nkag muaj tag nrho cov handlers uas yuav tsum tau mus rau hauv Internet rau cov ntsiab lus, thiab cov daemon nyob twj ywm tuav ib qho kev sib txuas rau lub database.
Tam sim no, es tsis txhob xaiv los ntawm cov ntaub ntawv, cov ids xav tau tom qab tso los ntawm tus neeg mob tsuas yog kis tau los ntawm lub npav mus rau txhua tus neeg ua haujlwm txaus siab.
Tau qhov twg los: www.hab.com