Tsis ntev los no, cov lus tshaj tawm txog Telegram zoo npaum li cas, qhov ci ntsa iab thiab kev paub txog Durov cov kwv tij nyob hauv lub tsev network, thiab lwm yam tau pib tshwm ntau zaus ntawm Habré. Nyob rau tib lub sijhawm, tsawg tus neeg tau nkag siab tiag tiag rau lawv tus kheej hauv cov cuab yeej siv - feem ntau, lawv siv qhov yooj yim (thiab txawv ntawm MTProto) Bot API raws li JSON, thiab feem ntau tsuas yog lees txais. ntawm kev ntseeg tag nrho cov qhuas thiab PR uas tig ib ncig ntawm tus tub txib. Yuav luag ib xyoos thiab ib nrab dhau los, kuv cov npoj yaig ntawm Eshelon NGO Vasily (hmoov tsis zoo, nws tus account ntawm Habré raug tshem tawm nrog rau cov ntawv sau) pib sau nws tus kheej Telegram tus neeg siv khoom los ntawm kos hauv Perl, thiab tom qab ntawd tus kws sau cov kab no tau koom nrog. Vim li cas Perl, qee leej yuav nug tam sim? Vim hais tias tej yaam num no twb muaj nyob rau hauv lwm yam lus. npaj ua tsev qiv ntawv, thiab raws li tus sau yuav tsum mus tag nrho txoj kev los ntawm kos. Ntxiv mus, cryptography yog ib qho teeb meem ntawm kev ntseeg, tab sis txheeb xyuas. Nrog rau cov khoom lag luam tsom rau kev ruaj ntseg, koj tsis tuaj yeem cia siab rau lub tsev qiv ntawv npaj tau los ntawm cov chaw tsim khoom thiab tsis ntseeg nws (txawm li cas los xij, qhov no yog lub ntsiab lus rau ntu thib ob). Tam sim no, lub tsev qiv ntawv ua haujlwm tau zoo nyob rau theem "nruab nrab" (tso cai rau koj ua txhua qhov kev thov API).

Txawm li cas los xij, yuav tsis muaj ntau yam crypto txiaj lossis lej hauv cov ntawv tshaj tawm no. Tab sis yuav muaj ntau lwm cov ntsiab lus thiab cov cuab yeej cuab tam (tseem muaj txiaj ntsig rau cov neeg uas yuav tsis sau los ntawm kos, tab sis yuav siv lub tsev qiv ntawv hauv txhua hom lus). Yog li, lub hom phiaj tseem ceeb yog sim siv tus neeg siv khoom los ntawm kos raws li cov ntaub ntawv raug cai. Ntawd yog, cia peb xav tias lub hauv paus cai ntawm cov neeg siv khoom raug kaw (dua, hauv ntu thib ob peb yuav piav qhia ntau ntxiv txog lub ntsiab lus ntawm qhov tseeb tias qhov no yog qhov tseeb. nws tshwm sim yog li), tab sis, raws li nyob rau hauv lub qub hnub, piv txwv li, muaj ib tug qauv zoo li RFC - nws puas tuaj yeem sau tus neeg siv khoom raws li qhov tshwj xeeb ib leeg, "tsis saib" ntawm qhov chaws, nws yog official (Telegram Desktop, mobile), lossis Telethon tsis raug cai?

Cov ntsiab lus:

Cov ntaub ntawv ... nws muaj, txoj cai? Puas muaj tseeb?..
Yuav pib qhov twg los ua tus sau ntawv hluas?
Binary serialization: TL (Hom Lus) thiab nws cov tswv yim, thiab txheej, thiab ntau lwm yam lus txaus ntshai
MT Proto
API?
Yuav tsum tau txuas ntxiv!

Cov ntaub ntawv ... nws muaj, txoj cai? Puas muaj tseeb?..

Fragments ntawm sau ntawv rau tsab xov xwm no tau pib sau lub caij ntuj sov dhau los. Txhua lub sijhawm no ntawm lub vev xaib official https://core.telegram.org Cov ntaub ntawv yog raws li txheej 23, i.e. daig qhov chaw hauv 2014 (nco ntsoov, tsis muaj cov channel rov qab los?). Tau kawg, nyob rau hauv txoj kev xav, qhov no yuav tsum tau tso cai rau peb siv tus neeg siv khoom nrog kev ua haujlwm ntawm lub sijhawm ntawd xyoo 2014. Tab sis txawm nyob rau hauv lub xeev no, cov ntaub ntawv yog, firstly, tsis tiav, thiab thib ob, nyob rau hauv qhov chaw nws contradicted nws tus kheej. Tsuas yog ib hlis dhau los, thaum lub Cuaj Hlis 2019, nws yog ua yuam kev Nws tau pom tias muaj qhov hloov tshiab loj ntawm cov ntaub ntawv ntawm lub xaib, rau Txheej Txheem 105 tsis ntev los no, nrog rau kev ceeb toom tias tam sim no txhua yam yuav tsum tau nyeem dua. Tseeb tiag, ntau cov ntawv tau hloov kho, tab sis ntau yam tseem tsis tau hloov. Yog li ntawd, thaum nyeem cov lus thuam hauv qab no txog cov ntaub ntawv, koj yuav tsum nco ntsoov tias qee yam ntawm cov no tsis muaj feem cuam tshuam, tab sis qee qhov tseem zoo. Tom qab tag nrho, 5 xyoo nyob rau hauv lub ntiaj teb niaj hnub no tsis yog lub sijhawm ntev, tab sis heev ntau ntau. Txij li thaum lub sijhawm ntawd (tshwj xeeb tshaj yog tias koj tsis suav nrog cov chaw pov tseg thiab rov kho dua qhov chaw geochat txij thaum ntawd los), tus naj npawb ntawm API txoj hauv kev tau nce los ntawm ib puas mus rau ntau tshaj ob puas thiab tsib caug!

Yuav pib qhov twg los ua tus sau ntawv hluas?

Nws tsis muaj teeb meem txawm tias koj sau los ntawm kos lossis siv, piv txwv li, npaj cov tsev qiv ntawv zoo li Telethon rau Python los yog Madeline rau PHP, nyob rau hauv txhua rooj plaub, koj yuav tsum tau ua ntej sau npe koj daim ntawv thov - tau parameters api_id и api_hash (cov neeg uas tau ua haujlwm nrog VKontakte API tam sim ntawd nkag siab) los ntawm cov neeg rau zaub mov yuav txheeb xyuas daim ntawv thov. Qhov no yuav tau ua nws vim kev cai lij choj, tab sis peb yuav tham ntau ntxiv txog vim li cas cov tsev qiv ntawv tsis tuaj yeem tshaj tawm nws hauv ntu thib ob. Tej zaum koj yuav txaus siab rau qhov ntsuas qhov tseem ceeb, txawm tias lawv muaj tsawg heev - qhov tseeb yog tias tam sim no koj tuaj yeem sau npe ib leeg xwb app, yog li tsis txhob maj mus rau hauv nws.

Tam sim no, los ntawm qhov kev pom zoo, peb yuav tsum xav txog qhov tseeb tias tom qab tso npe peb yuav tsum tau txais cov ntawv ceeb toom los ntawm Telegram txog kev hloov tshiab rau cov ntaub ntawv, raws tu qauv, thiab lwm yam. Ntawd yog, ib tus tuaj yeem xav tias qhov chaw nrog cov docks tsuas yog tso tseg thiab txuas ntxiv ua haujlwm tshwj xeeb nrog cov neeg pib ua cov neeg siv khoom, vim tias nws yooj yim dua. Tab sis tsis yog, tsis muaj dab tsi zoo li ntawd tau pom, tsis muaj ntaub ntawv tuaj.

Thiab yog tias koj sau los ntawm kos, tom qab ntawd siv cov kev ntsuas tau los yeej tseem muaj txoj hauv kev ntev. Txawm tias https://core.telegram.org/ thiab tham txog lawv hauv Kev Pib Ua ntej ntawm tag nrho cov, qhov tseeb, koj yuav tsum xub siv MTProto raws tu qauv - tab sis yog tias koj ntseeg layout raws li OSI qauv tom kawg ntawm nplooj ntawv rau cov lus piav qhia dav dav ntawm cov txheej txheem, ces nws yog qhov tsis muaj dab tsi.

Qhov tseeb, ob qho tib si ua ntej thiab tom qab MTProto, ntawm ntau theem ib zaug (raws li cov neeg txawv teb chaws ua haujlwm hauv OS ntsiav hais, txheej ua txhaum cai), lub ntsiab lus loj, mob thiab txaus ntshai yuav tau txais txoj hauv kev ...

Binary serialization: TL (Hom Lus) thiab nws cov tswv yim, thiab txheej, thiab ntau lwm yam lus txaus ntshai

Lub ntsiab lus no, qhov tseeb, yog tus yuam sij rau Telegram cov teeb meem. Thiab yuav muaj ntau cov lus txaus ntshai yog tias koj sim delve rau hauv nws.

Yog li, ntawm no yog daim duab. Yog hais tias lo lus no los rau hauv koj lub siab, hais, JSON Schema, Koj xav kom raug. Lub hom phiaj yog tib yam: qee hom lus los piav qhia txog cov ntaub ntawv sib kis tau. Qhov no yog qhov uas qhov zoo sib xws xaus. Yog los ntawm nplooj ntawv MTProto raws tu qauv, los yog los ntawm cov ntoo ntawm cov neeg siv khoom, peb yuav sim qhib qee qhov schema, peb yuav pom qee yam xws li:

int ? = Int;
long ? = Long;
double ? = Double;
string ? = String;

vector#1cb5c415 {t:Type} # [ t ] = Vector t;

rpc_error#2144ca19 error_code:int error_message:string = RpcError;

rpc_answer_unknown#5e2ad36e = RpcDropAnswer;
rpc_answer_dropped_running#cd78e586 = RpcDropAnswer;
rpc_answer_dropped#a43ad8b7 msg_id:long seq_no:int bytes:int = RpcDropAnswer;

msg_container#73f1f8dc messages:vector<%Message> = MessageContainer;

---functions---

set_client_DH_params#f5045f1f nonce:int128 server_nonce:int128 encrypted_data:bytes = Set_client_DH_params_answer;

ping#7abe77ec ping_id:long = Pong;
ping_delay_disconnect#f3427b8c ping_id:long disconnect_delay:int = Pong;

invokeAfterMsg#cb9f372d msg_id:long query:!X = X;
invokeAfterMsgs#3dc4b4f0 msg_ids:Vector<long> query:!X = X;

account.updateProfile#78515775 flags:# first_name:flags.0?string last_name:flags.1?string about:flags.2?string = User;
account.sendChangePhoneCode#8e57deb flags:# allow_flashcall:flags.0?true phone_number:string current_number:flags.0?Bool = auth.SentCode;

Ib tug neeg pom qhov no thawj zaug yuav intuitively muaj peev xwm paub txog tsuas yog ib feem ntawm cov ntawv sau - zoo, cov no yog cov qauv (txawm hais tias qhov twg yog lub npe, sab laug los yog sab xis?), muaj cov teb nyob rau hauv lawv, tom qab uas ib hom ua raws tom qab txoj hnyuv loj... tej zaum. Ntawm no hauv lub kaum sab xis muaj tej zaum cov qauv zoo li hauv C ++ (qhov tseeb, tsis yog). Thiab tag nrho lwm cov cim txhais li cas, cov lus nug, cov cim exclamation, feem pua, cov cim hash (thiab pom tseeb tias lawv txhais tau tias txawv ntawm qhov chaw sib txawv), qee zaum tam sim no thiab qee zaum tsis yog, tus lej hexadecimal - thiab qhov tseem ceeb tshaj, yuav ua li cas thiaj tau los ntawm qhov no qhov yog (uas yuav tsis raug tso tseg los ntawm lub server) byte kwj? Koj yuav tau nyeem cov ntaub ntawv (yog, muaj cov kev txuas mus rau schema hauv JSON version nyob ze - tab sis qhov ntawd tsis ua kom pom tseeb).

Qhib nplooj ntawv Binary Data Serialization thiab dhia mus rau hauv lub ntiaj teb tej yam yees siv ntawm nceb thiab discrete lej, ib yam dab tsi zoo li matan nyob rau hauv lub xyoo 4th. Cov tsiaj ntawv, hom, tus nqi, combinator, functional combinator, ib txwm daim ntawv, composite type, polymorphic type ... thiab qhov ntawd tsuas yog thawj nplooj ntawv! Tom ntej no tos koj TL Lus, uas, txawm hais tias nws twb muaj ib qho piv txwv ntawm qhov kev thov tsis tseem ceeb thiab cov lus teb, tsis muab cov lus teb rau txhua qhov xwm txheej, uas txhais tau hais tias koj yuav tsum tau taug kev los ntawm kev rov hais dua cov lej uas txhais los ntawm Lavxias mus rau Askiv rau lwm yim embedded. paj!

Cov neeg nyeem paub txog cov lus ua haujlwm thiab tsis siv neeg hom inference yuav, ntawm chav kawm, pom cov lus piav qhia hauv hom lus no, txawm tias los ntawm qhov piv txwv, paub ntau ntxiv, thiab tuaj yeem hais tias qhov no yog qhov tseeb tsis yog qhov phem hauv txoj cai. Cov kev tawm tsam rau qhov no yog:

yog, lub hom phiaj suab zoo, tab sis alas, nws tsis tiav
Kev kawm ntawv ntawm Lavxias teb sab universities sib txawv txawm nyob rau hauv IT tshwj xeeb - tsis yog txhua leej txhua tus tau kawm cov kev coj
Thaum kawg, raws li peb yuav pom, hauv kev xyaum nws yog tsis yuav tsum tau, txij li tsuas yog ib qho kev txwv ntawm cov TL uas tau piav qhia yog siv

Raws li tau hais LeoNerd ntawm channel #perl nyob rau hauv FreeNode IRC network, leej twg sim siv lub rooj vag ntawm Telegram rau Matrix (kev txhais lus ntawm qhov kev tsocai tsis raug ntawm lub cim xeeb):

Nws zoo nkaus li ib tug neeg tau qhia txog hom kev xav thawj zaug, zoo siab heev, thiab pib sim ua si nrog nws, tsis xav saib xyuas seb nws puas xav tau hauv kev xyaum.

Saib rau koj tus kheej, yog tias qhov xav tau rau hom liab qab (int, ntev, thiab lwm yam) raws li ib yam dab tsi hauv tsev kawm ntawv tsis muaj lus nug - thaum kawg lawv yuav tsum tau siv manually - piv txwv li, cia peb sim ua kom tau los ntawm lawv. vector. Ntawd yog, qhov tseeb, ib qho array, yog tias koj hu cov khoom tshwm sim los ntawm lawv cov npe kom raug.

Tab sis ua ntej

Cov lus piav qhia luv luv ntawm ib pawg ntawm TL syntax rau cov neeg uas tsis nyeem cov ntaub ntawv raug cai

constructor = Type;
myVec ids:Vector<long> = Type;

fixed#abcdef34 id:int = Type2;

fixedVec set:Vector<Type2> = FixedVec;

constructorOne#crc32 field1:int = PolymorType;
constructorTwo#2crc32 field_a:long field_b:Type3 field_c:int = PolymorType;
constructorThree#deadcrc bit_flags_of_what_really_present:# optional_field4:bit_flags_of_what_really_present.1?Type = PolymorType;

an_id#12abcd34 id:int = Type3;
a_null#6789cdef = Type3;

Kev txhais ib txwm pib kws qhia, tom qab uas optionally (hauv kev xyaum - ib txwm) los ntawm lub cim # yuav tsum CRC 32 los ntawm cov kab lus piav qhia zoo ntawm hom no. Tom ntej no tuaj ib qho kev piav qhia ntawm cov teb; yog tias lawv muaj, hom yuav khoob. Qhov no tag nrho xaus nrog qhov sib npaug ntawm kos npe, lub npe ntawm hom uas tus tsim qauv no - uas yog, qhov tseeb, subtype - belongs. Tus txiv leej tub mus rau sab xis ntawm daim ntawv sib npaug yog polymorphic - uas yog, ntau hom tshwj xeeb tuaj yeem sib haum rau nws.

Yog qhov txhais tau tshwm sim tom qab kab ---functions---, ces cov syntax yuav nyob twj ywm tib yam, tab sis lub ntsiab lus yuav txawv: tus constructor yuav dhau los ua lub npe ntawm RPC muaj nuj nqi, cov teb yuav ua tsis tau (zoo, uas yog, nws yuav nyob twj ywm raws nraim tib yam muab qauv, raws li tau piav qhia hauv qab no. , qhov no tsuas yog lub ntsiab lus tau muab rau), thiab "polymorphic type" - hom ntawm cov txiaj ntsig rov qab. Muaj tseeb, nws tseem yuav nyob twj ywm polymorphic - tsuas yog txhais hauv ntu ---types---, tab sis tus tsim qauv no yuav "tsis raug txiav txim siab". Overloading hom hu ua haujlwm los ntawm lawv cov lus sib cav, i.e. Rau qee qhov laj thawj, ntau lub luag haujlwm nrog tib lub npe tab sis sib txawv kos npe, xws li hauv C ++, tsis muab rau hauv TL.

Vim li cas "constructor" thiab "polymorphic" yog tias nws tsis yog OOP? Zoo, qhov tseeb, nws yuav yooj yim dua rau ib tus neeg xav txog qhov no hauv OOP cov ntsiab lus - hom polymorphic raws li chav kawm paub daws teeb meem, thiab cov neeg tsim khoom yog nws cov tub ntxhais kawm ncaj qha, thiab final nyob rau hauv lub terminology ntawm ntau yam lus. Qhov tseeb, tau kawg, ntawm no nkaus xwb zoo sib xws nrog tiag tiag overloaded constructor txoj kev nyob rau hauv OO programming lus. Txij li ntawm no tsuas yog cov ntaub ntawv cov qauv, tsis muaj txoj hauv kev (txawm hais tias qhov kev piav qhia ntawm cov haujlwm thiab cov txheej txheem ntxiv yog muaj peev xwm tsim kev tsis meej pem hauv lub taub hau uas lawv muaj, tab sis qhov ntawd yog qhov sib txawv) - koj tuaj yeem xav txog tus tsim ua tus nqi los ntawm uas tab tom tsim ntaus ntawv thaum nyeem ib byte kwj.

Qhov no tshwm sim li cas? Lub deserializer, uas ib txwm nyeem 4 bytes, pom tus nqi 0xcrc32 - thiab nkag siab tias yuav muaj dab tsi tshwm sim tom ntej field1 nrog hom int, i.e. nyeem raws nraim 4 bytes, ntawm no qhov overlying teb nrog hom PolymorType nyeem. Pom 0x2crc32 thiab nkag siab tias muaj ob thaj chaw ntxiv, ua ntej long, uas txhais tau tias peb nyeem 8 bytes. Thiab ces dua ib hom complex, uas yog deserialized nyob rau hauv tib txoj kev. Piv txwv li, Type3 tuaj yeem tshaj tawm nyob rau hauv Circuit Court sai li sai tau raws li ob tug constructors, feem, ces lawv yuav tsum tau raws li ib tug 0x12abcd34, tom qab ntawd koj yuav tsum tau nyeem 4 bytes ntxiv int, lossis 0x6789cdef, tom qab ntawd yuav tsis muaj dab tsi. Lwm yam - koj yuav tsum pov ib qho kev zam. Xijpeem, tom qab no peb rov qab mus nyeem 4 bytes int liaj teb field_c в constructorTwo thiab nrog qhov ntawd peb nyeem peb cov ntawv tiav PolymorType.

Thaum kawg, yog koj ntes tau 0xdeadcrc rau constructorThree, ces txhua yam yuav nyuaj dua. Peb thawj daim teb yog bit_flags_of_what_really_present nrog hom # - Qhov tseeb, qhov no tsuas yog lub npe hu ua hom nat, lub ntsiab lus "natural tooj". Ntawd yog, qhov tseeb, tsis tau kos npe int yog, los ntawm txoj kev, tsuas yog rooj plaub thaum tus lej tsis tau kos npe tshwm sim hauv cov khoom siv tiag tiag. Yog li, tom ntej no yog kev tsim kho nrog cov lus nug kos npe, txhais tau tias daim teb no - nws yuav muaj nyob rau ntawm cov hlau tsuas yog tias qhov sib thooj me ntsis tau teeb tsa hauv thaj teb raug xa mus rau (kwv yees li tus neeg teb xov tooj ternary). Yog li, cia peb xav tias qhov me me no tau teeb tsa, uas txhais tau hais tias ntxiv peb yuav tsum tau nyeem ib daim teb zoo li Type, uas nyob rau hauv peb piv txwv muaj 2 constructors. Ib qho yog khoob (tsuas yog tus lej cim), lwm qhov muaj daim teb ids nrog hom ids:Vector<long>.

Tej zaum koj yuav xav tias ob qho tib si templates thiab generics yog nyob rau hauv lub pros los yog Java. Tab sis tsis muaj. Yuav luag. Qhov no qhov tsuas yog rooj plaub ntawm kev siv lub kaum sab xis hauv lub voj voog tiag tiag, thiab nws tsuas yog siv rau Vector xwb. Hauv cov kwj byte, cov no yuav yog 4 CRC32 bytes rau Vector hom nws tus kheej, ib txwm zoo ib yam, tom qab ntawd 4 bytes - tus lej ntawm cov ntsiab lus, thiab tom qab ntawd cov ntsiab lus no lawv tus kheej.

Ntxiv rau qhov no qhov tseeb tias serialization ib txwm tshwm sim hauv cov lus ntawm 4 bytes, txhua hom yog ntau ntau ntawm nws - hom built-in kuj tau piav qhia bytes и string nrog phau ntawv serialization ntawm qhov ntev thiab qhov kev sib tw los ntawm 4 - zoo, nws zoo li suab zoo li qub thiab txawm tias muaj txiaj ntsig zoo? Txawm hais tias TL tau lees paub tias yog qhov zoo tshaj plaws binary serialization, mus rau ntuj txiag teb tsaus nrog lawv, nrog rau kev nthuav dav ntawm txhua yam, txawm tias Boolean qhov tseem ceeb thiab cov kab cim ib leeg rau 4 bytes, JSON puas tseem yuav tuab dua? Saib, txawm tias cov teb tsis tsim nyog tuaj yeem hla nrog cov chij me ntsis, txhua yam zoo heev, thiab tseem tuaj yeem txuas ntxiv mus rau yav tom ntej, yog li vim li cas ho tsis ntxiv cov kev xaiv tshiab rau cov neeg tsim khoom tom qab? ..

Tab sis tsis yog, yog tias koj tsis nyeem kuv cov lus piav qhia luv luv, tab sis cov ntaub ntawv tag nrho, thiab xav txog kev siv. Ua ntej, CRC32 ntawm tus tsim yog xam raws li cov kab lus ib txwm muaj ntawm cov lus piav qhia ntawm lub tswv yim (tshem tawm qhov chaw dawb ntxiv, thiab lwm yam) - yog li yog tias ib daim teb tshiab ntxiv, hom kab piav qhia yuav hloov, thiab yog li nws CRC32 thiab Yog li ntawd, serialization. Thiab tus neeg siv khoom qub yuav ua li cas yog tias nws tau txais daim teb nrog cov chij tshiab, thiab nws tsis paub yuav ua li cas rau lawv ntxiv?

Qhov thib ob, cia peb nco ntsoov CRC 32, uas yog siv ntawm no tseem ceeb li hash muaj nuj nqi txhawm rau txiav txim siab seb hom twg yog (de)serialized. Ntawm no peb tau ntsib teeb meem ntawm kev sib tsoo - thiab tsis yog, qhov tshwm sim tsis yog ib qho ntawm 232, tab sis ntau dua. Leej twg nco qab tias CRC32 yog tsim los xyuas (thiab kho) qhov tsis raug hauv kev sib txuas lus, thiab ua raws li cov khoom no ua rau muaj kev puas tsuaj rau lwm tus? Piv txwv li, nws tsis quav ntsej txog kev hloov kho bytes: yog tias koj xam CRC32 los ntawm ob kab, hauv qhov thib ob koj pauv thawj 4 bytes nrog rau 4 bytes tom ntej - nws yuav zoo ib yam. Thaum peb cov tswv yim yog cov kab ntawv los ntawm cov tsiaj ntawv Latin (thiab cov cim cim me ntsis), thiab cov npe no tsis yog tshwj xeeb random, qhov yuav tshwm sim ntawm qhov kev hloov kho no nce ntau.

Los ntawm txoj kev, leej twg tshawb xyuas dab tsi nyob ntawd? tiag tiag CRC 32? Ib qho ntawm cov cai thaum ntxov (txawm tias ua ntej Waltman) muaj lub luag haujlwm hash uas muab txhua tus cim los ntawm tus lej 239, yog li cov neeg nyiam no, ha ha!

Thaum kawg, ua li cas, peb pom tau hais tias cov neeg tsim kho nrog ib hom teb Vector<int> и Vector<PolymorType> yuav muaj qhov sib txawv CRC32. Yuav ua li cas txog kev ua haujlwm hauv online? Thiab los ntawm qhov kev xav theoretical, puas qhov no ua ib feem ntawm hom? Cia peb hais tias peb dhau ib array ntawm kaum txhiab tus lej, zoo nrog Vector<int> txhua yam yog qhov tseeb, qhov ntev thiab lwm 40000 bytes. Yuav ua li cas yog qhov no Vector<Type2>, uas muaj tsuas yog ib daim teb int thiab nws nyob ib leeg hauv hom - peb puas yuav tsum rov ua 10000xabcdef0 34 zaug thiab tom qab ntawd 4 bytes int, los yog cov lus muaj peev xwm INDEPEND nws rau peb los ntawm tus tsim fixedVec thiab hloov 80000 bytes, hloov dua tshiab tsuas yog 40000?

Qhov no tsis yog ib qho lus nug tsis txaus ntseeg txhua - xav txog tias koj tau txais cov npe ntawm cov neeg siv pab pawg, txhua tus ntawm lawv muaj id, thawj lub npe, lub xeem - qhov sib txawv ntawm cov ntaub ntawv xa mus rau kev sib txuas ntawm lub xov tooj ntawm tes yuav yog qhov tseem ceeb. Nws yog qhov ua tau zoo ntawm Telegram serialization uas tau tshaj tawm rau peb.

Yog li ntawd…

Vector, uas tsis tau tso tawm

Yog tias koj sim nkag mus rau cov nplooj ntawv piav qhia ntawm cov combinators thiab lwm yam, koj yuav pom tias lub vector (thiab txawm tias ib qho matrix) tau sim ua kom tau txais txiaj ntsig los ntawm tuples ntawm ob peb nplooj ntawv. Tab sis thaum kawg lawv tsis nco qab, cov kauj ruam kawg yog hla, thiab lub ntsiab lus ntawm vector tsuas yog muab, uas tseem tsis tau khi rau ib hom. Dab tsi yog qhov teeb meem? Hauv hom lus programming, tshwj xeeb tshaj yog cov ua haujlwm zoo, nws yog qhov zoo heev los piav qhia txog cov qauv rov ua dua - lub compiler nrog nws cov kev ntsuam xyuas tub nkeeg yuav nkag siab thiab ua txhua yam nws tus kheej. Hauv lus cov ntaub ntawv serialization dab tsi xav tau yog EFFICIENCY: nws yog txaus los piav qhia yooj yim daim ntawv teev, i.e. qauv ntawm ob lub ntsiab lus - thawj yog cov ntaub ntawv keeb, qhov thib ob yog tib lub qauv nws tus kheej lossis qhov chaw khoob rau tus Tsov tus tw (pob (cons) in Lisp). Tab sis qhov no yuav pom tseeb xav tau ntawm txhua tus element siv 4 bytes ntxiv (CRC32 hauv rooj plaub hauv TL) los piav txog nws hom. Ib qho array kuj tuaj yeem piav tau yooj yim tsau loj, tab sis nyob rau hauv cov ntaub ntawv ntawm ib tug array ntawm tsis paub ntev ua ntej, peb tawg tawm.

Yog li ntawd, txij li TL tsis tso cai tso tawm cov vector, nws yuav tsum tau ntxiv rau sab. Thaum kawg, cov ntaub ntawv hais tias:

Serialization ib txwm siv tib lub constructor "vector" (const 0x1cb5c415 = crc32("vector t:Type # [ t ] = Vector t") uas tsis yog nyob ntawm tus nqi tshwj xeeb ntawm qhov sib txawv ntawm hom t.

Tus nqi ntawm qhov kev xaiv parameter t tsis koom nrog hauv serialization vim nws tau muab los ntawm hom tshwm sim (ib txwm paub ua ntej deserialization).

Ua tib zoo saib: vector {t:Type} # [ t ] = Vector t - mas tsis pom qhov twg Qhov kev txhais no nws tus kheej tsis tau hais tias thawj tus lej yuav tsum sib npaug rau qhov ntev ntawm vector! Thiab nws tsis tuaj ntawm qhov twg. Qhov no yog ib qho uas yuav tsum tau khaws cia rau hauv siab thiab siv nrog koj txhais tes. Lwm qhov, cov ntaub ntawv txawm hais ncaj ncees hais tias hom tsis yog tiag:

Lub Vector t polymorphic pseudotype yog "hom" uas nws tus nqi yog ib ntu ntawm cov txiaj ntsig ntawm txhua hom t, ob lub thawv los yog liab qab.

... tab sis tsis tsom rau nws. Thaum koj, nkees ntawm kev taug kev los ntawm kev ncab ntawm kev ua lej (tej zaum txawm paub rau koj los ntawm kev kawm hauv tsev kawm ntawv), txiav txim siab tso tseg thiab saib xyuas yuav ua li cas ua haujlwm nrog nws hauv kev xyaum, lub tswv yim sab laug hauv koj lub taub hau yog qhov no hnyav heev. Kev ua lej ntawm lub hauv paus, nws tau tsim kom meej los ntawm Cov Neeg Txias (ob tus lej lej - ACM tus yeej), thiab tsis yog leej twg. Lub hom phiaj - qhia tawm - tau ua tiav.

Los ntawm txoj kev, hais txog tus lej. Cia peb nco ntsoov koj li ntawd # nws yog ib tug synonym nat, natural tooj:

Muaj hom kab zauv (yam-expr) thiab numeric expressions (nat exr). Txawm li cas los xij, lawv tau txhais tib yam nkaus.
type-expr ::= expr
nat-expr ::= expr
tab sis nyob rau hauv cov qauv sau ntawv lawv tau piav nyob rau hauv tib txoj kev, i.e. Qhov sib txawv no yuav tsum rov nco qab thiab muab tso rau hauv kev siv ntawm tes.

Zoo, yog, hom template (vector<int>, vector<User>) muaj ib tus neeg txheeb ze (#1cb5c415), i.e. yog koj paub tias hu xovtooj tshaj tawm li

users.getUsers#d91a548 id:Vector<InputUser> = Vector<User>;

ces koj tsis tau tos rau ib tug vector, tab sis ib vector ntawm cov neeg siv. Ntau precisely, yuav tsum tos - hauv txoj cai tiag tiag, txhua lub ntsiab lus, yog tias tsis yog hom liab qab, yuav muaj tus tsim, thiab hauv txoj hauv kev zoo hauv kev siv nws yuav tsum tau kuaj xyuas - tab sis peb raug xa mus raws nraim hauv txhua lub ntsiab lus ntawm cov vector no. yam li? Yuav ua li cas yog tias nws yog qee yam ntawm PHP, nyob rau hauv uas ib qho array tuaj yeem muaj ntau hom sib txawv?

Lub sijhawm no koj pib xav - puas yog qhov tsim nyog TL? Tej zaum rau lub laub nws yuav siv tau ib tug tib neeg serializer, tib protobuf uas twb muaj lawm ces? Qhov ntawd yog txoj kev xav, cia peb saib kev xyaum.

Kev siv TL uas twb muaj lawm hauv code

TL tau yug los nyob rau hauv qhov tob ntawm VKontakte txawm tias ua ntej cov xwm txheej nto moo nrog kev muag khoom ntawm Durov thiab (Tej zaum), txawm tias ua ntej txoj kev loj hlob ntawm Telegram pib. Thiab nyob rau hauv qhib qhov chaw qhov chaws ntawm thawj qhov kev siv koj tuaj yeem nrhiav tau ntau lub crutches funny. Thiab cov lus nws tus kheej tau ua tiav nyob rau ntawd ntau dua li tam sim no hauv Telegram. Piv txwv li, hashes tsis tau siv tag nrho hauv cov tswv yim (txhais tau tias yog ib qho pseudotype built-in (xws li vector) nrog tus cwj pwm txawv). Los yog

Templates are not used now. Instead, the same universal constructors (for example, vector {t:Type} [t] = Vector t) are used w

tab sis cia peb xav txog, rau qhov ua tiav, taug qab, yog li hais, evolution ntawm Giant of Thought.

#define ZHUKOV_BYTES_HACK

#ifdef ZHUKOV_BYTES_HACK

/* dirty hack for Zhukov request */

Los yog qhov zoo nkauj no:

    static const char *reserved_words_polymorhic[] = {

      "alpha", "beta", "gamma", "delta", "epsilon", "zeta", "eta", "theta", NULL

      };

Cov fragment no yog hais txog templates xws li:

intHash {alpha:Type} vector<coupleInt<alpha>> = IntHash<alpha>;

Qhov no yog lub ntsiab lus ntawm hashmap template hom raws li ib tug vector ntawm int - Hom khub. Hauv C ++ nws yuav zoo li no:

    template <T> class IntHash {
      vector<pair<int,T>> _map;
    }

yog li ntawd, alpha - lo lus tseem ceeb! Tab sis tsuas yog hauv C ++ koj tuaj yeem sau T, tab sis koj yuav tsum sau alpha, beta ... Tab sis tsis pub ntau tshaj 8 qhov tsis muaj, qhov ntawd yog qhov kev xav kawg. Nws zoo nkaus li tias ib zaug hauv St. Petersburg qee qhov kev sib tham zoo li no tau tshwm sim:

-- Надо сделать в TL шаблоны
-- Бл... Ну пусть параметры зовут альфа, бета,... Какие там ещё буквы есть... О, тэта!
-- Грамматика? Ну потом напишем

-- Смотрите, какой я синтаксис придумал для шаблонов и вектора!
-- Ты долбанулся, как мы это парсить будем?
-- Да не ссыте, он там один в схеме, захаркодить -- и ок

Tab sis qhov no yog hais txog thawj qhov kev siv ntawm TL "nyob rau hauv dav dav". Cia peb txav mus rau kev txiav txim siab siv hauv Telegram cov neeg siv khoom lawv tus kheej.

Lo lus rau Vasily:

Vasily, [09.10.18 17:07] Feem ntau ntawm tag nrho cov, lub nroog Yeiuxalees yog kub vim lawv tsim ib pawg ntawm abstractions, thiab ces hammered ib tug ntsia liaj qhov rooj rau lawv, thiab npog lub code generator nrog crutches.
Yog li ntawd, ua ntej ntawm dock pilot.jpg
Tom qab ntawd los ntawm tus lej dzhekichan.webp

Tau kawg, los ntawm cov neeg paub txog algorithms thiab lej, peb tuaj yeem cia siab tias lawv tau nyeem Aho, Ullmann, thiab paub txog cov cuab yeej uas tau dhau los ua tus qauv hauv kev lag luam ntau xyoo dhau los rau kev sau lawv cov DSL compilers, txoj cai?..

Los ntawm telegram-cli yog Vitaly Valtman, raws li tuaj yeem nkag siab los ntawm qhov tshwm sim ntawm TLO hom sab nraum nws (cli) ciam teb, ib tug tswv cuab ntawm pab pawg - tam sim no lub tsev qiv ntawv rau TL parsing tau faib nyias nyob nyias, qhov kev xav ntawm nws yog dab tsi TL parser? ..

16.12 04:18 Vasily: Kuv xav tias ib tug neeg tsis tau paub lex+yacc
16.12 04:18 Vasily: Kuv tsis tuaj yeem piav qhia lwm yam
16.12 04:18 Vasily: zoo, lossis lawv tau them rau cov kab hauv VK
16.12 04:19 Vasily: 3k+ kab thiab lwm yam.<censored> es tsis txhob parser

Tej zaum ib qho kev zam? Cia peb saib yuav ua li cas puas Nov yog OFFICIAL client - Telegram Desktop:

    nametype = re.match(r'([a-zA-Z.0-9_]+)(#[0-9a-f]+)?([^=]*)=s*([a-zA-Z.<>0-9_]+);', line);
    if (not nametype):
      if (not re.match(r'vector#1cb5c415 {t:Type} # [ t ] = Vector t;', line)):
         print('Bad line found: ' + line);

1100+ kab hauv Python, ob peb cov lus tsis tu ncua + cov xwm txheej tshwj xeeb zoo li vector, uas, tau kawg, tau tshaj tawm hauv cov tswv yim raws li nws yuav tsum ua raws li TL syntax, tab sis lawv tso siab rau cov syntax los txheeb xyuas nws ... Cov lus nug tshwm sim, yog vim li cas nws yog ib qho txuj ci tseem ceeb?иNws yog txheej txheej dua yog tias tsis muaj leej twg yuav txheeb nws raws li cov ntaub ntawv lawm?!

Los ntawm txoj kev ... Nco ntsoov peb tham txog kev kuaj CRC32? Yog li, hauv Telegram Desktop code generator muaj cov npe ntawm kev zam rau cov hom uas suav CRC32 tsis sib phim nrog rau qhov qhia hauv daim duab!

Vasily, [18.12/22 49:XNUMX] thiab ntawm no kuv yuav xav txog seb puas xav tau TL
yog tias kuv xav mess nrog lwm txoj kev siv, kuv yuav pib ntxig cov kab tawg, ib nrab ntawm cov parser yuav tawg ntawm ntau kab lus txhais.
tdesktop, txawm li cas los xij, ib yam nkaus

Nco ntsoov lub ntsiab lus ntawm ib-liner, peb yuav rov qab mus rau nws me ntsis tom qab.

Okay, telegram-cli tsis raug cai, Telegram Desktop yog official, tab sis dab tsi txog lwm tus? Leej twg paub?.. Hauv Android tus neeg siv code tsis muaj schema parser txhua (uas ua rau muaj lus nug txog qhib qhov chaw, tab sis qhov no yog rau ntu thib ob), tab sis muaj ob peb lwm yam kev lom zem ntawm code, tab sis ntxiv rau lawv hauv ntu ntu hauv qab no.

Lwm cov lus nug dab tsi ua serialization nce hauv kev xyaum? Piv txwv li, lawv tau ua ntau yam, ntawm chav kawm, nrog me me teb thiab thaj chaw muaj cai:

Vasily: flags.0? true
txhais tau hais tias daim teb tam sim no thiab sib npaug yog tias tus chij tau teeb tsa

Vasily: flags.1? int
txhais tau hais tias lub teb yog tam sim no thiab yuav tsum tau deserialized

Vasily: Ass, tsis txhob txhawj txog qhov koj ua!
Vasily: Muaj ib qho kev hais txog qhov chaw hauv doc uas muaj tseeb yog hom xoom-ntev ntev, tab sis nws tsis tuaj yeem sib sau ua ib yam dab tsi los ntawm lawv cov ntaub ntawv.
Vasily: Nyob rau hauv qhov qhib qhov kev siv qhov no tsis yog qhov teeb meem, tab sis muaj ntau cov crutches thiab kev txhawb nqa.

Yuav ua li cas yog Telethon? Saib ua ntej rau lub ntsiab lus ntawm MTProto, piv txwv - hauv cov ntaub ntawv muaj xws li daim, tab sis kos npe % nws tsuas yog piav raws li "xav mus rau ib hom liab qab", i.e. hauv cov piv txwv hauv qab no muaj qhov yuam kev lossis qee yam tsis muaj ntaub ntawv:

Vasily, [22.06.18 18:38] Ib qhov chaw:
msg_container#73f1f8dc messages:vector message = MessageContainer;
Hauv qhov sib txawv:
msg_container#73f1f8dc messages:vector<%Message> = MessageContainer;
Thiab cov no yog ob qhov sib txawv loj, hauv lub neej tiag tiag qee yam ntawm liab qab vector tuaj

Kuv tsis tau pom ib tug liab qab vector txhais thiab tsis tau tuaj hla ib

Kev tshuaj xyuas yog sau los ntawm tes hauv telethon

Nyob rau hauv nws daim duab lub ntsiab txhais tau hais tawm msg_container

Ib zaug ntxiv, cov lus nug tseem tshuav txog %. Nws tsis piav.

Vadim Goncharov, [22.06.18 19:22] and in tdesktop?

Vasily, [22.06.18 19:23] Tab sis lawv TL parser ntawm cov cav tsis tu ncua feem ntau yuav tsis noj qhov no.

// parsed manually

TL yog qhov kev xav zoo nkauj, tsis muaj leej twg ua tiav nws

Thiab % tsis yog nyob rau hauv lawv cov version ntawm lub tswv yim

Tab sis ntawm no cov ntaub ntawv contradicts nws tus kheej, yog li idk

Nws tau pom nyob rau hauv cov qauv sau ntawv, lawv tsuas tuaj yeem tsis nco qab piav txog cov semantics

Koj pom cov ntaub ntawv ntawm TL, koj tsis tuaj yeem xam nws yam tsis muaj ib nrab litres

"Zoo, cia peb hais," lwm tus nyeem yuav hais tias, "koj thuam ib yam dab tsi, yog li qhia kuv tias nws yuav tsum ua li cas."

Vasily teb hais tias: “Raws li tus parser, kuv nyiam yam zoo li

    args: /* empty */ { $$ = NULL; }
        | args arg { $$ = g_list_append( $1, $2 ); }
        ;

    arg: LC_ID ':' type-term { $$ = tl_arg_new( $1, $3 ); }
            | LC_ID ':' condition '?' type-term { $$ = tl_arg_new_cond( $1, $5, $3 ); free($3); }
            | UC_ID ':' type-term { $$ = tl_arg_new( $1, $3 ); }
            | type-term { $$ = tl_arg_new( "", $1 ); }
            | '[' LC_ID ']' { $$ = tl_arg_new_mult( "", tl_type_new( $2, TYPE_MOD_NONE ) ); }
            ;

cas zoo dua li os

struct tree *parse_args4 (void) {
  PARSE_INIT (type_args4);
  struct parse so = save_parse ();
  PARSE_TRY (parse_optional_arg_def);
  if (S) {
    tree_add_child (T, S);
  } else {
    load_parse (so);
  }
  if (LEX_CHAR ('!')) {
    PARSE_ADD (type_exclam);
    EXPECT ("!");
  }
  PARSE_TRY_PES (parse_type_term);
  PARSE_OK;
}

los yog

        # Regex to match the whole line
        match = re.match(r'''
            ^                  # We want to match from the beginning to the end
            ([w.]+)           # The .tl object can contain alpha_name or namespace.alpha_name
            (?:
                #             # After the name, comes the ID of the object
                ([0-9a-f]+)    # The constructor ID is in hexadecimal form
            )?                 # If no constructor ID was given, CRC32 the 'tl' to determine it

            (?:s              # After that, we want to match its arguments (name:type)
                {?             # For handling the start of the '{X:Type}' case
                w+            # The argument name will always be an alpha-only name
                :              # Then comes the separator between name:type
                [wd<>#.?!]+  # The type is slightly more complex, since it's alphanumeric and it can
                               # also have Vector<type>, flags:# and flags.0?default, plus :!X as type
                }?             # For handling the end of the '{X:Type}' case
            )*                 # Match 0 or more arguments
            s                 # Leave a space between the arguments and the equal
            =
            s                 # Leave another space between the equal and the result
            ([wd<>#.?]+)     # The result can again be as complex as any argument type
            ;$                 # Finally, the line should always end with ;
            ''', tl, re.IGNORECASE | re.VERBOSE)

Nov yog TAG NRHO lexer:

    ---functions---         return FUNCTIONS;
    ---types---             return TYPES;
    [a-z][a-zA-Z0-9_]*      yylval.string = strdup(yytext); return LC_ID;
    [A-Z][a-zA-Z0-9_]*      yylval.string = strdup(yytext); return UC_ID;
    [0-9]+                  yylval.number = atoi(yytext); return NUM;
    #[0-9a-fA-F]{1,8}       yylval.number = strtol(yytext+1, NULL, 16); return ID_HASH;

    n                      /* skip new line */
    [ t]+                  /* skip spaces */
    //.*$                 /* skip comments */
    /*.**/              /* skip comments */
    .                       return (int)yytext[0];

cov. yooj yim yog muab nws me ntsis. "

Feem ntau, raws li qhov tshwm sim, tus parser thiab code generator rau qhov tseeb siv subset ntawm TL haum rau kwv yees li 100 kab ntawm cov qauv sau ntawv thiab ~ 300 kab ntawm lub tshuab hluav taws xob (suav tag nrho print's generated code), nrog rau hom ntaub ntawv buns rau introspection nyob rau hauv txhua chav kawm. Txhua hom polymorphic hloov mus rau hauv chav kawm paub daws teeb meem, thiab cov neeg tsim khoom tau txais txiaj ntsig los ntawm nws thiab muaj txoj hauv kev rau serialization thiab deserialization.

Tsis muaj hom hauv hom lus

Kev ntaus ntawv muaj zog yog qhov zoo, txoj cai? Tsis yog, qhov no tsis yog holivar (txawm hais tias kuv nyiam cov lus dynamic), tab sis ib qho postulate nyob rau hauv lub moj khaum ntawm TL. Raws li nws, cov lus yuav tsum muab txhua yam kev kuaj xyuas rau peb. Zoo, ua li cas, tej zaum tsis yog nws tus kheej, tab sis qhov kev siv, tab sis nws yuav tsum tsawg kawg piav qhia lawv. Thiab peb xav tau lub cib fim zoo li cas?

Ua ntej tshaj plaws, kev txwv. Ntawm no peb pom nyob rau hauv cov ntaub ntawv rau uploading cov ntaub ntawv:

Cov ntaub ntawv binary cov ntsiab lus yog ces muab faib ua ntu. Txhua qhov chaw yuav tsum muaj qhov loj tib yam ( part_size ) thiab cov xwm txheej hauv qab no yuav tsum tau ua raws li:

part_size % 1024 = 0 (divisible los ntawm 1 KB)

524288 % part_size = 0 (512KB yuav tsum sib npaug sib faib los ntawm part_size)

Qhov kawg tsis tas yuav ua kom txaus siab rau cov xwm txheej no, muab nws qhov loj me me dua part_size.

Txhua ntu yuav tsum muaj tus lej sib txuas, file_part, nrog tus nqi ntawm 0 txog 2,999.

Tom qab cov ntaub ntawv tau muab faib koj yuav tsum xaiv ib txoj hauv kev txuag nws ntawm lub server. Siv upload.saveBigFilePart nyob rau hauv cov ntaub ntawv tag nrho loj ntawm cov ntaub ntawv yog ntau tshaj 10 MB thiab upload.saveFilePart rau cov ntaub ntawv me me.
[…] ib qho ntawm cov ntaub ntawv nkag yuam kev hauv qab no tuaj yeem xa rov qab:

FILE_PARTS_INVALID - Cov lej tsis raug. Tus nqi tsis nyob nruab nrab 1..3000

Puas muaj qhov no hauv daim duab? Puas yog qhov no qhia tau hais tias siv TL? Tsis muaj. Tab sis thov zam txim rau kuv, txawm tias yawg tus Turbo Pascal tuaj yeem piav qhia txog hom tau teev tseg thaj tsam. Thiab nws paub ib yam ntxiv, tam sim no zoo dua lub npe hu ua enum - ib hom uas suav nrog tus lej ntawm tus lej tas (me) ntawm qhov muaj nuj nqis. Hauv hom lus xws li C - tus lej, nco ntsoov tias tam sim no peb tsuas yog tham txog hom tooj. Tab sis kuj muaj cov arrays, cov hlua ... piv txwv li, nws yuav zoo los piav qhia tias txoj hlua no tsuas muaj tus xov tooj xwb, puas yog?

Tsis muaj qhov no nyob hauv TL. Tab sis muaj, piv txwv li, hauv JSON Schema. Thiab yog tias lwm tus neeg tuaj yeem sib cav txog qhov sib faib ntawm 512 KB, tias qhov no tseem yuav tsum tau kuaj xyuas hauv code, tom qab ntawd xyuas kom meej tias tus neeg siv khoom yooj yim. kuv ua tsis tau xa tus lej tawm ntawm qhov ntau 1..3000 (thiab qhov kev ua yuam kev tsis tuaj yeem tshwm sim) nws yuav ua tau, txoj cai? ..

Los ntawm txoj kev, hais txog qhov yuam kev thiab rov qab qhov tseem ceeb. Txawm tias cov uas tau ua hauj lwm nrog TL qhov muag plooj - nws tsis tau kaj ntug rau peb tam sim ntawd txhua kev ua haujlwm hauv TL tuaj yeem rov qab tsis tau tsuas yog cov lus piav qhia rov qab, tab sis kuj yog qhov yuam kev. Tab sis qhov no tsis tuaj yeem txiav tawm hauv txhua txoj kev siv TL nws tus kheej. Tau kawg, nws twb paub meej lawm thiab tsis tas yuav muaj dab tsi hauv kev xyaum (txawm hais tias qhov tseeb, RPC tuaj yeem ua tau ntau txoj hauv kev, peb yuav rov qab los rau qhov no tom qab) - tab sis dab tsi txog qhov Purity ntawm cov ntsiab lus ntawm Kev Ua lej ntawm Cov Lus Qhia Tsis Txaus Siab. los ntawm ntuj ceeb tsheej?... Kuv khaws tus tug - thiaj phim nws.

Thiab thaum kawg, ua li cas txog kev nyeem ntawv? Zoo, muaj, feem ntau, kuv xav piav qhia muaj nws txoj cai nyob rau hauv lub schema (hauv JSON schema, dua, nws yog), tab sis yog tias koj twb strained nrog nws, ces dab tsi txog cov tswv yim sab - tsawg kawg yog trivally saib qhov txawv thaum hloov tshiab? Saib rau koj tus kheej ntawm piv txwv tiag:

-channelFull#76af5481 flags:# can_view_participants:flags.3?true can_set_username:flags.6?true can_set_stickers:flags.7?true hidden_prehistory:flags.10?true id:int about:string participants_count:flags.0?int admins_count:flags.1?int kicked_count:flags.2?int banned_count:flags.2?int read_inbox_max_id:int read_outbox_max_id:int unread_count:int chat_photo:Photo notify_settings:PeerNotifySettings exported_invite:ExportedChatInvite bot_info:Vector<BotInfo> migrated_from_chat_id:flags.4?int migrated_from_max_id:flags.4?int pinned_msg_id:flags.5?int stickerset:flags.8?StickerSet available_min_id:flags.9?int = ChatFull;
+channelFull#1c87a71a flags:# can_view_participants:flags.3?true can_set_username:flags.6?true can_set_stickers:flags.7?true hidden_prehistory:flags.10?true can_view_stats:flags.12?true id:int about:string participants_count:flags.0?int admins_count:flags.1?int kicked_count:flags.2?int banned_count:flags.2?int online_count:flags.13?int read_inbox_max_id:int read_outbox_max_id:int unread_count:int chat_photo:Photo notify_settings:PeerNotifySettings exported_invite:ExportedChatInvite bot_info:Vector<BotInfo> migrated_from_chat_id:flags.4?int migrated_from_max_id:flags.4?int pinned_msg_id:flags.5?int stickerset:flags.8?StickerSet available_min_id:flags.9?int = ChatFull;

los yog

-message#44f9b43d flags:# out:flags.1?true mentioned:flags.4?true media_unread:flags.5?true silent:flags.13?true post:flags.14?true id:int from_id:flags.8?int to_id:Peer fwd_from:flags.2?MessageFwdHeader via_bot_id:flags.11?int reply_to_msg_id:flags.3?int date:int message:string media:flags.9?MessageMedia reply_markup:flags.6?ReplyMarkup entities:flags.7?Vector<MessageEntity> views:flags.10?int edit_date:flags.15?int post_author:flags.16?string grouped_id:flags.17?long = Message;
+message#44f9b43d flags:# out:flags.1?true mentioned:flags.4?true media_unread:flags.5?true silent:flags.13?true post:flags.14?true from_scheduled:flags.18?true id:int from_id:flags.8?int to_id:Peer fwd_from:flags.2?MessageFwdHeader via_bot_id:flags.11?int reply_to_msg_id:flags.3?int date:int message:string media:flags.9?MessageMedia reply_markup:flags.6?ReplyMarkup entities:flags.7?Vector<MessageEntity> views:flags.10?int edit_date:flags.15?int post_author:flags.16?string grouped_id:flags.17?long = Message;

Nws nyob ntawm txhua tus, tab sis GitHub, piv txwv li, tsis kam qhia txog kev hloov pauv hauv cov kab ntev. Qhov kev ua si "nrhiav 10 qhov sib txawv", thiab qhov uas lub hlwb pom tam sim ntawd yog qhov pib thiab qhov xaus ntawm ob qho piv txwv yog tib yam, koj yuav tsum tau nyeem ib qho chaw hauv nruab nrab ... Hauv kuv lub tswv yim, qhov no tsis yog nyob rau hauv txoj kev xav, tab sis yog pom tseeb qias neeg thiab sloppy.

Los ntawm txoj kev, hais txog purity ntawm txoj kev xav. Vim li cas peb thiaj xav tau me ntsis teb? Tsis zoo li lawv tsw phem los ntawm qhov pom ntawm hom kev xav? Cov lus piav qhia tuaj yeem pom hauv cov qauv ua ntej ntawm daim duab. Thaum xub thawj, yog, qhov ntawd yog li cas, rau txhua qhov txham ib hom tshiab tau tsim. Cov rudiments tseem muaj nyob rau hauv daim ntawv no, piv txwv li:

storage.fileUnknown#aa963b05 = storage.FileType;
storage.filePartial#40bc6f52 = storage.FileType;
storage.fileJpeg#7efe0e = storage.FileType;
storage.fileGif#cae1aadf = storage.FileType;
storage.filePng#a4f63c0 = storage.FileType;
storage.filePdf#ae1e508d = storage.FileType;
storage.fileMp3#528a0677 = storage.FileType;
storage.fileMov#4b09ebbc = storage.FileType;
storage.fileMp4#b3cea0e4 = storage.FileType;
storage.fileWebp#1081464c = storage.FileType;

Tab sis tam sim no xav txog, yog tias koj muaj 5 qhov kev xaiv hauv koj cov qauv, ces koj yuav xav tau 32 hom rau txhua qhov kev xaiv. Combinatorial tawg. Yog li, crystal purity ntawm TL txoj kev xav ib zaug ntxiv tawg tawm tsam cov cam khwb cia hlau lub nroog Yeiuxalees ntawm qhov kev muaj tiag hnyav ntawm serialization.

Tsis tas li ntawd, nyob rau qee qhov chaw cov txiv neej no lawv tus kheej ua txhaum lawv tus kheej typology. Piv txwv li, hauv MTProto ( tshooj tom ntej) cov lus teb tuaj yeem raug compressed los ntawm Gzip, txhua yam zoo - tshwj tsis yog tias cov khaubncaws sab nraud povtseg thiab Circuit Court raug ua txhaum. Ib zaug ntxiv, nws tsis yog RpcResult nws tus kheej uas tau sau tseg, tab sis nws cov ntsiab lus. Zoo, ua li cas qhov no?.. Kuv yuav tsum tau txiav mus rau hauv ib tug crutch kom lub compression yuav ua hauj lwm nyob qhov twg.

Los yog lwm qhov piv txwv, peb ib zaug pom qhov yuam kev - nws raug xa mus InputPeerUser es tsis txhob InputUser. Los yog vice versa. Tab sis nws ua haujlwm! Ntawd yog, tus neeg rau zaub mov tsis quav ntsej txog hom. Qhov no yuav ua li cas? Cov lus teb tuaj yeem muab rau peb los ntawm cov kab lus tawg ntawm telegram-cli:

  if (tgl_get_peer_type (E->id) != TGL_PEER_CHANNEL || (C && (C->flags & TGLCHF_MEGAGROUP))) {
    out_int (CODE_messages_get_history);
    out_peer_id (TLS, E->id);
  } else {    
    out_int (CODE_channels_get_important_history);

    out_int (CODE_input_channel);
    out_int (tgl_get_peer_id (E->id));
    out_long (E->id.access_hash);
  }
  out_int (E->max_id);
  out_int (E->offset);
  out_int (E->limit);
  out_int (0);
  out_int (0);

Hauv lwm lo lus, qhov no yog qhov serialization ua tiav manually, tsis generated code! Tej zaum tus neeg rau zaub mov tau siv los ua ib txoj hauv kev zoo sib xws?.. Hauv txoj ntsiab cai, qhov no yuav ua haujlwm yog tias ua tiav ib zaug, tab sis yuav ua li cas thiaj li txhawb nqa tom qab thaum hloov tshiab? Qhov no puas yog vim li cas lub tswvyim tau tsim? Thiab ntawm no peb mus rau lo lus nug tom ntej.

Versioning. Txheej

Vim li cas cov schematic versions hu ua txheej tsuas yog speculated raws li keeb kwm ntawm luam tawm schematics. Pom tseeb, thaum xub thawj cov kws sau ntawv xav tias tej yam yooj yim tuaj yeem ua tau los ntawm cov txheej txheem tsis hloov pauv, thiab tsuas yog qhov tsim nyog, rau kev thov tshwj xeeb, qhia tias lawv tau ua tiav los ntawm kev sib txawv. Hauv txoj ntsiab cai, txawm tias lub tswv yim zoo - thiab qhov tshiab yuav yog, zoo li nws yog, "sib xyaw", txheej rau saum cov qub. Tab sis cia saib seb nws ua li cas. Muaj tseeb, kuv tsis tuaj yeem saib nws txij thaum pib - nws lom zem, tab sis daim duab ntawm lub hauv paus txheej tsuas tsis muaj nyob. Cov txheej pib nrog 2. Cov ntaub ntawv qhia peb txog qhov tshwj xeeb TL feature:

Yog tias tus neeg siv khoom txhawb nqa Txheej 2, ces tus tsim qauv hauv qab no yuav tsum tau siv:
invokeWithLayer2#289dd1f6 {X:Type} query:!X = X;
Hauv kev xyaum, qhov no txhais tau tias ua ntej txhua tus API hu, ib qho int nrog tus nqi 0x289dd1f6 yuav tsum tau ntxiv ua ntej tus xov tooj.

Suab zoo li qub. Tab sis dab tsi tshwm sim tom ntej? Ces tshwm

invokeWithLayer3#b7475268 query:!X = X;

Yog li dab tsi ntxiv? Raws li koj tuaj yeem twv,

invokeWithLayer4#dea0d430 query:!X = X;

Funny? Tsis yog, nws ntxov dhau los luag, xav txog qhov tseeb txhua ib qho kev thov los ntawm lwm txheej yuav tsum tau muab qhwv rau hauv ib hom tshwj xeeb - yog tias koj muaj lawv txhua tus sib txawv, koj tuaj yeem paub qhov txawv ntawm lawv li cas? Thiab ntxiv 4 bytes rau pem hauv ntej yog ib txoj hauv kev zoo nkauj. Yog li ntawd,

invokeWithLayer5#417a57ae query:!X = X;

Tab sis nws yog pom tseeb tias tom qab ib ntus qhov no yuav dhau los ua qee yam ntawm bacchanalia. Thiab qhov kev daws teeb meem tuaj:

Hloov tshiab: Pib nrog Txheej 9, cov txheej txheem pab invokeWithLayerN tuaj yeem siv ua ke nrog initConnection

Hooray! Tom qab 9 versions, peb thaum kawg tuaj rau qhov ua tiav hauv Internet raws tu qauv rov qab rau hauv 80s - pom zoo rau lub version ib zaug thaum pib ntawm kev sib txuas!

Tom ntej no yog dab tsi?..

invokeWithLayer10#39620c41 query:!X = X;
...
invokeWithLayer18#1c900537 query:!X = X;

Tab sis tam sim no koj tseem tuaj yeem luag. Tsuas yog tom qab lwm 9 khaubncaws sab nraud povtseg, ib tug universal constructor nrog ib tug version thaum kawg tau ntxiv, uas yuav tsum tau hu ib zaug xwb thaum pib ntawm kev sib txuas, thiab lub ntsiab lus ntawm cov khaubncaws sab nraud povtseg zoo li tau ploj mus, tam sim no nws tsuas yog ib tug conditional version, xws li nyob txhua qhov chaw. Teeb meem daws tau.

Tiag?..

Vasily, [16.07.18 14:01] Txawm hnub Friday kuv xav tias:
Lub teleserver xa cov xwm txheej yam tsis tau thov. Kev thov yuav tsum tau qhwv hauv InvokeWithLayer. Tus neeg rau zaub mov tsis qhwv qhov hloov tshiab; tsis muaj cov qauv rau wrapping cov lus teb thiab hloov tshiab.

Cov. tus neeg siv khoom tsis tuaj yeem qhia cov txheej txheem uas nws xav tau hloov tshiab

Vadim Goncharov, [16.07.18 14:02] Tsis yog InvokeWithLayer yog tus ntoo khaub lig hauv txoj cai?

Vasily, [16.07.18 14:02] Qhov no yog tib txoj kev

Vadim Goncharov, [16.07.18 14:02] uas qhov tseem ceeb yuav tsum txhais tau tias pom zoo rau txheej thaum pib ntawm kev sib tham

Los ntawm txoj kev, nws ua raws li cov neeg siv khoom downgrade tsis muab

Hloov tshiab, i.e. hom Updates nyob rau hauv cov tswv yim, qhov no yog dab tsi tus neeg rau zaub mov xa mus rau tus neeg siv khoom tsis teb rau qhov kev thov API, tab sis ntawm nws tus kheej thaum muaj xwm txheej tshwm sim. Qhov no yog ib lub ntsiab lus nyuaj uas yuav tau tham hauv lwm tus tshaj tawm, tab sis tam sim no nws yog ib qho tseem ceeb kom paub tias cov neeg rau zaub mov txuag hloov tshiab txawm tias tus neeg siv khoom offline.

Yog li, yog tias koj tsis kam qhwv ntawm txhua tus pob los qhia nws cov version, qhov no logically ua rau cov teeb meem hauv qab no:

tus neeg rau zaub mov xa cov hloov tshiab rau tus neeg siv khoom txawm tias ua ntej tus neeg siv khoom tau ceeb toom rau qhov twg nws txhawb nqa
Kuv yuav ua li cas tom qab kho dua tus neeg siv khoom?
leej twg lavtias tus neeg rau zaub mov xav txog cov txheej txheem yuav tsis hloov thaum lub sijhawm ua haujlwm?

Koj puas xav tias qhov no yog qhov kev kwv yees nkaus xwb, thiab hauv kev xyaum, qhov no tsis tuaj yeem tshwm sim, vim hais tias tus neeg rau zaub mov tau sau kom raug (tsawg kawg, nws raug sim zoo)? Ha! Txawm li cas los xij!

Qhov no yog qhov peb tau khiav mus rau lub Yim Hli. Thaum Lub Yim Hli 14, muaj cov lus hais tias qee yam tau hloov kho ntawm Telegram servers ... thiab tom qab ntawd hauv cov cav:

2019-08-15 09:28:35.880640 MSK warn  main: ANON:87: unknown object type: 0x80d182d1 at TL/Object.pm line 213.
2019-08-15 09:28:35.751899 MSK warn  main: ANON:87: unknown object type: 0xb5223b0f at TL/Object.pm line 213.

thiab tom qab ntawd ob peb megabytes ntawm pawg kab (zoo, tib lub sijhawm cov ntawv kaw tau tsau). Tom qab tag nrho, yog tias ib yam dab tsi tsis lees paub hauv koj TL, nws yog binary los ntawm kos npe, txuas ntxiv mus rau kab TAG NRHO mus, decoding yuav ua tsis tau. Koj yuav tsum ua li cas rau qhov xwm txheej zoo li no?

Zoo, thawj yam uas los rau leej twg lub siab yog txiav thiab sim dua. Tsis tau pab. Peb google CRC32 - cov no tau los ua cov khoom los ntawm cov tswv yim 73, txawm hais tias peb ua haujlwm ntawm 82. Peb ua tib zoo saib ntawm cov cav - muaj cov cim los ntawm ob lub tswv yim sib txawv!

Tej zaum qhov teeb meem yog dawb huv hauv peb cov neeg siv khoom tsis raug cai? Tsis yog, peb tso Telegram Desktop 1.2.17 (version muab nyob rau hauv ib tug xov tooj ntawm Linux distributions), nws sau rau lub Exception cav: MTP Tsis npaj txhij hom id #b5223b0f nyeem hauv MTPMessageMedia…

Google tau qhia tias qhov teeb meem zoo sib xws twb tau tshwm sim rau ib qho ntawm cov neeg siv khoom tsis raug cai, tab sis tom qab ntawd cov lej version thiab, raws li, cov kev xav tau txawv ...

Yog li peb yuav tsum ua li cas? Vasily thiab kuv sib cais: nws sim hloov kho lub voj voog mus rau 91, Kuv txiav txim siab tos ob peb hnub thiab sim rau 73. Ob txoj kev ua haujlwm, tab sis txij li lawv yog cov empirical, tsis muaj kev nkag siab txog ntau npaum li cas versions nce lossis nqis koj xav tau. dhia, lossis ntev npaum li cas koj yuav tsum tau tos.

Tom qab ntawd kuv muaj peev xwm rov tsim dua qhov teeb meem: peb tso tus neeg siv khoom, tua nws, rov ua dua lub voj voog mus rau lwm txheej, rov pib dua, ntes qhov teeb meem dua, rov qab mus rau qhov qub - oops, tsis muaj qhov hloov pauv ntawm Circuit Court thiab tus neeg siv khoom rov pib dua. ob peb feeb yuav pab. Koj yuav tau txais cov ntaub ntawv sib xyaw ntawm cov txheej txheem sib txawv.

Kev piav qhia? Raws li koj tuaj yeem kwv yees los ntawm ntau cov tsos mob tsis ncaj ncees, cov neeg rau zaub mov muaj ntau cov txheej txheem ntawm ntau hom ntawm cov tshuab sib txawv. Feem ntau yuav, tus neeg rau zaub mov uas yog lub luag haujlwm rau "buffering" muab tso rau hauv cov kab uas nws cov thawj coj tau muab rau nws, thiab lawv tau muab rau hauv lub tswv yim uas nyob rau hauv qhov chaw ntawm lub sijhawm tiam. Thiab kom txog rau thaum cov kab no "rot", tsis muaj dab tsi ua tau txog nws.

Tej zaum... tab sis qhov no yog ib tug phem crutch?! Hauv Android version peb tsis pom TL parser, tab sis peb pom cov ntaub ntawv hnyav (GitHub tsis kam kov nws) nrog (de) serialization. Nov yog cov code snippet:

public static class TL_message_layer68 extends TL_message {
    public static int constructor = 0xc09be45f;
//...
//еще пачка подобных
//...
    public static class TL_message_layer47 extends TL_message {
        public static int constructor = 0xc992e15c;
        public static Message TLdeserialize(AbstractSerializedData stream, int constructor, boolean exception) {
            Message result = null;
            switch (constructor) {
                case 0x1d86f70e:
                    result = new TL_messageService_old2();
                    break;
                case 0xa7ab1991:
                    result = new TL_message_old3();
                    break;
                case 0xc3060325:
                    result = new TL_message_old4();
                    break;
                case 0x555555fa:
                    result = new TL_message_secret();
                    break;
                case 0x555555f9:
                    result = new TL_message_secret_layer72();
                    break;
                case 0x90dddc11:
                    result = new TL_message_layer72();
                    break;
                case 0xc09be45f:
                    result = new TL_message_layer68();
                    break;
                case 0xc992e15c:
                    result = new TL_message_layer47();
                    break;
                case 0x5ba66c13:
                    result = new TL_message_old7();
                    break;
                case 0xc06b9607:
                    result = new TL_messageService_layer48();
                    break;
                case 0x83e5de54:
                    result = new TL_messageEmpty();
                    break;
                case 0x2bebfa86:
                    result = new TL_message_old6();
                    break;
                case 0x44f9b43d:
                    result = new TL_message_layer104();
                    break;
                case 0x1c9b1027:
                    result = new TL_message_layer104_2();
                    break;
                case 0xa367e716:
                    result = new TL_messageForwarded_old2(); //custom
                    break;
                case 0x5f46804:
                    result = new TL_messageForwarded_old(); //custom
                    break;
                case 0x567699b3:
                    result = new TL_message_old2(); //custom
                    break;
                case 0x9f8d60bb:
                    result = new TL_messageService_old(); //custom
                    break;
                case 0x22eb6aba:
                    result = new TL_message_old(); //custom
                    break;
                case 0x555555F8:
                    result = new TL_message_secret_old(); //custom
                    break;
                case 0x9789dac4:
                    result = new TL_message_layer104_3();
                    break;

los yog

    boolean fixCaption = !TextUtils.isEmpty(message) &&
    (media instanceof TLRPC.TL_messageMediaPhoto_old ||
     media instanceof TLRPC.TL_messageMediaPhoto_layer68 ||
     media instanceof TLRPC.TL_messageMediaPhoto_layer74 ||
     media instanceof TLRPC.TL_messageMediaDocument_old ||
     media instanceof TLRPC.TL_messageMediaDocument_layer68 ||
     media instanceof TLRPC.TL_messageMediaDocument_layer74)
    && message.startsWith("-1");

Hm...zoo li qus. Tab sis, tej zaum, qhov no yog generated code, ces ok?.. Tab sis nws yeej txhawb tag nrho cov versions! Muaj tseeb, nws tsis paub meej vim li cas txhua yam sib xyaw ua ke, sib tham zais cia, thiab txhua yam _old7 cas tsis zoo li tshuab tiam... Txawm li cas los xij, feem ntau ntawm txhua yam kuv tau raug tshuab los ntawm

TL_message_layer104
TL_message_layer104_2
TL_message_layer104_3

Cov txiv neej, koj tsis tuaj yeem txiav txim siab dab tsi hauv ib txheej?! Zoo, ua li cas, cia peb hais tias "ob" raug tso tawm nrog qhov yuam kev, zoo, nws tshwm sim, tab sis peb?.. Tam sim ntawd, tib rake dua? Dab tsi yog duab liab qab no, thov txim?..

Nyob rau hauv lub hauv paus code ntawm Telegram Desktop, los ntawm txoj kev, ib yam dab tsi zoo sib xws tshwm sim - yog hais tias yog li ntawd, ob peb commits nyob rau hauv ib tug kab rau lub tswv yim tsis hloov nws cov txheej txheej, tab sis kho ib yam dab tsi. Hauv cov xwm txheej uas tsis muaj cov ntaub ntawv raug cai rau lub tswv yim, qhov twg tuaj yeem tau txais los ntawm, tshwj tsis yog cov cai ntawm tus neeg siv khoom? Thiab yog tias koj coj nws los ntawm qhov ntawd, koj tsis tuaj yeem paub tseeb tias lub tswv yim yog tag nrho kom txog thaum koj sim txhua txoj hauv kev.

Yuav ua li cas qhov no txawm raug kuaj? Kuv vam tias cov kiv cua ntawm chav tsev, kev ua haujlwm thiab lwm yam kev xeem yuav qhia hauv cov lus.

Okay, cia saib lwm daim code:

public static class TL_folders_deleteFolder extends TLObject {
    public static int constructor = 0x1c295881;

    public int folder_id;

    public TLObject deserializeResponse(AbstractSerializedData stream, int constructor, boolean exception) {
        return Updates.TLdeserialize(stream, constructor, exception);
    }

    public void serializeToStream(AbstractSerializedData stream) {
        stream.writeInt32(constructor);
        stream.writeInt32(folder_id);
    }
}

//manually created

//RichText start
public static abstract class RichText extends TLObject {
    public String url;
    public long webpage_id;
    public String email;
    public ArrayList<RichText> texts = new ArrayList<>();
    public RichText parentRichText;

    public static RichText TLdeserialize(AbstractSerializedData stream, int constructor, boolean exception) {
        RichText result = null;
        switch (constructor) {
            case 0x1ccb966a:
                result = new TL_textPhone();
                break;
            case 0xc7fb5e01:
                result = new TL_textSuperscript();
                break;

Cov lus no "manually tsim" qhia tias tsuas yog ib feem ntawm cov ntaub ntawv no tau sau manually (koj puas xav txog tag nrho cov kev saib xyuas npau suav phem?), thiab tus so yog tshuab tsim. Txawm li cas los xij, tom qab ntawd lwm lo lus nug tshwm sim - tias cov peev txheej muaj tsis tag (a la GPL blobs hauv Linux ntsiav), tab sis qhov no twb yog lub ntsiab lus rau ntu thib ob.

Tab sis txaus. Cia peb txav mus rau cov txheej txheem saum toj kawg nkaus uas txhua qhov serialization no khiav.

MT Proto

Yog li, cia peb qhib kev piav qhia dav dav и cov lus piav qhia ntxaws txog cov txheej txheem thiab thawj yam uas peb dawm dhau yog cov terminology. Thiab nrog kev nplua nuj ntawm txhua yam. Feem ntau, qhov no zoo li yog ib qho tshwj xeeb ntawm Telegram - hu cov khoom sib txawv hauv qhov chaw sib txawv, lossis ntau yam sib txawv nrog ib lo lus, lossis rov ua dua (piv txwv li, hauv API qib siab, yog tias koj pom cov ntawv nplaum, nws tsis yog. koj xav li cas).

Piv txwv li, "message" thiab "session" txhais tau tias qee yam txawv ntawm no dua li hauv Telegram tus neeg siv khoom siv niaj hnub. Zoo, txhua yam yog qhov tseeb nrog cov lus, nws tuaj yeem txhais hauv OOP cov ntsiab lus, lossis tsuas yog hu ua lo lus "pob ntawv" - qhov no yog qib qis, thauj, tsis muaj cov lus zoo ib yam li hauv interface, muaj ntau cov lus pabcuam. . Tab sis kev sib kho ... tab sis thawj yam ua ntej.

Thawv txheej

Thawj qhov yog kev thauj mus los. Lawv yuav qhia peb txog 5 txoj kev xaiv:

TCP
Websocket
Websocket dhau HTTPS
HTTP
HTTPS

Vasily, [15.06.18 15:04] Tseem muaj UDP thauj, tab sis nws tsis tau sau tseg

Thiab TCP nyob rau hauv peb variants

Thawj qhov zoo ib yam li UDP dhau TCP, txhua pob ntawv suav nrog tus lej lej thiab crc
Vim li cas nyeem cov ntaub ntawv ntawm lub laub mob heev?

Zoo, muaj tam sim no TCP twb nyob rau hauv 4 variants:

Ntshaiv
Intermediate
Padded nruab nrab
Tag nrho cov

Zoo, ok, Padded nruab nrab rau MTProxy, qhov no tau ntxiv tom qab vim muaj cov xwm txheej paub zoo. Tab sis vim li cas ob qho ntxiv (peb nyob rau hauv tag nrho) thaum koj tuaj yeem tau txais los ntawm ib qho? Tag nrho plaub qhov tseem ceeb sib txawv tsuas yog yuav ua li cas los teeb qhov ntev thiab them nyiaj ntawm lub ntsiab MTProto, uas yuav tau tham ntxiv:

hauv Abridged nws yog 1 lossis 4 bytes, tab sis tsis yog 0xef, ces lub cev
hauv Intermediate qhov no yog 4 bytes ntawm qhov ntev thiab ib daim teb, thiab thawj zaug tus neeg siv yuav tsum xa 0xeeeeeeee los qhia tias nws yog Intermediate
Nyob rau hauv Tag Nrho qhov ntau ntxiv, los ntawm qhov pom ntawm lub networker: ntev, tus lej sib lawv liag, thiab TSIS MUAJ ib qho uas yog MTProto, lub cev, CRC32. Yog, tag nrho cov no yog nyob saum TCP. Uas muab rau peb nrog txhim khu kev qha thauj nyob rau hauv daim ntawv ntawm ib tug sequential byte kwj; tsis muaj sequences yuav tsum tau, tshwj xeeb tshaj yog checksums. Okay, tam sim no ib tug neeg yuav tawm tsam kuv tias TCP muaj 16-ntsis checksum, yog li cov ntaub ntawv kev noj nyiaj txiag tshwm sim. Zoo heev, tab sis peb yeej muaj cov txheej txheem cryptographic nrog hashes ntev dua 16 bytes, tag nrho cov yuam kev no - thiab ntau dua - yuav raug ntes los ntawm SHA mismatch ntawm qib siab dua. Tsis muaj qhov taw tes hauv CRC32 saum toj no.

Cia peb piv Abridged, nyob rau hauv uas ib byte ntawm qhov ntev yog ua tau, nrog Intermediate, uas justifies "Yog hais tias 4-byte cov ntaub ntawv alignment yog xav tau," uas yog heev tsis muaj tseeb. Dab tsi, nws ntseeg tias Telegram programmers tsis muaj peev xwm nyeem tau cov ntaub ntawv los ntawm lub qhov (socket) rau hauv qhov tsis sib haum? Koj tseem yuav tau ua qhov no, vim tias kev nyeem ntawv tuaj yeem xa rov qab rau koj ib tus lej ntawm bytes (thiab tseem muaj npe servers, piv txwv li ...). Los yog ntawm qhov tod tes, yog vim li cas thaiv Abridged yog tias peb tseem yuav muaj ntau qhov padding saum 16 bytes - txuag 3 bytes qee zaus ?

Ib tug tau txais kev xav tias Nikolai Durov yeej nyiam rov tsim dua lub log, suav nrog cov txheej txheem network, tsis muaj qhov xav tau tiag tiag.

Lwm txoj kev thauj mus los, incl. Lub Vev Xaib thiab MTProxy, peb yuav tsis xav txog tam sim no, tej zaum hauv lwm tus ncej, yog tias muaj kev thov. Txog tib lub MTProxy no, cia peb tsuas yog nco ntsoov tam sim no tias tsis ntev tom qab nws tso tawm xyoo 2018, cov kws kho mob tau kawm sai sai los thaiv nws, npaj rau bypass thaivlos ntawm pob loj! Thiab tseem muaj qhov tseeb tias MTProxy server sau (dua los ntawm Waltman) hauv C tau dhau los khi rau Linux tshwj xeeb, txawm hais tias qhov no tsis tas yuav tsum tau ua (Phil Kulin yuav lees paub), thiab cov server zoo sib xws hauv Go lossis Node.js yuav. haum nyob rau hauv tsawg tshaj li ib puas kab.

Tab sis peb yuav txiav txim siab txog kev paub txog kev paub ntawm cov neeg no thaum kawg ntawm ntu, tom qab xav txog lwm yam teeb meem. Txog tam sim no, cia peb txav mus rau OSI txheej 5, kev sib kho - uas lawv tso MTProto kev sib kho.

Cov yuam sij, cov lus, ntu, Diffie-Hellman

Lawv muab tso rau qhov ntawd tsis yog tag nrho ... Ib qho kev sib kho tsis yog tib qhov kev sib tham uas pom nyob rau hauv lub interface hauv Active sessions. Tab sis nyob rau hauv kev txiav txim.

Yog li peb tau txais ib txoj hlua byte paub ntev los ntawm txheej thauj. Qhov no yog ib qho lus encrypted lossis plaintext - yog tias peb tseem nyob rau theem kev pom zoo tseem ceeb thiab tab tom ua tiag tiag. Lub ntsiab lus twg hu ua "qhov tseem ceeb" peb tham txog? Cia peb piav qhia qhov teeb meem no rau pab pawg Telegram nws tus kheej (Kuv thov txim rau kev txhais kuv cov ntaub ntawv los ntawm Askiv nrog lub hlwb nkees thaum 4 teev sawv ntxov, nws yooj yim dua los tso qee cov lus raws li lawv:

Muaj ob lub koom haum hu ua kev sib kho - ib qho hauv UI ntawm cov neeg siv khoom raug cai nyob rau hauv "kev sib tham tam sim no", qhov twg txhua qhov kev sib tham sib raug rau tag nrho cov cuab yeej / OS.
Thib ob - MTProto kev sib tham, uas muaj tus naj npawb ntawm cov lus (hauv kev nkag siab qis) hauv nws, thiab qhov twg Tej zaum yuav nyob ntawm qhov sib txawv TCP kev sib txuas. Ob peb MTProto ntu tuaj yeem ntsia tau rau tib lub sijhawm, piv txwv li, txhawm rau rub tawm cov ntaub ntawv sai.

Nruab nrab ntawm ob qho no zaug muaj tswvyim kev tso cai. Nyob rau hauv cov ntaub ntawv degenerate, peb tuaj yeem hais tias UI ntu yog tib yam li kev tso cai, tab sis alas, txhua yam yog qhov nyuaj. Cia peb saib:

Tus neeg siv ntawm lub cuab yeej tshiab ua ntej tsim auth_key thiab khi nws mus rau account, piv txwv li ntawm SMS - yog vim li cas kev tso cai

Nws tshwm sim hauv thawj MTProto kev sib tham, uas muaj session_id hauv koj tus kheej.

Hauv cov kauj ruam no, kev sib xyaw ua ke kev tso cai и session_id tuaj yeem hu tau txoj cai - Cov lus no tshwm sim hauv cov ntaub ntawv thiab cov cai ntawm qee tus neeg siv khoom

Tom qab ntawd, tus neeg siv khoom tuaj yeem qhib ntau MTProto ntu sub ib yam auth_key - rau tib DC.

Tom qab ntawd, muaj ib hnub tus neeg siv yuav tau thov cov ntaub ntawv los ntawm lwm dc - thiab rau qhov no DC ib qho tshiab yuav raug tsim tawm auth_key !

Qhia rau lub kaw lus tias nws tsis yog tus neeg siv tshiab uas tab tom sau npe, tab sis tib yam kev tso cai (UI ntu), tus neeg siv khoom siv API hu auth.exportAuthorization hauv DC auth.importAuthorization hauv DC tshiab.

Txhua yam yog tib yam, ob peb yuav qhib MTProto ntu (txhua tus nrog nws tus kheej session_id) rau qhov tshiab DC no, hauv qab nws auth_key.

Thaum kawg, tus neeg siv yuav xav tau Perfect Forward Secrecy. Txhua auth_key yog Ruaj khov tus yuam sij - ib DC - thiab tus neeg siv tuaj yeem hu auth.bindTempAuthKey rau siv ib ntus auth_key - thiab dua, tsuas yog ib qho temp_auth_key rau DC, ib txwm rau txhua tus MTProto ntu rau DC no.

ceeb toom, tias ntsev (thiab yav tom ntej ntsev) kuj yog ib qho ntawm auth_key cov. sib koom ntawm sawv daws MTProto ntu mus rau DC tib yam.

Qhov "nruab nrab ntawm TCP sib txuas" txhais li cas? Yog li qhov no txhais tau tias ib yam zoo li kev tso cai cov ncuav qab zib ntawm lub vev xaib - nws tseem muaj sia nyob (muaj sia nyob) ntau qhov kev sib txuas TCP rau lub server muab, tab sis muaj ib hnub nws mus tsis zoo. Tsuas yog tsis zoo li HTTP, hauv MTProto cov lus nyob rau hauv ib qho kev sib kho yog cov lej thiab paub tseeb; yog tias lawv nkag mus rau hauv qhov, qhov kev sib txuas tau tawg - tom qab tsim kev sib txuas tshiab, tus neeg rau zaub mov yuav ua siab zoo xa txhua yam hauv qhov kev sib kho no uas nws tsis tau xa hauv yav dhau los TCP kev twb kev txuas.

Txawm li cas los xij, cov ntaub ntawv saum toj no tau sau tseg tom qab ntau lub hlis ntawm kev tshawb nrhiav. Lub sijhawm no, peb puas siv peb cov neeg siv khoom los ntawm kos? - cia peb rov qab mus rau qhov pib.

Yog li cia peb tsim auth_key rau Diffie-Hellman versions los ntawm Telegram. Wb sim nkag siab cov ntaub ntawv...

Vasily, [19.06.18 20:05] data_with_hash := SHA1(data) + data + (ib yam random bytes); xws li qhov ntev sib npaug 255 bytes;
encrypted_data := RSA(data_with_hash, server_public_key); ib tug 255-byte ntev tus lej (loj endian) raug tsa mus rau lub hwj chim uas yuav tsum tau tshaj qhov yuav tsum tau ua, thiab cov txiaj ntsig tau muab khaws cia ua tus lej 256-byte.

Lawv muaj ib co dope DH

Tsis zoo li tus neeg noj qab haus huv DH
Tsis muaj ob tus yuam sij pej xeem hauv dx

Zoo, thaum kawg qhov no tau txheeb tawm, tab sis ib qho seem tseem tshuav - pov thawj ntawm kev ua haujlwm yog ua los ntawm tus neeg siv khoom uas nws muaj peev xwm ua tus lej. Hom kev tiv thaiv tawm tsam DoS. Thiab tus yuam sij RSA tsuas yog siv ib zaug hauv ib qho kev taw qhia, qhov tseem ceeb rau kev encryption new_nonce. Tab sis txawm tias qhov kev ua haujlwm yooj yim zoo li no yuav ua tiav, koj yuav tau ntsib dab tsi?

Vasily, [20.06.18/00/26 XNUMX:XNUMX] Kuv tseem tsis tau txais daim ntawv thov appid tseem

Kuv xa daim ntawv thov no mus rau DH

Thiab, hauv kev thauj khoom thauj nws hais tias nws tuaj yeem teb nrog 4 bytes ntawm tus lej yuam kev. Yog tag nrho

Zoo, nws hais rau kuv -404, yog li cas?

Yog li kuv tau hais rau nws: "Catch koj bullshit encrypted nrog tus yuam sij server nrog tus ntiv tes zoo li no, Kuv xav tau DH," thiab nws teb nrog 404 ruam.

Koj xav li cas txog cov lus teb server no? Yuav ua li cas? Tsis muaj leej twg nug (tab sis ntxiv rau qhov ntawd hauv ntu thib ob).

Ntawm no txhua qhov kev txaus siab yog ua tiav ntawm qhov chaw nres nkoj

Kuv tsis muaj dab tsi ntxiv ua, Kuv tsuas yog npau suav ntawm kev hloov cov lej rov qab thiab tawm mus

Ob tus lej 32 ntsis. Kuv ntim lawv ib yam li lwm tus

Tab sis tsis yog, ob qho no yuav tsum tau ntxiv rau kab ua ntej raws li BE

Vadim Goncharov, [20.06.18 15:49] thiab vim li no 404?

Vasily, [20.06.18 15:49] YOG!

Vadim Goncharov, [20.06.18 15:50] yog li kuv tsis nkag siab qhov nws tuaj yeem "tsis pom"

Vasily, [20.06.18 15:50] kwv yees li ntawm

Kuv tsis tuaj yeem pom qhov decomposition mus rau hauv cov ntsiab lus tseem ceeb%)

Peb tseem tsis tau tswj xyuas qhov yuam kev

Vasily, [20.06.18 20:18] Oh, tseem muaj MD5. Twb tau peb qhov sib txawv hashes

Tus ntiv tes tseem ceeb yog xam raws li hauv qab no:
digest = md5(key + iv)
fingerprint = substr(digest, 0, 4) XOR substr(digest, 4, 4)
SHA1 thiab 2

Yog li cia peb muab nws auth_key peb tau txais 2048 me me siv Diffie-Hellman. Tom ntej no yog dab tsi? Tom ntej no peb pom tias qhov qis dua 1024 ntsis ntawm tus yuam sij no tsis siv nyob rau hauv ib txoj kev ... tab sis cia peb xav txog qhov no rau tam sim no. Hauv cov kauj ruam no, peb muaj kev qhia zais cia nrog tus neeg rau zaub mov. Ib qho piv txwv ntawm TLS kev sib ntsib tau tsim, uas yog tus txheej txheem kim heev. Tab sis lub server tseem tsis paub dab tsi txog peb yog leej twg! Tsis tau, ua tau. kev tso cai. Cov. Yog tias koj xav txog "login-password", ib yam li koj tau ua hauv ICQ, lossis tsawg kawg "login-key", xws li hauv SSH (piv txwv li, ntawm qee qhov gitlab / github). Peb tau txais ib qho tsis qhia npe. Yuav ua li cas yog tus neeg rau zaub mov qhia peb "cov xov tooj no tau txais kev pabcuam los ntawm lwm tus DC"? Los yog txawm tias "koj tus lej xov tooj raug txwv"? Qhov zoo tshaj plaws uas peb tuaj yeem ua tau yog khaws tus yuam sij hauv kev cia siab tias nws yuav muaj txiaj ntsig thiab yuav tsis mus lwj thaum ntawd.

Los ntawm txoj kev, peb "tau txais" nws nrog kev tshwj tseg. Piv txwv li, peb puas ntseeg lub server? Yuav ua li cas yog nws yog fake? Cryptographic checks yuav xav tau:

Vasily, [21.06.18 17:53] Lawv muab cov neeg siv xov tooj ntawm tes los kuaj xyuas tus lej 2kbit rau primality%)

Tab sis nws tsis meej kiag li, nafeijoa

Vasily, [21.06.18 18:02] Cov ntaub ntawv tsis tau hais tias yuav ua li cas yog tias nws hloov mus ua tsis yooj yim

Tsis tau hais. Cia peb saib seb tus neeg siv khoom hauv Android ua li cas hauv qhov no? A yog li ntawd (thiab yog, tag nrho cov ntaub ntawv yog nthuav) - raws li lawv hais, kuv yuav cia li tawm ntawm no:

278     static const char *goodPrime = "c71caeb9c6b1c9048e6c522f70f13f73980d40238e3e21c14934d037563d930f48198a0aa7c14058229493d22530f4dbfa336f6e0ac925139543aed44cce7c3720fd51f69458705ac68cd4fe6b6b13abdc9746512969328454f18faf8c595f642477fe96bb2a941d5bcd1d4ac8cc49880708fa9b378e3c4f3a9060bee67cf9a4a4a695811051907e162753b56b0f6b410dba74d8a84b2a14b3144e0ef1284754fd17ed950d5965b4b9dd46582db1178d169c6bc465b0d6ff9ca3928fef5b9ae4e418fc15e83ebea0f87fa9ff5eed70050ded2849f47bf959d956850ce929851f0d8115f635b105ee2e4e15d04b2454bf6f4fadf034b10403119cd8e3b92fcc5b";
279   if (!strcasecmp(prime, goodPrime)) {

Tsis yog, tau kawg nws tseem nyob ntawd qee Muaj cov kev xeem rau qhov tseem ceeb ntawm tus lej, tab sis tus kheej kuv tsis muaj kev paub txog lej txaus lawm.

Okay, peb tau txais tus yuam sij master. Nkag mus rau hauv, i.e. xa cov lus thov, koj yuav tsum tau ua ntxiv encryption, siv AES.

Cov lus tseem ceeb yog txhais raws li 128 nruab nrab ntawm SHA256 ntawm lub cev cov lus (xws li kev sib tham, cov lus ID, thiab lwm yam), suav nrog cov padding bytes, prepended los ntawm 32 bytes coj los ntawm tus yuam sij tso cai.

Vasily, [22.06.18 14:08] Average, bitch, khoom

Tau txais auth_key. Tag nrho. Dhau ntawm lawv ... nws tsis meej ntawm cov ntaub ntawv. Xav tias dawb mus kawm qhov qhib qhov chaws.

Nco ntsoov tias MTProto 2.0 xav tau los ntawm 12 mus rau 1024 bytes ntawm padding, tseem raug rau cov xwm txheej uas qhov tshwm sim cov lus ntev yuav muab faib los ntawm 16 bytes.

Yog li ntau npaum li cas padding koj yuav tsum tau ntxiv?

Thiab yog, kuj tseem muaj 404 thaum muaj qhov yuam kev

Yog tias leej twg ua tib zoo kawm daim duab thiab cov ntawv ntawm cov ntaub ntawv, lawv pom tias tsis muaj MAC nyob ntawd. Thiab qhov ntawd AES siv nyob rau hauv qee yam IGE hom uas tsis siv nyob qhov twg. Lawv, tau kawg, sau txog qhov no hauv lawv cov FAQ... Ntawm no, zoo li, cov lus tseem ceeb nws tus kheej kuj yog SHA hash ntawm cov ntaub ntawv decrypted, siv los xyuas qhov ncaj ncees - thiab nyob rau hauv cov ntaub ntawv tsis sib haum, cov ntaub ntawv rau qee qhov laj thawj. pom zoo kom ntsiag to tsis quav ntsej lawv (tab sis yuav ua li cas txog kev ruaj ntseg, yog tias lawv ua txhaum peb?).

Kuv tsis yog cryptographer, tej zaum tsis muaj dab tsi tsis ncaj ncees lawm nrog hom no nyob rau hauv rooj plaub no los ntawm qhov kev xav ntawm kev xav. Tab sis kuv tuaj yeem hais meej lub npe qhov teeb meem, siv Telegram Desktop ua piv txwv. Nws encrypts lub zos cache (tag nrho cov no D877F783D5D3EF8C) ib yam li cov lus hauv MTProto (tsuas yog nyob rau hauv cov ntaub ntawv no version 1.0), i.e. thawj cov lus tseem ceeb, ces cov ntaub ntawv nws tus kheej (thiab qhov chaw ib sab ntawm lub ntsiab loj auth_key 256 bytes, tsis muaj qhov twg msg_key tsis siv). Yog li, qhov teeb meem dhau los ua pom ntawm cov ntaub ntawv loj. Namely, koj yuav tsum khaws ob daim ntawv theej ntawm cov ntaub ntawv - encrypted thiab decrypted. Thiab yog hais tias muaj megabytes, los yog streaming video, piv txwv li?.. Classic schemes nrog MAC tom qab lub ciphertext tso cai rau koj nyeem nws kwj, tam sim ntawd kis nws. Tab sis nrog MTProto koj yuav tau thaum xub thawj encrypt lossis decrypt tag nrho cov lus, tsuas yog tom qab ntawd hloov mus rau lub network lossis rau disk. Yog li ntawd, nyob rau hauv qhov tseeb versions ntawm Telegram Desktop nyob rau hauv lub cache nyob rau hauv user_data Lwm hom kuj siv - nrog AES hauv CTR hom.

Vasily, [21.06.18 01:27] Auj, Kuv pom tias IGE yog dab tsi: IGE yog thawj zaug sim ntawm "kev lees paub qhov encryption hom," keeb kwm rau Kerberos. Nws yog kev sim ua tsis tiav (nws tsis muaj kev tiv thaiv kev ncaj ncees), thiab yuav tsum tau muab tshem tawm. Qhov ntawd yog qhov pib ntawm 20 xyoo kev tshawb nrhiav rau kev txheeb xyuas qhov tseeb encryption hom uas ua haujlwm, uas nyuam qhuav ua tiav hauv hom xws li OCB thiab GCM.

Thiab tam sim no cov lus sib cav los ntawm lub laub sab:

Pab pawg tom qab Telegram, coj los ntawm Nikolai Durov, muaj rau ACM tus yeej, ib nrab ntawm lawv Ph.Ds hauv lej. Nws siv lawv li ob xyoos los dov tawm qhov tam sim no ntawm MTProto.

Uas funny. Ob xyoos nyob rau theem qis

Los yog koj tuaj yeem nqa tls

Zoo, cia peb hais tias peb tau ua qhov encryption thiab lwm yam nuances. Puas yog thaum kawg tuaj yeem xa cov lus thov serialized hauv TL thiab deserialize cov lus teb? Yog li dab tsi thiab koj yuav tsum xa li cas? Ntawm no, cia peb hais, txoj kev kev sib txuas, tej zaum qhov no yog nws?

Vasily, [25.06.18 18:46] Initializes kev sib txuas thiab txuag cov ntaub ntawv ntawm tus neeg siv lub cuab yeej thiab daim ntawv thov.

Nws txais app_id, device_model, system_version, app_version thiab lang_code.

Thiab qee cov lus nug

Cov ntaub ntawv ib txwm muaj. Xav tias dawb mus kawm qhov qhib qhov chaw

Yog tias txhua yam tau hais meej meej nrog invokeWithLayer, ces qhov no yog dab tsi? Nws hloov tawm, cia peb hais tias peb muaj - tus neeg siv khoom twb muaj ib yam dab tsi los nug tus neeg rau zaub mov txog - muaj kev thov uas peb xav xa:

Vasily, [25.06.18 19:13] Judging los ntawm cov cai, thawj hu yog qhwv nyob rau hauv no crap, thiab lub crap nws tus kheej yog qhwv nyob rau hauv invokewithlayer

Vim li cas ho tsis tuaj yeem initConnection yog ib qho kev hu sib cais, tab sis yuav tsum yog lub wrapper? Yog, raws li nws tau muab tawm, nws yuav tsum tau ua txhua zaus thaum pib ntawm txhua qhov kev sib tham, thiab tsis yog ib zaug, ib yam li tus yuam sij tseem ceeb. Tab sis! Nws tsis tuaj yeem hu los ntawm tus neeg siv tsis tau tso cai! Tam sim no peb tau mus txog theem uas nws siv tau Qhov no nplooj ntawv - thiab nws qhia peb tias ...

Tsuas yog ib feem me me ntawm API txoj kev muaj rau cov neeg siv tsis tau tso cai:

auth.sendCode
auth.resendCode
account.getPassword
auth.checkPassword
auth.checkPhone
auth.signUp
auth.signIn
auth.importAuthorization
help.getConfig
help.getNearestDc
help.getAppUpdate
help.getCdnConfig
langpack.getLangPack
langpack.getStrings
langpack.getDifference
langpack.getLanguages
langpack.getLanguage

Thawj thawj ntawm lawv, auth.sendCode, thiab muaj qhov cherished thawj qhov kev thov uas peb xa api_id thiab api_hash, thiab tom qab ntawd peb tau txais SMS nrog tus lej. Thiab yog tias peb nyob hauv DC tsis ncaj ncees lawm (cov xov tooj hauv lub tebchaws no tau txais kev pabcuam los ntawm lwm tus, piv txwv li), peb yuav tau txais qhov yuam kev nrog tus lej ntawm DC xav tau. Yog xav paub seb tus IP chaw nyob twg los ntawm DC tus lej koj xav tau txuas rau, pab peb help.getConfig. Nyob rau hauv ib lub sij hawm tsuas muaj 5 qhov nkag, tab sis tom qab cov xwm txheej nto moo ntawm 2018, tus lej tau nce ntau.

Tam sim no cia peb nco ntsoov tias peb tau mus rau theem no ntawm lub server tsis qhia npe. Tsis yog nws kim dhau los tsuas yog tau txais IP chaw nyob? Vim li cas ho tsis ua qhov no, thiab lwm yam kev ua haujlwm, hauv qhov tsis muaj ntaub ntawv ntawm MTProto? Kuv hnov qhov kev tsis pom zoo: "Peb yuav ua li cas thiaj paub tseeb tias nws tsis yog RKN leej twg yuav teb nrog qhov chaw tsis raug?" Rau qhov no peb nco ntsoov tias, feem ntau, cov neeg siv khoom raug cai RSA yuam sij yog embedded, i.e. tuaj yeem koj xwb subscribe cov ntaub ntawv no. Qhov tseeb, qhov no twb tau ua tiav rau cov ntaub ntawv ntawm kev hla kev thaiv cov neeg siv khoom tau txais los ntawm lwm cov channel (qhov tseeb, qhov no tsis tuaj yeem ua tiav hauv MTProto nws tus kheej; koj kuj yuav tsum paub qhov twg txuas).

OK. Nyob rau theem no ntawm kev tso cai cov neeg siv khoom, peb tseem tsis tau tso cai thiab tsis tau sau npe rau peb daim ntawv thov. Peb tsuas yog xav pom tam sim no dab tsi tus neeg rau zaub mov teb rau txoj hauv kev muaj rau tus neeg siv tsis tau tso cai. Thiab ntawm no…

Vasily, [10.07.18 14:45] https://core.telegram.org/method/help.getConfig
config#7dae33e0 [...] = Config;
help.getConfig#c4f9186b = Config;
https://core.telegram.org/api/datacenter
config#232d5905 [...] = Config;
help.getConfig#c4f9186b = Config;
Nyob rau hauv lub tswv yim, thawj los thib ob

Hauv tdesktop schema tus nqi thib peb yog

Yog lawm, txij thaum ntawd los, tau kawg, cov ntaub ntawv tau hloov kho. Txawm tias tsis ntev nws yuav dhau los ua tsis tseem ceeb dua. Yuav ua li cas ib tug novice developer paub? Tej zaum yog koj sau npe rau koj daim ntawv thov, lawv yuav qhia rau koj? Vasily tau ua qhov no, tab sis alas, lawv tsis xa nws dab tsi (dua, peb yuav tham txog qhov no hauv ntu thib ob).

...Koj pom tias peb twb tau tsiv mus rau API, piv txwv li. mus rau qib tom ntej, thiab nco ib yam dab tsi hauv MTProto lub ncauj lus? Tsis surprise:

Vasily, [28.06.18 02:04] Mm, lawv tab tom rummaging los ntawm qee qhov algorithms ntawm e2e

Mtproto txhais qhov encryption algorithms thiab cov yuam sij rau ob lub npe, nrog rau me ntsis ntawm cov qauv wrapper

Tab sis lawv txuas ntxiv sib txawv ntawm cov pawg, yog li nws tsis yog ib txwm paub meej qhov twg mtproto xaus thiab qib tom ntej pib

Lawv sib tov li cas? Zoo, ntawm no yog tib tus yuam sij rau PFS, piv txwv li (los ntawm txoj kev, Telegram Desktop tsis tuaj yeem ua nws). Nws raug tua los ntawm kev thov API auth.bindTempAuthKey, i.e. los ntawm theem sab saum toj. Tab sis tib lub sij hawm nws cuam tshuam nrog encryption ntawm qib qis - tom qab nws, piv txwv li, koj yuav tsum ua nws dua. initConnection etc., qhov no tsis yog xwb ib txwm thov. Dab tsi tseem ceeb yog tias koj tuaj yeem muaj IB tus yuam sij ib ntus rau DC, txawm tias thaj chaw auth_key_id hauv txhua cov lus tso cai rau koj hloov tus yuam sij tsawg kawg txhua cov lus, thiab cov neeg rau zaub mov muaj cai "tsis nco qab" tus yuam sij ib ntus txhua lub sijhawm - cov ntaub ntawv tsis tau hais tias yuav ua li cas rau qhov no ... zoo, vim li cas thiaj tuaj yeem ua tau. 'tsis koj muaj ob peb tus yuam sij, ib yam li cov txheej txheem ntsev yav tom ntej, thiab?..

Muaj ob peb lwm yam tsim nyog sau cia txog MTProto ntsiab lus.

Cov ntawv xov xwm, msg_id, msg_seqno, kev pom zoo, pings nyob rau hauv qhov tsis ncaj ncees lawm thiab lwm yam idiosyncrasies

Vim li cas koj yuav tsum paub txog lawv? Vim tias lawv "kua" mus rau qib siab dua, thiab koj yuav tsum paub txog lawv thaum ua haujlwm nrog API. Cia peb xav tias peb tsis nyiam msg_key; qib qis tau decrypted txhua yam rau peb. Tab sis sab hauv cov ntaub ntawv decrypted peb muaj cov nram qab no (kuj qhov ntev ntawm cov ntaub ntawv, yog li peb paub qhov twg padding yog, tab sis qhov tsis tseem ceeb):

ntsev - int64
session_id - int64
message_id — int64
seq_no - int32

Cia peb nco ntsoov tias tsuas muaj ib qho ntsev rau tag nrho DC. Vim li cas ho paub txog nws? Tsis yog vim muaj kev thov get_future_salts, uas qhia koj lub sijhawm twg yuav siv tau, tab sis kuj vim tias yog koj cov ntsev "roj", ces cov lus (thov) yuav ploj mus. Cov neeg rau zaub mov yuav, tau kawg, tshaj tawm cov ntsev tshiab los ntawm kev tshaj tawm new_session_created - tab sis nrog tus qub koj yuav tau rov xa nws li cas, piv txwv li. Thiab qhov teeb meem no cuam tshuam rau daim ntawv thov architecture.

Tus neeg rau zaub mov raug tso cai tso tseg kev sib tham tag nrho thiab teb rau hauv txoj kev no rau ntau yam. Qhov tseeb, qhov kev sib tham MTProto yog dab tsi los ntawm tus neeg siv khoom sab? Cov no yog ob tug lej session_id и seq_no cov lus hauv qhov kev sib tham no. Zoo, thiab hauv qab TCP kev twb kev txuas, tau kawg. Cia peb hais tias peb cov neeg siv khoom tseem tsis paub yuav ua li cas ntau yam, nws tau txiav thiab rov txuas dua. Yog tias qhov no tshwm sim sai sai - qhov kev sib tham qub txuas ntxiv hauv kev sib txuas TCP tshiab, nce ntxiv seq_no ntxiv. Yog tias nws siv sijhawm ntev, tus neeg rau zaub mov tuaj yeem tshem tawm, vim tias ntawm nws sab nws kuj yog kab, raws li peb pom.

Yuav tsum yog dab tsi seq_no? Auj, qhov no yog lo lus nug nyuaj. Sim ua ncaj ncees to taub dab tsi txhais tau tias:

Cov ntsiab lus hais txog lus

Cov lus xav tau kev lees paub meej. Cov no suav nrog tag nrho cov neeg siv thiab ntau cov lus pabcuam, zoo li txhua yam tshwj tsis yog cov thawv ntim khoom thiab kev lees paub.

Message Sequence Number (msg_seqno)

Tus lej 32-ntsis sib npaug li ob npaug ntawm tus lej ntawm "cov ntsiab lus ntsig txog" cov lus (cov uas xav tau kev lees paub, thiab tshwj xeeb tshaj yog cov uas tsis ntim khoom) tsim los ntawm tus neeg xa khoom ua ntej cov lus no thiab tom qab ntawd nce los ntawm ib qho yog tias cov lus tam sim no yog cov ntsiab lus hais txog cov lus. Lub thawv yog ib txwm tsim tom qab nws tag nrho cov ntsiab lus; yog li ntawd, nws tus lej ua ntu zus yog ntau dua lossis sib npaug ntawm cov lej ntawm cov lus uas muaj nyob hauv nws.

Dab tsi ntawm circus yog qhov no nrog qhov nce ntxiv los ntawm 1, thiab tom qab ntawd ntxiv los ntawm 2? ... Kuv xav tias thaum xub thawj lawv txhais tau tias "qhov tseem ceeb tshaj me ntsis rau ACK, tus so yog tus lej", tab sis qhov tshwm sim tsis zoo ib yam - tshwj xeeb, nws tawm los, tuaj yeem xa tuaj ntau kev pom zoo muaj tib yam seq_no! Yuav ua li cas? Zoo, piv txwv li, tus neeg rau zaub mov xa peb ib yam dab tsi, xa nws, thiab peb tus kheej nyob twj ywm, tsuas yog teb nrog cov lus pabcuam lees paub qhov tau txais nws cov lus. Hauv qhov no, peb qhov kev lees paub tawm yuav muaj tib tus lej tawm. Yog tias koj paub TCP thiab xav tias qhov no suab zoo li qus, tab sis nws zoo li tsis qus heev, vim tias hauv TCP seq_no tsis hloov, tab sis kev pom zoo mus rau seq_no ntawm lwm sab, kuv yuav maj nroos ua rau koj chim. Kev lees paub muaj nyob hauv MTProto TSIS rau seq_no, zoo li hauv TCP, tab sis los ntawm msg_id !

Qhov no yog dab tsi msg_id, qhov tseem ceeb tshaj ntawm cov teb no? Ib qho kev qhia tshwj xeeb, raws li lub npe qhia. Nws txhais tau tias yog tus lej 64-ntsis, qhov qis tshaj ntawm cov khoom uas rov muaj qhov "server-not-server" khawv koob, thiab tus so yog Unix timestamp, suav nrog cov feem feem, hloov 32 khoom mus rau sab laug. Cov. timestamp per se (thiab cov lus nrog lub sijhawm sib txawv ntau dhau yuav raug tsis lees paub los ntawm tus neeg rau zaub mov). Los ntawm qhov no nws hloov tawm tias feem ntau qhov no yog tus cim uas yog thoob ntiaj teb rau cov neeg siv khoom. Muab qhov ntawd - cia peb nco ntsoov session_id - Peb tau lav: Tsis muaj ib qho xwm txheej twg tuaj yeem xa cov lus rau ib qho kev sib tham. Ntawd yog, nws hloov tawm tias muaj lawm peb theem - kev sib kho, ntu ntu, xov id. Yog vim li cas thiaj li overcomplication, qhov no paub tsis meej yog zoo heev.

Thiab yog li ntawd, msg_id xav tau rau ...

RPC: thov, teb, yuam kev. Kev lees paub.

Raws li koj tuaj yeem pom, tsis muaj qhov tshwj xeeb "ua RPC thov" hom lossis ua haujlwm nyob qhov twg hauv daim duab, txawm tias muaj cov lus teb. Tom qab tag nrho, peb muaj cov ntsiab lus hais txog cov lus! Ntawd yog, twg cov lus yuav yog ib qho kev thov! Los tsis yog. Tom qab tag nrho, ntawm txhua tus yog msg_id. Tab sis muaj cov lus teb:

rpc_result#f35c6d01 req_msg_id:long result:Object = RpcResult;

Qhov no yog qhov uas nws tau qhia tias cov lus no yog cov lus teb rau. Yog li ntawd, nyob rau theem sab saum toj ntawm API, koj yuav tsum nco ntsoov tias tus lej ntawm koj qhov kev thov yog dab tsi - Kuv xav tias tsis muaj qhov yuav tsum tau piav qhia tias kev ua haujlwm yog asynchronous, thiab tuaj yeem muaj ntau qhov kev thov nyob rau tib lub sijhawm, cov lus teb rau qhov twg tuaj yeem xa rov qab rau hauv ib qho kev txiav txim? Raws li txoj cai, los ntawm qhov no thiab cov lus yuam kev zoo li tsis muaj cov neeg ua haujlwm, cov qauv hauv qab no tuaj yeem taug qab: tus neeg rau zaub mov uas tuav TCP kev sib txuas nrog koj yog tus sib npaug pem hauv ntej, nws xa mus thov rau cov backends thiab sau lawv rov qab los ntawm message_id. Nws zoo nkaus li tias txhua yam ntawm no yog qhov tseeb, muaj laj thawj thiab zoo.

Yog?.. Thiab yog koj xav li cas? Tom qab tag nrho, RPC teb nws tus kheej kuj muaj ib thaj chaw msg_id! Peb puas yuav tsum tau qw ntawm lub server "koj tsis teb kuv cov lus teb!"? Thiab yog, muaj dab tsi txog kev lees paub? Txog nplooj ntawv lus hais txog cov lus qhia peb tias yog dab tsi

msgs_ack#62d6b459 msg_ids:Vector long = MsgsAck;

thiab nws yuav tsum tau ua los ntawm txhua sab. Tab sis tsis yog ib txwm! Yog tias koj tau txais RpcResult, nws tus kheej ua qhov kev lees paub. Ntawd yog, tus neeg rau zaub mov tuaj yeem teb koj qhov kev thov nrog MsgsAck - zoo li, "Kuv tau txais nws." RpcResult tuaj yeem teb tam sim ntawd. Nws tuaj yeem yog ob qho tib si.

Thiab yog, koj tseem yuav tau teb cov lus teb! Kev lees paub. Txwv tsis pub, tus neeg rau zaub mov yuav xav tias nws tsis tuaj yeem xa thiab xa rov qab rau koj dua. Txawm tias tom qab reconnection. Tab sis ntawm no, tau kawg, qhov teeb meem ntawm lub sijhawm tshwm sim. Cia wb mus saib lawv me ntsis tom qab.

Lub sijhawm no, cia peb saib cov lus nug ua yuam kev.

rpc_error#2144ca19 error_code:int error_message:string = RpcError;

Auj, ib tug neeg yuav qw, ntawm no yog ib tug ntau humane hom - muaj ib txoj kab! Siv koj lub sijhawm. Ntawm no npe yuam kev, tab sis ntawm chav kawm tsis tiav. Los ntawm nws peb kawm tau hais tias lub code yog ib yam zoo li HTTP yuam kev (zoo, tau kawg, cov ntsiab lus ntawm cov lus teb tsis raug hwm, nyob rau qee qhov chaw lawv tau faib random ntawm cov lej), thiab kab zoo li CAPITAL_LETTERS_AND_NUMBERS. Piv txwv li, PHONE_NUMBER_OCCUPIED lossis FILE_PART_Х_MISSING. Zoo, uas yog, koj tseem yuav xav tau kab no parseCov. Piv txwv FLOOD_WAIT_3600 yuav txhais tau tias koj yuav tsum tau tos ib teev, thiab PHONE_MIGRATE_5, hais tias ib tug xov tooj nrog rau qhov prefix no yuav tsum tau sau npe nyob rau hauv lub 5th DC. Peb muaj hom lus, txoj cai? Peb tsis tas yuav muaj kev sib cav los ntawm txoj hlua, cov tsis tu ncua yuav ua tau, tsis yog.

Ib zaug ntxiv, qhov no tsis yog nyob rau nplooj ntawv pabcuam, tab sis, raws li twb tau ua nrog rau qhov project no, cov ntaub ntawv tuaj yeem pom ntawm lwm nplooj ntawv cov ntaub ntawv. Los yog pov tseg. Ua ntej, saib, ntaus ntawv / txheej ua txhaum cai - RpcError tuaj yeem ua zes hauv RpcResult. Vim li cas ho tsis nyob sab nraud? Dab tsi yog peb tsis coj mus rau hauv tus account?... Raws li, qhov twg yog qhov lav ntawd RpcError tej zaum yuav TSIS embedded RpcResult, tab sis ncaj qha los yog zes nyob rau hauv lwm hom? .. Thiab yog tias nws tsis tuaj yeem, vim li cas ho tsis nyob rau theem sab saum toj, i.e. nws ploj lawm req_msg_id ? ..

Tab sis cia peb txuas ntxiv txog kev pabcuam lus. Tus neeg siv yuav xav tias tus neeg rau zaub mov xav tau ntev thiab ua qhov kev thov zoo no:

rpc_drop_answer#58e4a740 req_msg_id:long = RpcDropAnswer;

Muaj peb lo lus teb rau lo lus nug no, rov sib tshuam nrog cov txheej txheem kev lees paub; sim nkag siab tias lawv yuav tsum yog dab tsi (thiab daim ntawv teev npe dav dav uas tsis tas yuav tsum tau lees paub) raug tso rau tus nyeem ntawv ua homework (ceeb toom: cov ntaub ntawv hauv Telegram Desktop qhov chaws tsis tiav).

Kev quav yeeb tshuaj: xov xwm xwm txheej

Feem ntau, ntau qhov chaw hauv TL, MTProto thiab Telegram feem ntau tawm ntawm kev xav tawv ncauj, tab sis tawm ntawm kev coj zoo, kev coj cwj pwm thiab lwm yam. kev txawj ntse Peb ua tib zoo nyob ntsiag to txog nws, thiab censored cov kev phem nyob rau hauv lub dialogues. Txawm li cas los xij, qhov chaw noОfeem ntau ntawm nplooj ntawv yog hais txog lus hais txog cov lus Nws tseem ceeb heev rau kuv, uas tau ua haujlwm nrog cov txheej txheem network tau ntev thiab tau pom cov tsheb kauj vab sib txawv ntawm qhov sib txawv.

Nws pib innocuously, nrog kev pom zoo. Tom ntej no lawv qhia peb txog

bad_msg_notification#a7eff811 bad_msg_id:long bad_msg_seqno:int error_code:int = BadMsgNotification;
bad_server_salt#edab447b bad_msg_id:long bad_msg_seqno:int error_code:int new_server_salt:long = BadMsgNotification;

Zoo, txhua tus neeg uas pib ua hauj lwm nrog MTProto yuav tsum tau nrog lawv; nyob rau hauv lub "kho - recompiled - launched" lub voj voog, tau txais cov lej yuam kev lossis ntsev uas tau tswj tsis zoo thaum lub sij hawm hloov kho yog ib qho tshwm sim. Txawm li cas los xij, muaj ob lub ntsiab lus ntawm no:

Qhov no txhais tau tias cov lus qub ploj lawm. Peb yuav tsum tsim qee cov kab, peb yuav saib tom qab ntawd.
Cov lej yuam kev txawv no yog dab tsi? 16, 17, 18, 19, 20, 32, 33, 34, 35, 48, 64... nyob qhov twg lwm tus lej, Tommy?

Cov ntaub ntawv hais tias:

Lub hom phiaj yog tias error_code qhov tseem ceeb yog pawg (error_code >> 4): piv txwv li, cov lis dej num 0x40 - 0x4f sib raug rau qhov yuam kev hauv lub thawv decomposition.

tab sis, thawj zaug, hloov mus rau lwm qhov kev taw qhia, thiab qhov thib ob, nws tsis muaj teeb meem, lwm cov lej nyob qhov twg? Nyob rau hauv tus sau lub taub hau?... Txawm li cas los, cov no yog trifles.

Kev quav yeeb tshuaj pib hauv cov lus hais txog cov xwm txheej thiab cov ntawv luam:

Thov kom Message Status Information
Yog tias ob tog tsis tau txais cov ntaub ntawv ntawm cov xwm txheej ntawm nws cov lus tawm mus ib ntus, nws tuaj yeem thov nws los ntawm lwm tus neeg sab nraud:
msgs_state_req#da69fb52 msg_ids:Vector long = MsgsStateReq;
Xov xwm qhia txog xwm txheej ntawm cov lus
msgs_state_info#04deb57d req_msg_id:long info:string = MsgsStateInfo;
Ntawm no, info yog ib txoj hlua uas muaj raws nraim ib byte ntawm cov xwm txheej rau txhua cov lus los ntawm cov npe msg_ids tuaj:
- 1 = tsis muaj dab tsi paub txog cov lus (msg_id tsawg dhau, lwm tus neeg yuav tsis nco qab nws)
- 2 = cov lus tsis tau txais (msg_id ntog nyob rau hauv qhov ntau ntawm cov ntaub ntawv khaws cia; txawm li cas los xij, lwm tus neeg tsis tau txais cov lus zoo li ntawd)
- 3 = cov lus tsis tau txais (msg_id siab dhau lawm; txawm li cas los xij, lwm tus neeg tseem tsis tau txais nws)
- 4 = cov lus tau txais (nco ntsoov tias cov lus teb no tseem nyob rau tib lub sijhawm lees paub lees paub)
- +8 = lus twb lees paub lawm
- +16 = cov lus tsis xav tau kev lees paub
- +32 = RPC cov lus nug uas muaj nyob rau hauv cov lus tau ua tiav lossis ua tiav lawm
- +64 = cov ntsiab lus hais txog cov lus teb rau cov lus twb tau tsim lawm
- +128 = Lwm tus neeg paub txog qhov tseeb tias cov lus twb tau txais lawm
  Cov lus teb no tsis xav tau kev lees paub. Nws yog kev lees paub ntawm qhov cuam tshuam msgs_state_req, hauv thiab ntawm nws tus kheej.
  Nco ntsoov tias yog tias nws hloov tawm tam sim ntawd tias lwm tog tsis muaj cov lus uas zoo li tau xa mus rau nws, cov lus tuaj yeem xa rov qab tau yooj yim. Txawm hais tias lwm tus neeg yuav tsum tau txais ob daim ntawv luam ntawm cov lus tib lub sijhawm, qhov sib npaug yuav raug tsis quav ntsej. (Yog tias lub sijhawm dhau mus, thiab tus thawj msg_id tsis siv tau lawm, cov lus yuav tsum muab qhwv rau hauv msg_copy).
Kev yeem sib txuas lus ntawm cov xwm txheej ntawm cov lus
Ib tog twg tuaj yeem yeem qhia rau lwm tus ntawm cov xwm txheej ntawm cov lus xa los ntawm lwm tus neeg.
msgs_all_info#8cc0d131 msg_ids:Vector long info:string = MsgsAllInfo
Txuas Ntxiv Kev Sib Txuas Lus ntawm Cov Txheej Txheem ntawm Ib Cov Lus
...
msg_detailed_info#276d3ec6 msg_id:long answer_msg_id:long bytes:int status:int = MsgDetailedInfo;
msg_new_detailed_info#809db6df answer_msg_id:long bytes:int status:int = MsgDetailedInfo;
Qhia meej Thov kom rov xa cov lus
msg_resend_req#7d861a08 msg_ids:Vector long = MsgResendReq;
Cov neeg nyob deb nroog tam sim teb los ntawm rov xa cov lus thov […]
Thov kom meej meej kom rov xa cov lus teb
msg_resend_ans_req#8610baeb msg_ids:Vector long = MsgResendReq;
Lub tog neeg nyob deb teb tam sim ntawd los ntawm kev xa rov qab cov lus teb rau cov lus thov […]
Cov ntawv luam
Hauv qee qhov xwm txheej, cov lus qub nrog msg_id uas tsis siv tau lawm yuav tsum tau xa rov qab. Tom qab ntawd, nws yog qhwv rau hauv ib lub thawv ntawv:
msg_copy#e06046b2 orig_message:Message = MessageCopy;
Thaum tau txais, cov lus tau ua tiav zoo li lub wrapper tsis nyob ntawd. Txawm li cas los xij, yog tias nws paub tseeb tias cov lus orig_message.msg_id tau txais, ces cov lus tshiab tsis ua tiav (thaum tib lub sijhawm, nws thiab orig_message.msg_id tau lees paub). Tus nqi ntawm orig_message.msg_id yuav tsum qis dua lub thawv msg_id.

Cia peb txawm nyob ntsiag to txog dab tsi msgs_state_info dua lub pob ntseg ntawm qhov tsis tiav TL tab tom tawm (peb xav tau ib qho vector ntawm bytes, thiab nyob rau hauv qis dua ob lub khoom muaj ib qho enum, thiab nyob rau hauv siab dua ob ntsis muaj chij). Lub ntsiab lus txawv. Puas muaj leej twg nkag siab tias vim li cas txhua qhov no nyob hauv kev xyaum? hauv tus neeg siv khoom tiag tsim nyog?.. Nrog nyuaj, tab sis ib tug yuav xav txog tej yam txiaj ntsim yog hais tias ib tug neeg yog koom nyob rau hauv debugging, thiab nyob rau hauv ib tug sib tham sib hom - nug tus neeg rau zaub mov dab tsi thiab yuav ua li cas. Tab sis ntawm no cov lus thov tau piav qhia ncig ncig.

Nws ua raws li hais tias txhua tog yuav tsum tsis tsuas yog encrypt thiab xa cov lus, tab sis kuj khaws cov ntaub ntawv hais txog lawv tus kheej, hais txog cov lus teb rau lawv, rau ib tug tsis paub lub sij hawm. Cov ntaub ntawv tsis tau piav qhia txog lub sijhawm lossis kev siv tau ntawm cov yam ntxwv no. tsis muaj txoj kev. Dab tsi yog qhov amazing tshaj plaws yog tias lawv tau siv tiag tiag hauv cov cai ntawm cov neeg siv khoom! Pom tau tias lawv tau hais ib yam dab tsi uas tsis suav nrog hauv cov ntaub ntawv pej xeem. Nkag siab los ntawm cov cai vim li cas, yog tsis yooj yim li nyob rau hauv cov ntaub ntawv ntawm TL - nws tsis yog ib tug (kuj) logically cais ib feem, tab sis ib daim khi rau daim ntawv thov architecture, i.e. yuav xav tau sijhawm ntau ntxiv kom nkag siab txog daim ntawv thov code.

Pings thiab sijhawm. Cov npe.

Los ntawm txhua yam, yog tias peb nco qab qhov kev kwv yees txog tus neeg rau zaub mov architecture (kev faib tawm ntawm kev thov thoob plaws backends), ib qho kev tu siab heev tom qab - txawm tias tag nrho cov khoom xa tuaj hauv TCP (xws li cov ntaub ntawv xa tuaj, lossis koj yuav raug ceeb toom txog qhov sib txawv, tab sis. cov ntaub ntawv yuav raug xa ua ntej qhov teeb meem tshwm sim), qhov kev lees paub hauv MTProto nws tus kheej - tsis muaj guarantees. Cov neeg rau zaub mov tuaj yeem poob lossis pov tawm koj cov lus tau yooj yim, thiab tsis muaj dab tsi tuaj yeem ua tau txog nws, tsuas yog siv ntau hom ntoo khaub lig.

Thiab ua ntej ntawm tag nrho cov - lus queues. Zoo, nrog ib yam dab tsi txhua yam tau pom tseeb txij thaum pib - cov lus tsis lees paub yuav tsum tau muab khaws cia thiab tsis txaus siab. Thiab tom qab lub sijhawm twg? Thiab tus jester paub nws. Tej zaum cov kev pabcuam kev quav yeeb quav tshuaj daws qhov teeb meem no nrog cov khoom siv, hais tias, hauv Telegram Desktop muaj txog 4 cov kab sib txuas rau lawv (tej zaum ntau dua, raws li tau hais tseg, rau qhov no koj yuav tsum delve rau hauv nws cov cai thiab architecture ntau dua; tib yam. lub sij hawm, Peb paub tias nws tsis tuaj yeem coj los ua qauv; qee yam ntawm cov hom los ntawm MTProto scheme tsis siv rau hauv nws).

Vim li cas qhov no tshwm sim? Tej zaum, tus neeg rau zaub mov programmers tsis tuaj yeem ua kom muaj kev ntseeg siab nyob rau hauv pawg, lossis txawm tias buffering nyob rau hauv pem hauv ntej balancer, thiab pauv qhov teeb meem no rau tus neeg siv khoom. Tawm ntawm kev poob siab, Vasily sim siv lwm txoj kev xaiv, tsuas yog ob kab, siv algorithms los ntawm TCP - ntsuas RTT rau tus neeg rau zaub mov thiab kho qhov loj ntawm "qhov rais" (hauv cov lus) nyob ntawm seb muaj pes tsawg tus thov tsis lees paub. Ntawd yog, xws li qhov hnyav hnyav rau kev ntsuas tus neeg rau zaub mov thauj khoom yog pes tsawg ntawm peb qhov kev thov nws tuaj yeem zom tib lub sijhawm thiab tsis poob.

Zoo, uas yog, koj nkag siab, txoj cai? Yog tias koj yuav tsum siv TCP dua nyob rau sab saum toj ntawm cov txheej txheem ua haujlwm dhau TCP, qhov no qhia txog kev tsim qauv tsis zoo.

Yog lawm, vim li cas koj thiaj xav tau ntau tshaj ib kab, thiab qhov no txhais li cas rau tus neeg ua haujlwm nrog API qib siab? Saib, koj thov, serialize nws, tab sis feem ntau koj tsis tuaj yeem xa nws tam sim ntawd. Vim li cas? Vim cov lus teb yuav yog msg_id, uas yog ib ntusаKuv yog daim ntawv lo, txoj haujlwm uas yog qhov zoo tshaj plaws ncua mus txog lig li sai tau - nyob rau hauv rooj plaub uas tus neeg rau zaub mov tsis lees txais nws vim yog lub sijhawm tsis sib haum xeeb ntawm peb thiab nws (qhov tseeb, peb tuaj yeem ua tus ntoo khaub lig uas hloov peb lub sijhawm los ntawm tam sim no. rau tus neeg rau zaub mov los ntawm kev ntxiv ib qho delta xam los ntawm cov neeg rau zaub mov cov lus teb - cov neeg siv khoom ua qhov no, tab sis nws yog crude thiab tsis raug vim buffering). Yog li ntawd, thaum koj thov nrog ib lub zos muaj nuj nqi hu los ntawm lub tsev qiv ntawv, cov lus mus rau cov theem hauv qab no:

Nws nyob rau hauv ib kab thiab tos encryption.
Teem caij msg_id thiab cov lus mus rau lwm txoj kab - tau xa mus; xa mus rau lub qhov (socket).
a) Tus neeg rau zaub mov teb MsgsAck - cov lus tau xa, peb tshem tawm ntawm "lwm kab".
b) Los yog vice versa, nws tsis nyiam ib yam dab tsi, nws teb badmsg - resend los ntawm "lwm kab"
c) Tsis muaj dab tsi paub, cov lus yuav tsum tau resent los ntawm lwm kab - tab sis nws tsis paub meej tias thaum twg.
Thaum kawg tus neeg rau zaub mov teb RpcResult - qhov tseeb teb (lossis yuam kev) - tsis yog xa tawm xwb, tab sis kuj ua tiav.

Tej zaum, kev siv cov thawv tuaj yeem daws qhov teeb meem ib nrab. Qhov no yog thaum ib pawg ntawm cov lus tau ntim rau hauv ib qho, thiab cov neeg rau zaub mov teb nrog kev lees paub rau lawv txhua tus ib zaug, hauv ib qho. msg_id. Tab sis nws kuj tseem yuav tsis lees txais cov pob no, yog tias muaj qee yam yuam kev, tag nrho.

Thiab nyob rau ntawm no cov kev txiav txim siab uas tsis yog kev xav tau los ua si. Los ntawm cov kev paub dhau los, peb tau pom ntau lub ru tsev, thiab ntxiv rau, tam sim no peb yuav pom ntau yam piv txwv ntawm cov lus qhia tsis zoo thiab kev tsim vaj tsev - nyob rau hauv cov xwm txheej zoo li no, puas tsim nyog tso siab rau thiab txiav txim siab zoo li no? Lo lus nug yog rhetorical (tawm tsis yog).

Peb tham txog dab tsi? Yog hais tias nyob rau hauv lub ntsiab lus ntawm "tshuaj yeeb tshuaj txog cov lus" koj tseem tuaj yeem kwv yees nrog kev tawm tsam xws li "koj ruam, koj tsis nkag siab peb txoj kev npaj ci ntsa iab!" (yog li sau cov ntaub ntawv ua ntej, raws li cov neeg ib txwm yuav tsum, nrog cov laj thawj thiab cov piv txwv ntawm kev sib pauv pob khoom, peb mam li tham), tom qab ntawd lub sijhawm / sijhawm yog cov lus nug uas siv tau thiab tshwj xeeb, txhua yam ntawm no tau paub ntev. Cov ntaub ntawv qhia li cas rau peb txog lub sijhawm?

Ib tus neeg rau zaub mov feem ntau lees paub qhov tau txais cov lus los ntawm tus neeg siv khoom (feem ntau, RPC lus nug) siv RPC cov lus teb. Yog tias cov lus teb tau ntev los, tus neeg rau zaub mov yuav xub xa daim ntawv lees paub, thiab qee zaum tom qab, RPC teb nws tus kheej.

Tus neeg siv khoom ib txwm lees paub qhov tau txais cov lus los ntawm tus neeg rau zaub mov (feem ntau, RPC cov lus teb) los ntawm kev ntxiv kev lees paub rau cov lus nug RPC tom ntej yog tias nws tsis tau xa tuaj lig (yog tias nws tsim, hais, 60-120 vib nas this tom qab tau txais ntawm cov lus los ntawm server). Txawm li cas los xij, yog tias lub sijhawm ntev tsis muaj laj thawj xa cov lus rau lub server lossis yog tias muaj ntau cov lus tsis lees paub los ntawm tus neeg rau zaub mov (hais, tshaj 16), tus neeg siv khoom xa tawm ib leeg lees paub.

... Kuv txhais: peb tus kheej tsis paub ntau npaum li cas thiab peb xav tau npaum li cas, yog li cia peb xav tias cia nws zoo li no.

Thiab hais txog pings:

Ping lus (PING / PONG)
ping#7abe77ec ping_id:long = Pong;
Ib qho lus teb feem ntau rov qab mus rau tib qhov kev sib txuas:
pong#347773c5 msg_id:long ping_id:long = Pong;
Cov lus no tsis xav tau kev lees paub. Lub pong tsuas yog kis tau los ntawm kev teb rau ping thaum lub ping tuaj yeem pib los ntawm ob sab.

Deferred Connection Kaw + PING
ping_delay_disconnect#f3427b8c ping_id:long disconnect_delay:int = Pong;
Ua haujlwm zoo li ping. Tsis tas li ntawd, tom qab qhov no tau txais, tus neeg rau zaub mov pib lub timer uas yuav kaw qhov kev sib txuas tam sim no disconnect_delay vib nas this tom qab tshwj tsis yog nws tau txais cov lus tshiab ntawm tib hom uas tau rov pib dua tag nrho cov timers dhau los. Yog tias tus neeg siv xa cov pings no ib zaug txhua 60 vib nas this, piv txwv li, nws yuav teem caij disconnect_delay sib npaug li 75 vib nas this.

Koj puas vwm?! Hauv 60 vib nas this, lub tsheb ciav hlau yuav nkag mus rau qhov chaw nres tsheb, tso tawm thiab tuaj tos cov neeg caij tsheb, thiab rov plam kev sib cuag hauv qhov av. Hauv 120 vib nas this, thaum koj hnov nws, nws yuav tuaj txog ntawm lwm tus, thiab kev sib txuas feem ntau yuav tawg. Zoo, nws yog qhov tseeb tias ob txhais ceg tuaj ntawm - "Kuv hnov ib lub suab nrov, tab sis tsis paub tias nws nyob qhov twg", muaj Nagl's algorithm thiab TCP_NODELAY kev xaiv, npaj rau kev sib tham ua haujlwm. Tab sis, zam txim rau kuv, tuav nws tus nqi pib - 200 Millivib nas this Yog tias koj xav piav qhia qee yam zoo sib xws thiab txuag ntawm ob peb lub pob ntawv, tom qab ntawd muab tso rau 5 vib nas this, lossis txawm li cas los xij "Tus neeg siv tau ntaus ntawv ..." lus timeout yog tam sim no. Tab sis tsis muaj ntxiv lawm.

Thiab thaum kawg, pings. Ntawd yog, tshawb xyuas lub neej ntawm TCP kev sib txuas. Nws lom zem, tab sis txog 10 xyoo dhau los kuv tau sau cov ntawv tseem ceeb hais txog tus xa xov ntawm peb cov kws qhia ntawv lub tsev - cov kws sau ntawv kuj tau pinged server los ntawm tus neeg siv khoom, thiab tsis ua rau lwm tus. Tab sis cov tub ntxhais kawm xyoo 3 yog ib yam, thiab lub chaw haujlwm thoob ntiaj teb yog lwm qhov, puas yog?..

Ua ntej, me ntsis kev kawm. Kev sib txuas TCP, thaum tsis muaj pob ntawv pauv, tuaj yeem nyob tau ntau lub lis piam. Qhov no yog qhov zoo thiab qhov tsis zoo, nyob ntawm lub hom phiaj. Nws yog qhov zoo yog tias koj muaj kev sib txuas SSH qhib rau lub server, koj tau sawv ntawm lub computer, rov pib lub router, rov qab mus rau koj qhov chaw - qhov kev sib kho los ntawm lub server no tsis raug torn (koj tsis tau ntaus dab tsi, tsis muaj pob ntawv) , nws yooj yim. Nws tsis zoo yog tias muaj ntau txhiab tus neeg siv khoom ntawm tus neeg rau zaub mov, txhua tus tau txais cov peev txheej (nyob zoo, Postgres!), thiab tus tswv tsev yuav tau rov pib dua ntev dhau los - tab sis peb yuav tsis paub txog nws.

Chat / IM systems poob rau hauv rooj plaub thib ob rau ib qho laj thawj ntxiv - cov xwm txheej hauv online. Yog hais tias tus neeg siv "poob", koj yuav tsum qhia nws interlocutors txog qhov no. Txwv tsis pub, koj yuav xaus nrog qhov yuam kev uas tus tsim ntawm Jabber tau ua (thiab kho rau 20 xyoo) - tus neeg siv tau txiav tawm, tab sis lawv tseem sau cov lus rau nws, ntseeg tias nws nyob hauv online (uas kuj tau ploj tag nrho hauv cov no. ob peb feeb ua ntej qhov kev txiav tawm tau pom). Tsis yog, qhov kev xaiv TCP_KEEPALIVE, uas ntau tus neeg tsis nkag siab yuav ua li cas TCP timers ua haujlwm cuam tshuam (los ntawm kev teeb tsa cov txiaj ntsig zoo li kaum vib nas this), yuav tsis pab ntawm no - koj yuav tsum paub tseeb tias tsis yog OS kernel nkaus xwb. ntawm tus neeg siv lub tshuab tseem ciaj sia, tab sis kuj ua haujlwm ib txwm, muaj peev xwm teb tau, thiab daim ntawv thov nws tus kheej (koj puas xav tias nws tsis tuaj yeem khov? Telegram Desktop ntawm Ubuntu 18.04 froze rau kuv ntau dua ib zaug).

Yog vim li cas koj yuav tsum ping neeg rau zaub mov tus neeg siv khoom, thiab tsis rov qab - yog tias tus neeg siv khoom ua qhov no, yog tias qhov kev sib txuas tau tawg, lub ping yuav tsis raug xa tawm, lub hom phiaj yuav tsis ua tiav.

Peb pom dab tsi hauv Telegram? Nws yog qhov opposite! Zoo, uas yog. Raws li txoj cai, tau kawg, ob tog tuaj yeem ping ib leeg. Hauv kev xyaum, cov neeg siv khoom siv lub cuab yeej ping_delay_disconnect, uas teeb lub timer ntawm lub server. Zoo, thov zam txim rau kuv, nws tsis yog nyob ntawm tus neeg siv khoom txiav txim siab ntev npaum li cas nws xav nyob ntawd yam tsis muaj ping. Lub server, raws li nws cov load, paub zoo dua. Tab sis, tau kawg, yog tias koj tsis mloog cov peev txheej, ces koj yuav yog koj tus kheej phem Pinocchio, thiab tus ntoo khaub lig yuav ua ...

Yuav tsum tau tsim li cas?

Kuv ntseeg tias qhov tseeb saum toj no qhia meej tias pab pawg Telegram / VKontakte tsis muaj peev xwm ua tau zoo hauv kev thauj mus los (thiab qis dua) qib ntawm cov khoos phis tawj thiab lawv cov kev tsim nyog tsawg hauv cov teeb meem cuam tshuam.

Vim li cas nws thiaj li nyuaj heev, thiab Telegram architects tuaj yeem sim tawm tsam li cas? Qhov tseeb tias lawv tau sim ua qhov kev sib tham uas muaj sia nyob TCP kev sib txuas tawg, piv txwv li, yam tsis tau xa tam sim no, peb yuav xa tom qab. Tej zaum lawv kuj tau sim ua UDP thauj, tab sis lawv ntsib teeb meem thiab tso tseg (yog vim li cas cov ntaub ntawv tsis muaj dab tsi - tsis muaj dab tsi los khav txog). Tab sis vim tsis muaj kev nkag siab txog yuav ua li cas kev sib txuas lus dav dav thiab TCP hauv kev ua haujlwm tshwj xeeb, qhov twg koj tuaj yeem tso siab rau nws, thiab qhov twg koj yuav tsum tau ua koj tus kheej (thiab yuav ua li cas), thiab kev sim ua ke qhov no nrog cryptography "ob tug noog nrog ib pob zeb”, qhov no yog qhov tshwm sim.

Nws tsim nyog li cas? Raws li qhov tseeb tias msg_id yog ib lub sij hawm tsim nyog los ntawm qhov kev pom cryptographic los tiv thaiv kev tawm tsam rov qab, nws yog qhov yuam kev los txuas lub cim tshwj xeeb rau nws. Yog li ntawd, tsis muaj kev hloov pauv ntawm cov qauv tam sim no (thaum Kev Hloov Kho kwj yog tsim, uas yog qib API qib siab rau lwm qhov ntawm cov ntawv tshaj tawm no), ib tus yuav xav tau:

Tus neeg rau zaub mov tuav TCP txuas rau tus neeg siv khoom siv lub luag haujlwm - yog tias nws tau nyeem los ntawm lub qhov (socket), thov lees paub, txheej txheem lossis xa rov qab qhov yuam kev, tsis muaj kev poob. Tom qab ntawd qhov kev lees paub tsis yog vector ntawm ids, tab sis tsuas yog "qhov kawg tau txais seq_no" - tsuas yog tus lej, xws li hauv TCP (ob tus lej - koj seq thiab tus lej lees paub). Peb ib txwm nyob hauv kev sib tham, puas yog?
Lub sij hawm los tiv thaiv replay tawm tsam dhau los ua ib qho chaw sib cais, ib qho tsis muaj. Nws raug kuaj, tab sis tsis cuam tshuam lwm yam. Txaus thiab uint32 - yog tias peb cov ntsev hloov tsawg kawg ib nrab ib hnub, peb tuaj yeem faib 16 khoom rau cov khoom qis ntawm ib feem ntawm lub sijhawm tam sim no, tus so - mus rau ib feem ntawm ib qho thib ob (raws li tam sim no).
Tshem tawm msg_id ntawm txhua qhov - los ntawm qhov pom ntawm qhov txawv qhov kev thov ntawm cov backends, muaj, thawj zaug, tus neeg siv khoom id, thiab qhov thib ob, qhov kev sib kho id, sib txuas rau lawv. Raws li, tsuas yog ib yam yog txaus raws li tus neeg thov kev thov seq_no.

Qhov no kuj tsis yog qhov kev xaiv ua tau zoo tshaj plaws; qhov ua tiav random tuaj yeem ua tus cim - qhov no tau ua tiav hauv qib siab API thaum xa lus, los ntawm txoj kev. Nws yuav zoo dua kom ua tiav cov qauv tsim los ntawm cov txheeb ze mus rau qhov tseeb, tab sis qhov no yog lub ntsiab lus rau lwm qhov, tsis yog qhov no.

API?

Taum! Yog li, tau tawm tsam los ntawm txoj hauv kev uas muaj kev mob thiab lub cuab yeej, thaum kawg peb tuaj yeem xa cov lus thov mus rau tus neeg rau zaub mov thiab tau txais cov lus teb rau lawv, nrog rau tau txais kev hloov tshiab los ntawm cov neeg rau zaub mov (tsis teb rau qhov kev thov, tab sis nws tus kheej. xa peb, zoo li PUSH, yog tias leej twg nws pom tseeb dua li ntawd).

Nco ntsoov, tam sim no yuav muaj tib qho piv txwv hauv Perl hauv kab lus! (rau cov neeg tsis paub txog cov syntax, thawj qhov kev sib cav ntawm koob hmoov yog cov khoom siv cov ntaub ntawv, qhov thib ob yog nws chav kawm):

2019.10.24 12:00:51 $1 = {
'cb' => 'TeleUpd::__ANON__',
'out' => bless( {
'filter' => bless( {}, 'Telegram::ChannelMessagesFilterEmpty' ),
'channel' => bless( {
'access_hash' => '-6698103710539760874',
'channel_id' => '1380524958'
}, 'Telegram::InputPeerChannel' ),
'pts' => '158503',
'flags' => 0,
'limit' => 0
}, 'Telegram::Updates::GetChannelDifference' ),
'req_id' => '6751291954012037292'
};
2019.10.24 12:00:51 $1 = {
'in' => bless( {
'req_msg_id' => '6751291954012037292',
'result' => bless( {
'pts' => 158508,
'flags' => 3,
'final' => 1,
'new_messages' => [],
'users' => [],
'chats' => [
bless( {
'title' => 'Хулиномика',
'username' => 'hoolinomics',
'flags' => 8288,
'id' => 1380524958,
'access_hash' => '-6698103710539760874',
'broadcast' => 1,
'version' => 0,
'photo' => bless( {
'photo_small' => bless( {
'volume_id' => 246933270,
'file_reference' => '
'secret' => '1854156056801727328',
'local_id' => 228648,
'dc_id' => 2
}, 'Telegram::FileLocation' ),
'photo_big' => bless( {
'dc_id' => 2,
'local_id' => 228650,
'file_reference' => '
'secret' => '1275570353387113110',
'volume_id' => 246933270
}, 'Telegram::FileLocation' )
}, 'Telegram::ChatPhoto' ),
'date' => 1531221081
}, 'Telegram::Channel' )
],
'timeout' => 300,
'other_updates' => [
bless( {
'pts_count' => 0,
'message' => bless( {
'post' => 1,
'id' => 852,
'flags' => 50368,
'views' => 8013,
'entities' => [
bless( {
'length' => 20,
'offset' => 0
}, 'Telegram::MessageEntityBold' ),
bless( {
'length' => 18,
'offset' => 480,
'url' => 'https://alexeymarkov.livejournal.com/[url_вырезан].html'
}, 'Telegram::MessageEntityTextUrl' )
],
'reply_markup' => bless( {
'rows' => [
bless( {
'buttons' => [
bless( {
'text' => '???? 165',
'data' => 'send_reaction_0'
}, 'Telegram::KeyboardButtonCallback' ),
bless( {
'data' => 'send_reaction_1',
'text' => '???? 9'
}, 'Telegram::KeyboardButtonCallback' )
]
}, 'Telegram::KeyboardButtonRow' )
]
}, 'Telegram::ReplyInlineMarkup' ),
'message' => 'А вот и новая книга! 
// [текст сообщения вырезан чтоб не нарушать правил Хабра о рекламе]
напечатаю.',
'to_id' => bless( {
'channel_id' => 1380524958
}, 'Telegram::PeerChannel' ),
'date' => 1571724559,
'edit_date' => 1571907562
}, 'Telegram::Message' ),
'pts' => 158508
}, 'Telegram::UpdateEditChannelMessage' ),
bless( {
'pts' => 158508,
'message' => bless( {
'edit_date' => 1571907589,
'to_id' => bless( {
'channel_id' => 1380524958
}, 'Telegram::PeerChannel' ),
'date' => 1571807301,
'message' => 'Почему Вы считаете Facebook плохой компанией? Можете прокомментировать? По-моему, это шикарная компания. Без долгов, с хорошей прибылью, а если решат дивы платить, то и еще могут нехило подорожать.
Для меня ответ совершенно очевиден: потому что Facebook делает ужасный по качеству продукт. Да, у него монопольное положение и да, им пользуется огромное количество людей. Но мир не стоит на месте. Когда-то владельцам Нокии было смешно от первого Айфона. Они думали, что лучше Нокии ничего быть не может и она навсегда останется самым удобным, красивым и твёрдым телефоном - и доля рынка это красноречиво демонстрировала. Теперь им не смешно.
Конечно, рептилоиды сопротивляются напору молодых гениев: так Цукербергом был пожран Whatsapp, потом Instagram. Но всё им не пожрать, Паша Дуров не продаётся!
Так будет и с Фейсбуком. Нельзя всё время делать говно. Кто-то когда-то сделает хороший продукт, куда всё и уйдут.
#соцсети #facebook #акции #рептилоиды',
'reply_markup' => bless( {
'rows' => [
bless( {
'buttons' => [
bless( {
'data' => 'send_reaction_0',
'text' => '???? 452'
}, 'Telegram::KeyboardButtonCallback' ),
bless( {
'text' => '???? 21',
'data' => 'send_reaction_1'
}, 'Telegram::KeyboardButtonCallback' )
]
}, 'Telegram::KeyboardButtonRow' )
]
}, 'Telegram::ReplyInlineMarkup' ),
'entities' => [
bless( {
'length' => 199,
'offset' => 0
}, 'Telegram::MessageEntityBold' ),
bless( {
'length' => 8,
'offset' => 919
}, 'Telegram::MessageEntityHashtag' ),
bless( {
'offset' => 928,
'length' => 9
}, 'Telegram::MessageEntityHashtag' ),
bless( {
'length' => 6,
'offset' => 938
}, 'Telegram::MessageEntityHashtag' ),
bless( {
'length' => 11,
'offset' => 945
}, 'Telegram::MessageEntityHashtag' )
],
'views' => 6964,
'flags' => 50368,
'id' => 854,
'post' => 1
}, 'Telegram::Message' ),
'pts_count' => 0
}, 'Telegram::UpdateEditChannelMessage' ),
bless( {
'message' => bless( {
'reply_markup' => bless( {
'rows' => [
bless( {
'buttons' => [
bless( {
'data' => 'send_reaction_0',
'text' => '???? 213'
}, 'Telegram::KeyboardButtonCallback' ),
bless( {
'data' => 'send_reaction_1',
'text' => '???? 8'
}, 'Telegram::KeyboardButtonCallback' )
]
}, 'Telegram::KeyboardButtonRow' )
]
}, 'Telegram::ReplyInlineMarkup' ),
'views' => 2940,
'entities' => [
bless( {
'length' => 609,
'offset' => 348
}, 'Telegram::MessageEntityItalic' )
],
'flags' => 50368,
'post' => 1,
'id' => 857,
'edit_date' => 1571907636,
'date' => 1571902479,
'to_id' => bless( {
'channel_id' => 1380524958
}, 'Telegram::PeerChannel' ),
'message' => 'Пост про 1С вызвал бурную полемику. Человек 10 (видимо, 1с-программистов) единодушно написали:
// [текст сообщения вырезан чтоб не нарушать правил Хабра о рекламе]
Я бы добавил, что блестящая у 1С дистрибуция, а маркетинг... ну, такое.'
}, 'Telegram::Message' ),
'pts_count' => 0,
'pts' => 158508
}, 'Telegram::UpdateEditChannelMessage' ),
bless( {
'pts' => 158508,
'pts_count' => 0,
'message' => bless( {
'message' => 'Здравствуйте, расскажите, пожалуйста, чем вредит экономике 1С?
// [текст сообщения вырезан чтоб не нарушать правил Хабра о рекламе]
#софт #it #экономика',
'edit_date' => 1571907650,
'date' => 1571893707,
'to_id' => bless( {
'channel_id' => 1380524958
}, 'Telegram::PeerChannel' ),
'flags' => 50368,
'post' => 1,
'id' => 856,
'reply_markup' => bless( {
'rows' => [
bless( {
'buttons' => [
bless( {
'data' => 'send_reaction_0',
'text' => '???? 360'
}, 'Telegram::KeyboardButtonCallback' ),
bless( {
'data' => 'send_reaction_1',
'text' => '???? 32'
}, 'Telegram::KeyboardButtonCallback' )
]
}, 'Telegram::KeyboardButtonRow' )
]
}, 'Telegram::ReplyInlineMarkup' ),
'views' => 4416,
'entities' => [
bless( {
'offset' => 0,
'length' => 64
}, 'Telegram::MessageEntityBold' ),
bless( {
'offset' => 1551,
'length' => 5
}, 'Telegram::MessageEntityHashtag' ),
bless( {
'length' => 3,
'offset' => 1557
}, 'Telegram::MessageEntityHashtag' ),
bless( {
'offset' => 1561,
'length' => 10
}, 'Telegram::MessageEntityHashtag' )
]
}, 'Telegram::Message' )
}, 'Telegram::UpdateEditChannelMessage' )
]
}, 'Telegram::Updates::ChannelDifference' )
}, 'MTProto::RpcResult' )
};
2019.10.24 12:00:51 $1 = {
'in' => bless( {
'update' => bless( {
'user_id' => 2507460,
'status' => bless( {
'was_online' => 1571907651
}, 'Telegram::UserStatusOffline' )
}, 'Telegram::UpdateUserStatus' ),
'date' => 1571907650
}, 'Telegram::UpdateShort' )
};
2019.10.24 12:05:46 $1 = {
'in' => bless( {
'chats' => [],
'date' => 1571907946,
'seq' => 0,
'updates' => [
bless( {
'max_id' => 141719,
'channel_id' => 1295963795
}, 'Telegram::UpdateReadChannelInbox' )
],
'users' => []
}, 'Telegram::Updates' )
};
2019.10.24 13:01:23 $1 = {
'in' => bless( {
'server_salt' => '4914425622822907323',
'unique_id' => '5297282355827493819',
'first_msg_id' => '6751307555044380692'
}, 'MTProto::NewSessionCreated' )
};
2019.10.24 13:24:21 $1 = {
'in' => bless( {
'chats' => [
bless( {
'username' => 'freebsd_ru',
'version' => 0,
'flags' => 5440,
'title' => 'freebsd_ru',
'min' => 1,
'photo' => bless( {
'photo_small' => bless( {
'local_id' => 328733,
'volume_id' => 235140688,
'dc_id' => 2,
'file_reference' => '
'secret' => '4426006807282303416'
}, 'Telegram::FileLocation' ),
'photo_big' => bless( {
'dc_id' => 2,
'file_reference' => '
'volume_id' => 235140688,
'local_id' => 328735,
'secret' => '71251192991540083'
}, 'Telegram::FileLocation' )
}, 'Telegram::ChatPhoto' ),
'date' => 1461248502,
'id' => 1038300508,
'democracy' => 1,
'megagroup' => 1
}, 'Telegram::Channel' )
],
'users' => [
bless( {
'last_name' => 'Panov',
'flags' => 1048646,
'min' => 1,
'id' => 82234609,
'status' => bless( {}, 'Telegram::UserStatusRecently' ),
'first_name' => 'Dima'
}, 'Telegram::User' )
],
'seq' => 0,
'date' => 1571912647,
'updates' => [
bless( {
'pts' => 137596,
'message' => bless( {
'flags' => 256,
'message' => 'Создать джейл с именем покороче ??',
'to_id' => bless( {
'channel_id' => 1038300508
}, 'Telegram::PeerChannel' ),
'id' => 119634,
'date' => 1571912647,
'from_id' => 82234609
}, 'Telegram::Message' ),
'pts_count' => 1
}, 'Telegram::UpdateNewChannelMessage' )
]
}, 'Telegram::Updates' )
};

Yog, tsis yog ib tug spoiler ntawm lub hom phiaj - yog tias koj tseem tsis tau nyeem nws, mus tom ntej thiab ua nws!

Oh, wai ~ ... qhov no zoo li cas? Ib yam dab tsi paub zoo ... tej zaum qhov no yog cov qauv ntaub ntawv ntawm ib qho Web API nyob rau hauv JSON, tshwj tsis yog cov chav kawm tseem txuas nrog cov khoom?..

Yog li qhov no yog yuav ua li cas nws tig tawm ... Nws yog dab tsi, comrades?... Kev siv zog ntau heev - thiab peb tau nres so qhov twg Web programmers nyuam qhuav pib?...Yuav tsis yog JSON dhau HTTPS yooj yim dua?! Peb tau txais dab tsi hauv kev pauv? Qhov kev siv zog puas tsim nyog?

Cia peb ntsuas seb TL + MTProto tau muab rau peb li cas thiab lwm txoj hauv kev tuaj yeem ua tau. Zoo, HTTP, uas tsom mus rau cov qauv kev thov teb, yog qhov tsis zoo, tab sis tsawg kawg yog qee yam nyob rau saum TLS?

Compact serialization. Pom cov qauv ntaub ntawv no, zoo ib yam li JSON, kuv nco ntsoov tias muaj binary versions ntawm nws. Cia peb cim MsgPack li tsis txaus txuas ntxiv, tab sis muaj, piv txwv li, CBOR - los ntawm txoj kev, tus qauv piav qhia hauv RFC 7049. Nws yog notable rau lub fact tias nws txhais cim npe, raws li ib tug expansion mechanism, thiab ntawm twb standardized muaj:

25 + 256 - hloov cov kab rov ua dua nrog kev siv rau kab xov tooj, xws li pheej yig compression txoj kev
26 - serialized Perl khoom nrog rau chav kawm lub npe thiab constructor sib cav
27 - serialized hom lus-yooj yim yam khoom nrog hom npe thiab tus tsim cov lus sib cav

Zoo, kuv tau sim serialize tib cov ntaub ntawv hauv TL thiab hauv CBOR nrog txoj hlua thiab cov khoom ntim tau qhib. Cov txiaj ntsig tau pib sib txawv hauv kev pom zoo ntawm CBOR qhov chaw ntawm ib megabyte:

cborlen=1039673 tl_len=1095092

Thiab yog li ntawd, xaus: Muaj ntau hom ntawv yooj yim uas tsis raug rau cov teeb meem ntawm synchronization tsis ua hauj lwm lossis tsis paub tus cim, nrog rau kev sib piv.

Kev sib txuas ceev ceev. Qhov no txhais tau tias xoom RTT tom qab kev sib txuas (thaum tus yuam sij twb tau tsim ib zaug) - siv tau los ntawm thawj MTProto cov lus, tab sis nrog qee qhov tshwj xeeb - ntaus tib ntsev, qhov kev sib kho tsis lwj, thiab lwm yam. TLS muab dab tsi rau peb? Quote ntawm lub ntsiab lus:

Thaum siv PFS hauv TLS, TLS daim pib pib (RFC 5077) kom rov pib qhov kev sib tham encrypted yam tsis muaj kev sib tham nrog cov yuam sij thiab tsis khaws cov ntaub ntawv tseem ceeb ntawm lub server. Thaum qhib thawj qhov kev sib txuas thiab tsim cov yuam sij, tus neeg rau zaub mov encrypts lub xeev kev sib txuas thiab xa mus rau tus neeg siv khoom (nyob rau hauv daim ntawv ntawm daim pib sib tham). Raws li, thaum qhov kev sib txuas tau rov pib dua, tus neeg siv yuav xa daim pib sib tham, suav nrog tus yuam sij kev sib kho, rov qab mus rau server. Daim pib nws tus kheej yog encrypted nrog ib ntus tus yuam sij (session ticket key), uas yog muab khaws cia rau ntawm tus neeg rau zaub mov thiab yuav tsum tau muab faib rau ntawm tag nrho cov frontend servers ua SSL nyob rau hauv pawg daws teeb meem.[10]. Yog li, kev taw qhia ntawm daim pib kev sib tham yuav ua txhaum PFS yog tias cov yuam sij server ib ntus raug cuam tshuam, piv txwv li, thaum lawv khaws cia ntev (OpenSSL, nginx, Apache cia lawv los ntawm lub neej ntawd rau tag nrho lub sijhawm ntawm qhov kev zov me nyuam; cov chaw nrov siv. tus yuam sij rau ob peb teev, mus txog hnub).

Ntawm no RTT tsis yog xoom, koj yuav tsum tau pauv yam tsawg kawg ClientHello thiab ServerHello, tom qab uas tus neeg siv tuaj yeem xa cov ntaub ntawv nrog rau Kev Ua tiav. Tab sis ntawm no peb yuav tsum nco ntsoov tias peb tsis muaj lub Vev Xaib, nrog nws pawg ntawm kev sib txuas tshiab, tab sis tus tub txib, kev sib txuas uas feem ntau yog ib qho thiab ntau dua lossis tsawg dua nyob ntev, luv luv thov rau cov nplooj ntawv Web - txhua yam yog multiplexed. sab hauv. Ntawd yog, nws tau txais txiaj ntsig zoo yog tias peb tsis tuaj hla qhov chaw subway tsis zoo.

Tsis nco qab lwm yam? Sau rau hauv cov lus.

Yuav tsum tau txuas ntxiv!

Hauv qhov thib ob ntawm cov ntawv tshaj tawm no peb yuav txiav txim siab tsis yog kev tshaj lij, tab sis cov teeb meem hauv lub koom haum - txoj hauv kev, kev xav, kev sib tham, tus cwj pwm ntawm cov neeg siv, thiab lwm yam. Raws li, txawm li cas los xij, ntawm cov ntaub ntawv kev tshaj lij uas tau nthuav tawm ntawm no.

Qhov thib peb yuav txuas ntxiv mus txheeb xyuas cov khoom siv / kev tsim kho. Koj yuav kawm, tshwj xeeb tshaj yog:

txuas ntxiv ntawm pandemonium nrog ntau hom TL
yam tsis paub txog channel thiab supergroups
yog vim li cas dialogs phem tshaj roster
hais txog absolute vs txheeb ze message addressing
Dab tsi yog qhov txawv ntawm daim duab thiab duab
yuav ua li cas emoji cuam tshuam nrog cov ntawv italic

thiab lwm yam crutches! Nyob twj ywm!

Tau qhov twg los: www.hab.com

Kev thuam ntawm cov txheej txheem thiab kev koom ua ke ntawm Telegram. Part 1, technical: kev sau ntawv tus neeg siv khoom los ntawm kos - TL, MT

Cov ntaub ntawv ... nws muaj, txoj cai? Puas muaj tseeb?..

Yuav pib qhov twg los ua tus sau ntawv hluas?

Binary serialization: TL (Hom Lus) thiab nws cov tswv yim, thiab txheej, thiab ntau lwm yam lus txaus ntshai

Cov lus piav qhia luv luv ntawm ib pawg ntawm TL syntax rau cov neeg uas tsis nyeem cov ntaub ntawv raug cai

Vector, uas tsis tau tso tawm

Kev siv TL uas twb muaj lawm hauv code

Tsis muaj hom hauv hom lus

Versioning. Txheej

MT Proto

Thawv txheej

Cov yuam sij, cov lus, ntu, Diffie-Hellman

Cov ntawv xov xwm, msg_id, msg_seqno, kev pom zoo, pings nyob rau hauv qhov tsis ncaj ncees lawm thiab lwm yam idiosyncrasies

RPC: thov, teb, yuam kev. Kev lees paub.

Kev quav yeeb tshuaj: xov xwm xwm txheej

Pings thiab sijhawm. Cov npe.

Yuav tsum tau tsim li cas?

API?

Yuav tsum tau txuas ntxiv!

Ntxiv ib saib Отменить ответ