Microsoft qhib lub tsev qiv ntawv vector nrhiav siv hauv Bing

Microsoft lub tuam txhab luam tawm machine learning library source codes SPTAG (Space Partition Tree And Graph) nrog rau kev siv cov approximate algorithm nrhiav neeg nyob ze. Lub tsev qiv ntawv tsim tawm nyob rau hauv kev tshawb fawb division ntawm Microsoft Kev Tshawb Fawb thiab kev tshawb fawb thev naus laus zis kev loj hlob chaw (Microsoft Search Technology Center). Hauv kev xyaum, SPTAG yog siv los ntawm Bing tshawb fawb cav los txiav txim siab cov txiaj ntsig zoo tshaj plaws raws li cov ntsiab lus ntawm cov lus nug tshawb fawb. Cov cai tau sau hauv C ++ thiab faib los ntawm nyob rau hauv MIT daim ntawv tso cai. Tsim rau Linux thiab Windows tau txais kev txhawb nqa. Muaj kev khi rau cov lus Python.

Txawm tias muaj tseeb hais tias lub tswv yim ntawm kev siv vector cia nyob rau hauv kev tshawb fawb xyaw tau ntab nyob ib ncig ntawm lub sij hawm ntev, nyob rau hauv kev xyaum, lawv cov kev siv yog hampered los ntawm cov kev pab cuam siab siv ntawm kev ua hauj lwm nrog vectors thiab scalability txwv. Kev sib xyaw ua ke ntawm kev kawm tshuab sib sib zog nqus nrog cov kwv yees ze ze tshaj plaws hauv zej zog tshawb nrhiav algorithms tau ua rau nws muaj peev xwm coj tau qhov kev ua tau zoo thiab kev ua kom muaj peev xwm ntawm vector systems mus rau qib uas siv tau rau cov tshuab tshawb fawb loj. Piv txwv li, nyob rau hauv Bing, rau ib tug vector Performance index ntawm ntau tshaj 150 billion vectors, lub sij hawm mus nqa cov txiaj ntsig zoo tshaj plaws yog nyob rau hauv 8 ms.

Lub tsev qiv ntawv suav nrog cov cuab yeej los tsim qhov Performance index thiab teeb tsa kev tshawb nrhiav vector, nrog rau cov txheej txheem rau kev tswj xyuas cov kev tshawb fawb hauv online uas suav nrog kev sau ntau ntawm vectors. Muab los cov nram qab no modules: index builder rau indexing, searcher rau kev tshawb fawb siv ib tug Performance index faib nyob rau hauv ib pawg ntawm ob peb nodes, neeg rau zaub mov rau khiav handlers ntawm nodes, Aggregator rau combining ob peb servers rau hauv ib tug, thiab cov neeg rau xa queries. Kev suav nrog cov vectors tshiab rau hauv qhov ntsuas thiab tshem tawm cov vectors ntawm ya tau txhawb nqa.

Lub tsev qiv ntawv txhais tau hais tias cov ntaub ntawv ua tiav thiab nthuav tawm hauv kev sau yog formatted nyob rau hauv daim ntawv ntawm cov vectors uas tuaj yeem muab piv los ntawm Euclidean (L2) lub cosine nyob deb Cov lus nug tshawb xyuas rov qab vectors uas nws qhov deb nruab nrab ntawm lawv thiab thawj vector yog tsawg. SPTAG muab ob txoj hauv kev los txhim kho qhov chaw vector: SPTAG-KDT (K-dimensional ntoo (kd cog) thiab txheeb ze ze daim duab) thiab SPTAG-BKT (k- txhais tau tias tsob ntoo (k- txhais tau tias tsob ntoo thiab cov duab txheeb ze ze). Thawj txoj kev yuav tsum muaj peev txheej tsawg dua thaum ua haujlwm nrog qhov ntsuas, thiab qhov thib ob qhia qhov tseeb dua ntawm cov txiaj ntsig kev tshawb fawb rau kev sau ntau ntawm vectors.

Nyob rau tib lub sijhawm, kev tshawb nrhiav vector tsis txwv rau cov ntawv nyeem thiab tuaj yeem siv rau cov ntaub ntawv xov xwm thiab cov duab, nrog rau hauv cov tshuab rau kev tsim cov lus pom zoo. Piv txwv li, ib qho ntawm cov qauv siv raws li PyTorch lub moj khaum siv lub vector system rau kev tshawb nrhiav raws li qhov sib xws ntawm cov khoom hauv cov duab, tsim siv cov ntaub ntawv los ntawm ntau cov ntaub ntawv sau nrog cov duab ntawm cov tsiaj, miv thiab dev, uas tau hloov mus rau hauv pawg vectors. . Thaum cov duab nkag tau txais rau kev tshawb fawb, nws tau hloov dua siab tshiab siv lub tshuab kev kawm qauv rau hauv vectors, raws li cov vectors zoo sib xws tau xaiv los ntawm qhov ntsuas siv SPTAG algorithm thiab cov duab cuam tshuam tau rov qab los ntawm qhov tshwm sim.

Tau qhov twg los: opennet.ru