Ua wehe ʻo Microsoft i ka waihona hulina vector i hoʻohana ʻia ma Bing

Microsoft paʻi ʻia nā code kumu waihona aʻo mīkini SPTAG (Space Partition Tree And Graph) me ka hoʻokō ʻana i ka algorithm pili huli hoalauna kokoke. Hale Waihona Puke kūkulu ʻia ma ka mahele noiʻi o Microsoft Research a me ke kikowaena hoʻomohala ʻenehana huli (Microsoft Search Technology Center). Ma ka hoʻomaʻamaʻa, hoʻohana ʻia ʻo SPTAG e ka ʻenekini ʻimi Bing no ka hoʻoholo ʻana i nā hopena kūpono loa e pili ana i ka pōʻaiapili o nā nīnau hulina. Ua kākau ʻia ke code ma C ++ a mahele ʻia e ma lalo o ka laikini MIT. Kākoʻo ʻia ke kūkulu ʻana no Linux a me Windows. Aia kahi paʻa no ka ʻōlelo Python.

ʻOiai ʻo ka manaʻo o ka hoʻohana ʻana i ka mālama ʻana i nā vector i nā ʻenekini ʻimi e lana ana a puni no ka manawa lōʻihi, i ka hoʻomaʻamaʻa ʻana, ua hoʻopilikia ʻia kā lākou hoʻokō ʻana e ka nui o ka waiwai o nā hana me nā vectors a me nā palena scalability. ʻO ka hoʻohui ʻana i nā ʻano aʻo mīkini hohonu me nā algorithm hulina pili kokoke loa i hiki ke lawe i ka hana a me ka scalability o nā ʻōnaehana vector i kahi pae e ʻae ʻia no nā ʻenekini huli nui. No ka laʻana, ma Bing, no kahi helu vector o ka 150 biliona vectors, ʻo ka manawa e kiʻi ai i nā hopena kūpono loa i loko o 8 ms.

Aia i loko o ka waihona nā mea hana no ke kūkulu ʻana i kahi papa kuhikuhi a me ka hoʻonohonoho ʻana i nā huli vector, a me kahi pūʻulu o nā mea hana no ka mālama ʻana i kahi ʻōnaehana huli pūnaewele puʻupuʻu e uhi ana i nā hōʻiliʻili nui loa o nā vectors. Hāʻawi ʻia ʻO nā modula aʻe: ka mea hana kuhikuhi no ka helu ʻana, ka mea ʻimi no ka ʻimi ʻana me ka hoʻohana ʻana i kahi kuhikuhi i puʻunaue ʻia i loko o kahi pūʻulu o nā nodes, server no ka holo ʻana i nā mea lawelawe ma nā nodes, Aggregator no ka hoʻohui ʻana i kekahi mau kikowaena i hoʻokahi, a me ka mea kūʻai no ka hoʻouna ʻana i nā nīnau. Kākoʻo ʻia ka hoʻokomo ʻana i nā vectors hou i ka index a me ka holoi ʻana i nā vectors ma ka lele.

Hōʻike ka waihona i ka ʻikepili i hana ʻia a hōʻike ʻia i ka hōʻiliʻili ʻana i ke ʻano o nā vectors pili i hiki ke hoʻohālikelike ʻia ma muli o Euclidean (L2) a i ʻole cosine mamao Hoʻihoʻi ka hulina hulina i nā vector nona ka mamao ma waena o lākou a me ka vector kumu ka liʻiliʻi. Hāʻawi ʻo SPTAG i ʻelua mau ala no ka hoʻonohonoho ʻana i kahi vector space: SPTAG-KDT (K-dimensional tree (lāʻau kd) a kiʻi pili pili) a me SPTAG-BKT (k-means tree (k-mea laau a me ka pakuhi pili pili). ʻO ke ala mua e koi ai i nā kumuwaiwai liʻiliʻi i ka wā e hana ai me ka index, a ʻo ka lua e hōʻike i ka pololei o nā hopena hulina no nā hōʻiliʻili nui loa o nā vectors.

Ma ka manawa like, ʻaʻole i kaupalena ʻia ka hulina vector i ka kikokikona a hiki ke hoʻohana ʻia i ka ʻike multimedia a me nā kiʻi, a me nā ʻōnaehana no ka hana ʻana i nā manaʻo. No ka laʻana, hoʻokahi o nā prototypes e pili ana i ka PyTorch framework i hoʻokō i kahi ʻōnaehana vector no ka huli ʻana e pili ana i ka like o nā mea i nā kiʻi, i kūkulu ʻia me ka hoʻohana ʻana i ka ʻikepili mai nā hōʻiliʻili kuhikuhi me nā kiʻi o nā holoholona, ​​nā pōpoki a me nā ʻīlio, i hoʻololi ʻia i mau pūʻulu vectors. . Ke loaʻa kahi kiʻi e hiki mai ana no ka ʻimi ʻana, hoʻololi ʻia ia me ka hoʻohana ʻana i ka mīkini aʻo ʻana i kahi vector, ma muli o ke koho ʻana i nā vectors like loa mai ka index me ka SPTAG algorithm a hoʻihoʻi ʻia nā kiʻi pili i ka hopena.

Source: opennet.ru