{"id":111103,"date":"2023-10-26T15:10:31","date_gmt":"2023-10-26T13:10:31","guid":{"rendered":"https:\/\/prohoster.info\/blog\/novosti-interneta\/otkryt-kod-jina-embedding-modeli-dlya-vektornogo-predstavleniya-smysla-teksta"},"modified":"2023-10-26T15:10:31","modified_gmt":"2023-10-26T13:10:31","slug":"otkryt-kod-jina-embedding-modeli-dlya-vektornogo-predstavleniya-smysla-teksta","status":"publish","type":"post","link":"https:\/\/prohoster.info\/hu\/blog\/news\/otkryt-kod-jina-embedding-modeli-dlya-vektornogo-predstavleniya-smysla-teksta","title":{"rendered":"Kinyitotta a Jina Embedding forr\u00e1st, egy modellt a sz\u00f6veg jelent\u00e9s\u00e9nek vektoralap\u00fa reprezent\u00e1l\u00e1s\u00e1hoz.","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

A Jina v\u00e1llalat az Apache 2.0 licenc alatt ny\u00fajtott g\u00e9pi tanul\u00e1si modellt a sz\u00f6veg vektoralak\u00fa reprezent\u00e1ci\u00f3j\u00e1ra — jina-embeddings-v2. A modell lehet\u0151v\u00e9 teszi, hogy tetsz\u0151leges sz\u00f6veget, amely legfeljebb 8192 karakterb\u0151l \u00e1ll, egy r\u00f6vid val\u00f3s sz\u00e1mokb\u00f3l \u00e1ll\u00f3 sorozatt\u00e1 alak\u00edtson, amely egy vektort k\u00e9pvisel, \u00f6sszhangban az eredeti sz\u00f6veggel, \u00e9s reproduk\u00e1lja annak szemantik\u00e1j\u00e1t (\u00e9rtelm\u00e9t). A Jina Embedding az els\u0151 ny\u00edlt forr\u00e1sk\u00f3d\u00fa g\u00e9pi tanul\u00e1si modell lett, mely olyan jellemz\u0151kkel b\u00edr, amelyek nem maradnak el a sz\u00f6veg vektoriz\u00e1l\u00f3 tulajdons\u00e1gaival rendelkez\u0151 OpenAI projekt (text-embedding-ada-002) v\u00e9dett modellj\u00e9t\u0151l, szint\u00e9n k\u00e9pes 8192 token sz\u00f6vegek feldolgoz\u00e1s\u00e1ra. <\/p>\n

A k\u00e9t l\u00e9trehozott vektor k\u00f6z\u00f6tti t\u00e1vols\u00e1g felhaszn\u00e1lhat\u00f3 az eredeti sz\u00f6vegek jelent\u00e9sbeli \u00f6sszef\u00fcgg\u00e9s\u00e9nek meghat\u00e1roz\u00e1s\u00e1ra. Gyakorlatilag a l\u00e9trehozott vektorok alkalmazhat\u00f3k a sz\u00f6vegek hasonl\u00f3s\u00e1g\u00e1nak elemz\u00e9s\u00e9re, tematikus anyagok keres\u00e9s\u00e9nek megszervez\u00e9s\u00e9re (az eredm\u00e9nyek rangsorol\u00e1sa szemantikai k\u00f6zels\u00e9g szerint), sz\u00f6vegek jelent\u00e9se szerinti csoportos\u00edt\u00e1s\u00e1ra, aj\u00e1nl\u00e1sok l\u00e9trehoz\u00e1s\u00e1ra (hasonl\u00f3 sz\u00f6vegsorok list\u00e1j\u00e1nak javaslat\u00e1ra), anom\u00e1li\u00e1k \u00e9szlel\u00e9s\u00e9re, pl\u00e1gium meghat\u00e1roz\u00e1s\u00e1ra \u00e9s tesztel\u00e9s klasszifik\u00e1l\u00e1s\u00e1ra. A felhaszn\u00e1l\u00e1si ter\u00fcletek p\u00e9ld\u00e1ik\u00e9nt szerepelnek a modell jogi dokumentumok elemz\u00e9s\u00e9ben, \u00fczleti elemz\u00e9sben, orvosi kutat\u00e1sok sor\u00e1n tudom\u00e1nyos cikkek feldolgoz\u00e1s\u00e1ban, irodalmi kritik\u00e1ban, p\u00e9nz\u00fcgyi jelent\u00e9sek elemz\u00e9s\u00e9ben \u00e9s a chatbotok \u00f6sszetett k\u00e9rd\u00e9sek kezel\u00e9s\u00e9nek jav\u00edt\u00e1s\u00e1ban. <\/p>\n

K\u00e9t modellv\u00e1ltozat \u00e9rhet\u0151 el a let\u00f6lt\u00e9sre: jina-embeddings (alap — 0,27 GB \u00e9s cs\u00f6kkentett — 0,07 GB), amelyek 400 milli\u00f3 angol nyelv\u0171 sz\u00f6vegsorozat p\u00e1rra lettek tan\u00edtva, k\u00fcl\u00f6nb\u00f6z\u0151 tudom\u00e1ny\u00e1gakat lefedve. A tan\u00edt\u00e1s sor\u00e1n 512 token m\u00e9ret\u0171 sorozatokat haszn\u00e1ltak, amelyeket ALiBi m\u00f3dszer (Figyelem Line\u00e1ris Elfogults\u00e1ggal) seg\u00edts\u00e9g\u00e9vel 8192 m\u00e9retre extrapol\u00e1ltak. <\/p>\n

A B\u00e1zis modell 137 milli\u00f3 param\u00e9tert tartalmaz, \u00e9s \u00e1ll\u00f3 rendszerekkel GPU-n val\u00f3 haszn\u00e1latra k\u00e9sz\u00fclt. A r\u00f6vid\u00edtett modell 33 milli\u00f3 param\u00e9tert k\u00edn\u00e1l, kisebb pontoss\u00e1got biztos\u00edt, \u00e9s mobil eszk\u00f6z\u00f6k\u00f6n, valamint kis mem\u00f3ri\u00e1val rendelkez\u0151 rendszereken val\u00f3 alkalmaz\u00e1sra c\u00e9loz. A k\u00f6zelj\u00f6v\u0151ben egy nagyobb modellt is terveznek, amely 435 milli\u00f3 param\u00e9tert \u00f6lel fel. Fejleszt\u00e9s alatt \u00e1ll egy t\u00f6bbnyelv\u0171 verzi\u00f3 is, amely jelenleg a n\u00e9met \u00e9s spanyol nyelvek t\u00e1mogat\u00e1s\u00e1ra \u00f6sszpontos\u00edt. K\u00fcl\u00f6n plugint fejlesztettek a jina-embeddings modell LLM eszk\u00f6zk\u00e9szlet\u00e9n kereszt\u00fcli haszn\u00e1lat\u00e1hoz.
\n
Forr\u00e1s: opennet.ru<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

\u041a\u043e\u043c\u043f\u0430\u043d\u0438\u044f Jina \u043e\u0442\u043a\u0440\u044b\u043b\u0430 \u043f\u043e\u0434 \u043b\u0438\u0446\u0435\u043d\u0437\u0438\u0435\u0439 Apache 2.0 \u043c\u043e\u0434\u0435\u043b\u044c \u043c\u0430\u0448\u0438\u043d\u043d\u043e\u0433\u043e \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f \u0434\u043b\u044f \u0432\u0435\u043a\u0442\u043e\u0440\u043d\u043e\u0433\u043e \u043f\u0440\u0435\u0434\u0441\u0442\u0430\u0432\u043b\u0435\u043d\u0438\u044f \u0442\u0435\u043a\u0441\u0442\u0430 — jina-embeddings-v2. \u041c\u043e\u0434\u0435\u043b\u044c \u043f\u043e\u0437\u0432\u043e\u043b\u044f\u0435\u0442 \u043f\u0440\u0435\u043e\u0431\u0440\u0430\u0437\u043e\u0432\u0430\u0442\u044c \u043f\u0440\u043e\u0438\u0437\u0432\u043e\u043b\u044c\u043d\u044b\u0439 \u0442\u0435\u043a\u0441\u0442, \u0432\u043a\u043b\u044e\u0447\u0430\u044e\u0449\u0438\u0439 \u0434\u043e 8192 \u0437\u043d\u0430\u043a\u043e\u0432, \u0432 \u043d\u0435\u0431\u043e\u043b\u044c\u0448\u0443\u044e \u043f\u043e\u0441\u043b\u0435\u0434\u043e\u0432\u0430\u0442\u0435\u043b\u044c\u043d\u043e\u0441\u0442\u044c \u0432\u0435\u0449\u0435\u0441\u0442\u0432\u0435\u043d\u043d\u044b\u0445 \u0447\u0438\u0441\u0435\u043b, \u043e\u0431\u0440\u0430\u0437\u0443\u044e\u0449\u0438\u0445 \u0432\u0435\u043a\u0442\u043e\u0440, \u0441\u043e\u043f\u043e\u0441\u0442\u0430\u0432\u043b\u0435\u043d\u043d\u044b\u0439 \u0441 \u0438\u0441\u0445\u043e\u0434\u043d\u044b\u043c \u0442\u0435\u043a\u0441\u0442\u043e\u043c \u0438 \u0432\u043e\u0441\u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u044f\u0449\u0438\u0439 \u0435\u0433\u043e \u0441\u0435\u043c\u0430\u043d\u0442\u0438\u043a\u0443 (\u0441\u043c\u044b\u0441\u043b). Jina Embedding \u0441\u0442\u0430\u043b\u0430 \u043f\u0435\u0440\u0432\u043e\u0439 \u043e\u0442\u043a\u0440\u044b\u0442\u043e\u0439 \u043c\u043e\u0434\u0435\u043b\u044c\u044e \u043c\u0430\u0448\u0438\u043d\u043d\u043e\u0433\u043e \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f, \u043e\u0431\u043b\u0430\u0434\u0430\u044e\u0449\u0435\u0439 \u0445\u0430\u0440\u0430\u043a\u0442\u0435\u0440\u0438\u0441\u0442\u0438\u043a\u0430\u043c\u0438, \u043d\u0435 \u0443\u0441\u0442\u0443\u043f\u0430\u044e\u0449\u0438\u043c\u0438 \u043f\u0440\u043e\u043f\u0440\u0438\u0435\u0442\u0430\u0440\u043d\u043e\u0439 \u043c\u043e\u0434\u0435\u043b\u0438 […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[702],"tags":[],"class_list":["post-111103","post","type-post","status-publish","format-standard","hentry","category-news"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47Megny\u00edlt a Jina Embedding k\u00f3dja, amely a sz\u00f6veg jelent\u00e9s\u00e9nek vektork\u00e9pz\u00e9s\u00e9re szolg\u00e1l\u00f3 modell | ProHoster","description":"A Jina c\u00e9g az Apache 2.0 licenc alatt megnyitotta a textus vektork\u00e9pz\u00e9s\u00e9re szolg\u00e1l\u00f3 g\u00e9pi tanul\u00e1si modellt - jina-embeddings-v2.","canonical_url":"https:\/\/prohoster.info\/hu\/blog\/news\/otkryt-kod-jina-embedding-modeli-dlya-vektornogo-predstavleniya-smysla-teksta","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"hu_HU","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041e\u0442\u043a\u0440\u044b\u0442 \u043a\u043e\u0434 Jina Embedding, \u043c\u043e\u0434\u0435\u043b\u0438 \u0434\u043b\u044f \u0432\u0435\u043a\u0442\u043e\u0440\u043d\u043e\u0433\u043e \u043f\u0440\u0435\u0434\u0441\u0442\u0430\u0432\u043b\u0435\u043d\u0438\u044f \u0441\u043c\u044b\u0441\u043b\u0430 \u0442\u0435\u043a\u0441\u0442\u0430 | ProHoster","og:description":"\u041a\u043e\u043c\u043f\u0430\u043d\u0438\u044f Jina \u043e\u0442\u043a\u0440\u044b\u043b\u0430 \u043f\u043e\u0434 \u043b\u0438\u0446\u0435\u043d\u0437\u0438\u0435\u0439 Apache 2.0 \u043c\u043e\u0434\u0435\u043b\u044c \u043c\u0430\u0448\u0438\u043d\u043d\u043e\u0433\u043e \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f \u0434\u043b\u044f \u0432\u0435\u043a\u0442\u043e\u0440\u043d\u043e\u0433\u043e \u043f\u0440\u0435\u0434\u0441\u0442\u0430\u0432\u043b\u0435\u043d\u0438\u044f \u0442\u0435\u043a\u0441\u0442\u0430 - jina-embeddings-v2.","og:url":"https:\/\/prohoster.info\/hu\/blog\/news\/otkryt-kod-jina-embedding-modeli-dlya-vektornogo-predstavleniya-smysla-teksta","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2023-10-26T13:10:31+00:00","article:modified_time":"2023-10-26T13:10:31+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":[],"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/hu\/wp-json\/wp\/v2\/posts\/111103","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/hu\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/hu\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/hu\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/hu\/wp-json\/wp\/v2\/comments?post=111103"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/hu\/wp-json\/wp\/v2\/posts\/111103\/revisions"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/hu\/wp-json\/wp\/v2\/media?parent=111103"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/hu\/wp-json\/wp\/v2\/categories?post=111103"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/hu\/wp-json\/wp\/v2\/tags?post=111103"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}