{"id":38448,"date":"2019-10-31T22:23:47","date_gmt":"2019-10-31T19:23:47","guid":{"rendered":"https:\/\/prohoster.info\/blog\/github-otkryl-narabotki-po-primeneniyu-mashinnogo-obucheniya-dlya-poiska-i-analiza-koda\/"},"modified":"2019-10-31T22:23:47","modified_gmt":"2019-10-31T19:23:47","slug":"github-otkryl-narabotki-po-primeneniyu-mashinnogo-obucheniya-dlya-poiska-i-analiza-koda","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/news\/github-otkryl-narabotki-po-primeneniyu-mashinnogo-obucheniya-dlya-poiska-i-analiza-koda","title":{"rendered":"In the mainline branch, nginx version 1.17.4 has been released.","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

GitHub presented<\/a><\/noindex> project CodeSearchNet<\/a><\/noindex>, which has prepared machine learning models and datasets necessary for parsing, classifying, and analyzing code in various programming languages. CodeSearchNet, analogous to ImageNet<\/a><\/noindex>, includes a large collection of code snippets accompanied by annotations that formalize the actions performed by the code. The components for training models and examples of using CodeSearchNet are written in Python using the Tensorflow framework and is distributed<\/a><\/noindex> is licensed under the MIT License.<\/p>\n

In creating CodeSearchNet, natural language text parsing technologies were utilized, allowing machine learning systems to take into account not only syntactic features but also the meaning of the actions performed by the code. On GitHub, the system is conducted<\/a><\/noindex> in experiments on organizing semantic code search using queries in natural language<\/a><\/noindex> (for example, when requesting 'sorting a list of strings', the code for implementing the corresponding algorithms is displayed).<\/p>\n

The proposed dataset includes over 2 million 'code-comment' pairs, prepared based on the source texts of existing open libraries. The code covers the complete source text of individual functions or methods, while the comment describes the actions performed by the function (detailed documentation is provided). Currently, datasets have been prepared for the languages Python, JavaScript, Ruby, Go, Java, and PHP. Examples of using the proposed datasets for training various types of neural networks are provided, including Neural-Bag-Of-Words<\/a><\/noindex>, RNN<\/a><\/noindex>, Self-Attention<\/a><\/noindex> (BERT) and 1D-CNN+Self-Attention Hybrid<\/a><\/noindex>.<\/p>\n

To enhance natural language search mechanisms, the CodeSearchNet Challenge dataset has also been prepared, including
\n99 typical<\/a><\/noindex> queries with about 4,000 expert annotations describing the most likely code bindings in the CodeSearchNet Corpus dataset, which covers approximately 6 million methods and functions (the dataset size<\/a><\/noindex> is about 20 GB). The CodeSearchNet Challenge can serve as a benchmark for evaluating the effectiveness of various methods for natural language code search. Using the tools of Kubeflow<\/a><\/noindex> prepared
\nan example<\/a><\/noindex> the code search engine.<\/p>\n

<\/a><\/noindex><\/p>\n

Source: opennet.ru<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

GitHub \u043f\u0440\u0435\u0434\u0441\u0442\u0430\u0432\u0438\u043b \u043f\u0440\u043e\u0435\u043a\u0442 CodeSearchNet, \u0432 \u0440\u0430\u043c\u043a\u0430\u0445 \u043a\u043e\u0442\u043e\u0440\u043e\u0433\u043e \u043f\u043e\u0434\u0433\u043e\u0442\u043e\u0432\u043b\u0435\u043d\u044b \u043c\u043e\u0434\u0435\u043b\u0438 \u043c\u0430\u0448\u0438\u043d\u043d\u043e\u0433\u043e \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f \u0438 \u043d\u0430\u0431\u043e\u0440\u044b \u0434\u0430\u043d\u043d\u044b\u0445, \u043d\u0435\u043e\u0431\u0445\u043e\u0434\u0438\u043c\u044b\u0435 \u0434\u043b\u044f \u0440\u0430\u0437\u0431\u043e\u0440\u0430, \u043a\u043b\u0430\u0441\u0441\u0438\u0444\u0438\u043a\u0430\u0446\u0438\u0438 \u0438 \u0430\u043d\u0430\u043b\u0438\u0437\u0430 \u043a\u043e\u0434\u0430 \u043d\u0430 \u0440\u0430\u0437\u043b\u0438\u0447\u043d\u044b\u0445 \u044f\u0437\u044b\u043a\u0430\u0445 \u043f\u0440\u043e\u0433\u0440\u0430\u043c\u043c\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f. CodeSearchNet, \u043f\u043e \u0430\u043d\u0430\u043b\u043e\u0433\u0438\u0438 \u0441 ImageNet, \u0432\u043a\u043b\u044e\u0447\u0430\u0435\u0442 \u0431\u043e\u043b\u044c\u0448\u0443\u044e \u043a\u043e\u043b\u043b\u0435\u043a\u0446\u0438\u044e \u043e\u0442\u0440\u044b\u0432\u043a\u043e\u0432 \u043a\u043e\u0434\u0430, \u0441\u043d\u0430\u0431\u0436\u0451\u043d\u043d\u044b\u0445 \u0430\u043d\u043d\u043e\u0442\u0430\u0446\u0438\u044f\u043c\u0438, \u0444\u043e\u0440\u043c\u0430\u043b\u0438\u0437\u0443\u044e\u0449\u0438\u043c\u0438 \u0432\u044b\u043f\u043e\u043b\u043d\u044f\u0435\u043c\u044b\u0435 \u043a\u043e\u0434\u043e\u043c \u0434\u0435\u0439\u0441\u0442\u0432\u0438\u044f. \u041a\u043e\u043c\u043f\u043e\u043d\u0435\u043d\u0442\u044b \u0434\u043b\u044f \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f \u043c\u043e\u0434\u0435\u043b\u0435\u0439 \u0438 \u043f\u0440\u0438\u043c\u0435\u0440\u044b \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u044f CodeSearchNet \u043d\u0430\u043f\u0438\u0441\u0430\u043d\u044b \u043d\u0430 \u044f\u0437\u044b\u043a\u0435 Python \u0441 \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u0435\u043c […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[702],"tags":[],"class_list":["post-38448","post","type-post","status-publish","format-standard","hentry","category-news"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47GitHub has released developments on the use of machine learning for code search and analysis | ProHoster","description":"GitHub has introduced a project","canonical_url":"https:\/\/prohoster.info\/en\/blog\/news\/github-otkryl-narabotki-po-primeneniyu-mashinnogo-obucheniya-dlya-poiska-i-analiza-koda","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"en_US","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47GitHub \u043e\u0442\u043a\u0440\u044b\u043b \u043d\u0430\u0440\u0430\u0431\u043e\u0442\u043a\u0438 \u043f\u043e \u043f\u0440\u0438\u043c\u0435\u043d\u0435\u043d\u0438\u044e \u043c\u0430\u0448\u0438\u043d\u043d\u043e\u0433\u043e \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f \u0434\u043b\u044f \u043f\u043e\u0438\u0441\u043a\u0430 \u0438 \u0430\u043d\u0430\u043b\u0438\u0437\u0430 \u043a\u043e\u0434\u0430 | ProHoster","og:description":"GitHub \u043f\u0440\u0435\u0434\u0441\u0442\u0430\u0432\u0438\u043b \u043f\u0440\u043e\u0435\u043a\u0442","og:url":"https:\/\/prohoster.info\/en\/blog\/news\/github-otkryl-narabotki-po-primeneniyu-mashinnogo-obucheniya-dlya-poiska-i-analiza-koda","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-10-31T19:23:47+00:00","article:modified_time":"2019-10-31T19:23:47+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"38448","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-23 22:05:20","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 01:09:22","updated":"2026-01-23 22:05:20","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/38448","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/comments?post=38448"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/38448\/revisions"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media?parent=38448"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/categories?post=38448"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/tags?post=38448"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}