Feature Selection in Machine Learning<\/a><\/noindex>. Shpresojm\u00eb se do t\u00eb jet\u00eb e dobishme p\u00ebr t\u00eb gjith\u00eb ata q\u00eb jan\u00eb t\u00eb interesuar p\u00ebr k\u00ebt\u00eb tem\u00eb.<\/p>\nN\u00eb bot\u00ebn reale, t\u00eb dh\u00ebnat nuk jan\u00eb gjithmon\u00eb aq t\u00eb pastra, sa ndonj\u00ebher\u00eb mendojn\u00eb porosit\u00ebsit e biznesit. Pik\u00ebrisht p\u00ebr k\u00ebt\u00eb arsye k\u00ebrkohet analiza inteligjente e t\u00eb dh\u00ebnave (data mining dhe data wrangling). Kjo ndihmon n\u00eb zbulimin e vlerave t\u00eb humbura dhe modeleve n\u00eb t\u00eb dh\u00ebnat e strukturuara, duke p\u00ebrdorur k\u00ebrkesa q\u00eb nuk mund t'i identifikoj\u00eb njeriu. P\u00ebr t\u00eb gjetur dhe p\u00ebrdorur k\u00ebto modele p\u00ebr parashikimin e rezultateve, p\u00ebrmes lidhjeve t\u00eb zbuluara n\u00eb t\u00eb dh\u00ebna, do t\u00eb nevojitet m\u00ebsimi i makinerive (Machine Learning).<\/p>\n
P\u00ebr t\u00eb kuptuar \u00e7do algoritem, \u00ebsht\u00eb e nevojshme t\u00eb shqyrtohen t\u00eb gjitha variabl\u00ebt n\u00eb t\u00eb dh\u00ebna dhe t\u00eb zbulohet se \u00e7far\u00eb p\u00ebrfaq\u00ebsojn\u00eb k\u00ebto variabla. Kjo \u00ebsht\u00eb jasht\u00ebzakonisht e r\u00ebnd\u00ebsishme, sepse arsyetimi i rezultateve bazohet n\u00eb kuptimin e t\u00eb dh\u00ebnave. N\u00ebse t\u00eb dh\u00ebnat p\u00ebrmbajn\u00eb 5 ose madje 50 variabla, mund t'i shqyrtoni t\u00eb gjitha. Por \u00e7far\u00eb n\u00ebse ka 200? At\u00ebher\u00eb thjesht nuk do t\u00eb mjaftoj\u00eb koha p\u00ebr t\u00eb shqyrtuar \u00e7do variab\u00ebl t\u00eb ve\u00e7ant\u00eb. M\u00eb shum\u00eb se kaq, disa algoritem nuk funksionojn\u00eb p\u00ebr t\u00eb dh\u00ebnat kategorike, dhe n\u00eb at\u00eb rast do t\u00eb duhet t'i kthejm\u00eb t\u00eb gjitha kolonat kategorike n\u00eb variabla num\u00ebror\u00eb (ato mund t\u00eb duken si num\u00ebror\u00eb, por metrikat do t\u00eb tregojn\u00eb se jan\u00eb kategorike), p\u00ebr t'i shtuar ato n\u00eb model. N\u00eb k\u00ebt\u00eb m\u00ebnyr\u00eb, numri i variabl\u00ebve rritet dhe ata b\u00ebhen rreth 500. \u00c7far\u00eb t\u00eb b\u00ebjm\u00eb tani? Mund t\u00eb mendohet se zgjidhja do t\u00eb ishte reduktimi i dimensionalitetit. Algoritmet p\u00ebr reduktimin e dimensionalitetit zvog\u00eblojn\u00eb numrin e parametrave, por ndikojn\u00eb negativisht n\u00eb interpretuarshm\u00ebri. \u00c7far\u00eb n\u00ebse ekzistojn\u00eb teknika t\u00eb tjera q\u00eb p\u00ebrjashtojn\u00eb karakteristikat dhe n\u00eb t\u00eb nj\u00ebjt\u00ebn koh\u00eb lejojn\u00eb q\u00eb t\u00eb kuptohen dhe interpretohen leht\u00ebsisht ato q\u00eb mbeten?<\/p>\n
N\u00eb var\u00ebsi t\u00eb asaj n\u00ebse analiza \u00ebsht\u00eb e bazuar n\u00eb regresion apo klasifikim, algoritmet e seleksionit t\u00eb ve\u00e7orave mund t\u00eb ndryshojn\u00eb, por ideja kryesore p\u00ebr realizimin e tyre mbetet e nj\u00ebjt\u00eb.<\/p>\n
Variablat e forta korrelacioni<\/h3>\n Variablat q\u00eb jan\u00eb shum\u00eb t\u00eb korrelacionuar me nj\u00ebri-tjetrin i japin modelit t\u00eb nj\u00ebjt\u00ebn informacion, prandaj, nuk \u00ebsht\u00eb e nevojshme t\u00eb p\u00ebrdoren t\u00eb gjitha p\u00ebr analiz\u00eb. P\u00ebr shembull, n\u00ebse t\u00eb dh\u00ebnat (dataset) p\u00ebrmbajn\u00eb ve\u00e7ori \"Koha n\u00eb rrjet\" dhe \"Trafiku i p\u00ebrdorur\", mund t\u00eb supozojm\u00eb se ato do t\u00eb jen\u00eb t\u00eb korrelacionuara n\u00eb nj\u00ebfar\u00eb m\u00ebnyre, dhe do t\u00eb shohim nj\u00eb korelim t\u00eb fort\u00eb, edhe n\u00ebse zgjedhim nj\u00eb most\u00ebr t\u00eb rast\u00ebsishme t\u00eb t\u00eb dh\u00ebnave. N\u00eb k\u00ebt\u00eb rast, modeli ka nevoj\u00eb vet\u00ebm p\u00ebr nj\u00eb nga k\u00ebto variabla. N\u00ebse p\u00ebrdorim t\u00eb dyja, modeli do t\u00eb jet\u00eb i mbingarkuar (overfit) dhe i prirur ndaj nj\u00eb ve\u00e7orie t\u00eb ve\u00e7ant\u00eb.<\/p>\n
Vlerat P<\/h3>\n N\u00eb algoritmet si regresioni liniar, nj\u00eb model statistik fillestar \u00ebsht\u00eb gjithmon\u00eb nj\u00eb ide e mir\u00eb. Ai ndihmon n\u00eb tregimin e r\u00ebnd\u00ebsis\u00eb s\u00eb karakteristikave p\u00ebrmes vlerave t\u00eb tyre p, t\u00eb cilat jan\u00eb nxjerr\u00eb nga ky model. Duke vendosur nj\u00eb nivel r\u00ebnd\u00ebsie, ne kontrollojm\u00eb vlerat e fituara p, dhe n\u00ebse ndonj\u00eb vler\u00eb rezulton t\u00eb jet\u00eb m\u00eb e ul\u00ebt se niveli i caktuar i r\u00ebnd\u00ebsis\u00eb, at\u00ebher\u00eb kjo karakteristik\u00eb shpallet e r\u00ebnd\u00ebsishme, dometh\u00ebn\u00eb ndryshimi i vler\u00ebs s\u00eb saj ka shum\u00eb mund\u00ebsi t\u00eb sjell\u00eb nj\u00eb ndryshim n\u00eb vler\u00ebn e targetit.<\/p>\n
Zgjedhja e drejtp\u00ebrdrejt\u00eb<\/h3>\n Selekcija e drejtp\u00ebrdrejt\u00eb \u00ebsht\u00eb nj\u00eb teknik\u00eb q\u00eb p\u00ebrfshin p\u00ebrdorimin e regresionit me hapa. Nd\u00ebrtimi i modelit fillon nga nj\u00eb pik\u00eb zero, dometh\u00ebn\u00eb nj\u00eb model bosh, dhe pastaj \u00e7do iteracion shton nj\u00eb variab\u00ebl q\u00eb sjell p\u00ebrmir\u00ebsim n\u00eb modelin n\u00eb nd\u00ebrtim. Variabli q\u00eb shtohet n\u00eb model p\u00ebrcaktohet nga r\u00ebnd\u00ebsia e tij. Kjo mund t\u00eb llogaritet duke p\u00ebrdorur metrika t\u00eb ndryshme. M\u00ebnyra m\u00eb e zakonshme \u00ebsht\u00eb p\u00ebrdorimi i vlerave p, t\u00eb cilat merren nga nj\u00eb model statistikor fillestar duke p\u00ebrdorur t\u00eb gjitha variablat. Disa her\u00eb, seleksioni i drejtp\u00ebrdrejt\u00eb mund t\u00eb \u00e7oj\u00eb n\u00eb mbifitim t\u00eb modelit, pasi n\u00eb model mund t\u00eb ndodhen variabla me korrelacion t\u00eb fort\u00eb, edhe n\u00ebse ata ofrojn\u00eb t\u00eb nj\u00ebjtin informacion p\u00ebr modelin (por modeli n\u00eb k\u00ebt\u00eb rast tregon p\u00ebrmir\u00ebsim).<\/p>\n
Seleksioni i kund\u00ebrt<\/h3>\n Zgjedhja e kthyer gjithashtu p\u00ebrfshin p\u00ebrjashtimin hap pas hapi t\u00eb karakteristikave, megjithat\u00eb n\u00eb drejtimin e kund\u00ebrt n\u00eb krahasim me t\u00eb drejtat. N\u00eb k\u00ebt\u00eb rast, modeli fillestar p\u00ebrfshin t\u00eb gjith\u00eb variablat e pavarur. M\u00eb pas, variablat p\u00ebrjashtohen (nj\u00eb nga nj\u00eb p\u00ebr iteracion), n\u00ebse ata nuk kan\u00eb vler\u00eb p\u00ebr modelin e ri regresion n\u00eb \u00e7do iteracion. N\u00eb themel t\u00eb p\u00ebrjashtimit t\u00eb karakteristikave q\u00ebndrojn\u00eb treguesit e vlerave p t\u00eb modelit fillestar. N\u00eb k\u00ebt\u00eb metod\u00eb ka gjithashtu pasiguri n\u00eb heqjen e variablave q\u00eb jan\u00eb fort t\u00eb korreluar.<\/p>\n
P\u00ebrjashtimi rekurziv i karakteristikave<\/h3>\n RFE \u00ebsht\u00eb nj\u00eb teknik\u00eb\/algotit\u00ebm i p\u00ebrdorur gjer\u00ebsisht p\u00ebr t\u00eb zgjedhur numrin e sakt\u00eb t\u00eb karakteristikave t\u00eb r\u00ebnd\u00ebsishme. Ndonj\u00ebher\u00eb metoda p\u00ebrdoret p\u00ebr t\u00eb shpjeguar nj\u00eb num\u00ebr \"m\u00eb t\u00eb r\u00ebnd\u00ebsishmish\" t\u00eb karakteristikave q\u00eb ndikojn\u00eb n\u00eb rezultatet; dhe ndonj\u00ebher\u00eb p\u00ebr t\u00eb reduktuar nj\u00eb num\u00ebr shum\u00eb t\u00eb madh variablish (rreth 200-400), duke l\u00ebn\u00eb vet\u00ebm ata q\u00eb kontribuojn\u00eb ndonj\u00ebher\u00eb n\u00eb model, nd\u00ebrsa t\u00eb tjer\u00ebt p\u00ebrjashtohen. RFE p\u00ebrdor nj\u00eb sistem renditjeje. Karakteristikave n\u00eb grupin e t\u00eb dh\u00ebnave u jepen renditje. M\u00eb pas k\u00ebto renditje p\u00ebrdoren p\u00ebr p\u00ebrjashtimin rekursiv t\u00eb karakteristikave, n\u00eb var\u00ebsi t\u00eb kollinearitetit midis tyre dhe r\u00ebnd\u00ebsis\u00eb s\u00eb k\u00ebtyre karakteristikave n\u00eb model. P\u00ebrve\u00e7 renditjes s\u00eb karakteristikave, RFE mund t\u00eb tregoj\u00eb n\u00ebse k\u00ebto karakteristika jan\u00eb t\u00eb r\u00ebnd\u00ebsishme apo jo, edhe p\u00ebr nj\u00eb num\u00ebr t\u00eb caktuar karakteristikash (sepse \u00ebsht\u00eb shum\u00eb e mundur q\u00eb numri i zgjedhur i karakteristikave mund t\u00eb mos jet\u00eb optimal, dhe numri optimal i karakteristikave mund t\u00eb jet\u00eb m\u00eb i madh ose m\u00eb i vog\u00ebl se ai i zgjedhuri).<\/p>\n
Diagrami i r\u00ebnd\u00ebsis\u00eb s\u00eb karakteristikave<\/h3>\n Duke p\u00ebr interpretimin e algoritmeve t\u00eb m\u00ebsimit t\u00eb makinerive, zakonisht diskutohet p\u00ebr regresionet linjare (t\u00eb cilat lejojn\u00eb t\u00eb analizohet r\u00ebnd\u00ebsia e karakteristikave p\u00ebrmes vlerave p) dhe pem\u00ebt e vendimeve (t\u00eb cilat tregojn\u00eb r\u00ebnd\u00ebsin\u00eb e karakteristikave n\u00eb form\u00ebn e nj\u00eb peme, si dhe hierarkin\u00eb e tyre). Nga ana tjet\u00ebr, n\u00eb algoritme si Random Forest, LightGBM dhe XG Boost, shpesh p\u00ebrdoret nj\u00eb diagram i r\u00ebnd\u00ebsis\u00eb s\u00eb karakteristikave, q\u00eb do t\u00eb thot\u00eb se nd\u00ebrtohet nj\u00eb diagram i variablave dhe \"sasit\u00eb e r\u00ebnd\u00ebsis\u00eb\" s\u00eb tyre. Kjo \u00ebsht\u00eb ve\u00e7an\u00ebrisht e dobishme kur duhet t\u00eb ofrohet nj\u00eb justifikim i strukturuar i r\u00ebnd\u00ebsis\u00eb s\u00eb karakteristikave n\u00eb lidhje me ndikimin e tyre n\u00eb biznes.<\/p>\n
Regjistrimi<\/h3>\n Regjistrimi b\u00ebhet p\u00ebr t\u00eb kontrolluar bilancin mes paragjykimit (bias) dhe devijimit (variance). Paragjykimi tregon se sa modeli \u00ebsht\u00eb p\u00ebrshtatur (overfit) n\u00eb grupin e t\u00eb dh\u00ebnave t\u00eb trajnimit. Devijimi tregon se sa t\u00eb ndryshme ishin parashikimet mes seteve t\u00eb t\u00eb dh\u00ebnave t\u00eb trajnimit dhe atyre p\u00ebr testim. Idealisht, si paragjykimi ashtu edhe devijimi duhet t\u00eb jen\u00eb t\u00eb vogla. K\u00ebtu vjen n\u00eb ndihm\u00eb regjistrimi! Ekzistojn\u00eb dy teknika kryesore:<\/p>\n
L1 Regularizimi \u2014 Lasso: Lasso penalizon coefficient\u00ebt e modelit p\u00ebr t\u00eb ndryshuar r\u00ebnd\u00ebsin\u00eb e tyre p\u00ebr modelin dhe madje mund t'i anuloj\u00eb ato (dmth. t'i heq\u00eb k\u00ebto variabla nga modeli p\u00ebrfundimtar). Zakonisht, Lasso p\u00ebrdoret n\u00ebse grupi i t\u00eb dh\u00ebnave p\u00ebrmban nj\u00eb num\u00ebr t\u00eb madh variablash dhe \u00ebsht\u00eb e nevojshme t\u00eb p\u00ebrjashtohen disa nga ata p\u00ebr t\u00eb kuptuar m\u00eb mir\u00eb se si karakteristikat e r\u00ebnd\u00ebsishme ndikojn\u00eb n\u00eb model (dmth. ato karakteristika q\u00eb jan\u00eb p\u00ebrzgjedhur prej Lasso dhe p\u00ebr t\u00eb cilat \u00ebsht\u00eb vendosur r\u00ebnd\u00ebsia).<\/p>\n
L2 Regularizimi \u2014 metoda Ridge: Q\u00ebllimi i Ridge \u00ebsht\u00eb t\u00eb mbaj\u00eb t\u00eb gjith\u00eb variablat dhe n\u00eb t\u00eb nj\u00ebjt\u00ebn koh\u00eb t'u caktoj\u00eb atyre r\u00ebnd\u00ebsi mbi baz\u00ebn e kontributit n\u00eb performanc\u00ebn e modelit. Ridge do t\u00eb ishte nj\u00eb zgjedhje e mir\u00eb n\u00ebse grupi i t\u00eb dh\u00ebnave p\u00ebrmban nj\u00eb num\u00ebr t\u00eb vog\u00ebl variablash dhe t\u00eb gjith\u00eb jan\u00eb t\u00eb nevojsh\u00ebm p\u00ebr interpretimin e p\u00ebrfundimeve dhe rezultateve t\u00eb marra.<\/p>\n
Duke pasur parasysh se Ridge l\u00eb t\u00eb gjith\u00eb variablat, nd\u00ebrsa Lasso vendos m\u00eb mir\u00eb r\u00ebnd\u00ebsin\u00eb e tyre, \u00ebsht\u00eb zhvilluar nj\u00eb algoritem q\u00eb bashkon ve\u00e7orit\u00eb m\u00eb t\u00eb mira t\u00eb t\u00eb dyja regularizimeve dhe njohur si Elastic-Net.<\/p>\n
Ka\u8fd8\u6709 shum\u00eb m\u00ebnyra p\u00ebr t\u00eb seleksionuar karakteristikat p\u00ebr m\u00ebsimin e makinerive, por ideja kryesore gjithmon\u00eb mbetet e nj\u00ebjt\u00eb: t\u00eb demonstrohet r\u00ebnd\u00ebsia e variablave dhe pastaj t\u00eb p\u00ebrjashtohen disa nga ata n\u00eb baz\u00eb t\u00eb r\u00ebnd\u00ebsis\u00eb s\u00eb marr\u00eb. R\u00ebnd\u00ebsia \u00ebsht\u00eb nj\u00eb term shum\u00eb subjektiv, pasi nuk \u00ebsht\u00eb nj\u00eb, por nj\u00eb grup i t\u00ebr\u00eb metrikash dhe grafik\u00ebsh q\u00eb mund t\u00eb p\u00ebrdoren p\u00ebr t\u00eb gjetur karakteristikat ky\u00e7e.<\/p>\n
Faleminderit p\u00ebr leximin! G\u00ebzuar m\u00ebsimin!<\/p>\n
Burimi: habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"\u041f\u0440\u0438\u0432\u0435\u0442, \u0425\u0430\u0431\u0440! \u041c\u044b \u0432 \u00ab\u0420\u0435\u043a\u0441\u043e\u0444\u0442\u00bb \u043f\u0435\u0440\u0435\u0432\u0435\u043b\u0438 \u043d\u0430 \u0440\u0443\u0441\u0441\u043a\u0438\u0439 \u044f\u0437\u044b\u043a \u0441\u0442\u0430\u0442\u044c\u044e Feature Selection in Machine Learning. \u041d\u0430\u0434\u0435\u0435\u043c\u0441\u044f, \u0431\u0443\u0434\u0435\u0442 \u043f\u043e\u043b\u0435\u0437\u043d\u043e \u0432\u0441\u0435\u043c, \u043a\u0442\u043e \u043d\u0435\u0440\u0430\u0432\u043d\u043e\u0434\u0443\u0448\u0435\u043d \u043a \u0442\u0435\u043c\u0435. \u0412 \u0440\u0435\u0430\u043b\u044c\u043d\u043e\u043c \u043c\u0438\u0440\u0435 \u0434\u0430\u043d\u043d\u044b\u0435 \u043d\u0435 \u0432\u0441\u0435\u0433\u0434\u0430 \u0442\u0430\u043a\u0438\u0435 \u0447\u0438\u0441\u0442\u044b\u0435, \u043a\u0430\u043a \u043f\u043e\u0440\u043e\u0439 \u0434\u0443\u043c\u0430\u044e\u0442 \u0431\u0438\u0437\u043d\u0435\u0441-\u0437\u0430\u043a\u0430\u0437\u0447\u0438\u043a\u0438. \u0418\u043c\u0435\u043d\u043d\u043e \u043f\u043e\u044d\u0442\u043e\u043c\u0443 \u0432\u043e\u0441\u0442\u0440\u0435\u0431\u043e\u0432\u0430\u043d \u0438\u043d\u0442\u0435\u043b\u043b\u0435\u043a\u0442\u0443\u0430\u043b\u044c\u043d\u044b\u0439 \u0430\u043d\u0430\u043b\u0438\u0437 \u0434\u0430\u043d\u043d\u044b\u0445 (data mining \u0438 data wrangling). \u041e\u043d \u043f\u043e\u043c\u043e\u0433\u0430\u0435\u0442 \u0432\u044b\u044f\u0432\u043b\u044f\u0442\u044c \u043d\u0435\u0434\u043e\u0441\u0442\u0430\u044e\u0449\u0438\u0435 \u0437\u043d\u0430\u0447\u0435\u043d\u0438\u044f \u0438 \u043f\u0430\u0442\u0442\u0435\u0440\u043d\u044b \u0432 \u0441\u0442\u0440\u0443\u043a\u0442\u0443\u0440\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u044b\u0445 […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-93155","post","type-post","status-publish","format-standard","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t