Mawu awiriwa a sayansi ya data amasokoneza anthu ambiri. Data Mining nthawi zambiri samamvetsetsa ngati kuchotsa ndi kubweza deta, koma zoona zake zimakhala zovuta kwambiri. Mu positiyi, tiyeni tiyike zomaliza pa Mining ndikupeza kusiyana pakati pa Kukumba kwa Data ndi Kuchotsa Data.
Kodi Data Mining ndi chiyani?
Data mining, amatchedwanso Kudziwa Kudziwa mu Database (KDD), ndi njira yomwe nthawi zambiri imagwiritsidwa ntchito posanthula deta yochuluka pogwiritsa ntchito njira zowerengera ndi masamu kuti apeze machitidwe obisika kapena machitidwe ndi kuchotsa mtengo kuchokera kwa iwo.
Kodi mungatani ndi Data Mining?
Pakupanga ndondomeko, imatha kusanthula ma database ndikuzindikira bwino mawonekedwe obisika. Kwa mabizinesi, migodi ya data nthawi zambiri imagwiritsidwa ntchito kuzindikira machitidwe ndi maubale mu data kuti athandizire kupanga zisankho zabwino zamabizinesi.
Zitsanzo za ntchito
Pambuyo pa migodi ya deta inakula kwambiri m'zaka za m'ma 1990, makampani omwe ali m'mafakitale osiyanasiyana, kuphatikizapo malonda, ndalama, chithandizo chamankhwala, kayendetsedwe ka mauthenga, mauthenga, e-commerce, ndi zina zotero, anayamba kugwiritsa ntchito njira zopangira migodi kuti apeze zambiri zokhudzana ndi deta. Kugulitsa kwa data kungathandize kugawa makasitomala, kuzindikira zachinyengo, kugulitsa zolosera, ndi zina zambiri.
- Gawo lamakasitomala
Posanthula zambiri zamakasitomala ndikuzindikira mawonekedwe amakasitomala omwe akuwafuna, makampani amatha kuwalunjika m'gulu losiyana ndikupereka zopereka zapadera zomwe zimakwaniritsa zosowa zawo. - Market Basket Analysis
Njira imeneyi imachokera pa chiphunzitso chakuti ngati mutagula gulu linalake lazinthu, mumatha kugula gulu lina lazinthu. Chitsanzo chimodzi chodziwika bwino: abambo akamagulira ana awo matewera, amakonda kugula mowa pamodzi ndi matewera. - Zolosera Zamalonda
Izi zingawoneke ngati zofanana ndi kusanthula basket basket, koma nthawi ino kusanthula deta kumagwiritsidwa ntchito kulosera pamene kasitomala adzagulanso mankhwala m'tsogolomu. Mwachitsanzo, mphunzitsi amagula chitini cha mapuloteni, chomwe chiyenera kukhala kwa miyezi 9. Sitolo yogulitsa puloteni iyi ikukonzekera kumasula yatsopano m'miyezi 9 kotero kuti wophunzitsa adzagulanso. - Kuzindikira Zachinyengo
Kuyika deta kumathandizira pamapangidwe omanga kuti azindikire zachinyengo. Potolera zitsanzo za malipoti achinyengo komanso ovomerezeka, mabizinesi amapatsidwa mphamvu kuti adziwe zomwe zikukayikitsa. - Kuzindikira kwa machitidwe pakupanga
M'makampani opanga zinthu, migodi ya data imagwiritsidwa ntchito kuthandizira kupanga dongosolo pozindikira mgwirizano pakati pa kapangidwe kazinthu, mbiri ndi zosowa za makasitomala. Kutsitsa kwa data kungathenso kulosera nthawi yopangira zinthu komanso mtengo wake.
Ndipo izi ndi zochepa chabe zogwiritsa ntchito migodi ya data.
Deta Mining Magawo
Kuyika deta ndi njira yonse yosonkhanitsira, kusankha, kuyeretsa, kusintha ndi kuchotsa deta kuti muwunikire machitidwe ndikuchotsa mtengo.
Monga lamulo, njira yonse yopangira migodi ya data imatha kufotokozedwa mwachidule m'magawo 7:
- Kuyeretsa deta
M'dziko lenileni, deta nthawi zonse imatsukidwa ndi kukonzedwa. Nthawi zambiri zimakhala zaphokoso, zosakwanira, ndipo zimatha kukhala ndi zolakwika. Kuonetsetsa kuti zotsatira za migodi ya deta ndi zolondola, choyamba muyenera kuyeretsa deta. Njira zina zoyeretsera zimaphatikizapo kudzaza zomwe zikusowa, kuyang'ana pamanja ndi pamanja, ndi zina. - Kuphatikiza kwa data
Iyi ndi sitepe yomwe deta yochokera kuzinthu zosiyanasiyana imachotsedwa, ikuphatikizidwa ndi kuphatikizidwa. Zochokera zitha kukhala nkhokwe, mafayilo amawu, maspredishiti, zikalata, ma data amitundu yosiyanasiyana, intaneti, ndi zina zotero. - Sampling ya data
Nthawi zambiri, sizinthu zonse zophatikizika zomwe zimafunikira pakupangira migodi. Sampling ya data ndi gawo lomwe deta yothandiza yokha imasankhidwa ndikuchotsedwa ku database yayikulu. - Kusintha kwa Data
Deta ikasankhidwa, imasinthidwa kukhala mafomu oyenera migodi. Izi zikuphatikizapo normalization, aggregation, generalization, etc. - Data Mining
Apa pakubwera gawo lofunika kwambiri la migodi ya data - kugwiritsa ntchito njira zanzeru kuti mupeze mawonekedwe momwemo. Njirayi imaphatikizapo kuyambiranso, kusanja, kulosera, kusanja, kuphunzira kuyanjana, ndi zina zambiri. - Kuwunika kwachitsanzo
Gawoli likufuna kuzindikira njira zomwe zingakhale zothandiza, zosavuta kuzimvetsetsa, komanso zochirikiza zongoyerekeza. - Chidziwitso choyimira
Pamapeto pake, zomwe zapezedwa zimaperekedwa mwanjira yowoneka bwino pogwiritsa ntchito chiwonetsero chazidziwitso ndi njira zowonera.
Kuipa kwa Data Mining
- Kugulitsa kwakukulu kwa nthawi ndi ntchito
Popeza migodi ya data ndi njira yayitali komanso yovuta, imafunikira ntchito zambiri kuchokera kwa anthu ogwira ntchito komanso aluso. Ogwiritsa ntchito data amatha kugwiritsa ntchito zida zamphamvu zopangira migodi, koma amafuna akatswiri kuti akonzekeretse deta ndikumvetsetsa zotsatira zake. Zotsatira zake, zingatenge nthawi kuti zidziwitso zonse zitheke. - Zazinsinsi ndi chitetezo cha data
Popeza migodi ya data imasonkhanitsa zambiri zamakasitomala kudzera munjira zamsika, zitha kuphwanya zinsinsi za ogwiritsa ntchito. Kuphatikiza apo, owononga amatha kupeza deta yosungidwa m'makina opangira migodi. Izi zikuwopseza chitetezo cha data yamakasitomala. Ngati deta yabedwa ikugwiritsidwa ntchito molakwika, ikhoza kuvulaza ena mosavuta.
Pamwambapa ndikufotokozera mwachidule za migodi ya data. Monga ndanenera kale, migodi ya deta imaphatikizapo njira yosonkhanitsa ndi kuphatikizira deta, yomwe imaphatikizapo ndondomeko yochotsa deta. Pankhaniyi, ndi bwino kunena kuti m'zigawo za deta akhoza kukhala mbali ya ndondomeko yaitali migodi deta.
Kodi Kuchotsa Data ndi chiyani?
Zomwe zimatchedwanso "web data mining" ndi "web scraping," njirayi ndi ntchito yochotsa deta kuchokera (nthawi zambiri yosasinthika kapena yosakonzedwa bwino) kumalo apakati ndikuwayika pamalo amodzi kuti asungidwe kapena kukonzedwanso. Makamaka, magwero osakhazikika a data amaphatikiza masamba, imelo, zikalata, mafayilo a PDF, zolemba zosakanizidwa, malipoti a mainframe, mafayilo a reel-to-reel, zotsatsa, ndi zina zambiri. Zosungirako zapakati zitha kukhala zakumalo, zamtambo, kapena zosakanizidwa. Ndikofunika kukumbukira kuti kuchotsa deta sikuphatikizapo kukonza kapena kusanthula kwina komwe kungachitike pambuyo pake.
Kodi mungatani ndi Data Extraction?
Kwenikweni, zolinga zochotsa deta zimagwera m'magulu atatu.
- Kusungidwa
Kuchotsa deta kungathe kusintha deta kuchokera ku maonekedwe enieni: mabuku, nyuzipepala, ma invoice kukhala mawonekedwe a digito, monga nkhokwe zosungirako kapena zosunga zobwezeretsera. - Kusintha mawonekedwe a data
Mukafuna kusamutsa deta kuchokera kutsamba lanu lamakono kupita kumalo atsopano omwe akukonzedwa, mukhoza kusonkhanitsa deta kuchokera pa tsamba lanu pochotsa. - Kusanthula deta
Kusanthula kowonjezereka kwa deta yochotsedwa kuti mudziwe zambiri ndizofala. Izi zingawoneke ngati zofanana ndi migodi ya deta, koma kumbukirani kuti migodi ya deta ndi cholinga cha migodi ya deta, osati mbali yake. Komanso, deta imawunikidwa mosiyana. Chitsanzo chimodzi: Eni ake ogulitsa pa intaneti amachotsa zidziwitso zamalonda kuchokera kumasamba a e-commerce monga Amazon kuti aziwunika njira za omwe akupikisana nawo munthawi yeniyeni. Monga migodi ya data, kuchotsa deta ndi njira yokhayo yomwe ili ndi ubwino wambiri. Mβmbuyomu, anthu ankakonda kukopera ndi kumata deta pamanja kuchokera kumalo ena kupita kwina, zomwe zinkatenga nthawi yambiri. Kuchotsa deta kumafulumizitsa kusonkhanitsa ndikuwongolera kwambiri kulondola kwazomwe zachotsedwa.
Zitsanzo zina zogwiritsira ntchito Data Extraction
Mofanana ndi migodi ya deta, migodi ya deta imagwiritsidwa ntchito kwambiri m'mafakitale osiyanasiyana. Kuphatikiza pa kuyang'anira mitengo pamalonda a e-commerce, migodi ya data imatha kukuthandizani pakufufuza kwanu, kuphatikizira nkhani, kutsatsa, kugulitsa nyumba, kuyenda ndi zokopa alendo, kufunsira, ndalama ndi zina zambiri.
- Mbadwo wotsogolera
Makampani amatha kutulutsa zambiri kuchokera kumakanema: Yelp, Crunchbase, Yellowpages ndikupanga zotsogola zachitukuko chabizinesi. Mutha kuwona kanema pansipa kuti mudziwe momwe mungatulutsire deta ku Yellowpages pogwiritsa ntchito . - Kuphatikiza zomwe zili ndi nkhani
Mawebusaiti ophatikiza zinthu amatha kulandira deta pafupipafupi kuchokera kumagwero angapo ndikusunga masamba awo kuti asinthe. - Kusanthula Maganizo
Potenga ndemanga, ndemanga, ndi ndemanga kuchokera pamasamba ochezera a pa Intaneti monga Instagram ndi Twitter, akatswiri amatha kusanthula zomwe zili m'munsimu ndikupeza chidziwitso cha momwe mtundu, malonda, kapena zochitika zimazindikiridwa.
Njira Zochotsera Data
Kuchotsa deta ndi gawo loyamba la ETL (chidule cha Extract, Transform, Load) ndi ELT (kutulutsa, katundu ndi kusintha). ETL ndi ELT nawonso ndi gawo la njira yophatikizira deta. Mwa kuyankhula kwina, kuchotsa deta kungakhale mbali ya migodi ya deta.
Chotsani, tembenuzani, tsegulani
Ngakhale migodi ya deta ikukhudza kuchotsa zambiri kuchokera kuzinthu zambiri, kuchotsa deta ndi njira yaifupi komanso yosavuta. Ikhoza kuchepetsedwa kukhala magawo atatu:
- Kusankha gwero la data
Sankhani gwero lomwe mukufuna kuchotsamo, monga tsamba lawebusayiti. - Kusonkhanitsa deta
Tumizani pempho la "GET" patsambali ndikuwonetsani chikalata cha HTML chotsatira pogwiritsa ntchito zilankhulo zamapulogalamu monga Python, PHP, R, Ruby, ndi zina. - Kusunga deta
Sungani deta mu database yanu yapafupi kapena kusungirako mitambo kuti mugwiritse ntchito mtsogolo. Ngati ndinu katswiri wodziwa mapulogalamu omwe akufuna kuchotsa deta, njira zomwe zili pamwambazi zingawoneke zosavuta kwa inu. Komabe, ngati mulibe code, njira yachidule ndiyo kugwiritsa ntchito zida zochotsa deta, mwachitsanzo. . Zida zochotsera zidziwitso, monga zida zochepetsera deta, zidapangidwa kuti zisunge mphamvu ndikupangitsa kuti ntchito ikhale yosavuta kwa aliyense. Zida izi sizongowonjezera ndalama komanso zokomera oyamba kumene. Amalola ogwiritsa ntchito kusonkhanitsa zidziwitso mkati mwa mphindi, kuzisunga mumtambo ndikuzitumiza kumitundu yambiri: Excel, CSV, HTML, JSON kapena patsamba lawebusayiti kudzera pa API.
Kuipa kwa Data m'zigawo
- Kuwonongeka kwa seva
Mukatenga zambiri pamlingo waukulu, seva yapaintaneti yomwe mukufuna ikhoza kudzaza, zomwe zingapangitse seva kugwa. Izi zidzawononga zokonda za eni webusayiti. - Kuletsa ndi IP
Munthu akasonkhanitsa zambiri pafupipafupi, masamba amatha kutsekereza ma adilesi awo a IP. Chothandiziracho chikhoza kukana kwathunthu adilesi ya IP kapena kuchepetsa mwayi wofikira, kupangitsa kuti deta ikhale yosakwanira. Kuti mutenge deta ndikupewa kutsekereza, muyenera kuchita pa liwiro laling'ono ndikugwiritsa ntchito njira zotsutsana ndi kutsekereza. - Mavuto ndi malamulo
Kuchotsa deta kuchokera pa intaneti kumagwera m'dera la imvi pankhani yovomerezeka. Masamba akulu monga Linkedin ndi Facebook amafotokoza momveka bwino momwe amagwiritsidwira ntchito kuti kuchotsa deta yodziwikiratu ndikoletsedwa. Pakhala pali milandu yambiri pakati pamakampani chifukwa cha zochita za bot.
Kusiyanitsa Kwakukulu Pakati pa Data Mining ndi Data Extraction
- Kutsitsa deta kumatchedwanso kupezedwa kwa chidziwitso m'madatabase, kuchotsa chidziwitso, kusanthula deta/chitsanzo, kusonkhanitsa zidziwitso. Kuchotsa deta kumagwiritsidwa ntchito mofanana ndi kuchotsa deta pa intaneti, kukwawa kwa intaneti, migodi ya deta, ndi zina zotero.
- Kafukufuku wa migodi ya data makamaka amachokera kuzinthu zosanjidwa bwino, pamene migodi ya data nthawi zambiri imatengedwa kuchokera kuzinthu zosalongosoka kapena zosalongosoka bwino.
- Cholinga cha migodi ya data ndikupangitsa kuti deta ikhale yothandiza kwambiri pakuwunika. Kuchotsa deta ndikusonkhanitsa deta kumalo amodzi komwe ingasungidwe kapena kukonzedwa.
- Kusanthula mumigodi ya data kumatengera njira zamasamu zodziwira masinthidwe kapena zochitika. Kutulutsa kwa data kumatengera zilankhulo zamapulogalamu kapena zida zochotsera zidziwitso zokwawa komwe kumayambira.
- Cholinga cha migodi ya deta ndikupeza mfundo zomwe poyamba sizinkadziwika kapena kunyalanyazidwa, pamene kuchotsa deta kumagwirizana ndi zomwe zilipo kale.
- Kuyika deta ndizovuta kwambiri ndipo kumafuna ndalama zambiri pophunzitsa anthu. Kuchotsa deta, mukagwiritsidwa ntchito ndi chida choyenera, kungakhale kosavuta komanso kopanda mtengo.
Timathandiza oyamba kumene kuti asasokonezeke mu Data. Tapanga khodi yotsatsira makamaka okhala ku Khabra HABR, kupereka kuchotsera kwa 10% kuchotsera komwe kwasonyezedwa pachikwangwanicho.
Maphunziro ambiri
Nkhani Zowonetsedwa
Source: www.habr.com