Hazvibviri kutsanangura chikonzero nei ndakaverenga izvi. Ndainge ndave nenguva uye ndaifarira kuti musika unoshanda sei. Uye iyi yatove musika wakazara sekureva kwaGartner kubvira 2018. Kubva 2014-2016 yakanzi advanced analytics (midzi muBI), muna 2017 - Data Science (ini handizivi kuti ndingashandura sei izvi muchiRussia). Kune avo vanofarira mafambiro evatengesi vakatenderedza sikweya, munogona tarisa. Uye ini ndichataura nezve 2020 square, kunyanya sezvo shanduko ipapo kubva 2019 idiki: SAP yakatama uye Altair yakatenga Datawatch.
Uku hakusi kuongorora kwakarongeka kana tafura. Maonero ega ega, zvakare kubva pakuona kwe geophysicist. Asi ini ndinogara ndichida kuverenga Gartner MQ, ivo vanogadzira mamwe mapoinzi zvakakwana. Saka hezvino zvinhu zvandakateerera kune zvese zvehunyanzvi, huchenjeri hwemusika, uye huzivi.
Izvi hazvisi zvevanhu vakadzama mumusoro weML, asi kune vanhu vanofarira izvo zvinowanzoitika mumusika.
Iyo DSML musika pachayo zvine musoro matendere pakati peBI uye Cloud AI yekuvandudza masevhisi.
Favourite quotes uye mazwi kutanga:
- "Mutungamiri anogona kunge asiri sarudzo yakanaka" - Mutungamiriri wemusika handizvo zvauri kuda. Chimbi-chimbi! Semhedzisiro yekushaikwa kwemutengi anoshanda, ivo vanogara vachitsvaga "yakanakisa" mhinduro, pane iyo "yakakodzera" imwe.
- "Model operationalization" - yakapfupikiswa seMOPs. Uye munhu wese ane nguva yakaoma nemapugs! - (inotonhorera pug theme inoita kuti modhi ishande).
- "Notebook environment" ipfungwa yakakosha apo kodhi, makomendi, data nemhedzisiro zvinouya pamwechete. Izvi zvakajeka, zvinovimbisa uye zvinogona kuderedza zvakanyanya huwandu hweUI kodhi.
- "Yakadzika midzi muOpenSource" - yakataurwa zvakanaka - inotora midzi mune yakavhurika sosi.
- "Citizen Data Scientists" - akadaro akareruka dudes, vakadaro vanochema, kwete nyanzvi, vanoda nharaunda yekuona uye marudzi ese ezvinhu ebetsero. Havazonyore.
- "Democracy" - inowanzo shandiswa kureva "kuita kuti kuwanikwa kune vanhu vakasiyana-siyana." Tinogona kuti "democratize iyo data" pachinzvimbo chenjodzi "yemahara iyo data" yataimboshandisa. "Democratise" inogara iri muswe wakareba uye vatengesi vese vanomhanya shure kwayo. Kurasikirwa neruzivo rwakasimba - kuwana mukuwanikwa!
- "Exploratory Data Analysis - EDA" - kutariswa kweidzi nzira dziripo. Dzimwe nhamba. Kuona zvishoma. Chinhu icho munhu wese anoita kune imwe dhigirii kana imwe. Handaiziva kuti pane zita reizvi
- "Reproducibility" - Kuchengetedzwa kwakanyanya kweese maparamendi ezvakatipoteredza, zvinopinda uye zvinobuda kuitira kuti kuedza kudzokororwe kana kwaitwa. Izwi rinonyanya kukosha renzvimbo yekuyedza bvunzo!
Saka:
Alteryx
Inotonhorera interface, kungofanana netoyi. Scalability, hongu, yakaoma zvishoma. Saizvozvo, iyo Citizen nharaunda yeinjiniya yakatenderedza zvakafanana netchotchkes kutamba. Analytics ndeyako yese mubhodhoro rimwe. Zvakandiyeuchidza nezve spectral-correlation data analysis , iyo yakatangwa muma90s.
Anaconda
Nharaunda yakatenderedza Python uye R nyanzvi. Open source yakakura saizvozvo. Zvakazoitika kuti vandinoshanda navo vanoishandisa nguva dzese. Asi ndakanga ndisingazivi.
DataBricks
Inoumbwa nemapurojekiti matatu akavhurika - vagadziri veSpark vakasimudza gehena remari yakawanda kubva 2013. Ini ndinofanira kunyatso tora wiki:
"Muna Gunyana 2013, Databricks yakazivisa kuti yakanga yakwidza $13.9 miriyoni kubva kuna Andreessen Horowitz. Iyo kambani yakasimudza mamwe madhora mazana matatu emamiriyoni muna 33, madhora mazana matanhatu emamiriyoni muna 2014, madhora zana nemakumi mana emamiriyoni muna 60, madhora mazana maviri nemakumi mashanu muna 2016 (Kukadzi) uye mazana mana emamiriyoni emadhora muna 140 (Gumiguru) β!!!
Vamwe vanhu vakuru vanocheka Spark. Handizivi, sorry!
Uye mapurojekiti ndeaya:
- Delta Lake - ACID paSpark ichangobva kuburitswa (zvatakarota nezvazvo neElasticsearch) - inoishandura kuita dhatabhesi: rigid schema, ACID, kuongororwa, shanduro ...
- ML Kuyerera -kutevera, kurongedza, manejimendi uye kuchengetedza mamodheru.
- koalas - Pandas DataFrame API paSpark - Pandas - Python API yekushanda nematafura uye data zvakazara.
Unogona kutarisa Spark kune avo vasingazive kana kukanganwa: . Ndakatarisa mavhidhiyo ane mienzaniso kubva kune zvishoma zvinofinha asi zvakadzama kubvunza huni: DataBricks yeData Science () uye yeData Engineering ().
Muchidimbu, Databricks inoburitsa Spark. Chero ani anoda kushandisa Spark kazhinji ari mugore anotora DataBricks pasina kuzeza, sezvakarongwa π Spark ndiye musiyanisi mukuru pano.
Ndakadzidza kuti Spark Streaming haisi yekunyepera realtime kana microbatching. Uye kana iwe uchida chaiyo Chaiyo nguva, iri muApache STORM. Munhu wese anotaurawo uye anonyora kuti Spark iri nani pane MapReduce. Iyi ndiyo slogan.
DATAIKU
Chinhu chinotonhorera-chekupedzisira. Kune zvakawanda zvekushambadzira. Handinzwisise kuti yakasiyana sei neAlteryx?
DataRobot
Paxata yekugadzirira data ikambani yakaparadzana yakatengwa neData Robots muna Zvita 2019. Takasimudza 20 MUSD tikatengesa. Zvese mumakore manomwe.
Kugadzirira kwedata muPaxata, kwete Excel - ona pano: .
Kune otomatiki lookups uye zvikumbiro zvekujoinha pakati pemataseti maviri. Chinhu chikuru - kuti unzwisise iyo data, paizove nekunyanya kukoshesa ruzivo rwemavara ().
Data Catalog ibhuku rakanakisa rezvisina basa "live" dataset.
Zvinonakidzawo kuti madhairekitori anoumbwa sei muPaxata ().
βMaererano nekambani yaanalyst , software inogoneka kuburikidza nekufambira mberi mukati , uye data caching methodology. Iyo software inoshandisa maalgorithms ekunzwisisa zvinorehwa nemakoramu etafura yedata uye maalgorithms ekuzivikanwa kwepateni kuti uwane angangove akadzokorora mune data-set. Inoshandisawo indexing, kucherechedzwa kwemavara mameseji uye mamwe matekinoroji anowanzo kuwanikwa mumasocial media uye software yekutsvaga. "
Data Robot's main product is . sirogani yavo inobva kuModel to Enterprise Application! Ndakawana kubvunza kune indasitiri yemafuta zvine chekuita nedambudziko, asi zvaive zvakanyanyisa uye zvisingafadze: . Ndakaona mavhidhiyo avo paMops kana MLops () Iyi iFrankenstein yakadaro yakaungana kubva ku6-7 kutora kwezvinhu zvakasiyana-siyana.
Ehe, zvinova pachena kuti timu hombe yeData Scientists inofanirwa kunge iine nharaunda yakadaro yekushanda nemamodheru, zvikasadaro ivo vanozoburitsa akawanda avo uye havambofi vakaisa chero chinhu. Uye mune yedu oiri uye gasi kumusoro kwemvura chokwadi, dai chete taigona kugadzira imwe yakabudirira modhi, ingave kufambira mberi kukuru!
Maitiro acho pachawo ainyanya kuyeuchidza nezvebasa nemagadzirirwo masisitimu mune geology-geophysics, semuenzaniso . Wese munhu asina usimbe anogadzira uye anogadzirisa mamodheru. Unganidza data mumuenzaniso. Vakabva vagadzira referensi modhi ndokuitumira kune kugadzirwa! Pakati, toti, geological modhi uye ML modhi, unogona kuwana zvakawanda zvakafanana.
Domino
Kusimbisa papuratifomu yakavhurika uye kubatana. Vashandisi vebhizinesi vanobvumidzwa mahara. Data Lab yavo yakafanana chaizvo nesharepoint. (Uye zita rinorova zvakanyanya IBM). Zvese zviedzo zvinobatanidza kune yekutanga dataset. Izvi zvinozivikanwa sei :) Sezvatinoita mukuita kwedu - imwe data yakadhonzwa mumuenzaniso, ichibva yacheneswa uye yakaiswa mukurongeka mumuenzaniso, uye izvi zvose zvinogara imomo mumuenzaniso uye migumo haigone kuwanikwa mune data data. .
Domino ine inotonhorera zvivakwa virtualization. Ndakaunganidza muchina wacho macores akawanda sezvinodiwa musekondi ndokuenda kunoverenga. Kuti zvakaitwa sei hazvisi kujeka. Docker iri kwese kwese. Rusununguko rwakawanda! Chero nzvimbo dzekushandira dzeshanduro dzichangoburwa dzinogona kubatanidzwa. Parallel kutanga kuedza. Kutevera uye kusarudzwa kwevakabudirira.
Zvakafanana neDataRobot - mhedzisiro inoburitswa kune vashandisi vebhizinesi muchimiro chekushandisa. Kune kunyanya vane zvipo "vane chekuita". Uye kushandiswa chaiko kwemamodheru kunotariswa zvakare. Zvese zvePugs!
Handinzwisisi zvizere kuti mamodheru akaoma anoguma sei mukugadzirwa. Imwe mhando yeAPI inopihwa kuvadyisa data uye kuwana mhinduro.
H2O
Driveless AI ndeye compact uye intuitive system yeYakatariswa ML. Zvese mubhokisi rimwe. Hazvina kunyatsojeka pakarepo nezve backend.
Iyo modhi inoiswa otomatiki muREST server kana Java App. Iyi ipfungwa huru. Zvakawanda zvakaitwa zvekududzira nekutsanangudza. Dudziro uye tsananguro yemhedzisiro yemuenzaniso (Chii chisingafanirwe kutsanangurwa, kana zvisina kudaro munhu anogona kuverenga zvakafanana?).
Kekutanga, nyaya yekudzidza nezve data isina kurongeka uye . Mufananidzo wepamusoro wekuvaka. Uye kazhinji ndaifarira mifananidzo.
Iko kune yakakura yakavhurika sosi H2O dhizaini isina kujeka zvachose (seti yealgorithms / maraibhurari?). Yako wega yekuona laptop isina programming seJupiter () Ini zvakare ndakaverenga nezve Pojo uye Mojo - H2O mhando dzakaputirwa muJava. Yekutanga yakatwasuka, yechipiri ine optimization. H20 ndidzo chete(!) kune Gartner akanyora zvinyorwa analytics uye NLP semasimba avo, pamwe nekuedza kwavo maererano neKutsanangurika. Chinokosha zvikuru!
Munzvimbo imwechete: kuita kwepamusoro, optimization uye indasitiri chiyero mumunda wekubatanidza nehardware nemakore.
Uye kushaya simba kune musoro - Driverles AI haina simba uye yakamanikana ichienzaniswa neyakavhurika sosi yavo. Kugadzirira kwedata kwakaremara kana kuenzaniswa nePaxata! Uye ivo vanofuratira data remaindasitiri - rukova, girafu, geo. Zvakanaka, zvese hazvigone kungoita zvakanaka.
KNIME
Ndakafarira 6 chaiyo, inonakidza bhizinesi kesi pane iyo huru peji. Yakasimba OpenSource.
Gartner akavadzikisira kubva kuvatungamiriri kuenda kuva vaoni. Kuwana mari zvisizvo chiratidzo chakanaka kune vashandisi, zvakapihwa kuti Mutungamiriri haagari ari sarudzo yakanaka.
Izwi rakakosha, senge muH2O, rinowedzerwa, zvinoreva kubatsira varombo vagari data masayendisiti. Aka ndekekutanga mumwe munhu kutsoropodzwa nekuda kwekuita muongororo! Inonakidza? Ndiko kuti, kune simba rakawanda rekombuta zvekuti kuita hakugone kuve dambudziko rehurongwa zvachose? Gartner ane nezve izwi iri "Augmented" , iyo yakanga isingasvikike.
Uye KNIME inoita kunge yekutanga isiri-America muongororo! (Uye vagadziri vedu vakanyatsofarira peji yavo yekumhara. Strange people.
MathWorks
MatLab ishamwari yekare inokudzwa inozivikanwa kune wese munhu! Mabhokisi emidziyo enzvimbo dzese dzehupenyu uye mamiriro. Chimwe chinhu chakasiyana zvikuru. Muchokwadi, mijenya uye yakawanda uye yakawanda yemasvomhu ezvese muhupenyu!
Chigadzirwa cheSimulink chekuwedzera chekugadzira system. Ndakachera mumabhokisi emidziyo yeDhijitari Mapatya - ini hapana chandinonzwisisa nezvazvo, asi zvakawanda zvakanyorwa. For . Kazhinji, ichi chigadzirwa chakasiyana kubva pakadzika kwemasvomhu uye engineering. Kusarudza chaiwo mathematics toolkits. Sekureva kwaGartner, matambudziko avo akafanana neaya einjiniya akangwara - hapana kubatana - munhu wese anotenderera nemuenzaniso wake, hapana democracy, hapana kujeka.
RapidMiner
Ndakauya ndikanzwa zvakawanda kare (pamwe naMatlab) mumamiriro ezvinhu akanaka akavhurika sosi. Ndakachera zvishoma muTurboPrep semazuva ese. Ndiri kufarira nzira yekuwana data yakachena kubva kune tsvina data.
Zvekare iwe unogona kuona kuti vanhu vakanaka zvichibva pane zvekutengesa zve2018 uye vanhu vanotyisa vanotaura Chirungu pane iyo demo demo.
Uye vanhu vanobva kuDortmund kubvira 2001 vane simba reGerman kumashure)
Ini handisati ndanzwisisa kubva kune saiti kuti chii chaizvo chinowanikwa mune yakavhurika sosi - iwe unofanirwa kuchera zvakadzika. Mavhidhiyo akanaka nezve deployment uye AutoML concepts.
Iko hakuna chinhu chakakosha nezve RapidMiner Server backend kana. Iyo inogona kunge iri compact uye inoshanda nemazvo pane premium kunze kwebhokisi. Iyo yakarongedzwa muDocker. Yakagovaniswa nharaunda chete paRapidMiner server. Uye zvakare kune Radoop, data kubva kuHadoop, kuverenga nhetembo kubva kuSpark muStudio workflow.
Sezvaitarisirwa, vatengesi vechiduku vanopisa βvatengesi vetsvimbo dzine mitsetseβ vakadziisa pasi. Gartner, zvisinei, anofanotaura kubudirira kwavo mune ramangwana munzvimbo yeBhizinesi. Unogona kuunganidza mari ipapo. VaGermany vanoziva kuita izvi, zvitsvene-zvitsvene :) Usataura nezveSAP !!!
Vanoita zvakawanda kune vagari vemo! Asi kubva pane peji iwe unogona kuona kuti Gartner anoti ivo vari kunetseka nekutengesa innovation uye havasi kurwira hupamhi hwekuvhara, asi kuti purofiti.
Kugara SAS ΠΈ Tibco vatengesi veBI kwandiri ... Uye vese vari pamusoro, izvo zvinosimbisa chivimbo changu chekuti yakajairika DataScience iri kukura zvine musoro.
kubva kuBI, uye kwete kubva kumakore uye Hadoop zvivakwa. Kubva kubhizinesi, ndiko, uye kwete kubva kuIT. Kufanana muGazpromneft semuenzaniso: , Nzvimbo yeDSML yakakura inokura kubva kune yakasimba BI maitiro. Asi pamwe iri smacky uye yakarerekera kune MDM nezvimwe zvinhu, ndiani anoziva.
SAS
Hapana zvakawanda zvekutaura. Zvinhu zviri pachena chete.
TIBCO
Iyo nzira inoverengwa mune yekutenga rondedzero pane peji-refu Wiki peji. Hongu, nyaya refu, asi 28 !!! Charles. Ndakatenga BI Spotfire (2007) kumashure kwangu techno-youth. Uye zvakare kushuma kubva kuJaspersoft (2014), zvino vanosvika vatatu vanofungidzira analytics vatengesi Insightful (S-plus) (2008), Statistica (2017) uye Alpine Data (2017), chiitiko kugadzirisa uye kutenderera Streambase System (2013), MDM Orchestra Networks (2018) uye Snappy Data (2019) mu-memory platform.
Mhoro Frankie!
Source: www.habr.com