Ndizosatheka kufotokoza chifukwa chomwe ndimawerengera izi. Ndinangokhala ndi nthawi ndipo ndinali ndi chidwi ndi momwe msika umagwirira ntchito. Ndipo uwu ndi msika wathunthu malinga ndi Gartner kuyambira 2018. Kuchokera ku 2014-2016 amatchedwa analytics apamwamba (mizu mu BI), mu 2017 - Data Science (sindikudziwa kumasulira izi mu Russian). Kwa iwo omwe ali ndi chidwi ndi kayendedwe ka ogulitsa kuzungulira bwaloli, mutha yang'anani. Ndipo ndilankhula za lalikulu la 2020, makamaka popeza zosintha kumeneko kuyambira 2019 ndizochepa: SAP idachoka ndipo Altair adagula Datawatch.
Uku si kusanthula mwadongosolo kapena tebulo. Kawonedwe kayekha, komanso kuchokera pamalingaliro a geophysicist. Koma nthawi zonse ndimakonda kuwerenga Gartner MQ, amapanga mfundo zina mwangwiro. Chifukwa chake nazi zinthu zomwe ndidasamala nazo zonse mwaukadaulo, mwanzeru zamsika, komanso mwanzeru.
Izi si za anthu omwe ali mozama pamutu wa ML, koma kwa anthu omwe ali ndi chidwi ndi zomwe zikuchitika pamsika.
Msika wa DSML wokha umakhala pakati pa BI ndi Cloud AI developer services.
Mawu omwe mumakonda komanso mawu oyamba:
- "Mtsogoleri sangakhale chisankho chabwino kwambiri" - Mtsogoleri wamsika sizomwe mukufunikira. Mwachangu kwambiri! Chifukwa cha kusowa kwa kasitomala wogwira ntchito, nthawi zonse amafunafuna njira "yabwino", osati "yoyenera".
- "Model ntchito" - achidule ngati MOPs. Ndipo aliyense amavutika ndi ma pugs! - (mutu wozizira wa pug umapangitsa kuti chitsanzocho chigwire ntchito).
- "Notebook chilengedwe" ndi lingaliro lofunikira pomwe code, ndemanga, deta ndi zotsatira zimabwera palimodzi. Izi ndizomveka bwino, zolonjeza ndipo zingathe kuchepetsa kwambiri chiwerengero cha UI code.
- "Mizu mu OpenSource" - zonenedwa bwino - zimakhazikika pagwero lotseguka.
- "Citizen Data Scientists" - ma dudes osavuta otero, amalira, osati akatswiri, omwe amafunikira malo owoneka bwino ndi mitundu yonse ya zinthu zothandizira. Iwo sangalembe.
- "Demokalase" - nthawi zambiri amagwiritsidwa ntchito kutanthauza "kupanga kupezeka kwa anthu osiyanasiyana." Titha kunena kuti "democratize data" m'malo mwa "kumasula deta" yomwe tinkagwiritsa ntchito. "Democratise" nthawi zonse imakhala mchira wautali ndipo ogulitsa onse amathamangira pambuyo pake. Kutaya chidziwitso champhamvu - kupeza mwayi!
- "Exploratory Data Analysis - EDA" - kulingalira za njira zomwe zilipo. Ziwerengero zina. Kuwona pang'ono. Chinachake chimene aliyense amachita ku digiri imodzi kapena imzake. Sindimadziwa kuti pali dzina la izi
- "Reproducibility" - Kusungidwa kwakukulu kwa magawo onse a chilengedwe, zolowetsa ndi zotulukapo kuti kuyesako kubwerezedwe kamodzi. Mawu ofunika kwambiri pa malo oyesera oyesera!
Kotero:
Alteryx
Kuzizira mawonekedwe, monga chidole. Scalability, ndithudi, ndizovuta pang'ono. Chifukwa chake, gulu la Citizen la mainjiniya omwe ali ndi ma tchotchkes oti azisewera. Analytics ndi yanu yonse mu botolo limodzi. Zandikumbutsa za kusanthula kwa data kowoneka bwino , yomwe idakhazikitsidwa m'ma 90s.
Anaconda
Anthu ozungulira Python ndi R akatswiri. Open source ndi yayikulu molingana. Zinapezeka kuti anzanga amazigwiritsa ntchito nthawi zonse. Koma sindimadziwa.
Zithunzi za DataBricks
Zili ndi mapulojekiti atatu otsegulira - oyambitsa Spark apeza ndalama zambiri kuyambira 2013. Ndiyenera kunena mawu a wiki:
"Mu Seputembala 2013, Databricks idalengeza kuti idakweza $13.9 miliyoni kuchokera kwa Andreessen Horowitz. Kampaniyo idakweza $33 miliyoni mu 2014, $60 miliyoni mu 2016, $140 miliyoni mu 2017, $250 miliyoni mu 2019 (Feb) ndi $400 miliyoni mu 2019 (Oct)β!!!
Anthu ena otchuka amadula Spark. Sindikudziwa, pepani!
Ndipo ma projekiti ndi:
- Delta Lake - ACID pa Spark idatulutsidwa posachedwa (zomwe timalota ndi Elasticsearch) - imasandutsa nkhokwe: schema yolimba, ACID, kuwunika, mitundu...
- Kuthamanga kwa ML - kutsatira, kulongedza, kuyang'anira ndi kusunga zitsanzo.
- koalas - Pandas DataFrame API pa Spark - Pandas - Python API yogwira ntchito ndi matebulo ndi deta yonse.
Mutha kuyang'ana Spark kwa omwe sakudziwa kapena ayiwala: . Ndidawonera makanema okhala ndi zitsanzo kuchokera ku zopala matabwa zotopetsa pang'ono koma zatsatanetsatane: DataBricks for Data Science () ndi Data Engineering ().
Mwachidule, Databricks imatulutsa Spark. Aliyense amene akufuna kugwiritsa ntchito Spark nthawi zonse pamtambo amatenga DataBricks mosazengereza, monga momwe amafunira π Spark ndiye wosiyanitsa wamkulu pano.
Ndinaphunzira kuti Spark Streaming si nthawi yeniyeni yeniyeni kapena microbatching. Ndipo ngati mukufuna Real Real Time, ili mu Apache STORM. Aliyense akunenanso ndikulemba kuti Spark ndiyabwino kuposa MapReduce. Ili ndiye slogan.
DATAIKU
Chinthu chozizira chakumapeto. Pali zambiri zotsatsa. Sindikumvetsa kuti zimasiyana bwanji ndi Alteryx?
DataRobot
Paxata yokonzekera deta ndi kampani ina yomwe idagulidwa ndi Data Robots mu Disembala 2019. Tinakweza 20 MUSD ndikugulitsa. Zonse m'zaka 7.
Kukonzekera kwa data ku Paxata, osati Excel - onani apa: .
Pali zofufuza zokha ndi malingaliro olumikizirana pakati pa ma dataset awiri. Chinthu chachikulu - kumvetsetsa zambiri, pangakhale kutsindika kwambiri pamawu ().
Data Catalog ndi mndandanda wabwino kwambiri wamaseti "amoyo" opanda ntchito.
Ndizosangalatsanso momwe zolemba zimapangidwira ku Paxata ().
"Malinga ndi analyst firm , pulogalamuyo imatheka chifukwa cha kupita patsogolo mu , ndi njira yosungira deta. Pulogalamuyi imagwiritsa ntchito ma aligorivimu kuti mumvetsetse tanthauzo la mizati ya tebulo la data ndi ma aligorivimu ozindikira mapateni kuti mupeze zobwereza zomwe zingatheke mu seti ya data. Imagwiritsanso ntchito indexing, kuzindikira malembedwe ndi matekinoloje ena omwe nthawi zambiri amapezeka pamasamba ochezera komanso mapulogalamu osakira. β
Chinthu chachikulu cha Data Robot ndi . Mawu awo akuchokera ku Model kupita ku Enterprise Application! Ndinapeza kufunsira kwa makampani amafuta okhudzana ndi vutoli, koma zinali zoletsedwa komanso zosasangalatsa: . Ndidawonera makanema awo pa Mops kapena MLops (). Izi ndi Frankenstein anasonkhana kuchokera 6-7 kugula zinthu zosiyanasiyana.
Inde, zikuwonekeratu kuti gulu lalikulu la Data Scientists liyenera kukhala ndi malo oterowo kuti azigwira ntchito ndi zitsanzo, apo ayi adzatulutsa zambiri ndipo sangatumize kalikonse. Ndipo mu zenizeni zathu zamafuta ndi gasi kumtunda, ngati titha kupanga chitsanzo chimodzi chopambana, chingakhale kupita patsogolo kwakukulu!
Njira yokhayo inali yokumbutsa kwambiri ntchito ndi machitidwe opangira mu geology-geophysics, mwachitsanzo . Aliyense amene si waulesi amapanga ndi kusintha zitsanzo. Sungani deta mu chitsanzo. Kenako adapanga choyimira ndikuchitumiza kukupanga! Pakati, titi, chitsanzo cha geological ndi chitsanzo cha ML, mungapeze zambiri zofanana.
Domino
Kugogomezera pa nsanja yotseguka ndi mgwirizano. Ogwiritsa ntchito bizinesi amaloledwa kwaulere. Data Lab yawo ndi yofanana kwambiri ndi sharepoint. (Ndipo dzinali limamenya kwambiri IBM). Zoyeserera zonse zimalumikizana ndi dataset yoyambirira. Izi ndizodziwika bwanji :) Monga momwe timachitira - deta ina inakokedwa mu chitsanzo, ndiye idatsukidwa ndikuyikidwa mu dongosolo, ndipo zonsezi zimakhala kale mu chitsanzo ndipo mapeto sangapezeke mu data yomwe imachokera. .
Domino ili ndi magwiridwe antchito abwino. Ndinasonkhanitsa makina ambiri momwe amafunikira mumphindi imodzi ndikupita kuwerengera. Sizikudziwikiratu mmene zinachitikira. Docker ili paliponse. Ufulu wambiri! Malo aliwonse ogwirira ntchito amitundu yaposachedwa akhoza kulumikizidwa. Kuyambitsa kofananira kwa zoyeserera. Kutsata ndi kusankha opambana.
Zomwezo monga DataRobot - zotsatira zimasindikizidwa kwa ogwiritsa ntchito malonda mu mawonekedwe a mapulogalamu. Kwa "okhudzidwa" omwe ali ndi mphatso. Ndipo kugwiritsidwa ntchito kwenikweni kwa zitsanzozo kumayang'aniridwanso. Zonse za Pugs!
Sindikumvetsa bwino momwe zitsanzo zovuta zimathera pakupanga. Mtundu wina wa API umaperekedwa kuti uziwadyetsa deta ndikupeza zotsatira.
H2O
Driveless AI ndi njira yophatikizika komanso yowoneka bwino ya Supervised ML. Zonse mu bokosi limodzi. Sizikudziwika bwino nthawi yomweyo za backend.
Mtunduwu umangoyikidwa mu seva ya REST kapena Java App. Ili ndi lingaliro labwino. Zambiri zachitidwa pa Kutanthauzira ndi Kufotokozera. Kutanthauzira ndi kufotokozera zotsatira za chitsanzo (Ndi chiyani chomwe sichiyenera kufotokozedwa, mwinamwake munthu akhoza kuwerengera zomwezo?).
Kwa nthawi yoyamba, kafukufuku wokhudza deta yosasinthika ndi . Chithunzi chapamwamba cha zomangamanga. Ndipo zambiri ndimakonda zithunzi.
Pali lalikulu lotseguka gwero H2O chimango kuti si bwino bwino (a aligorivimu / malaibulale?). Laputopu yanu yowonera popanda mapulogalamu ngati Jupiter (). Ndinawerenganso za mitundu ya Pojo ndi Mojo - H2O yokulungidwa mu Java. Yoyamba ndi yolunjika, yachiwiri ndi kukhathamiritsa. H20 ndi okhawo (!) kwa omwe Gartner adawalembera zowerengera ndi NLP monga mphamvu zawo, komanso zoyesayesa zawo zokhudzana ndi Kufotokozera. Ndizofunika kwambiri!
Pamalo omwewo: magwiridwe antchito apamwamba, kukhathamiritsa ndi muyezo wamakampani pantchito yophatikizika ndi zida ndi mitambo.
Ndipo kufooka ndikomveka - Driverles AI ndi yofooka komanso yopapatiza poyerekeza ndi gwero lawo lotseguka. Kukonzekera kwa data ndikopunduka poyerekeza ndi Paxata! Ndipo amanyalanyaza deta yamakampani - mtsinje, graph, geo. Chabwino, zonse sizingakhale bwino.
KNIME
Ndidakonda milandu 6 yeniyeni, yosangalatsa kwambiri patsamba lalikulu. Wamphamvu OpenSource.
Gartner adawatsitsa kuchoka pa atsogoleri kukhala owonera masomphenya. Kupeza ndalama movutikira ndichizindikiro chabwino kwa ogwiritsa ntchito, chifukwa chakuti Mtsogoleri si nthawi zonse kusankha bwino.
Mawu ofunikira, monga mu H2O, amawonjezeredwa, zomwe zikutanthauza kuthandiza asayansi osauka a data. Aka ndi koyamba kuti wina akudzudzulidwa chifukwa chakuchita bwino pakuwunika! Zosangalatsa? Ndiye kuti, pali mphamvu zambiri zamakompyuta zomwe magwiridwe antchito sangakhale vuto ladongosolo konse? Gartner ali ndi mawu akuti "Augmented" , zomwe sizikanatheka.
Ndipo KNIME ikuwoneka ngati yoyamba yosakhala yaku America pakuwunikanso! (Ndipo okonza athu adakonda kwambiri tsamba lawo lofikira. Anthu achilendo.
MathWorks
MatLab ndi mnzake wakale wolemekezeka yemwe amadziwika kwa aliyense! Mabokosi a zida zamagawo onse amoyo ndi zochitika. Chinachake chosiyana kwambiri. M'malo mwake, zambiri ndi masamu ambiri pa chilichonse m'moyo!
Chowonjezera cha Simulink pamapangidwe adongosolo. Ndinakumba m'mabokosi a zida za Digital Twins - sindikumvetsa kalikonse za izi, koma zambiri zalembedwa. Za . Mwambiri, ichi ndi chinthu chosiyana kwambiri ndi kuya kwa masamu ndi uinjiniya. Kusankha zida zapadera za masamu. Malinga ndi Gartner, mavuto awo ndi ofanana ndi a mainjiniya anzeru - palibe mgwirizano - aliyense amangoyang'ana mwachitsanzo chake, palibe demokalase, palibe kutanthauzira.
Chithunzi cha RapidMiner
Ndakumanapo ndikumva zambiri m'mbuyomu (pamodzi ndi Matlab) pamayendedwe abwino otseguka. Ndinakumba pang'ono mu TurboPrep monga mwachizolowezi. Ndili ndi chidwi ndi momwe ndingapezere deta yoyera kuchokera ku data yonyansa.
Apanso mutha kuwona kuti anthuwo ndi abwino kutengera zida zamalonda za 2018 komanso anthu oyipa olankhula Chingerezi pachiwonetsero.
Ndipo anthu ochokera ku Dortmund kuyambira 2001 okhala ndi mbiri yaku Germany)
Sindikumvetsabe kuchokera patsambali zomwe zimapezeka patsamba lotseguka - muyenera kukumba mozama. Makanema abwino okhudza kutumiza ndi malingaliro a AutoML.
Palibe chapadera pa RapidMiner Server backend mwina. Itha kukhala yaying'ono ndipo imagwira ntchito bwino pamtengo wotsika kwambiri. Imayikidwa mu Docker. Malo omwe amagawidwa pa seva ya RapidMiner yokha. Ndiyeno pali Radoop, deta kuchokera ku Hadoop, kuwerengera nyimbo zochokera ku Spark mu Studio workflow.
Monga momwe amayembekezeredwa, ogulitsa otentha achichepere βogulitsa timitengo tamizeremizereβ anawatsitsa pansi. Gartner, komabe, amalosera kupambana kwawo kwamtsogolo mu malo a Enterprise. Mutha kupeza ndalama kumeneko. A Germany amadziwa momwe angachitire izi, woyera-woyera :) Osatchula SAP !!!
Amachitira nzika zambiri! Koma kuchokera patsambalo mutha kuwona kuti Gartner akunena kuti akulimbana ndi zatsopano zogulitsa ndipo sakulimbana ndi kufalikira, koma phindu.
Anatsalira SAS ΠΈ Tibco mavenda wamba a BI kwa ineβ¦
kuchokera ku BI, osati kuchokera ku mitambo ndi zida za Hadoop. Kuchokera ku bizinesi, ndiko, osati kuchokera ku IT. Monga Gazpromneft mwachitsanzo: , Malo okhwima a DSML amakula kuchokera ku machitidwe amphamvu a BI. Koma mwina ndizovuta komanso zokondera ku MDM ndi zinthu zina, ndani akudziwa.
SAS
Palibe zambiri zoti munene. Zinthu zoonekeratu zokha.
Mtengo wa TIBCO
Njirayi imawerengedwa pamndandanda wazogula patsamba la Wiki lalitali. Inde, nkhani yayitali, koma 28 !!! Charles. Ndinagula BI Spotfire (2007) kubwerera ku techno-youth yanga. Komanso lipoti kuchokera ku Jaspersoft (2014), ndiye ochuluka mpaka atatu olosera analytics ogulitsa Insightful (S-plus) (2008), Statistica (2017) ndi Alpine Data (2017), kukonza zochitika ndi kukhamukira Streambase System (2013), MDM Orchestra Networks (2018) ndi Snappy Data (2019) mu-memory nsanja.
Hello Frankie!
Source: www.habr.com