Ka ho batla R kapa Python Marang-rang, u tla fumana limilione tsa lingoliloeng le lik'hilomithara tsa lipuisano mabapi le hore na ke efe e molemo, e potlakileng le e loketseng ho sebetsa ka data. Empa ka bomalimabe, lihlooho tsena tsohle le likhohlano ha li na thuso haholo.

Sepheo sa sengoloa sena ke ho bapisa mekhoa ea mantlha ea ts'ebetso ea data ka har'a liphutheloana tse tsebahalang haholo tsa lipuo ka bobeli. Le ho thusa babali kapele ho tseba ntho eo ba e-so tsebe. Bakeng sa ba ngolang Python, fumana mokhoa oa ho etsa se tšoanang ho R, le ka tsela e fapaneng.

Nakong ea sengoloa re tla sekaseka syntax ea liphutheloana tse tsebahalang haholo ho R. Tsena ke liphutheloana tse kenyellelitsoeng laeboraring. tidyversele sephutheloana data.table. Le ho bapisa syntax ea bona le pandas, sephutheloana se tsebahalang haholo sa tlhahlobo ea data ho Python.

Re tla tsamaea mohato ka mohato tseleng eohle ea tlhahlobo ea data ho tloha ho e kenya ho fihlela ho etsa mesebetsi ea tlhahlobo ea fensetere re sebelisa Python le R.

Tse ka hare

Sengoliloeng sena se ka sebelisoa e le letlapa la cheat haeba u lebetse mokhoa oa ho etsa ts'ebetso ea ts'ebetso ea data ho e 'ngoe ea liphutheloana tse ntseng li nahanoa.

1. Phapang ea mantlha ea syntax lipakeng tsa R le Python
   1.1. Ho fihlella Mesebetsi ea Sephutheloana
   1.2. Mosebetsi
   1.3. Indexing
   1.4. Mekhoa le OOP
   1.5. Liphaephe
   1.6. Mehaho ea Boitsebiso
2. Mantsoe a seng makae ka liphutheloana tseo re tla li sebelisa
   2.1. hloekile
   2.2. data.tafole
   2.3. panda
3. Ho kenya liphutheloana
4. Loading Data
5. Ho theha liforomo tsa data
6. Ho Khetha Likholomo Tseo U li Hlokang
7. Ho sefa mela
8. Ho hlopha le ho Kopanya
9. Thepa e kopaneng ea litafole (UNION)
10. Litafole tse tšekaletseng (JOIN)
11. Mesebetsi ea mantlha ea lifensetere le likholomo tse baloang
12. Tafole ea mangolo pakeng tsa mekhoa ea ts'ebetso ea data ho R le Python
13. fihlela qeto e
14. Phuputso e khutšoane mabapi le hore na u sebelisa sephutheloana sefe

Haeba u thahasella tlhahlobo ea data, u ka fumana ea ka thelekramo и mang youtube dikanale. Boholo ba litaba bo nehetsoe puong ea R.

Phapang ea mantlha ea syntax lipakeng tsa R le Python

Ho etsa hore ho be bonolo ho uena ho tloha ho Python ho ea ho R, kapa ka tsela e fapaneng, ke tla fana ka lintlha tse seng kae tsa bohlokoa tseo u lokelang ho li ela hloko.

Ho fihlella Mesebetsi ea Sephutheloana

Hang ha sephutheloana se kentsoe ho R, ha ho hlokahale hore u hlalose lebitso la sephutheloana ho fihlella mesebetsi ea sona. Hangata sena ha se tloaelehe ho R, empa sea amoheleha. Ha ho hlokahale hore u kenye sephutheloana ho hang haeba u hloka e 'ngoe ea mesebetsi ea eona ka khoutu ea hau, empa e letse feela ka ho bolela lebitso la sephutheloana le lebitso la mosebetsi. Karohano lipakeng tsa mabitso a sephutheloana le tšebetso ho R ke kolone e habeli. package_name::function_name().

Ho Python, ho fapana le hoo, ho nkoa e le khale ho bitsa mesebetsi ea sephutheloana ka ho hlakisa lebitso la eona. Ha sephutheloana se kopitsoa, ​​hangata se fuoa lebitso le khutsufalitsoeng, mohlala. pandas hangata ho sebelisoa lebitso la maiketsetso pd. Mosebetsi oa sephutheloana o fumaneha ka letheba package_name.function_name().

Mosebetsi

Ho R, ho tloaelehile ho sebelisa motsu ho fana ka boleng ho ntho. obj_name <- value, leha lets'oao le le leng le lekanang le lumelletsoe, lets'oao le le leng le lekanang ho R le sebelisoa haholo ho fetisa boleng ho sebetsa likhang.

Ho Python, kabelo e etsoa feela ka lets'oao le le leng le lekanang obj_name = value.

Indexing

Ho boetse ho na le liphapang tse kholo mona. Ho R, indexing e qala ho e 'ngoe mme e kenyelletsa likarolo tsohle tse boletsoeng lethathamong le hlahang,

Ho Python, indexing e qala ho tloha ho zero mme mofuta o khethiloeng ha o kenyelle karolo ea ho qetela e boletsoeng ho index. Kahoo moralo x[i:j] ho Python e ke ke ea kenyelletsa karolo ea j.

Ho boetse ho na le liphapang tsa indexing tse mpe, ho R notation x[-1] e tla khutlisa likarolo tsohle tsa vector ntle le ea ho qetela. Ho Python, notation e ts'oanang e tla khutlisa karolo ea ho qetela feela.

Mekhoa le OOP

R e sebelisa OOP ka tsela ea eona, ke ngotse ka sena sehloohong "OOP ka puo ea R (karolo ea 1): litlelase tsa S3". Ka kakaretso, R ke puo e sebetsang, 'me tsohle tse ho eona li hahiloe holim'a mesebetsi. Ka hona, ho etsa mohlala, bakeng sa basebelisi ba Excel, e ea ho tydiverse ho tla ba bonolo ho feta pandas. Le hoja ena e ka ba maikutlo a ka a ikemetseng.

Ka bokhutšoanyane, lintho tse ho R ha li na mekhoa (haeba re bua ka lihlopha tsa S3, empa ho na le lisebelisoa tse ling tsa OOP tse sa tloaelehang haholo). Ho na le mesebetsi e akaretsang feela e e sebetsanang ka tsela e fapaneng ho latela sehlopha sa ntho.

Liphaephe

Mohlomong lena ke lebitso la pandas E ke ke ea nepahala ka botlalo, empa ke tla leka ho hlalosa moelelo.

E le hore u se ke ua boloka lipalo tse bohareng le hore u se ke ua hlahisa lintho tse sa hlokahaleng sebakeng sa mosebetsi, u ka sebelisa mofuta oa pipeline. Tseo. fetisa sephetho sa lipalo ho tloha tšebetsong e 'ngoe ho ea ho e 'ngoe, 'me u se ke ua boloka liphetho tsa lipakeng.

Ha re nke mohlala o latelang oa khoutu, moo re bolokang lipalo tsa mahareng ka lintho tse fapaneng:

temp_object <- func1()
temp_object2 <- func2(temp_object )
obj <- func3(temp_object2 )

Re entse ts'ebetso ea 3 ka tatellano, 'me sephetho sa e' ngoe le e 'ngoe se bolokiloe nthong e fapaneng. Empa ha e le hantle, ha re hloke lintho tsena tse mahareng.

Kapa ho hobe le ho feta, empa ho tloaelehile ho basebelisi ba Excel.

obj  <- func3(func2(func1()))

Tabeng ena, ha rea ​​boloka liphetho tsa lipalo tsa mahareng, empa khoutu ea ho bala e nang le mesebetsi e hlophisitsoeng e thata haholo.

Re tla sheba mekhoa e 'maloa ea ts'ebetso ea data ho R,' me ba etsa ts'ebetso e ts'oanang ka mekhoa e fapaneng.

Liphaephe tse laebraring tidyverse kenngwa tshebetsong ke mosebeletsi %>%.

obj <- func1() %>% 
            func2() %>%
            func3()

Kahoo re nka sephetho sa mosebetsi func1() le ho e fetisa e le khang ea pele ho func2(), ebe re fetisa sephetho sa palo ena e le khang ea pele func3(). 'Me qetellong, re ngola lipalo tsohle tse entsoeng ka ntho obj <-.

Tsohle tse kaholimo li bonts'oa hantle ho feta mantsoe a meme ena:

В data.table liketane li sebelisoa ka mokhoa o ts'oanang.

newDT <- DT[where, select|update|do, by][where, select|update|do, by][where, select|update|do, by]

Ho e 'ngoe le e' ngoe ea mabakete a sekwere u ka sebelisa sephetho sa ts'ebetso e fetileng.

В pandas ts'ebetso e joalo e arotsoe ka letheba.

obj = df.fun1().fun2().fun3()

Tseo. re nka tafole ea rona df le ho sebelisa mokhoa oa hae fun1(), joale re sebelisa mokhoa ho sephetho se fumanoeng fun2(), kamora fun3(). Sephetho se bolokiloeng ho ntho sepheo .

Mehaho ea Boitsebiso

Mehaho ea data ho R le Python e tšoana, empa e na le mabitso a fapaneng.

tlhaloso
Lebitso ka R
Lebitso ka Python/pandas

Sebopeho sa tafole
data.frame, data.table, tibble
DataFrame

Lenane la boleng bo le bong
Vector
Letoto la li-pandas kapa lethathamong la Python e hloekileng

Sebopeho sa maemo a mangata a se nang tabular
Lenane
Bukantswe (dict)

Re tla sheba likarolo tse ling le liphapang tsa syntax ka tlase.

Mantsoe a seng makae ka liphutheloana tseo re tla li sebelisa

Taba ea pele, ke tla u joetsa hanyane ka liphutheloana tseo u tla tloaelana le tsona nakong ea sengoloa sena.

hloekile

Websaete ea molao: tidyverse.org

laebrari tidyverse e ngotsoe ke Hedley Wickham, Senior Research Scientist ho RStudio. tidyverse e na le sete e khahlang ea liphutheloana tse nolofatsang ts'ebetso ea data, tse 5 tsa tsona li kenyellelitsoe ho tse 10 tse jarollotsoeng tse holimo ho tsoa polokelong ea CRAN.

Moko oa laebrari o na le liphutheloana tse latelang: ggplot2, dplyr, tidyr, readr, purrr, tibble, stringr, forcats. E 'ngoe le e' ngoe ea liphutheloana tsena e reretsoe ho rarolla bothata bo itseng. Ka mohlala dplyr e etselitsoe ho qhekella data, tidyr ho tlisa data ka mokhoa o hlakileng, stringr e nolofatsa ho sebetsa ka likhoele, le ggplot2 ke e 'ngoe ea lisebelisoa tse tsebahalang haholo tsa pono ea data.

molemo tidyverse ke mokhoa o bonolo le o bonolo oa ho bala oa syntax, oo ka litsela tse ngata o tšoanang le puo ea SQL ea potso.

data.tafole

Websaete ea molao: r-datatable.com

Ka data.table ke Matt Dole oa H2O.ai.

Khatiso ea pele ea laebrari e etsahetse ka 2006.

Syntax ea sephutheloana ha e bonolo joalo ka ho tidyverse 'me e re hopotsa haholoanyane liforomo tsa khale tsa data ho R, empa ka nako e ts'oanang li atolohile haholo ts'ebetsong.

Liphetoho tsohle tse nang le tafole ka har'a sephutheloana sena li hlalositsoe ka masakaneng a lisekoere, 'me haeba u fetolela syntax. data.table ho SQL, o fumana ntho e kang ena: data.table[ WHERE, SELECT, GROUP BY ]

Matla a sephutheloana sena ke lebelo la ho sebetsana le lintlha tse ngata.

panda

Websaete ea molao: pandas.pydata.org

Lebitso la laebrari le tsoa lentsoeng la econometric "data ea phanele", le sebelisetsoang ho hlalosa mefuta e mengata e hlophisitsoeng ea tlhahisoleseling.

Ka pandas ke Wes McKinney oa Amerika.

Ha ho tluoa tabeng ea tlhahlobo ea data ho Python, ho lekana pandas Che. Sephutheloana se nang le mesebetsi e mengata haholo, se maemong a holimo se u lumellang ho etsa manonyeletso ka data, ho tloha ho kenya data ho tsoa mehloling efe kapa efe ho ea ho e bona ka mahlo.

Ho kenya liphutheloana tse eketsehileng

Liphutheloana tseo ho buuoeng ka tsona sehloohong sena ha li kenyelelitsoe kabong ea mantlha ea R le Python. Leha ho na le tlhokomeliso e nyane, haeba u kentse kabo ea Anaconda, ebe u kenya hape pandas ha ea hlokeha.

Ho kenya liphutheloana ho R

Haeba u butse tikoloho ea ntlafatso ea RStudio bonyane hang, mohlomong u se u ntse u tseba ho kenya sephutheloana se hlokahalang ho R. Ho kenya liphutheloana, sebelisa taelo e tloaelehileng. install.packages() ka ho e tsamaisa ka kotloloho ho R ka boeona.

# установка пакетов
install.packages("vroom")
install.packages("readr")
install.packages("dplyr")
install.packages("data.table")

Ka mor'a ho kenya, liphutheloana li hloka ho kopanngoa, tseo hangata ho sebelisoang taelo library().

# подключение или импорт пакетов в рабочее окружение
library(vroom)
library(readr)
library(dplyr)
library(data.table)

Ho kenya liphutheloana ho Python

Kahoo, haeba u na le Python e hloekileng e kentsoeng, joale pandas o hloka ho e kenya ka letsoho. Bula mola oa taelo, kapa terminal, ho latela sistimi ea hau ea ts'ebetso ebe u kenya taelo e latelang.

pip install pandas

Ebe re khutlela Python ebe re kenya sephutheloana se kentsoeng ka taelo import.

import pandas as pd

Loading Data

Merafo ea data ke e 'ngoe ea mehato ea bohlokoa ka ho fetisisa ea ho hlahloba lintlha. Ka bobeli Python le R, haeba li lakatsa, li u fa menyetla e mengata ea ho fumana lintlha ho tsoa mehloling efe kapa efe: lifaele tsa lehae, lifaele tse tsoang Marang-rang, liwebsaete, mefuta eohle ea database.

Ho pholletsa le sengoloa re tla sebelisa li-dataset tse 'maloa:

1. Litharollo tse peli ho tsoa ho Google Analytics.
2. Lethathamo la Bapalami ba Titanic.

Lintlha tsohle li ho ea ka GitHub ka sebopeho sa lifaele tsa csv le tsv. Re tla li kopa hokae?

E kenya data ho R: tidyverse, vroom, reader

Ho kenya data laebraring tidyverse Ho na le liphutheloana tse peli: vroom, readr. vroom ea morao-rao, empa nakong e tlang liphutheloana li ka kopanngoa.

Qotsa ho tsoa ho litokomane tsa molao vroom.

vroom vs 'mali
Ho lokolloa ha eng vroom bolela bakeng sa readr? Hajoale re rera ho tlohella liphutheloana tse peli hore li fetohe ka thoko, empa mohlomong re tla kopanya liphutheloana nakong e tlang. Phoso e 'ngoe ea ho bala ka botsoa ha vroom ke mathata a itseng a data a ke keng a tlalehoa ka pele, kahoo mokhoa oa ho a kopanya o hloka mohopolo o itseng.

vroom vs 'mali
Ho lokolloa ho bolela eng? vroom etsoe readr? Hajoale re rera ho nts'etsapele liphutheloana ka bobeli ka thoko, empa mohlomong re tla li kopanya nakong e tlang. E 'ngoe ea mathata a ho bala ka botsoa vroom ke hore mathata a mang a data a ke ke a tlalehoa esale pele, kahoo o hloka ho nahana ka mokhoa o motle oa ho a kopanya.

Sehloohong sena re tla sheba liphutheloana tse peli tsa ho kenya data:

E kenya data ho sephutheloana sa R: vroom

# install.packages("vroom")
library(vroom)

# Чтение данных
## vroom
ga_nov  <- vroom("https://raw.githubusercontent.com/selesnow/publications/master/data_example/r_python_data/ga_nowember.csv")
ga_dec  <- vroom("https://raw.githubusercontent.com/selesnow/publications/master/data_example/r_python_data/ga_december.csv")
titanic <- vroom("https://raw.githubusercontent.com/selesnow/publications/master/data_example/r_python_data/titanic.csv")

E kenya data ho R: reader

# install.packages("readr")
library(readr)

# Чтение данных
## readr
ga_nov  <- read_tsv("https://raw.githubusercontent.com/selesnow/publications/master/data_example/r_python_data/ga_nowember.csv")
ga_dec  <- read_tsv("https://raw.githubusercontent.com/selesnow/publications/master/data_example/r_python_data/ga_december.csv")
titanic <- read_csv("https://raw.githubusercontent.com/selesnow/publications/master/data_example/r_python_data/titanic.csv")

Ka har'a sephutheloana vroom, ho sa tsotelehe mofuta oa data oa csv / tsv, ho kenya ho etsoa ka ts'ebetso ea lebitso le le leng vroom(), ka har'a sephutheloana readr re sebelisa ts'ebetso e fapaneng bakeng sa sebopeho ka seng read_tsv() и read_csv().

E kenya data ho R: data.table

В data.table ho na le ts'ebetso ea ho kenya data fread().

E kenya data ho R: data.table package

# install.packages("data.table")
library(data.table)

## data.table
ga_nov  <- fread("https://raw.githubusercontent.com/selesnow/publications/master/data_example/r_python_data/ga_nowember.csv")
ga_dec  <- fread("https://raw.githubusercontent.com/selesnow/publications/master/data_example/r_python_data/ga_december.csv")
titanic <- fread("https://raw.githubusercontent.com/selesnow/publications/master/data_example/r_python_data/titanic.csv")

E kenya data ho Python: pandas

Haeba re bapisa le liphutheloana tsa R, joale tabeng ena syntax e haufi haholo le pandas tla ba readr, hobane pandas e ka kopa data ho tsoa kae kapa kae, 'me ho na le lelapa lohle la mesebetsi ka har'a sephutheloana sena read_*().

• read_csv()
• read_excel()
• read_sql()
• read_json()
• read_html()

Le mesebetsi e meng e mengata e etselitsoeng ho bala data ho tsoa mefuteng e fapaneng. Empa molemong oa rona ho lekane read_table() kapa read_csv() ho sebedisa khang Sep ho hlalosa karohano ea kholomo.

E kenya data ho Python: pandas

import pandas as pd

ga_nov  = pd.read_csv("https://raw.githubusercontent.com/selesnow/publications/master/data_example/russian_text_in_r/ga_nowember.csv", sep = "t")
ga_dec  = pd.read_csv("https://raw.githubusercontent.com/selesnow/publications/master/data_example/russian_text_in_r/ga_december.csv", sep = "t")
titanic = pd.read_csv("https://raw.githubusercontent.com/selesnow/publications/master/data_example/russian_text_in_r/titanic.csv")

Ho theha liforomo tsa data

Tafole Titanic, eo re e belesitseng, ho na le tšimo Sex, e bolokang boitsebiso ba bong ba mopalami.

Empa bakeng sa tlhahiso e bonolo haholoanyane ea data mabapi le bong ba bapalami, u lokela ho sebelisa lebitso ho fapana le khoutu ea bong.

Ho etsa sena, re tla theha bukana e nyane, tafole eo ho eona ho tla ba le likholomo tse 2 feela (khoutu le lebitso la bong) le mela e 2, ka ho latellana.

Ho theha dataframe ho R: tidyverse, dplyr

Mohlala oa khoutu o ka tlase, re theha dataframe e lakatsehang re sebelisa ts'ebetso tibble() .

Ho theha dataframe ho R: dplyr

## dplyr
### создаём справочник
gender <- tibble(id = c(1, 2),
                 gender = c("female", "male"))

Ho theha dataframe ho R: data.table

Ho theha dataframe ho R: data.table

## data.table
### создаём справочник
gender <- data.table(id = c(1, 2),
                    gender = c("female", "male"))

Ho theha dataframe ho Python: pandas

В pandas Ho theoa ha liforeimi ho etsoa ka mekhahlelo e mengata, pele re theha bukana e hlalosang mantsoe, ebe re fetolela bukana ho ba dataframe.

Ho theha dataframe ho Python: pandas

# создаём дата фрейм
gender_dict = {'id': [1, 2],
               'gender': ["female", "male"]}
# преобразуем словарь в датафрейм
gender = pd.DataFrame.from_dict(gender_dict)

Ho Khetha Likholomo

Litafole tseo u sebetsang le tsona li ka ba le lintlha tse ngata kapa tse makholo. Empa ho etsa tlhahlobo, joalo ka molao, ha ho hlokahale litšiea tsohle tse fumanehang tafoleng ea mohloli.

Ka hona, e 'ngoe ea ts'ebetso ea pele eo u tla e etsa ka tafole ea mohloli ke ho e hlakola tlhahisoleseling e sa hlokahaleng le ho lokolla mohopolo oo tlhahisoleseling ena e leng ho ona.

Ho khetha litšiea ho R: tidyverse, dplyr

hokahanngoang dplyr e ts'oana haholo le puo ea potso ea SQL, haeba u e tseba u tla tseba sephutheloana sena kapele.

Ho khetha litšiea, sebelisa ts'ebetso select().

Ka tlase ke mehlala ea khoutu eo ka eona u ka khethang likholomo ka litsela tse latelang:

• Ho thathamisa mabitso a likholomo tse hlokahalang
• Sheba mabitso a likholomo u sebelisa lipolelo tse tloaelehileng
• Ka mofuta oa data kapa thepa efe kapa efe ea data e fumanehang kholomong

Ho khetha litšiea ho R: dplyr

# Выбор нужных столбцов
## dplyr
### выбрать по названию столбцов
select(ga_nov, date, source, sessions)
### исключь по названию столбцов
select(ga_nov, -medium, -bounces)
### выбрать по регулярному выражению, стобцы имена которых заканчиваются на s
select(ga_nov, matches("s$"))
### выбрать по условию, выбираем только целочисленные столбцы
select_if(ga_nov, is.integer)

Ho khetha litšiea ho R: data.table

Ts'ebetso e ts'oanang ho data.table li etsoa ka tsela e fapaneng hanyane, qalong ea sengoloa ke fane ka tlhaloso ea hore na likhang li teng ka har'a masakaneng a lisekoere data.table.

DT[i,j,by]

Moo:
ke - kae, ke. ho tlhotla ka mela
j - khetha|ntlafatsa| etsa, ke hore. ho kgetha dikholomo le ho di fetola
ka - data grouping

Ho khetha litšiea ho R: data.table

## data.table
### выбрать по названию столбцов
ga_nov[ , .(date, source, sessions) ]
### исключь по названию столбцов
ga_nov[ , .SD, .SDcols = ! names(ga_nov) %like% "medium|bounces" ]
### выбрать по регулярному выражению
ga_nov[, .SD, .SDcols = patterns("s$")]

E fapaneng .SD e o lumella ho fihlella litšiea tsohle, le .SDcols sefa likholomo tse hlokahalang u sebelisa lipolelo tse tloaelehileng, kapa mesebetsi e meng ho sefa mabitso a likholomo tseo u li hlokang.

Ho khetha litšiea ho Python, pandas

Ho khetha likholomo ka mabitso ho pandas ho lekane ho fana ka lethathamo la mabitso a bona. 'Me ho khetha kapa ho qhelela ka thoko likholomo ka mabitso u sebelisa mantsoe a tloaelehileng, u hloka ho sebelisa lits'ebetso drop() и filter(), le khang setsi=1, eo ka eona u bontšang hore hoa hlokahala ho sebetsa litšiea ho e-na le mela.

Ho khetha sebaka ka mofuta oa data, sebelisa ts'ebetso select_dtypes(), le likhang kenyeletsa kapa kenyeletsa fetisa lethathamo la mefuta ea data e tsamaellanang le hore na ke likarolo life tseo u lokelang ho li khetha.

Ho khetha litšiea ho Python: li-pandas

# Выбор полей по названию
ga_nov[['date', 'source', 'sessions']]
# Исключить по названию
ga_nov.drop(['medium', 'bounces'], axis=1)
# Выбрать по регулярному выражению
ga_nov.filter(regex="s$", axis=1)
# Выбрать числовые поля
ga_nov.select_dtypes(include=['number'])
# Выбрать текстовые поля
ga_nov.select_dtypes(include=['object'])

Ho sefa mela

Mohlala, tafole ea mohloli e kanna ea ba le lilemo tse 'maloa tsa data, empa o hloka feela ho sekaseka khoeli e fetileng. Hape, mela e meng e tla fokotsa ts'ebetso ea ts'ebetso ea data le ho koala memori ea PC.

Ho sefa mela ho R: tydyverse, dplyr

В dplyr mosebetsi o sebedisoa ho sefa mela filter(). Ho nka dataframe joalo ka khang ea pele, ebe u thathamisa maemo a ho sefa.

Ha u ngola lipolelo tse utloahalang ho sefa tafole, tabeng ena, bolela mabitso a kholomo ntle le mantsoe a qotsitsoeng le ntle le ho phatlalatsa lebitso la tafole.

Ha u sebelisa mantsoe a mangata a utloahalang ho sefa, sebelisa lisebelisoa tse latelang:

• & kapa phegelwana - e utlwahala LE
• | - e utloahalang KAPA

Ho sefa mela ho R: dplyr

# фильтрация строк
## dplyr
### фильтрация строк по одному условию
filter(ga_nov, source == "google")
### фильтр по двум условиям соединённым логическим и
filter(ga_nov, source == "google" & sessions >= 10)
### фильтр по двум условиям соединённым логическим или
filter(ga_nov, source == "google" | sessions >= 10)

Ho hlopha mela ho R: data.table

Joalokaha ke se ke ngotse ka holimo, ka data.table Syntax ea phetoho ea data e kenyelelitsoe ka masakaneng a sekoere.

DT[i,j,by]

Moo:
ke - kae, ke. ho tlhotla ka mela
j - khetha|ntlafatsa| etsa, ke hore. ho kgetha dikholomo le ho di fetola
ka - data grouping

Khang e sebelisoa ho sefa mela i, e nang le boemo ba pele ka masakaneng a lisekoere.

Likholomo li fumaneha ka lipolelo tse utloahalang ntle le matšoao a qotso le ntle le ho bolela lebitso la tafole.

Lipolelo tse utloahalang li amana ka tsela e tšoanang le ea ho dplyr ka & le | basebelisi.

Ho hlopha mela ho R: data.table

## data.table
### фильтрация строк по одному условию
ga_nov[source == "google"]
### фильтр по двум условиям соединённым логическим и
ga_nov[source == "google" & sessions >= 10]
### фильтр по двум условиям соединённым логическим или
ga_nov[source == "google" | sessions >= 10]

Ho sefa likhoele ho Python: li-pandas

Sefa ka mela pandas e ts'oanang le ho sefa data.table, 'me e etsoa ka masakaneng a lisekoere.

Tabeng ena, phihlello ea likholomo e etsoa ka ho bonts'a lebitso la dataframe; ebe lebitso la kholomo le ka bonts'oa ka matšoao a qotsulo ka masakaneng a lisekoere (mohlala df['col_name']), kapa ntle le mantsoe a qotsitsoeng ka mor'a nako (mohlala df.col_name).

Haeba o hloka ho sefa dataframe ka maemo a 'maloa, boemo bo bong le bo bong bo tlameha ho beoa ka masakaneng. Maemo a utloahalang a hokahanngoa ke ba sebetsang & и |.

Ho sefa likhoele ho Python: li-pandas

# Фильтрация строк таблицы
### фильтрация строк по одному условию
ga_nov[ ga_nov['source'] == "google" ]
### фильтр по двум условиям соединённым логическим и
ga_nov[(ga_nov['source'] == "google") & (ga_nov['sessions'] >= 10)]
### фильтр по двум условиям соединённым логическим или
ga_nov[(ga_nov['source'] == "google") | (ga_nov['sessions'] >= 10)]

Ho hlopha le ho kopanya lintlha

E 'ngoe ea lits'ebetso tse sebelisoang haholo tlhahlobong ea data ke ho kopanya le ho kopanya.

Syntax ea ho etsa ts'ebetso ena e hasane ho liphutheloana tsohle tseo re li hlahlobang.

Tabeng ena, re tla nka dataframe e le mohlala Titanic, 'me u bale palo le karolelano ea litšenyehelo tsa litekete ho latela sehlopha sa cabin.

Ho hlopha le ho kopanya data ho R: tidyverse, dplyr

В dplyr mosebetsi o sebedisoa bakeng sa ho kopanya group_by(), le bakeng sa ho kopanya summarise(). Haele hantle, dplyr ho na le lelapa lohle la mesebetsi summarise_*(), empa morero oa sehlooho sena ke ho bapisa mantsoe a motheo, kahoo re ke ke ra kena morung o joalo.

Mesebetsi ea mantlha ea ho kopanya:

• sum() - kakaretso
• min() / max() – bonyane le boholo ba boleng
• mean() - karolelano
• median() - ea mahareng
• length() - bongata

Ho kopanya le ho kopanya ho R: dplyr

## dplyr
### группировка и агрегация строк
group_by(titanic, Pclass) %>%
  summarise(passangers = length(PassengerId),
            avg_price  = mean(Fare))

Ho sebetsa group_by() ra feta tafole e le khang ea pele Titanic, ebe o bontša sebaka Pclass, eo ka eona re tla hlophisa tafole ea rona. Sephetho sa ts'ebetso ena ho sebelisoa opareitara %>% e fetisitsoe joalo ka khang ea pele ea tšebetso summarise(), le ho eketsa likarolo tse ling tse 2: bapalami и avg_theko. Ntlha ea pele, sebelisa sesebelisoa length() bala palo ea litekete, 'me ea bobeli sebelisa tshebetso mean() e fumane kakaretso ea theko ea litekete.

Ho hlopha le ho kopanya lintlha ho R: data.table

В data.table khang e sebedisoa bakeng sa ho kopanya j e nang le boemo ba bobedi ka masakaneng a sekwere, le bakeng sa ho hlopha by kapa keyby, tse nang le boemo ba boraro.

Lethathamo la mesebetsi ea aggregation tabeng ena le ts'oana le le hlalositsoeng ho dplyr, hobane tsena ke mesebetsi ho tsoa ho syntax ea mantlha ea R.

Ho hlopha le ho kopanya ho R: data.table

## data.table
### фильтрация строк по одному условию
titanic[, .(passangers = length(PassengerId),
            avg_price  = mean(Fare)),
        by = Pclass]

Ho kopanya le ho kopanya data ho Python: pandas

Ho kopanya ka lihlopha pandas tšoanang le dplyr, empa kakaretso ha e tšoane le dplyr eseng on data.table.

Ho sehlopha, sebelisa mokhoa groupby(), eo ho eona u hlokang ho fetisa lethathamo la litšiea tseo dataframe e tla aroloa ka tsona.

Bakeng sa ho kopanya, o ka sebelisa mokhoa agg()e amohelang bukantswe. Linotlolo tsa dikishinari ke litšiea tseo ho tsona u tla sebelisa mesebetsi ea ho kopanya, 'me boleng ke mabitso a mesebetsi ea ho kopanya.

Mesebetsi ea ho kopanya:

• sum() - kakaretso
• min() / max() – bonyane le boholo ba boleng
• mean() - karolelano
• median() - ea mahareng
• count() - bongata

Mosebetsi reset_index() mohlaleng o ka tlase e sebelisoa ho reset li-index tsa sehlaha hore pandas defaults ho kamora ho kopanya data.

Letšoao e o lumella ho ea moleng o latelang.

Ho hlopha le ho kopanya ho Python: li-pandas

# группировка и агрегация данных
titanic.groupby(["Pclass"]).
    agg({'PassengerId': 'count', 'Fare': 'mean'}).
        reset_index()

Khokahano e otlolohileng ea litafole

Ts'ebetso eo ho eona u kopanyang litafole tse peli kapa ho feta tsa sebopeho se le seng. Lintlha tseo re li kentseng li na le litafole kha_nov и ga_dec. Litafole tsena li tšoana ka sebopeho, i.e. ba le litšiea tse tšoanang, le mefuta ea data likholomong tsena.

Ena ke tlhahiso e tsoang ho Google Analytics bakeng sa khoeli ea November le December, karolong ena re tla kopanya lintlha tsena tafoleng e le 'ngoe.

Ho kopanya litafole ho R: tidyverse, dplyr

В dplyr O ka kopanya litafole tse 2 ho e le 'ngoe u sebelisa ts'ebetso bind_rows(), ho fetisa litafole e le mabaka a eona.

Ho sefa mela ho R: dplyr

# Вертикальное объединение таблиц
## dplyr
bind_rows(ga_nov, ga_dec)

Ho kopanya litafole ho R: data.table

Hape ha se letho le rarahaneng, a re sebeliseng rbind().

Ho hlopha mela ho R: data.table

## data.table
rbind(ga_nov, ga_dec)

Ho kopanya litafole ho Python: li-pandas

В pandas mosebetsi o sebedisoa ho kopanya litafole concat(), eo ho eona u hlokang ho fetisa lethathamo la liforeimi ho li kopanya.

Ho sefa likhoele ho Python: li-pandas

# вертикальное объединение таблиц
pd.concat([ga_nov, ga_dec])

Khokahano e otlolohileng ea litafole

Ts'ebetso eo ho eona litšiea tse tsoang ho ea bobeli li kenyelletsoang tafoleng ea pele ka senotlolo. Hangata e sebelisoa ha ho ntlafatsoa tafole ea lintlha (mohlala, tafole e nang le data ea thekiso) e nang le lintlha tse ling tsa litšupiso (mohlala, litšenyehelo tsa sehlahisoa).

Ho na le mefuta e 'maloa ea likamano:

Tafoleng e kentsoeng pele Titanic re na le kholomo Sex, e tsamaellanang le khoutu ea bong ba mopalami:

1 - e motšehali
2 - monna

Hape, re thehile tafole - buka ea litšupiso tekano. Bakeng sa tlhahiso e bonolo haholoanyane ea datha mabapi le bong ba bapalami, re hloka ho kenyelletsa lebitso la bong ho tsoa bukeng. tekano tafoleng Titanic.

Tafole e otlolohileng e kenella ho R: tidyverse, dplyr

В dplyr Ho na le lelapa lohle la mesebetsi bakeng sa ho kopanya ho rapameng:

• inner_join()
• left_join()
• right_join()
• full_join()
• semi_join()
• nest_join()
• anti_join()

E sebelisoang haholo ts'ebetsong ea ka ke left_join().

E le likhang tse peli tsa pele, mesebetsi e thathamisitsoeng ka holimo e nka litafole tse peli ho kopanya, 'me e le khang ea boraro by o tlameha ho totobatsa litšiea tseo u lokelang ho li kopanya.

Tafole e otlolohileng e kenella ho R: dplyr

# объединяем таблицы
left_join(titanic, gender,
          by = c("Sex" = "id"))

Khokahano e tšekaletseng ea litafole ho R: data.table

В data.table U hloka ho kopanya litafole ka senotlolo u sebelisa ts'ebetso merge().

Likhang tsa ho kopanya() tshebetso ho data.table

• x, y - Litafole tsa ho kopanya
• ka - Column eo ke senotlolo sa ho kopanya haeba e na le lebitso le tšoanang litafoleng ka bobeli
• ka.x, ka.y - Mabitso a kholomo a tla kopanngoa, haeba a na le mabitso a fapaneng litafoleng
• all, all.x, all.y — Kopanya mofuta, bohle ba tla kgutlisetsa mela yohle ho tswa ho ditafole ka bobedi, all.x e tsamaellana le tshebetso ya LEFT JOIN (e tla siya mela yohle ya tafole ya pele), all.y — e tsamaellana le Ts'ebetso e nepahetseng ea JOIN (e tla siea mela eohle ea tafole ea bobeli).

Khokahano e tšekaletseng ea litafole ho R: data.table

# объединяем таблицы
merge(titanic, gender, by.x = "Sex", by.y = "id", all.x = T)

Tafole e otlolohileng e ikopanya le Python: li-pandas

Joalo ka ho data.table, ho pandas mosebetsi o sebedisoa ho kopanya litafole merge().

Likhang tsa ts'ebetso ea merge() ho li-pandas

• joang - Mofuta oa khokahanyo: ka ho le letšehali, ho le letona, ka ntle, ka hare
• on - Kholomo eo ke senotlolo haeba e na le lebitso le tšoanang litafoleng ka bobeli
• left_on, right_on - Mabitso a likholomo tsa bohlokoa, haeba a na le mabitso a fapaneng litafoleng

Tafole e otlolohileng e ikopanya le Python: li-pandas

# объединяем по ключу
titanic.merge(gender, how = "left", left_on = "Sex", right_on = "id")

Mesebetsi ea mantlha ea lifensetere le likholomo tse baloang

Mesebetsi ea lifensetere e tšoana ka moelelo le mesebetsi ea ho kopanya, hape e sebelisoa hangata tlhahlobong ea data. Empa ho fapana le mesebetsi ea ho kopanya, mesebetsi ea lifensetere ha e fetole palo ea mela ea dataframe e tsoang.

Ha e le hantle, ha re sebelisa ts'ebetso ea fensetere, re arola "dataframe" e kenang likarolo ho latela maemo a itseng, ke hore. ka boleng ba tšimo, kapa masimo a mmaloa. 'Me re etsa mesebetsi ea lipalo fensetereng e' ngoe le e 'ngoe. Sephetho sa ts'ebetso ena se tla khutlisoa moleng o mong le o mong, ke hore. ntle le ho fetola palo yohle ya mela e tafoleng.

Ka mohlala, ha re nke tafole Titanic. Re ka bala hore na theko ea tekete ka 'ngoe e ne e le liperesente tse kae ka har'a sehlopha sa eona sa khabinete.

Ho etsa sena, re hloka ho fumana moleng o mong le o mong kakaretso ea litšenyehelo tsa tekete bakeng sa sehlopha sa hajoale sa khabinete seo tekete e leng ho sona, ebe re arola litjeo tsa tekete ka 'ngoe ka kakaretso ea litšenyehelo tsa litekete tsohle tsa sehlopha se le seng. .

Mesebetsi ea lifensetere ho R: tidyverse, dplyr

Ho kenya likholomo tse ncha, ntle le ho sebelisa lihlopha tsa mela, ho dplyr e sebeletsa mosebetsi mutate().

U ka rarolla bothata bo hlalositsoeng ka holimo ka ho hlophisa lintlha ka lihlopha Pclass le ho akaretsa tšimo ka kholomong e ncha etsa. Ka mor'a moo, tlosa tafole 'me u arole litekanyetso tsa tšimo etsa ho se etsahetseng mohatong o fetileng.

Mesebetsi ea fensetere ho R: dplyr

group_by(titanic, Pclass) %>%
  mutate(Pclass_cost = sum(Fare)) %>%
  ungroup() %>%
  mutate(ticket_fare_rate = Fare / Pclass_cost)

Mesebetsi ea lifensetere ho R: data.table

Algorithm ea tharollo e ntse e tšoana le ho dplyr, re hloka ho arola tafole ka lifensetere ka tšimo Pclass. Hlahisa ka kholomong e ncha chelete ea sehlopha se tsamaellanang le mola o mong le o mong, 'me u kenye kholomo eo ho eona re balang karolo ea litšenyehelo tsa tekete ka' ngoe sehlopheng sa eona.

Ho kenya likholomo tse ncha ho data.table mosebeletsi o teng :=. Ka tlase ke mohlala oa ho rarolla bothata ka ho sebelisa sephutheloana data.table

Mesebetsi ea lifensetere ho R: data.table

titanic[,c("Pclass_cost","ticket_fare_rate") := .(sum(Fare), Fare / Pclass_cost), 
        by = Pclass]

Mesebetsi ea lifensetere ho Python: li-pandas

Mokhoa o mong oa ho eketsa kholomo e ncha ho pandas - sebelisa tshebetso assign(). Ho akaretsa litšenyehelo tsa litekete ka sehlopha sa cabin, ntle le mela ea lihlopha, re tla sebelisa ts'ebetso transform().

Ka tlase ke mohlala oa tharollo eo re e eketsang tafoleng Titanic litšiea tse 2 tse tšoanang.

Mesebetsi ea lifensetere ho Python: li-pandas

titanic.assign(Pclass_cost      =  titanic.groupby('Pclass').Fare.transform(sum),
               ticket_fare_rate = lambda x: x['Fare'] / x['Pclass_cost'])

Tafole ea mangolo a mesebetsi le mekhoa

Ka tlase ke lethathamo la mangolo a mangolo a lipakeng tsa mekhoa ea ho etsa ts'ebetso e fapaneng ka data ho liphutheloana tseo re li hlahlobileng.

tlhaloso
hloekile
data.tafole
panda

Loading Data
vroom()/ readr::read_csv() / readr::read_tsv()
fread()
read_csv()

Ho theha liforomo tsa data
tibble()
data.table()
dict() + from_dict()

Ho Khetha Likholomo
select()
ngangisano j, boemo ba bobeli ka masakaneng a sekwere
re fetisa lenane la litšiea tse hlokahalang ka masakaneng a lisekoere / drop() / filter() / select_dtypes()

Ho sefa mela
filter()
ngangisano i, boemo ba pele ka masakaneng a sekwere
Re thathamisa maemo a ho sefa ka masakaneng a lisekoere / filter()

Ho hlopha le ho Kopanya
group_by() + summarise()
likhang j + by
groupby() + agg()

Thepa e kopaneng ea litafole (UNION)
bind_rows()
rbind()
concat()

Litafole tse tšekaletseng (JOIN)
left_join() / *_join()
merge()
merge()

Mesebetsi ea mantlha ea lifensetere le ho eketsa likholomo tse baloang
group_by() + mutate()
ngangisano j ho sebedisa mosebeletsi := + khang by
transform() + assign()

fihlela qeto e

Mohlomong sengolong seo ke se hlalositseng eseng ts'ebetsong e nepahetseng ka ho fetesisa ea ts'ebetso ea data, ka hona, ke tla thaba haeba u lokisa liphoso tsa ka litlhalosong, kapa u tlatselletsa tlhahisoleseling e fanoeng sengolong ka mekhoa e meng ea ho sebetsa le data ho R / Python.

Joalokaha ke ngotse ka holimo, morero oa sehlooho sena e ne e se ho qobella maikutlo a motho hore na ke puo efe e molemo, empa ho nolofatsa monyetla oa ho ithuta lipuo tse peli, kapa, haeba ho hlokahala, ho falla pakeng tsa tsona.

Haeba u ratile sengoloa, ke tla thabela ho ba le ba ngolisitseng ba bacha ho ea ka mang youtube и thelekramo dikanale.

Poll

Ke liphutheloana life ho tse latelang tseo u li sebelisang mosebetsing oa hau?

Litlhalosong u ka ngola lebaka la khetho ea hau.

Ke basebelisi ba ngolisitsoeng feela ba ka kenyang letsoho phuputsong. kenaka kopo.

U sebelisa sephutheloana sefe sa ts'ebetso ea data (o ka khetha likhetho tse 'maloa)

• 45,2%tidyverse19

• 33,3%data.tafole14

• 54,8%pandas23

Basebelisi ba 42 ba ile ba khetha. Basebelisi ba 9 ba ile ba hana.

Source: www.habr.com