Los ntawm kev tshawb nrhiav R lossis Python hauv Is Taws Nem, koj yuav pom ntau lab ntawm cov kab lus thiab mais ntawm kev sib tham ntawm cov ncauj lus uas ib qho zoo dua, sai dua thiab yooj yim dua rau kev ua haujlwm nrog cov ntaub ntawv. Tab sis hmoov tsis, tag nrho cov kab lus thiab kev tsis sib haum xeeb no tsis muaj txiaj ntsig tshwj xeeb.

Lub hom phiaj ntawm tsab xov xwm no yog los sib piv cov txheej txheem ua cov ntaub ntawv hauv cov pob nrov tshaj plaws ntawm ob hom lus. Thiab pab cov neeg nyeem sai sai ua ib yam dab tsi uas lawv tsis paub. Rau cov neeg uas sau hauv Python, nrhiav seb yuav ua li cas tib yam hauv R, thiab rov ua dua.

Nyob rau hauv tsab xov xwm peb yuav txheeb xyuas cov syntax ntawm cov pob khoom nrov tshaj plaws hauv R. Cov no yog cov pob khoom suav nrog hauv tsev qiv ntawv tidyversethiab kuj lub pob data.table. Thiab sib piv lawv syntax nrog pandas, qhov nrov tshaj plaws cov ntaub ntawv tsom xam pob hauv Python.

Peb yuav mus rau ib kauj ruam dhau los ntawm tag nrho txoj hauv kev ntawm kev txheeb xyuas cov ntaub ntawv los ntawm kev thauj khoom mus rau kev ua haujlwm analytical qhov rais siv Python thiab R.

Txheem

Cov kab lus no tuaj yeem siv los ua ib daim ntawv khib nyiab yog tias koj tsis nco qab yuav ua li cas ua qee cov ntaub ntawv ua haujlwm hauv ib qho ntawm cov pob khoom raws li kev xav.

Lub ntsiab syntax sib txawv ntawm R thiab Python
1.1. Nkag mus rau pob Functions
1.2. Txoj hauj lwm
1.3. Indexing
1.4. Methods thiab OOP
1.5. Cov kav dej
1.6. Cov qauv ntaub ntawv
Ob peb lo lus hais txog cov pob peb yuav siv
2.1. tidyverse
2.2. cov ntaub ntawv
2.3. panda
Txhim kho pob
Chaw thau khoom
Tsim dataframes
Xaiv cov kab uas koj xav tau
Lim kab
Grouping thiab Aggregation
Vertical union ntawm lub rooj (UNION)
Kab rov tav sib koom ntawm cov rooj (JOIN)
Lub qhov rais yooj yim ua haujlwm thiab suav cov kab
Cov lus sib tham ntawm cov ntaub ntawv ua cov txheej txheem hauv R thiab Python
xaus
Kev tshawb fawb luv luv txog cov pob twg koj siv

Yog tias koj txaus siab rau cov ntaub ntawv txheeb xyuas, koj tuaj yeem nrhiav kuv telegram и youtube cov channel. Cov ntsiab lus feem ntau yog mob siab rau R hom lus.

Lub ntsiab syntax sib txawv ntawm R thiab Python

Txhawm rau ua kom yooj yim rau koj hloov ntawm Python rau R, lossis rov ua dua, kuv yuav muab ob peb lub ntsiab lus tseem ceeb uas koj yuav tsum tau them sai sai rau.

Nkag mus rau pob Functions

Thaum ib lub pob ntim rau hauv R, koj tsis tas yuav qhia lub npe pob kom nkag mus rau nws cov haujlwm. Feem ntau qhov no tsis tshwm sim hauv R, tab sis nws tau txais. Koj tsis tas yuav import ib pob tag nrho yog tias koj xav tau ib qho ntawm nws cov haujlwm hauv koj cov cai, tab sis tsuas yog hu nws los ntawm kev qhia lub npe ntawm pob thiab lub npe ntawm txoj haujlwm. Qhov sib cais ntawm pob thiab cov npe ua haujlwm hauv R yog ob txoj hnyuv loj. package_name::function_name().

Hauv Python, ntawm qhov tsis sib xws, nws suav hais tias yog classic hu ua lub luag haujlwm ntawm lub pob los ntawm kev qhia meej meej nws lub npe. Thaum lub pob rub tawm, nws feem ntau muab lub npe luv, piv txwv li. pandas feem ntau yog siv pseudonym pd. Ib pob muaj nuj nqi yog nkag los ntawm ib qho chaw package_name.function_name().

Txoj hauj lwm

Hauv R, nws yog ib txwm siv tus xub los muab tus nqi rau ib qho khoom. obj_name <- value, txawm hais tias ib qho kev sib npaug sib npaug tau tso cai, ib qho sib npaug kos npe hauv R yog siv feem ntau kom dhau qhov tseem ceeb rau kev sib cav.

Hauv Python, kev ua haujlwm tsuas yog ua tiav nrog ib qho kos npe sib npaug obj_name = value.

Indexing

Kuj tseem muaj qhov sib txawv tseem ceeb ntawm no. Hauv R, indexing pib ntawm ib qho thiab suav nrog tag nrho cov ntsiab lus teev tseg hauv qhov tshwm sim,

Hauv Python, indexing pib ntawm xoom thiab qhov kev xaiv ntau yam tsis suav nrog cov ntsiab lus kawg uas tau teev tseg hauv qhov ntsuas. Yog li tsim x[i:j] hauv Python yuav tsis suav nrog cov ntsiab lus j.

Kuj tseem muaj qhov sib txawv ntawm qhov tsis zoo indexing, hauv R sau x[-1] yuav rov qab tag nrho cov ntsiab lus ntawm vector tsuas yog qhov kawg. Hauv Python, cov ntawv sau zoo sib xws yuav rov qab tsuas yog lub caij kawg nkaus xwb.

Methods thiab OOP

R siv OOP hauv nws tus kheej txoj kev, kuv tau sau txog qhov no hauv tsab xov xwm "OOP hauv R hom (ib ntu 1): S3 chav kawm". Feem ntau, R yog lus ua haujlwm, thiab txhua yam hauv nws yog tsim los ntawm kev ua haujlwm. Yog li ntawd, piv txwv li, rau cov neeg siv Excel, mus rau tydiverse nws yuav yooj yim dua pandas. Txawm hais tias qhov no yuav yog kuv qhov kev xav.

Hauv luv luv, cov khoom hauv R tsis muaj txoj hauv kev (yog tias peb tham txog S3 chav kawm, tab sis muaj lwm yam kev siv OOP uas tsis tshua muaj ntau). Tsuas muaj kev ua haujlwm dav dav uas ua rau lawv txawv nyob ntawm chav kawm ntawm cov khoom.

Cov kav dej

Tej zaum qhov no yog lub npe rau pandas Nws yuav tsis yog tag nrho, tab sis kuv yuav sim piav qhia lub ntsiab lus.

Txhawm rau kom tsis txhob txuag cov kev suav nruab nrab thiab tsis tsim cov khoom siv tsis tsim nyog hauv qhov chaw ua haujlwm, koj tuaj yeem siv hom kav dej. Cov. dhau qhov txiaj ntsig ntawm kev suav ntawm ib qho haujlwm mus rau qhov txuas ntxiv, thiab tsis txhob khaws cov txiaj ntsig nruab nrab.

Cia peb ua tus piv txwv hauv qab no, qhov twg peb khaws cov kev suav nruab nrab hauv cov khoom sib cais:

temp_object <- func1()
temp_object2 <- func2(temp_object )
obj <- func3(temp_object2 )

Peb tau ua 3 txoj haujlwm ua ntu zus, thiab qhov tshwm sim ntawm txhua qhov tau txais kev cawmdim hauv ib qho khoom cais. Tab sis qhov tseeb, peb tsis xav tau cov khoom nruab nrab no.

Los yog txawm phem dua, tab sis paub ntau dua rau cov neeg siv Excel.

obj  <- func3(func2(func1()))

Hauv qhov no, peb tsis tau txuag cov txiaj ntsig nruab nrab ntawm kev suav, tab sis kev nyeem cov lej nrog cov haujlwm nested yog qhov tsis yooj yim heev.

Peb yuav saib ntau txoj hauv kev rau kev ua cov ntaub ntawv hauv R, thiab lawv ua haujlwm zoo sib xws hauv ntau txoj kev.

Cov kav dej hauv tsev qiv ntawv tidyverse siv los ntawm tus neeg teb xov tooj %>%.

obj <- func1() %>% 
            func2() %>%
            func3()

Yog li peb coj qhov tshwm sim ntawm kev ua haujlwm func1() thiab dhau nws raws li thawj qhov kev sib cav rau func2(), ces peb dhau qhov tshwm sim ntawm qhov kev suav no ua thawj qhov kev sib cav func3(). Thiab thaum kawg, peb sau tag nrho cov kev suav ua rau hauv cov khoom obj <-.

Tag nrho cov saum toj no yog piav qhia zoo dua li cov lus los ntawm no meme:

В data.table chains yog siv nyob rau hauv ib txoj kev zoo sib xws.

newDT <- DT[where, select|update|do, by][where, select|update|do, by][where, select|update|do, by]

Hauv txhua qhov square brackets koj tuaj yeem siv qhov tshwm sim ntawm kev ua haujlwm dhau los.

В pandas xws li kev khiav hauj lwm yog cais los ntawm ib tug dot.

obj = df.fun1().fun2().fun3()

Cov. peb muab peb lub rooj df thiab siv nws txoj kev fun1(), ces peb siv txoj kev mus rau qhov tau txais fun2(), tom qab fun3(). Cov txiaj ntsig tau txais kev cawmdim rau hauv ib qho khoom ob .

Cov qauv ntaub ntawv

Cov ntaub ntawv hauv R thiab Python zoo sib xws, tab sis muaj cov npe sib txawv.

piav qhia
Lub npe hauv R
Lub npe hauv Python/pandas

Rooj qauv
data.frame, data.table, tib
DataFrame

Ib-dimensional daim ntawv teev cov nqi
Vector
Series hauv pandas lossis npe hauv Python ntshiab

Multi-level non-tabular qauv
Sau npe
Phau ntawv txhais lus (dict)

Peb yuav saib qee qhov nta thiab qhov sib txawv ntawm cov syntax hauv qab no.

Ob peb lo lus hais txog cov pob peb yuav siv

Ua ntej, kuv yuav qhia koj me ntsis txog cov pob khoom uas koj yuav paub txog thaum lub sijhawm tsab xov xwm no.

tidyverse

Official lub website: tidyverse.org

tsev qiv ntawv tidyverse sau los ntawm Hedley Wickham, Senior Research Scientist ntawm RStudio. tidyverse muaj cov txheej txheem zoo ntawm cov pob khoom uas yooj yim ua cov ntaub ntawv, 5 ntawm cov uas tau suav nrog hauv 10 rub tawm sab saum toj los ntawm CRAN repository.

Lub hauv paus ntawm lub tsev qiv ntawv muaj cov pob hauv qab no: ggplot2, dplyr, tidyr, readr, purrr, tibble, stringr, forcats. Txhua pob khoom no yog tsom rau kev daws teeb meem tshwj xeeb. Piv txwv li dplyr tsim rau cov ntaub ntawv manipulation, tidyr coj cov ntaub ntawv mus rau ib daim ntawv zoo, stringr simplifies ua hauj lwm nrog cov hlua, thiab ggplot2 yog ib qho ntawm cov ntaub ntawv nrov tshaj plaws pom cov cuab yeej.

Qhov zoo tidyverse yog qhov yooj yim thiab yooj yim nyeem syntax, uas yog nyob rau hauv ntau txoj kev zoo ib yam li SQL lus nug.

cov ntaub ntawv

Official lub website: r-datatable.com ua

Los ntawm data.table yog Matt Dole ntawm H2O.ai.

Thawj qhov kev tso tawm ntawm lub tsev qiv ntawv tau tshwm sim hauv xyoo 2006.

Lub pob syntax tsis yooj yim li hauv tidyverse thiab yog ntau reminiscent ntawm classic dataframes nyob rau hauv R, tab sis nyob rau tib lub sij hawm ho nthuav nyob rau hauv functionality.

Tag nrho cov manipulations nrog lub rooj nyob rau hauv lub pob no yog piav nyob rau hauv square brackets, thiab yog hais tias koj txhais cov syntax data.table hauv SQL, koj tau txais qee yam zoo li no: data.table[ WHERE, SELECT, GROUP BY ]

Lub zog ntawm lub pob no yog qhov ceev ntawm kev ua cov ntaub ntawv loj.

panda

Official lub website: pandas.pydata.org

Lub npe ntawm lub tsev qiv ntawv los ntawm lo lus econometric "cov ntaub ntawv vaj huam sib luag", siv los piav qhia ntau yam txheej txheem ntawm cov ntaub ntawv.

Los ntawm pandas yog American Wes McKinney.

Thaum nws los txog rau kev txheeb xyuas cov ntaub ntawv hauv Python, sib npaug pandas Tsis muaj. Multifunctional heev, qib siab pob uas tso cai rau koj los ua txhua yam kev tswj hwm nrog cov ntaub ntawv, los ntawm kev thauj cov ntaub ntawv los ntawm txhua qhov chaw kom pom nws.

Txhim kho cov pob khoom ntxiv

Cov pob khoom sib tham hauv kab lus no tsis suav nrog hauv R thiab Python kev faib tawm yooj yim. Txawm hais tias muaj qhov tsis txaus ntseeg me me, yog tias koj tau teeb tsa Anaconda faib, ces nruab ntxiv pandas tsis xav tau.

Txhim kho pob khoom hauv R

Yog tias koj tau qhib RStudio kev txhim kho ib puag ncig tsawg kawg ib zaug, tej zaum koj twb paub yuav ua li cas rau nruab cov pob yuav tsum tau ua hauv R. Txhawm rau nruab cov pob, siv cov lus txib txheem install.packages() los ntawm kev khiav nws ncaj qha hauv R nws tus kheej.

# установка пакетов
install.packages("vroom")
install.packages("readr")
install.packages("dplyr")
install.packages("data.table")

Tom qab kev teeb tsa, cov pob yuav tsum tau txuas nrog, uas feem ntau siv cov lus txib library().

# подключение или импорт пакетов в рабочее окружение
library(vroom)
library(readr)
library(dplyr)
library(data.table)

Txhim kho pob hauv Python

Yog li, yog tias koj muaj Python ntshiab ntsia, ces pandas koj yuav tsum nruab nws manually. Qhib cov kab hais kom ua, lossis lub davhlau ya nyob twg, nyob ntawm koj qhov kev khiav haujlwm thiab nkag mus rau cov lus txib hauv qab no.

pip install pandas

Tom qab ntawd peb rov qab mus rau Python thiab import lub pob teeb tsa nrog cov lus txib import.

import pandas as pd

Chaw thau khoom

Kev tshawb nrhiav cov ntaub ntawv yog ib qho tseem ceeb tshaj plaws hauv kev txheeb xyuas cov ntaub ntawv. Ob leeg Python thiab R, yog tias xav tau, muab sijhawm rau koj kom tau txais cov ntaub ntawv los ntawm txhua qhov chaw: cov ntaub ntawv hauv zos, cov ntaub ntawv hauv Is Taws Nem, cov vev xaib, txhua yam ntawm cov ntaub ntawv.

Nyob rau hauv tsab xov xwm peb yuav siv ob peb datasets:

Ob qhov rub tawm los ntawm Google Analytics.
Titanic Neeg nrog caij Dataset.

Tag nrho cov ntaub ntawv yog nyob ntawm kuv GitHub in the form of csv and tsv files. Peb yuav thov lawv qhov twg los?

Thauj cov ntaub ntawv rau hauv R: tidyverse, vroom, readr

Txhawm rau thauj cov ntaub ntawv mus rau hauv lub tsev qiv ntawv tidyverse Muaj ob lub pob: vroom, readr. vroom niaj hnub ntau dua, tab sis yav tom ntej cov pob khoom tuaj yeem ua ke.

Quote los ntawm cov ntaub ntawv raug cai vroom.

vroom vs nyeem
Dab tsi yog qhov tso tawm ntawm vroom txhais rau readr? Txog tam sim no peb npaj cia ob pob khoom hloov pauv sib txawv, tab sis tej zaum peb yuav sib sau ua ke cov pob khoom yav tom ntej. Ib qho tsis zoo rau vroom tus tub nkees nyeem ntawv yog qee cov ntaub ntawv teeb meem tsis tuaj yeem tshaj tawm pem hauv ntej, yog li yuav ua li cas zoo tshaj los koom ua ke lawv yuav tsum muaj qee qhov kev xav.

vroom vs readr
Kev tso tawm txhais li cas? vroom rau readr? Tam sim no peb npaj yuav tsim ob pob khoom sib cais, tab sis peb yuav zaum ua ke rau yav tom ntej. Ib qho tsis zoo ntawm tub nkeeg nyeem ntawv vroom yog tias qee qhov teeb meem nrog cov ntaub ntawv tsis tuaj yeem tshaj tawm ua ntej, yog li koj yuav tsum xav txog yuav ua li cas zoo tshaj los muab lawv.

Hauv tsab xov xwm no peb yuav saib ob qho tib si cov ntaub ntawv thauj khoom:

Loading cov ntaub ntawv rau hauv R: vroom pob

# install.packages("vroom")
library(vroom)

# Чтение данных
## vroom
ga_nov  <- vroom("https://raw.githubusercontent.com/selesnow/publications/master/data_example/r_python_data/ga_nowember.csv")
ga_dec  <- vroom("https://raw.githubusercontent.com/selesnow/publications/master/data_example/r_python_data/ga_december.csv")
titanic <- vroom("https://raw.githubusercontent.com/selesnow/publications/master/data_example/r_python_data/titanic.csv")

Loading cov ntaub ntawv rau hauv R: readr

# install.packages("readr")
library(readr)

# Чтение данных
## readr
ga_nov  <- read_tsv("https://raw.githubusercontent.com/selesnow/publications/master/data_example/r_python_data/ga_nowember.csv")
ga_dec  <- read_tsv("https://raw.githubusercontent.com/selesnow/publications/master/data_example/r_python_data/ga_december.csv")
titanic <- read_csv("https://raw.githubusercontent.com/selesnow/publications/master/data_example/r_python_data/titanic.csv")

Hauv pob vroom, tsis hais txog cov ntaub ntawv csv / tsv, kev thauj khoom yog nqa tawm los ntawm kev ua haujlwm ntawm tib lub npe vroom(), hauv pob readr peb siv cov haujlwm sib txawv rau txhua hom ntawv read_tsv() и read_csv().

Loading cov ntaub ntawv rau hauv R: data.table

В data.table muaj kev ua haujlwm rau thauj cov ntaub ntawv fread().

Thauj cov ntaub ntawv rau hauv R: data.table pob

# install.packages("data.table")
library(data.table)

## data.table
ga_nov  <- fread("https://raw.githubusercontent.com/selesnow/publications/master/data_example/r_python_data/ga_nowember.csv")
ga_dec  <- fread("https://raw.githubusercontent.com/selesnow/publications/master/data_example/r_python_data/ga_december.csv")
titanic <- fread("https://raw.githubusercontent.com/selesnow/publications/master/data_example/r_python_data/titanic.csv")

Thauj cov ntaub ntawv hauv Python: pandas

Yog tias peb piv nrog R pob, ces qhov no cov syntax yog ze tshaj plaws pandas yuav readr, vim pandas tuaj yeem thov cov ntaub ntawv los ntawm txhua qhov chaw, thiab muaj tag nrho tsev neeg ntawm kev ua haujlwm hauv pob no read_*().

read_csv()
read_excel()
read_sql()
read_json()
read_html()

Thiab ntau lwm txoj haujlwm tsim los nyeem cov ntaub ntawv los ntawm ntau hom ntawv. Tab sis rau peb lub hom phiaj nws yog txaus read_table() los yog read_csv() siv kev sib cav Sep txhawm rau txheeb xyuas kab sib cais.

Thauj cov ntaub ntawv hauv Python: pandas

import pandas as pd

ga_nov  = pd.read_csv("https://raw.githubusercontent.com/selesnow/publications/master/data_example/russian_text_in_r/ga_nowember.csv", sep = "t")
ga_dec  = pd.read_csv("https://raw.githubusercontent.com/selesnow/publications/master/data_example/russian_text_in_r/ga_december.csv", sep = "t")
titanic = pd.read_csv("https://raw.githubusercontent.com/selesnow/publications/master/data_example/russian_text_in_r/titanic.csv")

Tsim dataframes

Rooj Titanic, uas peb loaded, muaj ib daim teb Pw ua niam txiv, uas khaws cov neeg caij tsheb tus cim poj niam txiv neej.

Tab sis rau kev nthuav qhia yooj yim dua ntawm cov ntaub ntawv hais txog poj niam txiv neej cov neeg caij tsheb, koj yuav tsum siv lub npe es tsis yog poj niam txiv neej code.

Ua li no, peb yuav tsim ib phau ntawv me me, ib lub rooj uas tsuas muaj 2 kab (code thiab poj niam txiv neej lub npe) thiab 2 kab, feem.

Tsim ib dataframe hauv R: tidyverse, dplyr

Hauv qhov piv txwv code hauv qab no, peb tsim qhov xav tau dataframe siv cov haujlwm tibble() .

Tsim ib dataframe hauv R: dplyr

## dplyr
### создаём справочник
gender <- tibble(id = c(1, 2),
                 gender = c("female", "male"))

Tsim ib dataframe hauv R: data.table

Tsim ib dataframe hauv R: data.table

## data.table
### создаём справочник
gender <- data.table(id = c(1, 2),
                    gender = c("female", "male"))

Tsim ib dataframe hauv Python: pandas

В pandas tsim thav ntawv yog nqa tawm nyob rau hauv ob peb theem, ua ntej peb tsim ib phau ntawv txhais lus, thiab ces peb hloov cov phau ntawv txhais lus rau hauv ib dataframe.

Tsim ib dataframe hauv Python: pandas

# создаём дата фрейм
gender_dict = {'id': [1, 2],
               'gender': ["female", "male"]}
# преобразуем словарь в датафрейм
gender = pd.DataFrame.from_dict(gender_dict)

Xaiv Kab

Cov ntxhuav koj ua haujlwm nrog yuav muaj kaum lossis ntau pua kab ntawm cov ntaub ntawv. Tab sis txhawm rau ua tiav kev txheeb xyuas, raws li txoj cai, koj tsis tas yuav tag nrho cov kab ntawv uas muaj nyob rau hauv lub ntsiab lus.

Yog li ntawd, ib qho ntawm thawj cov haujlwm uas koj yuav ua nrog lub rooj sib tham yog kom tshem tawm cov ntaub ntawv tsis tsim nyog thiab tso lub cim xeeb uas cov ntaub ntawv no nyob.

Xaiv cov kab hauv R: tidyverse, dplyr

syntax dplyr Nws zoo ib yam li cov lus nug SQL, yog tias koj paub txog nws koj yuav ceev cov pob no.

Txhawm rau xaiv kab, siv qhov ua haujlwm select().

Hauv qab no yog cov piv txwv ntawm cov cai uas koj tuaj yeem xaiv cov kab hauv qab no:

Sau cov npe ntawm cov kab uas xav tau
Xa mus rau kab npe siv cov kab lus tsis tu ncua
Los ntawm hom ntaub ntawv lossis lwm yam khoom ntiag tug ntawm cov ntaub ntawv muaj nyob hauv kab

Xaiv cov kab hauv R: dplyr

# Выбор нужных столбцов
## dplyr
### выбрать по названию столбцов
select(ga_nov, date, source, sessions)
### исключь по названию столбцов
select(ga_nov, -medium, -bounces)
### выбрать по регулярному выражению, стобцы имена которых заканчиваются на s
select(ga_nov, matches("s$"))
### выбрать по условию, выбираем только целочисленные столбцы
select_if(ga_nov, is.integer)

Xaiv kab hauv R: data.table

Tib yam haujlwm hauv data.table tau ua txawv me ntsis, thaum pib ntawm tsab xov xwm kuv tau muab cov lus piav qhia ntawm cov lus sib cav dab tsi nyob rau hauv square brackets hauv data.table.

DT[i,j,by]

Qhov twg:
kuv, i.e. lim los ntawm kab
j - xaiv|hloov tshiab|ua, i.e. xaiv kab thiab hloov lawv
los ntawm - data grouping

Xaiv kab hauv R: data.table

## data.table
### выбрать по названию столбцов
ga_nov[ , .(date, source, sessions) ]
### исключь по названию столбцов
ga_nov[ , .SD, .SDcols = ! names(ga_nov) %like% "medium|bounces" ]
### выбрать по регулярному выражению
ga_nov[, .SD, .SDcols = patterns("s$")]

Kuj tsis paub meej .SD tso cai rau koj nkag mus rau txhua kab, thiab .SDcols lim cov kab uas xav tau siv cov kab lus tsis tu ncua, lossis lwm yam haujlwm los lim cov npe ntawm kab koj xav tau.

Xaiv kab hauv Python, pandas

Txhawm rau xaiv kab los ntawm lub npe hauv pandas Nws yog txaus los muab ib daim ntawv teev npe ntawm lawv cov npe. Thiab txhawm rau xaiv lossis cais cov kab los ntawm lub npe siv cov kab lus tsis tu ncua, koj yuav tsum siv cov haujlwm drop() и filter(), thiab kev sib cav axis = 1, uas koj qhia tias nws yog ib qho tsim nyog los ua cov kab ntau dua li cov kab.

Txhawm rau xaiv ib daim teb los ntawm hom ntaub ntawv, siv cov haujlwm select_dtypes(), thiab hauv kev sib cav suav nrog los yog tsis suav hla ib daim ntawv teev cov ntaub ntawv hom sib raug rau cov teb uas koj xav tau xaiv.

Xaiv kab hauv Python: pandas

# Выбор полей по названию
ga_nov[['date', 'source', 'sessions']]
# Исключить по названию
ga_nov.drop(['medium', 'bounces'], axis=1)
# Выбрать по регулярному выражению
ga_nov.filter(regex="s$", axis=1)
# Выбрать числовые поля
ga_nov.select_dtypes(include=['number'])
# Выбрать текстовые поля
ga_nov.select_dtypes(include=['object'])

Lim kab

Piv txwv li, lub hauv paus ntsiab lus yuav muaj ob peb xyoos ntawm cov ntaub ntawv, tab sis koj tsuas yog yuav tsum tau soj ntsuam lub hli tas los. Ntxiv dua thiab, cov kab ntxiv yuav ua rau cov txheej txheem ua cov ntaub ntawv qeeb thiab txhaws lub PC nco.

Lim kab hauv R: tydyverse, dplyr

В dplyr txoj haujlwm yog siv los lim kab filter(). Nws yuav siv lub dataframe ua thawj qhov kev sib cav, ces koj sau cov kev lim dej.

Thaum sau cov lus qhia kev xav kom lim lub rooj, hauv qhov no, qhia cov npe ntawm kab ntawv yam tsis muaj nqe lus thiab tsis tshaj tawm lub npe lub rooj.

Thaum siv ntau qhov kev xav tau los lim, siv cov neeg ua haujlwm hauv qab no:

& lossis comma - logical THIAB
| - logical LOS YOG

Lim kab hauv R: dplyr

# фильтрация строк
## dplyr
### фильтрация строк по одному условию
filter(ga_nov, source == "google")
### фильтр по двум условиям соединённым логическим и
filter(ga_nov, source == "google" & sessions >= 10)
### фильтр по двум условиям соединённым логическим или
filter(ga_nov, source == "google" | sessions >= 10)

Lim kab hauv R: data.table

Raws li kuv tau sau saum toj no, hauv data.table cov ntaub ntawv hloov dua siab tshiab syntax yog nyob rau hauv square brackets.

DT[i,j,by]

Qhov twg:
kuv, i.e. lim los ntawm kab
j - xaiv|hloov tshiab|ua, i.e. xaiv kab thiab hloov lawv
los ntawm - data grouping

Cov lus sib cav yog siv los lim kab i, uas muaj thawj txoj hauj lwm nyob rau hauv square brackets.

Cov kab yog nkag mus rau hauv cov lus piav qhia tsis muaj cov lus hais thiab tsis qhia lub npe lub rooj.

Logical expressions muaj feem xyuam rau ib leeg ib yam li nyob rau hauv dplyr los ntawm & thiab | tus neeg ua haujlwm.

Lim kab hauv R: data.table

## data.table
### фильтрация строк по одному условию
ga_nov[source == "google"]
### фильтр по двум условиям соединённым логическим и
ga_nov[source == "google" & sessions >= 10]
### фильтр по двум условиям соединённым логическим или
ga_nov[source == "google" | sessions >= 10]

Lim cov hlua hauv Python: pandas

Lim los ntawm kab hauv pandas zoo ib yam li lim hauv data.table, thiab yog ua nyob rau hauv square brackets.

Hauv qhov no, kev nkag mus rau txhua kab yog ua tiav los ntawm kev qhia lub npe ntawm dataframe; tom qab ntawd lub npe kab kuj tseem tuaj yeem qhia hauv cov lus hais hauv cov kab lus square (Piv txwv df['col_name']), los yog tsis muaj quotes tom qab lub sij hawm (Piv txwv df.col_name).

Yog tias koj xav tau lim cov dataframe los ntawm ntau qhov xwm txheej, txhua qhov xwm txheej yuav tsum tau muab tso rau hauv kab lus. Logical tej yam kev mob yog txuas nrog ib leeg los ntawm cov tswv & и |.

Lim cov hlua hauv Python: pandas

# Фильтрация строк таблицы
### фильтрация строк по одному условию
ga_nov[ ga_nov['source'] == "google" ]
### фильтр по двум условиям соединённым логическим и
ga_nov[(ga_nov['source'] == "google") & (ga_nov['sessions'] >= 10)]
### фильтр по двум условиям соединённым логическим или
ga_nov[(ga_nov['source'] == "google") | (ga_nov['sessions'] >= 10)]

Grouping thiab aggregation ntawm cov ntaub ntawv

Ib qho ntawm cov haujlwm feem ntau siv hauv kev txheeb xyuas cov ntaub ntawv yog kev sib koom ua ke thiab sib sau ua ke.

Cov syntax rau kev ua cov haujlwm no tau tawg mus thoob plaws txhua pob peb tshuaj xyuas.

Hauv qhov no, peb yuav coj tus dataframe ua piv txwv Titanic, thiab suav tus lej thiab tus nqi nruab nrab ntawm daim pib nyob ntawm chav tsev nyob.

Grouping thiab aggregation ntawm cov ntaub ntawv nyob rau hauv R: tidyverse, dplyr

В dplyr txoj haujlwm yog siv rau kev ua pab pawg group_by(), thiab rau aggregation summarise(). Qhov tseeb, dplyr muaj tag nrho tsev neeg ntawm kev ua haujlwm summarise_*(), tab sis lub hom phiaj ntawm tsab xov xwm no yog los sib piv cov syntax yooj yim, yog li peb yuav tsis mus rau hauv hav zoov zoo li no.

Basic aggregation functions:

sum() — summation
min() / max() - yam tsawg kawg nkaus thiab siab tshaj tus nqi
mean() - nruab nrab
median() - nruab nrab
length() - kom muaj nuj nqis

Grouping thiab aggregation nyob rau hauv R: dplyr

## dplyr
### группировка и агрегация строк
group_by(titanic, Pclass) %>%
  summarise(passangers = length(PassengerId),
            avg_price  = mean(Fare))

Ua haujlwm group_by() peb dhau lub rooj ua thawj qhov kev sib cav Titanic, thiab tom qab ntawd qhia lub teb Pclass, los ntawm qhov uas peb yuav pab pawg peb lub rooj. Qhov tshwm sim ntawm qhov kev ua haujlwm no siv tus neeg teb xov tooj %>% dhau raws li thawj qhov kev sib cav rau lub luag haujlwm summarise(), thiab ntxiv 2 qhov chaw ntxiv: cov neeg caij tsheb и avg_nqi. Hauv thawj zaug, siv cov haujlwm length() xam tus naj npawb ntawm daim pib, thiab nyob rau hauv lub thib ob siv lub functionality mean() tau txais daim pib nqi nruab nrab.

Grouping thiab aggregation ntawm cov ntaub ntawv nyob rau hauv R: data.table

В data.table kev sib cav yog siv rau kev sib sau j uas muaj ob txoj hauj lwm nyob rau hauv square brackets, thiab rau pab pawg by los yog keyby, uas muaj qhov thib peb txoj haujlwm.

Cov npe ntawm kev sib sau ua haujlwm nyob rau hauv rooj plaub no zoo ib yam li tau piav qhia hauv dplyr, vim cov no yog cov haujlwm los ntawm qhov yooj yim R syntax.

Grouping thiab aggregation nyob rau hauv R: data.table

## data.table
### фильтрация строк по одному условию
titanic[, .(passangers = length(PassengerId),
            avg_price  = mean(Fare)),
        by = Pclass]

Pawg thiab kev sib sau ntawm cov ntaub ntawv hauv Python: pandas

Pab pawg hauv pandas zoo ib yam li dplyr, tab sis kev sib sau ua ke tsis zoo ib yam li dplyr tsis yog kiag li data.table.

Rau pab pawg, siv txoj kev groupby(), nyob rau hauv uas koj yuav tsum dhau ib daim ntawv teev cov kab uas cov dataframe yuav muab pawg.

Rau kev sib sau koj tuaj yeem siv txoj kev agg()uas lees txais phau ntawv txhais lus. Cov yuam sij phau ntawv txhais lus yog cov kab uas koj yuav siv cov kev sib sau ua ke, thiab cov txiaj ntsig yog cov npe ntawm cov haujlwm sib sau ua ke.

Aggregation muaj nuj nqi:

sum() — summation
min() / max() - yam tsawg kawg nkaus thiab siab tshaj tus nqi
mean() - nruab nrab
median() - nruab nrab
count() - kom muaj nuj nqis

muaj nuj nqi reset_index() nyob rau hauv cov piv txwv hauv qab no nws yog siv los pib dua nested indexes uas pandas defaults rau tom qab cov ntaub ntawv aggregation.

Cim tso cai rau koj txav mus rau kab tom ntej.

Grouping thiab aggregation nyob rau hauv Python: pandas

# группировка и агрегация данных
titanic.groupby(["Pclass"]).
    agg({'PassengerId': 'count', 'Fare': 'mean'}).
        reset_index()

Vertical koom ntawm rooj

Ib qho kev ua haujlwm uas koj koom nrog ob lossis ntau lub rooj ntawm tib lub qauv. Cov ntaub ntawv peb thauj khoom muaj cov ntxhuav ga_nov и ga_ lis. Cov rooj no zoo ib yam hauv cov qauv, i.e. muaj tib kab, thiab cov ntaub ntawv hom nyob rau hauv cov kab no.

Qhov no yog ib qho upload los ntawm Google Analytics rau lub hli ntawm Kaum Ib Hlis thiab Kaum Ob Hlis, hauv ntu no peb yuav muab cov ntaub ntawv no tso rau hauv ib lub rooj.

Vertically koom cov rooj hauv R: tidyverse, dplyr

В dplyr Koj tuaj yeem ua ke 2 lub rooj rau hauv ib qho uas siv cov haujlwm bind_rows(), dhau lub rooj raws li nws cov lus sib cav.

Lim kab hauv R: dplyr

# Вертикальное объединение таблиц
## dplyr
bind_rows(ga_nov, ga_dec)

Vertically koom cov rooj hauv R: data.table

Nws kuj tsis muaj dab tsi nyuab, cia peb siv rbind().

Lim kab hauv R: data.table

## data.table
rbind(ga_nov, ga_dec)

Vertically koom cov rooj hauv Python: pandas

В pandas txoj haujlwm yog siv los koom nrog cov rooj concat(), nyob rau hauv uas koj yuav tsum dhau ib daim ntawv teev cov thav duab los ua ke lawv.

Lim cov hlua hauv Python: pandas

# вертикальное объединение таблиц
pd.concat([ga_nov, ga_dec])

Kab rov tav sib koom ntawm cov rooj

Ib qho kev ua haujlwm uas cov kab los ntawm qhov thib ob ntxiv rau thawj lub rooj los ntawm qhov tseem ceeb. Nws yog feem ntau siv thaum ntxiv cov lus tseeb (piv txwv li, lub rooj nrog cov ntaub ntawv muag khoom) nrog qee cov ntaub ntawv siv (piv txwv li, tus nqi ntawm cov khoom).

Muaj ntau ntau hom kev koom nrog:

Nyob rau hauv lub rooj ua ntej loaded Titanic peb muaj ib kem Pw ua niam txiv, uas sib haum rau tus neeg caij tsheb tus tub los ntxhais code:

1 - poj niam
2 - txiv neej

Tsis tas li ntawd, peb tau tsim ib lub rooj - ib phau ntawv siv poj niam txiv neej. Rau kev nthuav qhia yooj yim dua ntawm cov ntaub ntawv ntawm poj niam txiv neej ntawm cov neeg caij tsheb, peb yuav tsum ntxiv lub npe ntawm poj niam los txiv neej los ntawm phau ntawv teev npe poj niam txiv neej mus rau lub rooj Titanic.

Kab rov tav rooj koom nrog R: tidyverse, dplyr

В dplyr Muaj tag nrho tsev neeg ntawm kev ua haujlwm rau kev sib koom ua ke kab rov tav:

inner_join()
left_join()
right_join()
full_join()
semi_join()
nest_join()
anti_join()

Feem ntau siv nyob rau hauv kuv xyaum yog left_join().

Raws li thawj ob qhov kev sib cav, cov haujlwm tau teev tseg saum toj no coj ob lub rooj los koom, thiab raws li qhov kev sib cav thib peb by koj yuav tsum qhia kom meej cov kab mus koom.

Kab rov tav rooj koom nrog R: dplyr

# объединяем таблицы
left_join(titanic, gender,
          by = c("Sex" = "id"))

Kab rov tav sib koom ntawm cov ntxhuav hauv R: data.table

В data.table Koj yuav tsum koom nrog cov ntxhuav los ntawm qhov tseem ceeb uas siv cov haujlwm merge().

Cov lus sib cav rau kev sib koom ua ke () ua haujlwm hauv data.table

x, y — Cov rooj sib koom
los ntawm - Kem uas yog tus yuam sij rau koom yog tias nws muaj tib lub npe hauv ob lub rooj
by.x, by.y — Kem cov npe yuav tsum muab sib xyaw, yog tias lawv muaj cov npe sib txawv hauv cov ntxhuav
tag nrho, all.x, all.y — Koom hom, tag nrho yuav rov qab tag nrho cov kab ntawm ob lub rooj, all.x sib raug rau LEFT JOIN ua haujlwm (yuav tawm tag nrho cov kab ntawm thawj lub rooj), all.y — sib raug rau RIGHT JOIN ua haujlwm (yuav tawm tag nrho cov kab ntawm lub rooj thib ob).

Kab rov tav sib koom ntawm cov ntxhuav hauv R: data.table

# объединяем таблицы
merge(titanic, gender, by.x = "Sex", by.y = "id", all.x = T)

Kab rov tav rooj koom nrog Python: pandas

Zoo li hauv data.table, nyob rau hauv pandas txoj haujlwm yog siv los koom nrog cov rooj merge().

Kev sib cav ntawm kev sib koom ua ke () ua haujlwm hauv pandas

yuav ua li cas - Kev sib txuas hom: sab laug, sab xis, sab nraud, sab hauv
ntawm - Kem uas yog tus yuam sij yog tias nws muaj tib lub npe hauv ob lub rooj
left_on, right_on - Cov npe ntawm cov kab ntawv tseem ceeb, yog tias lawv muaj cov npe sib txawv hauv cov lus

Kab rov tav rooj koom nrog Python: pandas

# объединяем по ключу
titanic.merge(gender, how = "left", left_on = "Sex", right_on = "id")

Lub qhov rais yooj yim ua haujlwm thiab suav cov kab

Lub qhov rais ua haujlwm zoo ib yam hauv lub ntsiab lus rau kev sib sau ua ke, thiab feem ntau siv rau hauv kev tshuaj xyuas cov ntaub ntawv. Tab sis tsis zoo li aggregation functions, qhov rais functions tsis hloov tus naj npawb ntawm kab ntawm cov ntaub ntawv tawm.

Qhov tseem ceeb, siv lub qhov rais ua haujlwm, peb faib cov ntaub ntawv nkag mus rau hauv qhov chaw raws li qee qhov kev ntsuas, i.e. los ntawm tus nqi ntawm ib daim teb, los yog ob peb lub teb. Thiab peb ua haujlwm lej lej ntawm txhua lub qhov rais. Cov txiaj ntsig ntawm cov haujlwm no yuav raug xa rov qab rau hauv txhua kab, i.e. tsis hloov tag nrho cov kab hauv lub rooj.

Piv txwv li, cia peb coj lub rooj Titanic. Peb tuaj yeem xam qhov feem pua ntawm cov nqi ntawm txhua daim pib nyob hauv nws chav tsev nyob.

Ua li no, peb yuav tsum tau txais nyob rau hauv txhua kab ntawm tag nrho cov nqi ntawm daim pib rau chav tsev nyob tam sim no, uas daim pib nyob rau hauv kab no belongs, ces faib tus nqi ntawm txhua daim pib los ntawm tag nrho cov nqi ntawm tag nrho cov daim pib ntawm tib chav tsev cabin. .

Qhov rai ua haujlwm hauv R: tidyverse, dplyr

Txhawm rau ntxiv cov kab ntawv tshiab, tsis tas siv kab ua pawg, hauv dplyr ua haujlwm mutate().

Koj tuaj yeem daws qhov teeb meem tau piav qhia saum toj no los ntawm kev muab cov ntaub ntawv los ntawm thaj chaw Pclass thiab summing lub teb nyob rau hauv ib kab tshiab Nqi. Tom ntej no, ungroup lub rooj thiab faib cov nqi teb Nqi rau yam uas tau tshwm sim hauv cov kauj ruam dhau los.

Qhov rai ua haujlwm hauv R: dplyr

group_by(titanic, Pclass) %>%
  mutate(Pclass_cost = sum(Fare)) %>%
  ungroup() %>%
  mutate(ticket_fare_rate = Fare / Pclass_cost)

Qhov rai ua haujlwm hauv R: data.table

Kev daws algorithm tseem zoo ib yam li hauv dplyr, peb yuav tsum tau faib lub rooj rau hauv qhov rais los ntawm teb Pclass. Muab tso rau hauv ib kem tshiab tus nqi rau cov pab pawg sib raug rau txhua kab, thiab ntxiv ib kem nyob rau hauv uas peb xam qhov feem ntawm tus nqi ntawm txhua daim pib nyob rau hauv nws pab pawg neeg.

Ntxiv cov kab tshiab rau data.table tus neeg teb xov tooj tam sim no :=. Hauv qab no yog ib qho piv txwv ntawm kev daws teeb meem uas siv lub pob data.table

Qhov rai ua haujlwm hauv R: data.table

titanic[,c("Pclass_cost","ticket_fare_rate") := .(sum(Fare), Fare / Pclass_cost), 
        by = Pclass]

Qhov rai ua haujlwm hauv Python: pandas

Ib txoj hauv kev ntxiv ib kab tshiab rau pandas - siv lub luag haujlwm assign(). Txhawm rau txiav txim siab tus nqi ntawm daim pib los ntawm chav tsev cabin, tsis muaj pawg kab, peb yuav siv txoj haujlwm transform().

Hauv qab no yog ib qho piv txwv ntawm kev daws teeb meem uas peb ntxiv rau lub rooj Titanic tib 2 kem.

Qhov rai ua haujlwm hauv Python: pandas

titanic.assign(Pclass_cost      =  titanic.groupby('Pclass').Fare.transform(sum),
               ticket_fare_rate = lambda x: x['Fare'] / x['Pclass_cost'])

Lub luag haujlwm thiab cov txheej txheem sau ntawv

Hauv qab no yog lub rooj sib tham ntawm cov txheej txheem rau kev ua haujlwm ntau yam nrog cov ntaub ntawv hauv cov pob peb tau txiav txim siab.

piav qhia
tidyverse
cov ntaub ntawv
panda

Chaw thau khoom
vroom()/ readr::read_csv() / readr::read_tsv()
fread()
read_csv()

Tsim dataframes
tibble()
data.table()
dict() + from_dict()

Xaiv Kab
select()
kev sib cav j, thib ob txoj hauj lwm nyob rau hauv square brackets
peb dhau daim ntawv teev cov kab uas yuav tsum tau muaj nyob rau hauv square brackets / drop() / filter() / select_dtypes()

Lim kab
filter()
kev sib cav i, thawj txoj hauj lwm hauv square brackets
Peb teev cov xwm txheej lim dej hauv square brackets / filter()

Grouping thiab Aggregation
group_by() + summarise()
cov kev sib cav j + by
groupby() + agg()

Vertical union ntawm lub rooj (UNION)
bind_rows()
rbind()
concat()

Kab rov tav sib koom ntawm cov rooj (JOIN)
left_join() / *_join()
merge()
merge()

Lub qhov rais yooj yim ua haujlwm thiab ntxiv cov lej suav
group_by() + mutate()
kev sib cav j siv tus neeg teb xov tooj := + kev sib cav by
transform() + assign()

xaus

Tej zaum hauv tsab xov xwm kuv tau piav qhia tsis yog qhov ua tau zoo tshaj plaws ntawm kev ua cov ntaub ntawv, yog li kuv yuav zoo siab yog tias koj kho kuv qhov yuam kev hauv cov lus, lossis tsuas yog ntxiv cov ntaub ntawv muab rau hauv tsab xov xwm nrog rau lwm cov txheej txheem rau kev ua haujlwm nrog cov ntaub ntawv hauv R / Python.

Raws li kuv tau sau saum toj no, lub hom phiaj ntawm tsab xov xwm tsis yog los tsim ib qho kev xav ntawm hom lus twg zoo dua, tab sis kom yooj yim rau kev kawm ob hom lus, lossis, yog tias tsim nyog, tsiv ntawm lawv.

Yog koj nyiam zaj no, kuv yuav zoo siab uas muaj cov neeg tshiab rau kuv youtube и xov tooj cov channel.

Poll

Cov pob khoom twg hauv qab no koj siv hauv koj txoj haujlwm?

Hauv cov lus koj tuaj yeem sau qhov laj thawj rau koj xaiv.

Tsuas yog cov neeg siv sau npe tuaj yeem koom nrog hauv daim ntawv ntsuam xyuas. Kos npe rau hauvthov.

Cov ntaub ntawv dab tsi koj siv (koj tuaj yeem xaiv ntau txoj kev xaiv)

45,2%tidyverse19
33,3%data.tab 14
54,8%pandas 23

42 cov neeg siv pov npav. 9 cov neeg siv txwv tsis pub siv.

Tau qhov twg los: www.hab.com

Hom lus twg los xaiv rau kev ua haujlwm nrog cov ntaub ntawv - R lossis Python? Ob leeg! Migrating los ntawm pandas mus tidyverse thiab data.table thiab rov qab