🥇 Keeb kwm ntawm peb qhov chaw qhib: yuav ua li cas peb ua qhov kev tshuaj ntsuam xyuas hauv Go thiab ua kom muaj kev tshaj tawm

Tam sim no, yuav luag txhua lub tuam txhab hauv ntiaj teb sau cov txheeb cais txog cov neeg siv ua haujlwm ntawm lub vev xaib. Qhov kev txhawb siab yog qhov tseeb - cov tuam txhab xav paub seb lawv cov khoom siv / lub vev xaib siv li cas thiab nkag siab zoo dua rau lawv cov neeg siv. Tau kawg, muaj ntau cov cuab yeej hauv kev ua lag luam los daws qhov teeb meem no - los ntawm kev tshuaj ntsuam xyuas uas muab cov ntaub ntawv nyob rau hauv daim ntawv ntawm dashboards thiab graphs (piv txwv li. Google Analytics) rau Cov Neeg Siv Khoom Platform, uas tso cai rau koj los sau thiab sau cov ntaub ntawv los ntawm ntau qhov chaw hauv ib lub tsev khaws khoom (piv txwv li Segment).

Tab sis peb pom ib qho teeb meem uas tseem tsis tau daws. Li no yug EventNative - qhov kev pabcuam qhib qhov kev tshuaj xyuas. Nyeem txog vim li cas peb txiav txim siab los tsim peb tus kheej cov kev pabcuam, nws tau muab rau peb li cas, thiab qhov kawg tshwm sim yog dab tsi (nrog cov lej).

Vim li cas peb yuav tsum txhim kho peb tus kheej kev pabcuam?

Nws yog lub cuaj caum, peb muaj sia nyob raws li qhov zoo tshaj plaws peb ua tau. 2019, peb tsim API Thawj Cov Neeg Siv Khoom Platform kSense, uas ua rau nws muaj peev xwm sau cov ntaub ntawv los ntawm ntau qhov chaw (Facebook tshaj tawm, Stripe, Salesforce, Google ua si, Google Analytics, thiab lwm yam) kom yooj yim dua cov ntaub ntawv txheeb xyuas, txheeb xyuas cov neeg nyob, thiab lwm yam. Peb tau pom tias ntau tus neeg siv peb lub platform rau kev txheeb xyuas cov ntaub ntawv tshwj xeeb Google Analytics (tom qab GA). Peb tau tham nrog qee tus neeg siv thiab pom tias lawv xav tau cov ntaub ntawv txheeb xyuas rau lawv cov khoom uas lawv tau txais siv GA, tab sis Google ua piv txwv cov ntaub ntawv thiab rau ntau, GA Tus Neeg Siv interface tsis yog tus qauv yooj yim. Peb muaj kev sib tham txaus nrog peb cov neeg siv thiab pom tau tias ntau tus kuj tau siv Segment platform (uas, los ntawm txoj kev, tsuas yog hnub dhau los. muag rau $ 3.2 billion).

Lawv tau teeb tsa Segment javascript pixel ntawm lawv lub vev xaib thiab cov ntaub ntawv hais txog tus cwj pwm ntawm lawv cov neeg siv tau raug thauj mus rau hauv cov ntaub ntawv teev tseg (piv txwv li Postgres). Tab sis Segment kuj muaj nws qhov downside - tus nqi. Piv txwv li, yog tias lub vev xaib muaj peev xwm muaj 90,000 MTU (cov neeg siv taug qab txhua hli), ces koj yuav tsum them ~ 1,000 $ ib hlis rau tus neeg txais nyiaj. Kuj tseem muaj qhov teeb meem thib peb - qee qhov browser txuas ntxiv (xws li AdBlock) tau thaiv kev sau cov kev tshuaj ntsuam vim ... http thov los ntawm browser raug xa mus rau GA thiab Segment domains. Raws li qhov kev xav tau ntawm peb cov neeg siv khoom, peb tau tsim cov kev pabcuam analytics uas sau cov ntaub ntawv tiav (tsis muaj qauv), yog dawb thiab tuaj yeem ua haujlwm ntawm peb tus kheej cov txheej txheem.

Qhov kev pabcuam ua haujlwm li cas

Cov kev pabcuam muaj peb ntu: javascript pixel (uas tom qab peb rov sau dua hauv typescript), cov neeg rau zaub mov yog siv hauv GO hom lus, thiab nws tau npaj siv Redshift thiab BigQuery ua cov ntaub ntawv hauv tsev (tom qab peb ntxiv kev txhawb nqa rau Postgres, ClickHouse thiab Snowflake).

Nws tau txiav txim siab tawm ntawm cov qauv ntawm GA thiab Segment cov xwm txheej tsis hloov. Txhua yam uas xav tau yog kom duplicate tag nrho cov xwm txheej los ntawm lub vev xaib qhov chaw uas lub pixel raug teeb tsa rau peb qhov backend. Raws li nws hloov tawm, qhov no tsis yooj yim ua. Lub Javascript pixel overridden tus thawj GA tsev qiv ntawv txoj kev nrog ib tug tshiab, uas duplicated qhov kev tshwm sim rau hauv peb lub cev.

//'ga' - стандартное название переменной Google Analytics
if (window.ga) {
    ga(tracker => {
        var originalSendHitTask = tracker.get('sendHitTask');
        tracker.set('sendHitTask', (model) => {
            var payLoad = model.get('hitPayload');
            //отправка оригинального события в GA
            originalSendHitTask(model);
            let jsonPayload = this.parseQuery(payLoad);
            //отправка события в наш сервис
            this.send3p('ga', jsonPayload);
        });
    });
}

Nrog rau Segment pixel txhua yam yooj yim dua; nws muaj cov txheej txheem nruab nrab, ib qho uas peb siv.


//'analytics' - стандартное название переменной Segment
if (window.analytics) {
    if (window.analytics.addSourceMiddleware) {
        window.analytics.addSourceMiddleware(chain => {
            try {
		//дублирование события в наш сервис
                this.send3p('ajs', chain.payload);
            } catch (e) {
                LOG.warn('Failed to send an event', e)
            }
	    //отправка оригинального события в Segment
            chain.next(chain.payload);
        });
    } else {
        LOG.warn("Invalid interceptor state. Analytics js initialized, but not completely");
    }
} else {
    LOG.warn('Analytics.js listener is not set.');
}

Ntxiv nrog rau kev luam cov xwm txheej, peb tau ntxiv lub peev xwm xa json arbitrary:


//Отправка событий с произвольным json объектом
eventN.track('product_page_view', {
    product_id: '1e48fb70-ef12-4ea9-ab10-fd0b910c49ce',
    product_price: 399.99,
    price_currency: 'USD'
    product_release_start: '2020-09-25T12:38:27.763000Z'
});

Tom ntej no, cia peb tham txog qhov server. Lub backend yuav tsum lees txais http thov, sau lawv nrog cov ntaub ntawv ntxiv, piv txwv li, cov ntaub ntawv geo (ua tsaug maxmind ua rau qhov no) thiab sau nws hauv database. Peb xav ua kom cov kev pabcuam yooj yim li sai tau kom nws tuaj yeem siv nrog kev teeb tsa tsawg. Peb tau siv qhov kev ua haujlwm ntawm kev txiav txim siab cov ntaub ntawv schema raws li cov qauv ntawm cov khoom json tuaj. Cov ntaub ntawv hom yog txhais los ntawm cov nqi. Nested khoom yog decomposed thiab txo mus rau ib tug tiaj tus qauv:

//входящий json
{
  "field_1":  {
    "sub_field_1": "text1",
    "sub_field_2": 100
  },
  "field_2": "text2",
  "field_3": {
    "sub_field_1": {
      "sub_sub_field_1": "2020-09-25T12:38:27.763000Z"
    }
  }
}

//результат
{
  "field_1_sub_field_1":  "text1",
  "field_1_sub_field_2":  100,
  "field_2": "text2",
  "field_3_sub_field_1_sub_sub_field_1": "2020-09-25T12:38:27.763000Z"
}

Txawm li cas los xij, arrays tam sim no tsuas yog hloov mus rau cov hlua vim Tsis yog txhua qhov kev sib raug zoo databases txhawb cov teb rov qab. Nws kuj tseem tuaj yeem hloov cov npe teb lossis tshem tawm lawv siv cov cai tswj kev xaiv. Lawv tso cai rau koj hloov cov ntaub ntawv schema yog tias tsim nyog lossis hloov ib hom ntaub ntawv mus rau lwm qhov. Piv txwv li, yog json teb muaj ib txoj hlua nrog timestamp (field_3_sub_field_1_sub_sub_field_1 los ntawm qhov piv txwv saum toj no), tom qab ntawd txhawm rau tsim ib daim teb hauv cov ntaub ntawv nrog lub sijhawm sau ntawv, koj yuav tsum tau sau ib daim ntawv teev cov cai hauv kev teeb tsa. Hauv lwm lo lus, hom ntaub ntawv ntawm daim teb yog txiav txim siab ua ntej los ntawm tus nqi json, thiab tom qab ntawd siv txoj cai hom casting (yog tias teeb tsa) raug siv. Peb tau txheeb xyuas 4 hom ntaub ntawv tseem ceeb: STRING, FLOAT64, INT64 thiab TIMESTAMP. Daim ntawv qhia thiab hom kev cai casting zoo li no:

rules:
  - "/field_1/subfield_1 -> " #правило удаления поля
  - "/field_2/subfield_1 -> /field_10/subfield_1" #правило переноса поля
  - "/field_3/subfield_1/subsubfield_1 -> (timestamp) /field_20" #правило переноса поля и приведения типа

Algorithm rau kev txiav txim siab hom ntaub ntawv:

hloov json qauv rau cov qauv tiaj tus
txiav txim siab cov ntaub ntawv hom teb los ntawm cov nqi
siv daim ntawv qhia thiab hom kev cai casting

Tom qab ntawd los ntawm cov qauv json tuaj:

{
    "product_id":  "1e48fb70-ef12-4ea9-ab10-fd0b910c49ce",
    "product_price": 399.99,
    "price_currency": "USD",
    "product_type": "supplies",
    "product_release_start": "2020-09-25T12:38:27.763000Z",
    "images": {
      "main": "picture1",
      "sub":  "picture2"
    }
}

cov ntaub ntawv schema yuav tau txais:

"product_id" character varying,
"product_price" numeric (38,18),
"price_currency" character varying,
"product_type" character varying,
"product_release_start" timestamp,
"images_main" character varying,
"images_sub" character varying

Peb kuj tau xav tias tus neeg siv yuav tsum tuaj yeem teeb tsa kev faib lossis faib cov ntaub ntawv hauv cov ntaub ntawv raws li lwm cov qauv thiab siv lub peev xwm los teeb tsa lub npe lub rooj nrog qhov tsis tu ncua lossis kev qhia nyob rau hauv lub configuration. Hauv qhov piv txwv hauv qab no, qhov kev tshwm sim yuav raug cawm rau lub rooj nrog lub npe suav raws li qhov tseem ceeb ntawm cov khoom_type thiab _timestamp teb (piv txwv li khoom_2020_10):

tableName: '{{.product_type}}_{{._timestamp.Format "2006_01"}}'

Txawm li cas los xij, cov qauv ntawm cov xwm txheej tuaj yeem hloov pauv thaum lub sijhawm ua haujlwm. Peb tau siv lub algorithm los xyuas qhov sib txawv ntawm cov qauv ntawm lub rooj uas twb muaj lawm thiab cov qauv ntawm qhov kev tshwm sim tuaj. Yog tias pom qhov sib txawv, lub rooj yuav raug kho nrog cov teb tshiab. Txhawm rau ua qhov no, siv thaj SQL lus nug:

#Пример для Postgres
ALTER TABLE "schema"."table" ADD COLUMN new_column character varying

architecture

Vim li cas koj thiaj li yuav tsum tau sau cov xwm txheej rau cov ntaub ntawv kaw lus, thiab tsis yog sau ncaj qha rau hauv cov ntaub ntawv? Databases tsis tas yuav ua tau zoo thaum cuam tshuam nrog ntau cov ntaub ntawv ntxig (Postgres cov lus pom zoo). Ua li no, Logger sau cov xwm txheej tuaj rau hauv cov ntaub ntawv thiab hauv ib qho kev sib cais (xov) Cov ntaub ntawv nyeem cov ntaub ntawv, ces cov ntaub ntawv hloov dua siab tshiab thiab txiav txim siab. Tom qab tus thawj tswj Table ua kom paub tseeb tias lub rooj schema yog hloov tshiab, cov ntaub ntawv yuav raug sau rau hauv cov ntaub ntawv hauv ib pawg. Tom qab ntawd, peb ntxiv lub peev xwm los sau cov ntaub ntawv ncaj qha rau hauv cov ntaub ntawv, tab sis peb siv hom no rau cov xwm txheej uas tsis muaj ntau - piv txwv li kev hloov pauv.

Qhib Qhov Chaw thiab npaj rau yav tom ntej

Qee lub sij hawm, cov kev pabcuam tau pib zoo li cov khoom lag luam puv ntoob thiab peb tau txiav txim siab tso tawm rau Open Source. Tam sim no, kev koom ua ke nrog Postgres, ClickHouse, BigQuery, Redshift, S3, Snowflake tau ua tiav. Txhua qhov kev sib koom ua ke txhawb nqa ob qho tib si batch thiab streaming hom ntawm cov ntaub ntawv thauj khoom. Ntxiv kev txhawb nqa rau kev thov ntawm API.

Txoj kev sib koom ua ke tam sim no zoo li no:

Txawm hais tias qhov kev pabcuam tuaj yeem siv nws tus kheej (piv txwv li siv Docker), peb kuj muaj hosted version, nyob rau hauv uas koj tuaj yeem teeb tsa kev koom ua ke nrog lub tsev khaws ntaub ntawv, ntxiv CNAME rau koj lub npe thiab saib cov txheeb cais ntawm cov xwm txheej. Peb cov phiaj xwm tam sim no yog ntxiv lub peev xwm los sib sau ua ke tsis yog tsuas yog txheeb cais los ntawm lub vev xaib, tab sis kuj tseem muaj cov ntaub ntawv los ntawm cov ntaub ntawv sab nraud thiab khaws cia rau txhua qhov chaw koj xaiv!

→ GitHub
→ Ntaub ntawv
→ slack

Peb yuav zoo siab yog tias EventNative pab daws koj cov teeb meem!

Tsuas yog cov neeg siv sau npe tuaj yeem koom nrog hauv daim ntawv ntsuam xyuas. Kos npe rau hauvthov.

Dab tsi txheeb cais sau system siv hauv koj lub tuam txhab?

48,0%Google Analytics12
4,0%Ntu 1
16,0%Lwm tus (sau hauv cov lus) 4
32,0%Ua raws li koj qhov kev pabcuam 8

25 cov neeg siv pov npav. 6 cov neeg siv txwv tsis pub siv.

Tau qhov twg los: www.hab.com

Keeb kwm ntawm peb qhov chaw qhib: yuav ua li cas peb tau ua qhov kev tshuaj ntsuam xyuas hauv Go thiab ua rau nws muaj rau pej xeem

Vim li cas peb yuav tsum txhim kho peb tus kheej kev pabcuam?

Qhov kev pabcuam ua haujlwm li cas

architecture

Qhib Qhov Chaw thiab npaj rau yav tom ntej

Dab tsi txheeb cais sau system siv hauv koj lub tuam txhab?

Ntxiv ib saib Отменить ответ